Text Embeddings Inference (TEI)
Hugging Face's dedicated server for embedding and reranking models, built for throughput: batching, quantisation and hardware-specific kernels to turn text into vectors quickly at scale. It does one job, serving embeddings and rerankers behind an API, and does it fast, which matters because embedding is often the bottleneck in a RAG pipeline.
Strengths
- High performance via Flash Attention and dynamic token-based batching
- Runs on CPU, NVIDIA GPUs, and Apple Silicon, with small images and fast startup
- Production features: OpenTelemetry tracing, Prometheus metrics, HTTP and gRPC
Weaknesses
- Some GPU paths are experimental (Turing, Blackwell, ROCm)
- GPU use needs a recent CUDA
- Scope is limited to embeddings and reranking, not general LLM serving
At a glance
- Licence
- Apache-2.0
- Pricing
- Free
- Platforms
- macOS, Linux, Docker
- GPU required
- No
Links
Works with
Related guides
Glossary
Entry last verified 18 August 2026.