Embeddings
Transforming text into a numeric vector that captures its semantic meaning. Two similar texts have close vectors in embedding space. The foundation of RAG and semantic search. The vector comes out of a dedicated embedding model, not the LLM that answers: Cohere and OpenAI offer them through an API, as [Bashiri Smith](https://www.instagram.com/reel/Dd9QONJlGEG/) points out. One rule is unforgiving: the user's question must go through the same model as the indexed documents. Two different models produce two spaces that do not compare, and the search still returns results, just the wrong ones. Corollary: switching embedding model means re-embedding the whole base, not only the new documents.
Strengths
- Search by meaning rather than keywords, much more relevant results
Limitations
- Generation cost: each text must be vectorized via API (OpenAI, Voyage, etc.)
Best for
- Understanding how semantic search and RAG work