Imagine searching your company’s IT helpdesk knowledge base for “My laptop won’t turn on.” One document describes a computer that fails to start when you press the power button. Another recommends laptop sleeves for international travel.

A simple filter for the word laptop would return the article about sleeves. The document about startup problems is the one you’d want to read first. How can a search system find it when the wording is different?

From Token IDs to Text Embeddings

In my earlier article on tokenization, I explained how text splits into tokens and how each token maps to an ID. A token ID identifies a token in the tokenizer’s vocabulary. For semantic search, we feed a sentence or short passage into an embedding model. The model returns a text embedding, a vector of numbers that we can compare with vectors for other texts.

Ranking Results by Vector Similarity

We use the same embedding model for the query and the documents. Our application can then compare the query vector with each document vector using cosine similarity and rank the results by score.

In the helpdesk example, the document about a computer failing to start can score higher against the query because it describes the same kind of problem, even though it uses different words. The score helps us decide what to read first. The document itself tells us whether it addresses the problem.

How Semantic Search Fits into RAG

Embeddings can help recommend articles on similar topics or find relevant passages for retrieval-augmented generation (RAG). The embedding model produces the vectors, and the search layer retrieves related passages. In a RAG application, a language model can then use those passages to answer the question.

In the AI Espresso video, I use four short helpdesk documents to compare the results of a simple keyword filter with those of a vector similarity search.