Embeddings & Vector Spaces: What They Actually Mean

How text becomes numbers, and how to reason about vector search. Part of AI Engineering for Backend Developers.

Imagine searching a help centre for “How do I stop a payment being processed twice?” The useful guide is titled “Idempotency keys for retry-safe requests.” Those phrases use different words, but they describe the same problem.

An idempotency key is a label a payment service can use to recognise a repeated request and avoid charging twice. You do not need to know the term to ask the question. A search tool should help you find the guide anyway.

Embeddings give a search system a way to compare text beyond exact word matches. A model converts each piece of text into a list of numbers. The system compares those lists and returns the closest matches. The useful trick is in how the model learns to produce the numbers.

You do not need machine learning experience or advanced math to follow this article. I will build the idea from a small example. The formulas and Python exercises are optional; the later sections cover the trade-offs that matter when building a real service.

Three terms will recur: the query is what someone searches for, a document is something they might want to find, and retrieval is selecting documents to return. A document could be a whole page or just one useful paragraph.

I separate the work into three steps: a model turns text into numbers, a scoring rule compares the numbers, and a search system selects results. Keeping these steps separate helps explain both useful matches and surprising mistakes.

From text to a vector

A vector is an ordered list of numbers. For example, [0.2, -0.7, 0.4] is a vector with three entries. Each entry is a coordinate; the number of entries is its dimension. A 384-dimensional vector simply has 384 entries.

An embedding is a vector used to represent something, such as a sentence. An embedding model is a trained program that calculates that vector from its input. The three numbers above are invented for illustration. In a real model, the numbers come from calculations using settings learned during training.

It helps to imagine a function: embed("avoid charging twice") → a list of numbers. Give the same model another sentence and it produces another list of the same length. A useful retrieval model produces vectors that receive higher similarity scores for the kinds of matches it was trained to find. This is learned behaviour, not a guarantee that every match is correct.

Text usually goes through several steps before it becomes one searchable vector. These are related, but they are not the same thing:

• Split text into tokens: a tokenizer breaks text into pieces, often words or parts of words, and gives each piece an integer ID. These IDs are labels. Two nearby IDs do not necessarily refer to similar words.
• Build token representations: the model looks up a starting vector for each token. Transformer layers then update the vectors using the surrounding text. These updated representations are often called hidden states.
• Produce one retrieval vector: the model combines or selects information from the token representations to represent the whole sentence or passage. An encoder is the part that turns the input into a representation.

The tokenization article and transformer architecture article explain those earlier steps in more detail. You can keep reading here with just the basic idea: text pieces go in, one vector comes out.

The final combining step is sometimes called pooling. One method, mean pooling, averages the token vectors coordinate by coordinate. It ignores padding, the blank positions added to make inputs fit a batch. A readout is the broader name for the step that produces the final vector; averaging is only one option.

For implementation: follow the embedding model’s documented readout. Some models select a particular token or apply a learned transformation called a projection. Averaging arbitrary vectors from a chat model does not automatically make a good search model. Sentence-BERT is an example of training a transformer to produce sentence vectors that can be compared usefully.

Where the geometry comes from

The model does not start out knowing that “duplicate payment” and “idempotency” belong together. Training changes its internal numerical settings, called parameters, using examples of useful and less useful matches.

For our payment question, a useful training example could pair the query with a guide to avoiding duplicate charges. A competing passage might discuss counting duplicate payments in a report. Both mention payments, but only one answers the question.

One common training approach encourages the relevant pair to score higher than the competing pair. This is called contrastive training: learning through comparisons. A misleading passage with similar vocabulary is a difficult negative example. The E5 paper describes one approach to learning embeddings this way.

After many comparisons, the model can learn useful relationships even when the wording changes. But what counts as “useful” depends on the training task. Finding two sentences with the same meaning is not quite the same as finding a passage that answers a question. A model that works well for one task may need evaluation or adaptation for another.

An embedding also leaves information out. This is what lossy means here: the vector is a compact representation, not a copy of the text. It may preserve the general topic while giving too little importance to a version number or a word such as “not.” That can change whether a result answers your question.

What a vector space actually means

With two coordinates, we can draw a vector as a point on a page. In [0.8, 0.6], the first number gives the horizontal position and the second gives the vertical position. We can also draw an arrow from zero to that point. Both pictures represent the same vector.

A vector space, for this discussion, is the mathematical setting in which these same-length lists can be added or multiplied by a number. A search system adds a rule for comparing them. With hundreds of coordinates, the arithmetic still works, even though we cannot draw every direction on a page.

Think of a map as a limited visual aid: points that are close under the chosen rule are possible matches. This is not a map of guaranteed meaning. Training determines which relationships the model represents, and the comparison rule determines what “close” means.

Does one coordinate measure “payment knowledge” and another “database knowledge”? Usually, no. Useful information is spread across combinations of coordinates. Looking at a single number rarely tells you what the sentence means.

Optional mathematical detail: the notation ℝⁿ means the set of all vectors with n real-number coordinates. A model’s outputs occupy only part of it. Applying the same rotation to every vector preserves dot products and Euclidean distances, even though the coordinates change. That is one reason to focus on relationships between vectors rather than naming individual axes.

Keep the original text alongside its vector. The vector is useful for comparisons; it is not a reliable way to reconstruct the source. Adding two embeddings is mathematically valid, but it does not guarantee a sentence that combines their facts.

For implementation: use query and document representations designed to work together. Two models can each output 384 numbers while placing related text in completely different locations. Matching the list length is not enough.

Keep the model version, readout, and input instructions compatible. Some models deliberately use different instructions or encoder paths for questions and documents. Sentence Transformers explains this distinction. “Symmetric” search compares similar kinds of text; “asymmetric” search can compare a short question with a longer answer passage.

Cosine, dot product, and Euclidean distance

Imagine two arrows that start at the same point. We can compare where their tips end up, how long they are, or which way they point. These are different questions, which is why there is more than one similarity measure:

• Dot product: multiply matching coordinates and add the results. For example, [1, 2] · [3, 4] = 1×3 + 2×4 = 11. Both direction and length affect this score. Larger values rank higher.
• Cosine similarity: compare the direction of the arrows, ignoring their lengths. A score of 1 means the same direction, 0 means a right angle, and −1 means opposite directions. Larger values rank higher. A vector containing only zeros has no direction, so cosine is undefined for it.
• Euclidean distance: measure the straight-line distance between the arrow tips. Smaller values rank higher. Moving a tip further away can change this distance even if the arrow keeps pointing the same way.

For example, [1, 0] and [10, 0] both point right. Their cosine similarity is 1, but their tips are 9 units apart. They have the same direction, not the same coordinates. Also, opposite directions in a learned space do not automatically mean opposite meanings in language.

Normalization is a way to put vectors on a common scale. L2 normalization changes each nonzero arrow to length 1 while keeping its direction. For example, it changes [10, 0] to [1, 0]. A length-1 vector is called a unit vector.

When both vectors have length 1, dot product and cosine give the same score. Euclidean distance gives a different number but the same exact ranking, apart from ties and rounding effects. This is why you will often see normalized vectors compared with a dot product. Sentence Transformers documents that equivalence.

Optional formulas: write the query vector as q and a document vector as v. The symbol · means dot product, and ‖q‖ means the length of q. Then cosine is (q · v) / (‖q‖ ‖v‖), and Euclidean distance is ‖q − v‖. For two unit vectors, ‖q − v‖² = 2 − 2(q · v): a larger dot product means a smaller distance.

For implementation: use the scoring and normalization rules recommended for your chosen model. Removing length information can hurt a model that uses it as part of its intended score.

A numerical example you can run

Let us compare three possible results using numbers small enough to draw. Our query is q = [1, 0]. The documents are A = [0.8, 0.6], B = [0.6, 0.8], and C = [−1, 0]. All four arrows have length 1. These are invented vectors for learning the arithmetic, not embeddings generated from text.

Because the query is [1, 0], its dot product with A is 1×0.8 + 0×0.6 = 0.8. The same calculation gives 0.6 for B and −1.0 for C. The vectors already have length 1, so these are also their cosine scores. A ranks first, then B, then C.

The distances from the query are approximately 0.632, 0.894, and 2.000. Choosing the smallest distance produces the same order. The picture and the arithmetic are telling the same story.

Optional Python exercise: save the following as vector_demo.py and run python3 vector_demo.py with Python 3.10 or newer. It uses only Python’s built-in standard library. You can skip the code and continue with the result below.

The cosine function calculates one score. sorted puts the highest score first, and each assert checks an expected result. The extra guards reject invalid vectors. Scaling before calculating the length avoids numerical problems with extremely large or small values.

from math import fsum, hypot, isclose, isfinite


def cosine(a, b):
    if not a or len(a) != len(b):
        raise ValueError("Expected equal, non-empty vectors")
    if not all(isfinite(x) for x in (*a, *b)):
        raise ValueError("Coordinates must be finite")
    sa, sb = max(map(abs, a)), max(map(abs, b))
    if not sa or not sb:
        raise ValueError("Expected nonzero vectors")
    # Scale first to avoid overflow or underflow in the norm.
    a, b = [x / sa for x in a], [x / sb for x in b]
    na, nb = hypot(*a), hypot(*b)
    return fsum((x / na) * (y / nb)
                for x, y in zip(a, b, strict=True))


q = [1.0, 0.0]
documents = {
    "A": [0.8, 0.6],
    "B": [0.6, 0.8],
    "C": [-1.0, 0.0],
}
ranked = sorted(
    ((name, cosine(q, v))
     for name, v in documents.items()),
    key=lambda item: item[1], reverse=True,
)
assert [name for name, _ in ranked] == ["A", "B", "C"]
assert isclose(ranked[0][1], 0.8)
assert isclose(cosine(q, [10.0, 0.0]), 1.0)
for magnitude in (5e-324, 1.7e308):
    v = [magnitude, magnitude]
    assert isclose(cosine(v, v), 1.0)
for invalid in ([], [1.0], [0.0, 0.0], [float("nan"), 0.0]):
    try:
        cosine(q, invalid)
    except ValueError:
        pass
    else:
        raise AssertionError("Invalid vector accepted")
for name, score in ranked:
    print(f"{name}: {score:.3f}")

Output: A: 0.800, B: 0.600, C: -1.000. The ranking code never reads a sentence. It only compares numbers. The embedding model has to make those numbers useful before the search step begins.

Generating real embeddings

Optional hands-on example: now we can replace our invented vectors with real text embeddings. I use a small, ready-made model called intfloat/e5-small-v2. You can use a pretrained model without training one yourself.

Its model card, the documentation supplied with it, specifies English input, 384 numbers per embedding, and a limit of 512 tokens per input. It also requires the prefixes query: and passage: for retrieval. These labels tell the model which kind of input it is receiving. See the E5-small-v2 model card.

This exercise assumes you can run a Python script and install packages. Use Python 3.12 in a separate environment, so the exercise’s packages do not interfere with another project. Install the tested versions with python -m pip install sentence-transformers==6.0.1 transformers==5.16.1 torch==2.13.0 numpy==2.5.2. Save the example as embedding_search_demo.py and run python embedding_search_demo.py.

The first run downloads the model files from Hugging Face. The example then runs locally on the CPU, so no paid embedding API is needed. The long revision value fixes the exact model version used in this test.

Read the code in four steps: load the model, supply one question and three passages, convert them into vectors, then sort the passage scores. normalize_embeddings=True makes the vectors length 1. The shape (4, 384) means four vectors, each containing 384 numbers. The @ operation calculates all three document-to-query dot products.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "intfloat/e5-small-v2",
    revision="ffb93f3bd4047442299a41ebb6fa998a38507c52",
    device="cpu",
)
docs = [
    "Use an idempotency key to avoid processing a payment twice.",
    "A payment report lists daily totals and duplicate counts.",
    "A database connection pool reuses open connections.",
]
query = "How can I safely retry a payment request?"
inputs = ["query: " + query]
inputs += ["passage: " + text for text in docs]
vectors = model.encode(inputs, normalize_embeddings=True)
assert vectors.shape == (4, 384)
scores = vectors[1:] @ vectors[0]
assert int(scores.argmax()) == 0
for i in scores.argsort()[::-1]:
    print(f"{scores[i]:.3f}  {docs[i]}")

In the CPU test on September 2, 2026 (Python 3.12.13), the passage about avoiding duplicate payment processing ranked first at 0.859. The reporting passage scored 0.800, and the connection-pool passage scored 0.787, rounded to three decimals. These are measured results for this example and model revision, not a quality benchmark.

The model found a useful passage even though it did not repeat the question’s wording. But notice the fairly high score for the unrelated connection-pool passage. A score is not a percentage of correctness. Try another question, or add a passage that contradicts the useful one. A single successful search does not establish overall quality.

From vectors to a search system

A basic vector search service does the same work as the Python example, with more documents. It has two stages:

Before a search: split long documents into useful passages, create an embedding for each passage, and store it alongside the original text. Also keep where it came from, its version, and who is allowed to read it.

When someone searches: embed their question with the compatible model settings, compare it with the permitted document vectors, and return the best matching passages. Asking for the top 5 means returning the five highest-ranked candidates. This is often written as top-k, where k is the number requested. This is the basic semantic-search workflow.

Vector search returns existing material; it does not itself write a new answer. A system can pass the results to a language model to help generate an answer. That extra step is part of retrieval-augmented generation (RAG), and it needs its own checks.

The simplest approach checks every permitted vector. This is brute-force exact search. It is a useful starting point because you can see what the model and scoring rule produce without an index skipping candidates.

For larger systems: comparing N vectors with n coordinates each takes work proportional to N × n, before selecting the best results. Ten times as many vectors means roughly ten times as much scoring work in this simple scan.

An index is an extra data structure that helps locate results faster. An approximate nearest neighbor (ANN) index usually searches only part of the stored vectors. It trades the possibility of missing some exact nearest matches for faster retrieval.

One approach is HNSW, short for Hierarchical Navigable Small World. It links nearby vectors into several layers of a graph, then follows promising links during a search. You can understand vector search without implementing that graph. Its role is to find candidate vectors efficiently, not to interpret their documents. The HNSW paper explains the structure.

Two separate things can go wrong: the index may miss the nearest vector, or the nearest vector may belong to an unhelpful document. Exact search avoids the first kind of approximation, but it cannot guarantee the second kind of relevance.

For a small experiment, comparing vectors in memory may be enough. When you need storage, updates, and filtering, choose tools that fit those requirements. For example, pgvector adds vector search to PostgreSQL, with exact search and approximate indexes. A separate vector database is one option, not a prerequisite for learning the idea.

Failure modes that the geometry cannot solve

A score is not a probability

Cosine 0.8 does not mean “80% likely to answer the question.” It is a geometric score, not a calibrated confidence estimate. Its useful range depends on the model and data. The E5 FAQ, for example, explains why its scores can cluster at relatively high values. Do not copy a threshold from another model or tutorial.

Top-k search will still return the best available candidates when none is useful. If you search a cooking library for a payment-processing question, the closest recipe is still the wrong answer. Test questions with no answer in your collection, and allow the application to return “No useful result found.”

Related text can still be the wrong answer

“Refunds are available after 30 days” and “Refunds are not available after 30 days” share most of their words and subject matter. An embedding may represent that shared topic strongly. Whether it also preserves the critical distinction must be tested. Similar problems arise with version numbers, dates, API symbols, product codes, and exception clauses.

My starting point would be to keep exact matching for identifiers and try a combination of word-based search, also called lexical retrieval, and vector search for prose. This combination is often called hybrid search. It gives exact words and learned similarity a chance to contribute.

For harder ranking problems, a reranker can take the initial candidates and score them again. A cross-encoder reranker reads the question and each candidate together, instead of comparing separately computed embeddings. It adds work for every candidate, so measure whether better results justify the extra delay. Sentence Transformers describes this pattern.

The retrieval unit changes the result

Chunking means splitting a document into the pieces you will retrieve. One vector for an entire handbook has to represent many topics at once. A tiny fragment might lose the heading or condition that makes it understandable.

Start with coherent sections and keep useful headings. Check the model’s token limit before embedding. A section titled “Refunds for international orders” is more useful with its heading than a detached sentence saying “Allow 30 days.” The right chunk size depends on the documents and questions.

Keep a source ID or link and a version with each passage. This record of where information came from is called provenance. If the text changes, create a fresh embedding. If the document is deleted or access is removed, stop returning it. A stored vector is derived from a document; it does not decide whether that document is still current or permitted.

Filters and model changes are part of correctness

A good similarity score never grants permission to read a document. For example, a result belonging to another customer must not leave the trusted service just because it matches the query. Keep access rules with each passage and check authorization before returning text or sending it to another model.

Filtering also affects search quality. An approximate search can find several candidates and then lose most of them to a filter, leaving too few results. pgvector documents this behaviour. Test the actual permission, date, and category filters your application uses.

When changing models: treat the stored embeddings as data that needs rebuilding. Record the model name and version, vector length, text preparation, chunking rules, input prefixes, and normalization. Generate and evaluate the new vectors before directing queries to them. Rebuilding an index over old vectors does not move those vectors into the new model’s space.

What I would measure before adding complexity

Start with a small collection of questions and passages you have judged useful by reading them. Include different wording, exact identifiers, ambiguous questions, negation, and questions your documents cannot answer. Compare word-based search with exact vector search before adding an approximate index.

For a first experiment, inspect the first few results for each question and ask: “Would these help someone answer it?” For a repeatable evaluation, keep the same questions and record measures such as these:

• Relevance recall@k: how many of the known useful passages appear in the first k results? If there are four useful passages and the top five contain three of them, recall@5 is 3/4 = 75%. People judge usefulness; the model’s score does not define it.
• ANN recall@k: how many of the exact search’s top k results did the approximate search recover? If it recovers four of the exact top five, that is 4/5 = 80%. This measures the index’s accuracy, not whether the passages answer the question.
• Service behaviour: measure response time, results after filtering, handling of unanswered questions, document updates, and permission checks under realistic load.

This gives a practical debugging order. If exact search returns poor matches, investigate the model, documents, chunking, and task. If exact results are good but approximate results are poor, investigate the index and filters. If useful passages are retrieved but a generated answer is wrong, investigate the answer-generation step separately.

The mental model to keep

Text becomes a vector. Vectors receive comparison scores. Search returns the best available matches. Training makes those comparisons useful, but no step guarantees truth, permission, or a complete answer.

For a first project, use a few passages and one query, print the scores, and read the results. You have already built the essential mechanism. For a production service, add evaluation, compatible model versions, document updates, and access control. The core idea stays the same as the system grows.

Continue with the AI Engineering for Backend Developers series, or revisit transformer architecture to connect retrieval embeddings with the token representations inside a model.