Embeddings Explained for Software Engineers

Why embeddings matter even if you never train a model

Most software engineers now touch embeddings indirectly, whether or not they ever train a model themselves. A search box that understands synonyms, a recommendation widget, a chatbot that can answer questions about your internal documentation, a feature that flags duplicate support tickets: all of these are commonly built today by calling an embeddings API or library, storing the resulting vectors somewhere, and comparing them. You do not need a machine learning background to use embeddings, but you do need to understand what they are, because the moment something goes wrong, the debugging questions are specific: why did the search return irrelevant results, which distance metric should I use, how big should my vectors be, why did results change after I swapped models.

This article builds the idea from scratch. We start with the problem embeddings solve, build intuition with a tiny worked example you can follow by hand, give a precise definition, look at how embeddings are actually learned, work through a short runnable code example, tour the places embeddings show up in real systems, and finish with the mistakes that trip people up most often.

The problem embeddings solve

Computers are naturally good at exact comparisons: string equality, hash lookups, a WHERE clause in SQL. This works well for a huge number of problems but fails for an equally important class of problems: matching things that mean the same thing but are expressed differently. A help desk search for "cannot log in" should find a ticket whose text says "unable to sign into my account", even though the two phrases share no words at all. A product search for "sneakers" should surface listings titled "running shoes". A code search for "parse a JSON file" should find a function whose docstring says it "loads a JSON document from disk" and never uses the word parse. Traditional full text search, even with stemming, synonym lists and fuzzy matching, is still fundamentally comparing surface forms: characters and words. It runs out of road quickly once two pieces of text that mean the same thing stop looking alike.

Embeddings reframe the problem entirely. Instead of asking whether two pieces of text share the same words, you ask whether they land close together in a space that was built so that nearby points mean similar things. That shift, from comparing strings to comparing geometry, is the single idea underneath modern search, recommendation systems and retrieval-augmented generation.

From symbols to numbers: building intuition with a toy example

Before any formal definition, it helps to build intuition with a deliberately tiny, made-up example. Imagine describing foods using two invented numeric features: sweetness, from 0 (not sweet) to 1 (very sweet), and size, from 0 (small) to 1 (large). Give apple the coordinates (0.6, 0.3), banana (0.65, 0.35), and, for contrast, a car (0.0, 0.9) since a car is large but not sweet at all. Each item is now represented as a short list of numbers, a vector, placed at a specific point in a two-dimensional space. That is, at its core, what an embedding is: a fixed-length list of numbers standing in for something that is not naturally numeric.

Now we can measure how close two items are using plain arithmetic. One common measure is cosine similarity: take the dot product of the two vectors and divide by the product of their lengths. For apple (0.6, 0.3) and banana (0.65, 0.35), the dot product is 0.6 times 0.65 plus 0.3 times 0.35, which is 0.39 plus 0.105, or 0.495. The length of the apple vector is the square root of 0.6 squared plus 0.3 squared, which is the square root of 0.45, about 0.671. The length of the banana vector works out to about 0.738. Dividing 0.495 by 0.671 times 0.738 gives roughly 1.00, meaning apple and banana point in almost exactly the same direction. Compare apple to car: the dot product is 0.6 times 0 plus 0.3 times 0.9, which is 0.27, and the car vector has length 0.9, so the cosine similarity is 0.27 divided by 0.671 times 0.9, about 0.45, clearly lower. A second common measure, Euclidean distance, the ordinary straight-line distance, tells the same story: apple to banana is the square root of 0.0025 plus 0.0025, about 0.07, while apple to car is the square root of 0.72, about 0.85. Both measures agree: apple and banana are close, car is far away, purely because of the numbers we chose.

• Similar items end up close together in the vector space, and dissimilar items end up far apart.
• Different distance measures (cosine similarity, Euclidean distance) can agree on which pairs are close, though they do not always behave identically, as we will see later.
• Nobody told the computer that apples and bananas are both fruit. The closeness fell directly out of the numbers we assigned to each feature.
• Real embeddings are not limited to two hand-picked features like sweetness and size. They typically have anywhere from a few dozen to a few thousand dimensions, learned automatically rather than chosen by a person, which lets them capture far more subtle shades of meaning than any human-designed feature set could.

A precise definition

An embedding is a mapping from a discrete object, such as a word, a sentence, a document, an image, a user, or a product, to a vector of real numbers of a fixed length, constructed or learned so that geometric relationships in that vector space, such as distance or angle, correspond to some meaningful relationship between the original objects, such as similar meaning, similar topic, or frequent co-purchase. Two properties are worth separating out. First, the length is fixed regardless of the size or complexity of the input: a single word and a two-thousand-word document can both become, say, a 384-number vector. Second, the representation is dense: in most embeddings, nearly every one of those numbers is a nonzero floating point value, in contrast to sparse representations where almost all values are zero.

• Fixed length: every object, however long or short, becomes a vector of the same size.
• Dense: most or all dimensions hold meaningful nonzero values, rather than mostly zeros.
• Usually learned from data rather than hand designed, through a training process described in the next section.
• Only meaningful relative to other vectors produced by the same model; a vector from one embedding model cannot be sensibly compared to a vector from a different model.
• The number of dimensions is a design and training choice, commonly ranging from around one hundred to a few thousand for text.

Embeddings vs one-hot encoding

It helps to see embeddings against the baseline they replaced: one-hot encoding. Given a vocabulary of distinct words, assign each word an index, then represent that word as a vector of zeros with a single 1 at its index. This is simple and perfectly captures identity, but it throws away any notion of similarity. If your vocabulary has five words, cat, dog, car, apple and banana, then cat might be [1, 0, 0, 0, 0] and dog might be [0, 1, 0, 0, 0]. Their dot product is zero and the Euclidean distance between any two distinct one-hot vectors is always the square root of 2, about 1.41. Cat is exactly as far from dog as it is from car. For a realistic vocabulary of fifty thousand words, every vector would be fifty thousand numbers long with only one of them nonzero, which is both wasteful and semantically flat: there is no way to ask "which words are similar to cat" using these vectors at all. Embeddings solve both problems at once: far fewer dimensions, and distances that actually mean something.

import numpy as np

vocab = ["cat", "dog", "car", "apple", "banana"]

def one_hot(word):
    vec = np.zeros(len(vocab))
    vec[vocab.index(word)] = 1
    return vec

cat = one_hot("cat")
dog = one_hot("dog")
car = one_hot("car")

print(np.dot(cat, dog))            # 0.0
print(np.dot(cat, car))            # 0.0
print(np.linalg.norm(cat - dog))   # 1.4142135623730951
print(np.linalg.norm(cat - car))   # 1.4142135623730951

How embeddings are learned

Most embedding methods build on an old idea in linguistics, the distributional hypothesis: a word is known by the company it keeps, meaning words that tend to appear in similar surrounding contexts tend to have similar meaning. The word2vec method, published by researchers at Google in 2013, turned this idea into a concrete training procedure: train a small neural network to predict a word from the words around it, or the words around a word from the word itself. Crucially, nobody actually wants the network's predictions once training finishes. What gets kept is the hidden layer of weights, because each word's row in that weight matrix ends up sitting near the rows of words used in similar contexts. This pattern repeats across almost every embedding method you will encounter: define an auxiliary task that is cheap to generate training examples for, train a network on it, then discard the output and keep the internal representation.

A frequently cited illustration from the word2vec research is vector arithmetic on word embeddings: the vector for "king", minus the vector for "man", plus the vector for "woman", lands close to the vector for "queen". Treat this as a demonstration of what embedding spaces can capture, namely that a relationship like gender can show up as a roughly consistent direction in the space, rather than as a guaranteed property you should expect from every embedding model on every pair of words.

It is worth separating static embeddings from contextual embeddings. Word2vec and a related method called GloVe produce one fixed vector per word, regardless of context, so the word "bank" gets a single vector that has to blend the "river bank" sense and the "savings bank" sense together. Transformer-based language models, such as BERT and the GPT family, produce contextual embeddings instead: the vector for "bank" differs depending on the surrounding sentence, because the model processes the whole sequence of words together through a mechanism called self-attention. For whole sentences or documents rather than single words, sentence embedding models, such as those in the widely used sentence-transformers library, take a pretrained transformer and add a pooling step, often averaging the per-word vectors into a single sentence vector, along with additional training. That additional training frequently uses contrastive learning: the model is shown many pairs of texts known to be similar, such as a question and its correct answer or two paraphrases of the same sentence, alongside pairs known to be dissimilar, and its weights are adjusted so similar pairs end up close together in the vector space while dissimilar pairs end up far apart.

Commercial embedding APIs work the same way in spirit: you send text, you get a fixed-length vector back. Their exact training data and procedure are usually not published in full detail, so it is reasonable to treat them as black boxes. The practical rule is simply to pick one embedding model or API for a given system and stay consistent, since vectors from different models are not comparable to each other.

Measuring similarity: the geometry of embedding space

Cosine similarity takes the dot product of two vectors and divides it by the product of their lengths, which produces a value between -1 and 1. A value near 1 means the vectors point in almost the same direction, near 0 means no particular relationship was picked up, and near -1 means opposite directions. Cosine similarity cares about direction, not magnitude: scale a vector by any positive number and its cosine similarity to everything else stays exactly the same. That is usually what you want for text, because the raw length of an embedding vector often correlates with incidental factors like sentence length or how common the words are, rather than with meaning.

Euclidean distance, the ordinary straight-line distance between two points, is the other common option. It squares the difference along each dimension, sums those squares, and takes the square root, so unlike cosine similarity it is sensitive to both direction and magnitude. If an embedding model's vectors are L2-normalized, meaning each vector is rescaled to have length exactly 1, then cosine similarity and Euclidean distance rank pairs in the same order, because the two measures become mathematically tied together for unit-length vectors. Many sentence embedding models, including the default behavior of sentence-transformers, produce normalized vectors for exactly this reason. The plain dot product, with no division at all, is the cheapest of the three to compute and is often what vector databases use internally once they know the stored vectors are normalized.

The practical rule is to check the documentation of whatever embedding model you use for the distance metric it was designed and evaluated around, and then use that metric consistently throughout your system. Most text embedding models are built with cosine similarity in mind.

Seeing it in code

The following example uses sentence-transformers, an open source Python library that runs locally without an API key, to turn four sentences into vectors and compare them. The example assumes sentence-transformers version 2.x and a recent version of PyTorch; check your installed versions if the exact output shape or numbers look different, since minor library updates can shift results slightly.

# pip install sentence-transformers
# tested with sentence-transformers==2.7.0

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

sentences = [
    "The cat sat on the mat.",
    "A kitten is resting on the rug.",
    "The stock market fell sharply today.",
    "Shares dropped after the earnings report.",
]

embeddings = model.encode(sentences)
print(embeddings.shape)

Running this prints (4, 384): four sentences, each turned into 384 numbers. The dimension count, 384 in this case, is a fixed property of the chosen model rather than something you choose per call; a different model might produce 768 numbers or some other fixed size instead.

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

labels = ["cat/mat", "kitten/rug", "stocks fell", "shares dropped"]

for i in range(len(sentences)):
    for j in range(i + 1, len(sentences)):
        sim = cosine_similarity(embeddings[i], embeddings[j])
        print(f"{labels[i]:15s} vs {labels[j]:15s}: {sim:.2f}")

You should see a clear pattern rather than specific memorized numbers: "cat/mat" compared with "kitten/rug" should come out noticeably similar, despite sharing almost no words, because both describe a small animal resting on a textile surface. "Stocks fell" compared with "shares dropped" should likewise come out similar, for the same reason applied to financial news. The cross pairs, such as the cat sentence compared with either stock market sentence, should come out clearly lower, since the topics are unrelated. The exact decimal values depend on the model and even on the specific library version, so do not hard-code similarity thresholds without checking them against your own data first. The reliable part is the ranking, which pairs are more similar than which others, not the precise number.

Where embeddings show up in real systems

Semantic search and retrieval-augmented generation

Retrieval-augmented generation, usually shortened to RAG, is one of the most common reasons engineers reach for embeddings today. The pipeline runs in two phases. In the indexing phase, you take your documents, split each one into chunks of a manageable size, such as a paragraph or a few hundred words, embed each chunk with an embedding model, and store the resulting vector alongside the original chunk text and any metadata in a vector index. In the query phase, you embed the user's question using the same embedding model, search the index for the stored vectors nearest to the query vector, retrieve the corresponding text chunks, and insert that text into the prompt sent to a language model, which then generates an answer grounded in the retrieved content rather than relying purely on what it memorized during its own training. The word same in that description matters: the query and the documents must be embedded with the same model, otherwise their vectors live in incompatible spaces and the nearest neighbor search becomes meaningless.

Recommendations

Recommendation systems often learn an embedding for each user and each item from historical interaction data, such as purchases, clicks or ratings, so that users who interact with similar items end up with similar embeddings, and items that are frequently bought or viewed together end up close to each other as well. Generating a recommendation then becomes a nearest neighbor search: find items whose embedding is close to a given user's embedding, or close to items that user has already shown interest in.

Deduplication and clustering

Embeddings are a natural tool for finding near-duplicate content, such as support tickets describing the same underlying issue in different words, or crawled web pages that are mostly repeated content. You embed each item and flag pairs whose similarity exceeds some threshold as likely duplicates. The same vectors can be fed into a clustering algorithm, such as k-means, to group similar feedback or tickets into topics automatically, without needing a predefined set of categories up front.

Classification and anomaly detection

Embeddings also work well as ready-made features for classic machine learning. Instead of training a deep model from scratch on a small labeled dataset, you can embed each piece of text with a pretrained embedding model and train a much simpler classifier, such as logistic regression, on top of those vectors. This often performs surprisingly well with modest amounts of labeled data, because the embedding model has already learned a great deal about general language structure. Anomaly detection follows a similar idea in reverse: flag a point as an outlier if its nearest neighbor in embedding space is unusually far away compared to typical distances in the dataset.

Storing and searching embeddings at scale

Comparing a query vector against every stored vector, known as brute force or exact nearest neighbor search, is fine for a collection of a few thousand items but becomes slow once you have millions of vectors and need to answer queries repeatedly, since the cost of each query grows directly with the number of stored vectors. Approximate nearest neighbor algorithms trade a small, usually acceptable, amount of accuracy for a large gain in speed by avoiding a full comparison against every stored vector. Two families are common in practice: graph-based methods, such as HNSW, which build a graph connecting each vector to a handful of its closest neighbors and then navigate that graph at query time instead of scanning everything, and partition-based methods, such as IVF, which pre-group vectors into clusters and only search the clusters most likely to contain the answer.

Several concrete tools exist for this. FAISS is a widely used library for approximate nearest neighbor search that you run yourself. Pgvector adds vector columns and approximate nearest neighbor indexes directly to PostgreSQL, which is convenient if you already run Postgres. Managed vector databases such as Pinecone, Weaviate and Milvus offer this as a hosted service. This area changes quickly, with new index types and features appearing regularly, so check each tool's current documentation for supported distance metrics, index types and scaling limits rather than treating any particular configuration as fixed.

It is worth doing the storage arithmetic by hand before committing to a dimension size. A 1536-dimension embedding stored as 32-bit floating point numbers takes 1536 times 4 bytes, which is 6144 bytes, roughly 6 kilobytes, per vector. For one million documents, that works out to roughly 6 gigabytes just for the raw vectors, before accounting for index overhead, metadata, or replication across servers. Choosing a smaller embedding model, when accuracy allows it, is a real cost and performance lever, not just a default setting to accept without thinking about it.

Common mistakes and pitfalls

Mixing embeddings from different models or versions. Vectors produced by two different embedding models, or even two versions of the same model, are generally not comparable: distances and similarity scores only carry meaning relative to other vectors produced by that exact same model with the same preprocessing. A common bug in practice is re-indexing only newly added documents after upgrading to a new embedding model while older documents keep their vectors from the previous model, silently corrupting search quality because old and new vectors now live in different, incompatible spaces.

Using the wrong distance metric. If a model was designed and evaluated using cosine similarity but your system stores raw, unnormalized vectors and computes Euclidean distance instead, result rankings can shift in ways that quietly make search worse, without producing any error message to alert you.

Chunking text badly for retrieval-augmented generation. Chunks that are too large blend several unrelated ideas into a single vector, blurring its meaning and making it a worse match for any one specific question. Chunks that are too small lose the surrounding context needed to make sense of a sentence; a chunk that just says "it increased by 12 percent" is close to meaningless on its own. There is no universal correct chunk size, it needs to be tuned and evaluated against your own documents and the kinds of questions users actually ask.

Treating similarity as truth. A retrieved chunk being the closest match in vector space does not mean it is correct, current, or actually relevant to what the user needs. Embeddings capture statistical patterns in how language is used, not verified facts. A retrieval-augmented system can confidently retrieve a wrong or outdated chunk, and a language model can then generate a fluent, confident answer built on top of that wrong chunk.

Skipping evaluation. It is tempting to look at a handful of search results, decide they look reasonable, and move on. A more reliable approach is to build a small labeled test set of queries paired with the document that should be retrieved for each one, and measure something concrete, such as recall at k, how often the correct document appears somewhere in the top k results, before and after any change to chunking strategy, embedding model, or preprocessing.

Ignoring preprocessing consistency. If text is lowercased, stripped of HTML, or truncated differently at indexing time compared to query time, the embedding model is effectively being given systematically different inputs at the two stages, which quietly degrades match quality in ways that are hard to notice without directly comparing the two code paths.

Summary and what to explore next

Embeddings turn discrete objects, words, sentences, documents, users, products, into fixed-length vectors of numbers arranged so that geometric closeness reflects some meaningful similarity. They are usually learned rather than hand designed, either as a side effect of training a model on an auxiliary task and keeping its internal representation, or more directly through contrastive learning on pairs of known similar and dissimilar examples. Cosine similarity and Euclidean distance are the two most common ways to compare embedding vectors, and the two agree on ranking when vectors are normalized to unit length. Embeddings power semantic search, retrieval-augmented generation, recommendations, clustering and lightweight classification, and at scale they require approximate nearest neighbor indexes along with real engineering decisions about dimension size, storage cost and consistency between indexing and querying.

• Run the sentence-transformers example on your own text to build direct intuition for how similarity scores behave.
• Read further on contrastive learning to understand how modern sentence embedding models are actually trained.
• Try a vector database; pgvector is a low-friction starting point if you already run PostgreSQL.
• Build a small labeled test set and measure recall at k before trusting a retrieval system in production.
• Look into hybrid search, which combines classic keyword search, such as BM25, with embedding-based search to get the strengths of both.