Objects become comparable only when their vectors share a coordinate system and preprocessing contract.
The actual problem finally arrives
Real search systems need to compare text, images, audio, or entire database records. Since computers can't directly multiply two paragraphs or subtract two photographs, a model first maps each object into an ordered list of numbers.
Think of that mapping process like a compiler boundary. A compiler doesn't preserve the readable surface spelling of your program; it produces a representation strictly suited to later machine operations. Likewise, an embedding model doesn't preserve the original sentence as readable text. It produces a numerical representation specifically suited to similarity comparison. The analogy stops there, though: an embedding is learned directly from data, and it absolutely does not promise exact round-trip reconstruction.
The model completely decides the coordinates. Coordinate 17 usually does not have a simple human label like “quietness.” The meaning is distributed holistically across the entire list. This is exactly why we treat the result as a whole ordered object, rather than trying to interpret each entry independently.
Worked example
Do not read an embedding as a feature table
Suppose two sentences receive [0.81, −0.22, 0.04] and [0.79, −0.19, 0.08] in a toy three-coordinate model. Their closeness is incredibly useful evidence that the model placed them similarly. It does NOT justify announcing that the first coordinate means “topic,” the second means “tone,” and the third means “length.” Those human labels were never part of the model's contract.
Dimension is just list length
If an embedding contains D numbers, its dimension is simply D. Chapter 1 already gave us the valid positions for this list: 0 through D−1. In this system, supported values of D are 128, 256, 512, 1024, 2048, and 4096.
Dimension is a structural property, not a quality score. A 4096-dimensional embedding is not automatically “four times smarter” than a 1024-dimensional one. It just occupies four times as many coordinate positions, which radically changes storage, arithmetic cost, and packing density. Whether it actually represents meaning better is an empirical question you have to test.
Worked example
The same model is part of the contract
A query embedded by one model cannot be safely compared with a corpus embedded by a completely unrelated model. Even if both lists have a length of 768, their coordinate systems won't mean the same thing. Model identity and preprocessing both belong strictly in the data contract.
Check your understanding
Why must the query and corpus embeddings share a model and preprocessing contract?
Transfer check
Two embedding models both return 768 numbers. Is that enough to compare their outputs coordinate by coordinate?
Section summary
- An embedding maps a rich, complex object to an ordered numerical list.
- D is simply the list length.
- Model and preprocessing identity fully define the coordinate system.
- An embedding can absolutely still be sensitive private data.
Repository layer · second pass
What contract turns text, images, or records into comparable lists?
An embedding model is a deterministic coordinate-system builder. Given an object and an exact preprocessing pipeline, it emits a fixed-length list. Individual coordinates rarely have human-readable names; their meaning lives in the model as a whole. Comparability therefore requires the same model version, tokenization, normalization, and coordinate order on both sides.
Treat an embedding as an interface value with provenance, not as an arbitrary float array. Length is necessary but insufficient: two models can both emit 768 numbers whose positions mean unrelated things.
Reasoning chain
- 1
Define the source-object preprocessing.
- 2
Pin the model and tokenizer versions.
- 3
Record the output dimension and numeric type.
- 4
Apply the identical contract to queries and corpus items.
- 5
Reject vectors whose provenance does not match.
Worked trace
Compatible shape, incompatible meaning
- Model A and Model B both emit length 384.
- A query is encoded by A; a document was encoded by B.
- Array multiplication succeeds mechanically.
- The positions do not belong to the same learned coordinate system.
Result. The score is numerically defined but semantically meaningless.
Executable lens · Python
Make the hidden state visible
from dataclasses import dataclass
@dataclass(frozen=True)
class Embedding:
values: tuple[float, ...]
model_id: str
def comparable(a, b):
return a.model_id == b.model_id and len(a.values) == len(b.values)Retype this example, predict each intermediate value, and then change one input that touches a boundary.
Misconception clinic
Tempting mistakes
- Assuming equal dimensions imply compatible coordinates.
- Re-embedding only the query after changing model versions.
Retrieval and transfer
Close the book first
- Design metadata that makes an embedding self-describing.
- List three preprocessing changes that require corpus re-embedding.
- Explain why shuffling every vector with the same permutation preserves dot products.