Skip to section
Foundationsfor rotation-free search
Section 15 of 5229% of course
Contents
Chapter 3 Section 3.2 60 min

Part II · The problem and its constraints

Similarity as a dot product

Connect the Chapter 1 operation to comparison and ranking.

Reuse an operation we already built

Let q be a query embedding and v be one corpus embedding. Their dot product multiplies matching coordinates and adds the products together:

score(q,v) = q · v = ∑i=0D−1qivi

The operation itself is no longer new to you. Only its job is new: it produces a comparison score. Each individual product asks whether one coordinate contributes agreement or opposition; the final sum collects all those local contributions into one single scalar.

Worked example

One comparison, one contribution at a time

For q=[2, −1, 3] and v=[3, 0, 1], coordinate 0 contributes 2×3=6, coordinate 1 contributes (−1)×0=0, and coordinate 2 contributes 3×1=3. The total score is 6+0+3=9. Writing out the contribution table is definitely slower than mental arithmetic, but it makes the matching-position rule beautifully explicit—and that's the exact rule our later layout must preserve.

A score is meaningful only relative to alternatives

The number 9 has no universal semantic label. It only becomes useful when compared with other scores produced by the exact same model, preprocessing, and normalization policy. If another corpus vector scores 4 and a third scores 11, the relative ordering 11 > 9 > 4 is incredibly useful, unlike trying to interpret a naked “9” in isolation.

Length can seriously distort a raw score

Multiplying a vector by 10 multiplies its raw dot product by 10 without changing its direction at all. A large-magnitude vector can therefore easily win a search because it is long, not because it points in a more similar direction. Most embedding pipelines normalize each list so its length is exactly one. For unit-length vectors, the dot product perfectly equals cosine similarity and naturally lies between −1 and 1.

Worked example

Magnitude can fully reverse a ranking

Let q=[1, 0]. Candidate a=[0.9, 0.1] points almost the exact same way as q. Candidate b=[4, 3] is far less directionally aligned but has a raw dot product of 4, compared with just 0.9 for a. If direction is your intended notion of similarity, you must normalize before scoring. After normalization, a will correctly end up much closer to q.

Optional geometry refresher

The length of a is ‖a‖=√(a·a). Cosine similarity is defined as (a·b)/(‖a‖‖b‖). If both lengths are exactly one, the denominator beautifully disappears. The cryptographic construction needs only the dot product; normalization can easily happen in plaintext before encoding.

Check your understanding

If normalized embeddings have unit length, why is their dot product so convenient?

Interpretation check

A candidate receives the largest correctly computed score. What has actually been proved?

Section summary

  • The comparison score is just a Chapter 1 dot product.
  • Normalization prevents magnitude from distorting the results.
  • Arithmetic correctness does not magically guarantee model quality.
Reactive Python laboratory · marimo + PyodideVectors from boxes to similarity

Edit coordinates and inspect alignment, contribution traces, norms, dot products, and cosine similarity.

Open full-screen lab ↗

Runs entirely in this browser. Python executes in Pyodide WebAssembly with no remote kernel. The construction code stays visible while reactive dependents recompute whenever you change an input.

Repository layer · second pass

When does a dot product behave like a similarity measure?

A raw dot product rewards both directional agreement and large magnitudes. Cosine similarity divides by the two vector lengths so that only the angle remains. If the embedding pipeline normalizes every vector to unit length, the denominator is one and the dot product already equals cosine similarity.

Finite precision means normalization and scoring are approximate. The useful systems question is not whether every bit is exact, but whether the numerical error is small compared with the margin separating plausible neighbors.

Reasoning chain

  1. 1

    Compute or verify each vector norm.

  2. 2

    Normalize if the model contract requires it.

  3. 3

    Form matching-coordinate contributions.

  4. 4

    Sum to obtain the score.

  5. 5

    Interpret the scale only under the declared normalization contract.

Worked trace

Magnitude can disguise direction

  1. q=[1,0].
  2. a=[2,0] has raw dot 2 and cosine 1.
  3. b=[100,1] has raw dot 100 and cosine just under 1.
  4. Raw magnitude makes b look much larger despite a being perfectly aligned.

Result. Normalization determines whether raw dot scores are comparable as direction-only similarity.

Executable lens · Python

Make the hidden state visible

from math import sqrt
def cosine(a, b):
    dot = sum(x*y for x,y in zip(a,b))
    na = sqrt(sum(x*x for x in a))
    nb = sqrt(sum(y*y for y in b))
    return dot / (na * nb)

Retype this example, predict each intermediate value, and then change one input that touches a boundary.

Misconception clinic

Tempting mistakes

  • Calling every dot product cosine similarity.
  • Normalizing after encryption without budgeting the much harder inverse-square-root circuit.

Retrieval and transfer

Close the book first

  1. Find two vectors with the same cosine but different raw dot products.
  2. Explain how score margins govern acceptable approximation.
  3. State what a negative cosine says geometrically.