Why build hardware for this computation

Software schedules instructions; hardware arranges machinery

If you're coming from a software background, you're used to writing a program that hands instructions to a processor. The processor decides how to schedule and execute them. But in hardware design (specifically Register Transfer Level, or RTL), you aren't writing instructions. You're arranging the physical machinery—the registers, the arithmetic units, and the wires connecting them. That's why thinking in terms of "running the next line of code" will trip you up. In hardware, multiple pieces of logic are active simultaneously, and registers are the gatekeepers that decide which results get saved for the next clock cycle.

Look for the repetitive heavy lifting

Think about our scoring path. We're repeating the exact same modular math operations over thousands of polynomial positions, for thousands of corpus objects. It's incredibly regular, the data widths never change, and the corpus streams in steadily. That kind of predictable, high-volume repetition is exactly what custom hardware is hungry for.

Don't just count the multipliers

It's tempting to think you can just stamp out a bunch of multipliers to go faster. But a multiplier that can crunch 16 values per cycle is just going to sit there twiddling its thumbs if memory can only feed it one value at a time. On the flip side, paying for ultra-wide memory bandwidth is a waste if your math units keep stalling. Hardware is all about balancing the triangle:

def offload_time(batch, setup_ms=3, kernel_ms=.02):
    return setup_ms + batch * kernel_ms
def cpu_time(batch, per_ms=.05):
    return batch * per_ms