Designing for latency and research iteration

Daniel B. is an engineering team lead at Optiver Shanghai, focused on building low-latency trading systems that help bring research into production.

Most teams treat a ground-up rebuild as a last resort. In the kind of systems I work on, you sometimes do it anyway because the market changes, models get heavier, and you eventually run into a different bottleneck. As trading has become more systematic and model-driven, we've been running into those bottlenecks more often. I’ve spent most of the last decade on APAC markets watching that play out across several generations of our trading stack. The details are specific to these markets, but the engineering problems aren’t unique. The trade-offs are relevant anywhere you're balancing latency with the speed of turning research into production.

If you don't work in trading, it's probably useful to start with a bit of context: a high-frequency trading algorithm is what turns live market data and your models into orders, very quickly. It has to work with research, risk, and everything else that changes when someone ships a new idea. How you build it comes down to the venue's rules, the market data you get, and how you balance latency against iteration speed.

Latency and fast iteration of trading ideas pull on each other. A stack that is easy to experiment in tends to grow layers; a stack that is ruthlessly minimal can slow research down. Where you draw boundaries, what you cut and whether you treat iteration cost as a first-class design input matter as much as any single performance optimization.

Three generations

Large trading stacks rarely die of a single bug. They die of accumulated constraints: the abstraction that helped you scale year one becomes a tax by year five. Our system went through roughly three generations, each one fixing the biggest bottleneck at that time. As the system became more capable, the team found new ways to use it until they ran into the limits of the design itself.
First: traders composed logic in large visual graphs. The engineering bet was legibility and how quickly traders could turn a new idea into something running. The platform kept growing new node types and paths as traders asked more of it. Early on that worked well: the valuation is easy to construct and you ship fast. It gets painful later: graphs grow into forests, new ideas start needing platform changes to express, and running all of it on the live path gets too heavy. Latency was not competitive by later standards.

Second: we moved to a compiled approach: valuation written in a higher-level language that compiled to fast production code, running inside a thinner runtime. Engineering owned compiler performance, the interface between language and runtime, and how far the language could stretch before you started embedding lower-level code. That’s what really unlocked machine learning in research and production, and pushed latency into a much faster band. Initially, this seemed pretty good, but over time, the compiler became part of the bottleneck.  The new friction was as much organizational as technical: trading and engineering were split across a boundary (strategy surface vs runtime), and the toolchain itself became something you had to keep feeding. Slower builds were the obvious symptom. The real cost was the compiler itself: keeping it expressive enough for new strategy abstractions and fast enough as trading logic grew more complex took a steadily growing amount of engineering effort.

Third: the direction was native code end-to-end, collapsing the layers between market data arriving and orders going out. This time, we wanted expressiveness and latency together: keep the tick-to-trade path fast while giving researchers and traders abstractions rich enough to express real ideas at close to zero run-time cost, so flexibility is no longer paid for in latency.

The key to the design is deciding where complexity is allowed to live, not only optimizing the parts that have to be fast. Keeping simulation honest relative to production matters here too, so research does not “work” in a way production cannot reproduce. That's the kind of foundation that systematic trading needs – new ideas can be tested and moved into production quickly, without giving up latency to do it. It also blurs the boundary between developer and researcher. New features often need shared ownership across research and engineering, because a single design choice affects both model behavior and production constraints.

	First generation: Graphs	Second generation: Compiled	Third generation: Native
Flexibility to iterate	Low	High	Higher
Time-to-production for new ideas	Slow	Faster	Fastest
Relative latency	Slow	Middle	Fastest
Research and engineering collaboration	Tight coupling	Split by boundary	More shared ownership of the stack

Outcomes

We were aiming for meaningfully lower end-to-end latency, latency that does not degrade as features and models accumulate, and smoother promotion from

. In early deployment, we saw improvements in the performance metrics we care about (specifics subject to disclosure policy). The bigger point is that the wins tracked together: when the stack got easier to change safely, the latency story and the research story stopped fighting each other as much.

Scope

Whether this makes sense depends on your market structure, research velocity, toolchain cost, and reproducibility requirements. The stack is still rolling forward; the next bottlenecks will show up in the usual way, once enough real traffic and enough new ideas have passed through.