Unlike fields like physics or mathematics, where the gap between a research result and any real consequence can stretch for years, trading gives you immediate, clear feedback. It's adversarial, competitive and dynamic, with a direct relationship between an idea and its outcome. In most areas of machine learning, the data comes from a system that doesn't know or care that you're studying it. In trading, every other participant is trying to solve the same problem you are.
That environment rewards intuition and creativity, not a grinding search for truth. So applying machine learning to it is a genuine intellectual endeavor – which is why it interests mathematicians, physicists, coders, and academics as much as those already in finance.
Modelling in a competitive system
Optiver operates as a market maker, continuously buying and selling financial instruments and setting the prices at which we're willing to trade, with a spread between the two offers. That pricing is a research problem, and it sits inside a live, competitive system.
Your model is part of the system it's trying to predict. In standard ML, the data and the environment's behavior are assumed to be largely known, and you don't affect either. In trading, you do. If your model predicts a trade is profitable and tries to execute it, one of two things happen:
- You don't get it, because a competitor got there before you.
- You get it. But perhaps because a competitor didn't like it. Is your model correct and the competitor's model wrong?
When your strategy acts on a signal, other participants respond within microseconds. They pull quotes, widen spreads, revise their own models. The edge you identified exists because of how participants are collectively behaving at that moment – and the moment you act on it, you're one of those participants. The research problem is pricing accurately when the system you're pricing into is actively adapting to you.
Why machine learning fits the problem
Markets generate vast streams of structured data: prices, orders, trades, volumes, and other activity across a long list of instruments and exchanges. In high-frequency trading (HFT), buying and selling decisions are made repeatedly, at scale, and at breakneck speed. The objective is clear: did the model help us price more accurately, manage risk better, and execute more profitable transactions?
Machine learning fits this problem because that data isn't summarizable in one rule. A price can be influenced by hundreds of weak signals at once: what just traded, how the order book is shifting, how related instruments are moving, how competitors respond. Individually those signals don't say much. But combined correctly, they can:
- Uncover relationships too complex to express as simple rules
- Assign accurate weights to many competing signals
- Update a prediction as new information arrives
- Provide feedback on what's working – and do it fast
With classical statistical approaches, you specify the structure of a relationship upfront. In market microstructure, that structure changes depending on the regime, the product, and the time horizon. ML lets the data surface structure a researcher might not think to look for; in a feature space this large, with interactions between signals that aren't obvious ahead of time, this is important.
Backtesting and the silent counterfactual
Before any model touches a live market, a researcher can test it against huge quantities of historical data before risking real money. Researchers often start with the counterfactual: using historical market data, they check what a proposed model would have done had it been running at the time. Would it have predicted price moves more accurately, quoted more competitive spreads, avoided bad trades?
At each point, the model can only use information genuinely available at that moment. Let tomorrow's price leak into today's numbers and that's data leakage, so the backtest isn't valid. Then there's execution realism. A great price means nothing if another participant got there first, or if the order book had already shifted while the order was in transit. Other frictions come from fees, latency, and how the exchange you're trading on functions.
Take a passive order resting in the book at the best bid. In historical data, that price level traded, so the backtest records a fill. But in a live market, that fill probably happened because an aggressive seller swept through the level – meaning the price was already moving against you at the moment you got filled. The backtest sees a completed trade. The reality was adverse selection. You got filled because others chose not to be there.
And there's a deeper complication. The data you're testing against did not contain the strategy you're testing. Even the best backtest is not a proof. Human judgment is needed to decide whether it learned something about the market or just learned the backtest.
Non-stationarity and model decay
When a pattern is real but the data landscape it was built on has changed, you've got non-stationarity. And it's a key reason quantitative researchers – many of whom come to us from slower-moving fields where assumptions don't change much – find their jobs interesting.
Often it's the most cutting-edge models that decay fastest – the ones that found a genuine relationship and thrived through backtesting without overfitting. Other players found the same signal, their trading sucked away the opportunity your model was exploiting, and what was once an edge became common knowledge. The original idea was right; the market adapted around it. Your model didn't change, but the world around it did. By the time you see aggregate performance nosediving, the underlying problem may have been around for eons. Or a few days, which is an eon in HFT.
The best model is not the one with the most impressive origin story. It's the one that works right now.
Same game, different rules
The nature of the exchange itself works against you too. Every venue has its own mix of participants, fee structures, tick sizes, order types, and matching engine rules, and in HFT those distinctions shape what the data means and whether a strategy executes as designed.
Queue position is one example. When multiple participants quote the same price, the exchange decides whose order fills first, and the rules governing that vary — sometimes changing what the data means entirely. Whether your order fills before a competitor's, or whether you're the last one resting at a price everyone else has moved away from, depends on those rules. Your model needs to account for how likely an order is to complete as planned, because even a small rule change can alter behavior throughout the exchange.
The same futures contract can also trade on two exchanges with different tick sizes. A smaller tick means tighter price granularity, which changes spread dynamics, queue depth, and the economics of providing liquidity at each level. A model trained on one venue's data can draw conclusions that are technically correct for that venue and meaningfully wrong for the other, even though the underlying instrument is the same.
Domain knowledge, what you know about a specific trading environment, remains important. Sometimes success depends on modeling one venue in extraordinary detail, and it's an area where the scale of ML works closely with the experience of researchers who know those markets well.
A change in fees, an updated order type, a new limit on how participants can quote. Any of these can kill off a strategy that was otherwise working. At human timescales, these distinctions look negligible. In HFT, they are part of the market.
Why the research doesn't stop
Once a model is live, you see what a backtest can't show you. It's the first time a hypothesis meets a competitive system and the first time other participants can respond to your presence. The data tells you something changed. Working out what, and whether it matters, is where judgment and domain knowledge earn their place.
A trading model is never truly finished, which means a quant researcher's work is never done. You test an idea, observe how it behaves inside the actual system, and use what you find to decide what to investigate next.
And the adversary who challenges you most is often an earlier version of yourself.
