During a recent fireside chat, Optiver's Head of Quant Engineering, Scott McKenzie, sat down with NVIDIA's Global Head of Capital Markets Strategy, Ioana Boier, and Optiver Software Engineer Josh Durham to discuss how accelerated computing is changing engineering in trading. Here's an edited excerpt from the conversation.
When Scott opened the discussion, he framed it in a way that probably sounded familiar to anyone who's worked in trading.
“Speed for a trading firm has often meant low latency... these days it means many more things than that.”
Ioana Boier described where NVIDIA is seeing most accelerated compute in capital markets today, and it isn't primarily in production inference. Much of it still goes into research. She also described NVIDIA as a full-stack company, and the conversation quickly moved beyond GPUs to how hardware, networking, and software fit together. One sentence captured that shift:
“It's not the FLOPS. It's the researcher time.”
She described firms investing in shortening the entire research cycle, moving from raw data to ideas, from ideas to strategy, and eventually into production.
Scott then asked Josh how unified memory was changing the way engineers build systems at Optiver. Historically, preparing datasets meant spreading work across thousands of CPU cores. Josh described how that workflow is changing. Instead of distributing tiny data chunks across thousands of CPUs, a few GPUs with large, unified memory can process the same dataset more efficiently in large blocks before feeding the results into model training. Rather than synchronizing data across separate memory spaces, engineers can focus more on how the data is processed and where the bottlenecks are.
Later, when the discussion moved into production inference, Scott asked whether
“I want my GPU software and model weights preloaded and ready, waiting for the market data to come in and immediately trigger a decision.”
Josh explained that production systems don't have the luxury of launching work when data is ready. The application is already waiting for the next market event.
Scott also asked how much engineers still need to think about memory movement now that unified memory has become more common. Ioana cautioned that “unified memory is not the same as uniform performance.”
She described unified memory as removing some of the friction between CPU and GPU, but not the need to understand where data lives or how it moves through a system. Josh picked up on the same point from an engineering perspective, describing benchmarking and profiling as just as important as ever when trying to understand where the bottlenecks are.
By the end of the discussion, the conversation had moved well beyond GPUs to the software, systems, and engineering decisions that make accelerated computing practical.
