When Laude Institute launched Terminal Bench, a joint project with Stanford University that benchmarks AI agents' terminal mastery, it quickly became a key performance metric for frontier labs like Anthropic, OpenAI, and Google. But as adoption surged ahead of infrastructure capacity, running thousands of parallel agent-model experiments created compute and orchestration demands that Laude wasn’t yet equipped to support.

Initially, Laude’s Founding Member of Technical Staff, Alex Shaw, conducted experiments using Docker sandboxes on Laude’s local machines. While this setup supported four to six containers, CPU, memory, and I/O constraints made it impractical for running the thousands of tests on Terminal Bench tasks.

Sandbox volume was only part of the equation. Each task had unique dependencies and specifications, such as custom frameworks and CLIs, calling for clean sandboxes provisioned on demand. Experiments ran for hours or days at a time, so the sandboxes needed reliable uptime. At this scale, provisioning speed was also essential to keep testing cycles on track and increase throughput.

None of these capabilities could come at the expense of benchmark accuracy. Because researchers needed valid results without risking unpredictable agent behavior, each environment had to securely run the same way it originally did on Laude’s local machines. 

Building and maintaining infrastructure that met all these demands was a non-starter. Managing containerization and cloud provider intricacies would divert resources away from key research initiatives. Plus, Terminal Bench’s async Python architecture required careful integration to avoid pre-configuration overhead. So Alex set out to find a managed cloud runtime that could create thousands of isolated, parallel environments.

While several solutions appeared promising on the surface, each had critical limitations. Some installed pre-configured dependencies that compromised unique task execution, while others required manual container configuration, all with minimal customer support.

That’s when Alex discovered Daytona. Their agent-native runtime platform was exactly what he needed to support AI research teams relying on Terminal Bench.