Terminal-Bench Science 0.1 (Vals AI)

Agentic scientific research

Terminal-Bench Science is an open-source benchmark of 70 research-workflow tasks contributed by practicing scientists across the life, physical, mathematical, engineering and earth sciences. Each task asks a model to carry out real research work inside a sandboxed terminal — reconstructing a signal, calibrating a model, reducing an instrument’s output, proving a bound — and is graded strictly pass/fail by verifier tests written by the task’s own author, with up to eight hours per task. Scores here are Vals AI’s run, which puts every model on the same general-purpose mini-swe-agent harness at pass@1, so they sit below the official leaderboard’s figures, where each model uses its vendor’s own agent.