All benchmarks

DeepSWE v1.1

Agentic coding

DeepSWE (Datacurve) is a contamination-resistant coding-agent benchmark of 113 original, long-horizon software-engineering tasks across TypeScript, Go, Python, JavaScript and Rust, each graded in an isolated environment by hand-written, program-based verifiers. Because every task is authored from scratch rather than scraped from public repositories, it resists the training-data leakage that inflates older SWE benchmarks. It shares its name with — but is unrelated to — DeepSWE-Preview, the open-source RL-trained coding model from Agentica and Together AI.

Model scores

  • Fable 569.7%
  • Opus 4.859.0%
  • Sonnet 5
  • GPT-5.6 Sol72.7%
  • GPT-5.567.0%
  • Composer 2.5
  • Opus 4.7
  • Gemini 3.1 Pro11.8%
  • Mythos Preview

Official source: DeepSWE benchmark (Datacurve)

Related reading