All benchmarks

Chartography

Visual reasoning

Chartography is a chart-understanding benchmark from Surge AI covering 100 tasks on specialised chart types that other suites rarely test — Kaplan–Meier curves, candlestick charts, contour maps, wind rose diagrams, Sankey diagrams, Bode plots and 3D surface plots. Because different chart types can be read to different precisions, each answer is graded against an expert-set acceptable range rather than a fixed tolerance. Scores are reported with and without an image-cropping and code-execution tool.

Model scores

  • Fable 5.144.8% (no tools) / 88.4% (with tools)
  • Fable 5—
  • Opus 5.564.4% (no tools) / 89.0% (with tools)
  • Opus 529.8% (no tools) / 83.4% (with tools)
  • Opus 4.8—
  • Sonnet 5.561.6% (no tools) / 90.2% (with tools)
  • Sonnet 5—
  • GPT-6 Astra—
  • GPT-6.1 Sol—
  • GPT-5.6 Sol—
  • Grok 4.7—
  • GPT-5.5—
  • Composer 2.5—
  • Opus 4.7—
  • Gemini 3.1 Pro—
  • Mythos Preview—

Official source: Chartography (Surge AI)

Related reading