Finance Agent v2.1
Professional Work · 2026-10-19
Finance Agent v2.1 revises Finance Agent v2 after expert review: some test questions were reworded or removed and grading checks revised, a three-judge jury (GPT-6.1 Sol, Gemini 3.8 Flash and Claude Sonnet 5.5) replaces the v2 jury, and hedged answers that give several candidate values now fail. Agents use the shared FAB v2 harness with six tools and a two-hour limit per task on the 449-question private test split. The headline is the unweighted mean of nine category means of dealbreaker-gated, severity-weighted partial credit; the 449 distinct questions are not the aggregation denominator. Kept separate from Finance Agent v2 because the questions and grading changed.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 4 Argon | 67.7 |
| Muse Wonder Full | 67.6 |
| Gemini 3.8 Flash | 65.8 |
| Muse Wonder | 65.5 |
| DeepSeek V4.1 Flash | 65.4 |
| Claude Haiku 5.5 | 65.4 |
| GLM-5.3 | 65.0 |
| MiMo-V2.6-Pro | 64.3 |
| MiMo-V2.6-Flash | 63.3 |
| DeepSeek V4.1 Flash 1017 | 62.9 |
| GPT-6.1 Sol | 61.8 |
| Hy4 Preview | 61.6 |
| Mistral Large 4 Preview | 61.4 |
| Muse Spark Open | 60.7 |
| GLM-5.3 Flash | 60.6 |
| Grok 4.7 | 60.6 |
| GPT-6 Astra | 60.4 |
| Kimi K3 | 60.0 |
| Step 5 Preview | 58.9 |
| MiniMax M3.1 Flash | 58.8 |
| DeepSeek V4 Pro 0813 | 57.1 |
| GPT-6 Luna | 56.8 |
| Mercury 3 | 54.7 |
| MiniMax M3 | 54.3 |
| Beam 1.0 501B A23B | 51.8 |
| Inkling | 49.9 |
| Inkling-Small | 44.8 |