BigFinanceBench private holdout (final-answer accuracy)
Professional Work
Rogo's BigFinanceBench evaluates financial-research agents on workflow-grounded finance tasks. This row is the separate 139-task private, non-public holdout set run with Rogo's official agent loop and grading methodology, scored as final-answer accuracy: the share of task-runs a Kimi K3 judge marks correct, averaged over three runs or trials. It is distinct from the paper's public 928-item rubric-scored release.
Top models (higher is better)
| Model | Score |
|---|---|
| Gemini 4 Argon | 54.7 |
| GPT-6 Astra | 54.6 |
| Muse Spark Open | 53.3 |
| Claude Opus 5 | 49.2 |
| Claude Opus 5.5 | 48.2 |
| GPT-6 Sol | 47.1 |
| GLM-5.3 | 45.8 |
| GPT-5.6 Sol | 44.2 |
| Kimi K3 | 41.3 |
| DeepSeek V4.1 Flash | 40.6 |
| DeepSeek V4 Pro 0813 | 39.3 |
| Ember-1 | 39.0 |
| GLM-5.3 Flash | 37.2 |
| GPT-6 Luna | 35.3 |