CTF-Archive (CAISI 577-task set)
Agents · 2025-09-30
CAISI's 2025 CTF-Archive evaluation contains 577 challenges drawn from 79 competitions held from 2011 through 2025. Agents receive the original challenge description, relevant files, and an Inspect-based container environment, and scores report the percentage of flags retrieved.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5 | 50.6 |
| gpt-oss-120b | 34.3 |
| Opus 4 | 34.1 |
| DeepSeek-V3.1 | 28.2 |
| DeepSeek-R1-0528 | 26.5 |
| DeepSeek-R1 | 8.5 |