Ruby LLM Program Fixer — Calendar
Code · 2025-08-01
The Calendar Program Fixer task asks a model to repair a broken Ruby calendar implementation against 23 tests. Its composite is 90% test success plus 10% RuboCop quality, with each offense subtracting two quality points down to zero.
Top models (higher is better)
| Model | Score |
|---|---|
| gpt-oss-120b | 92.1 |
| Gemma 4 26B A4B IT | 91.9 |
| Grok 4.5 | 91.7 |
| Opus 4 | 91.3 |
| Haiku 4.5 | 90.9 |
| Opus 4.1 | 90.9 |
| Gemini 2.0 Flash 001 | 88.6 |
| GPT-4 Turbo | 87.0 |
| GPT-5.1-Codex mini | 86.7 |
| Kimi K2.6 | 86.1 |
| Claude Opus 5 | 85.9 |
| GPT-4.1 | 85.6 |
| Sonnet 4.6 | 85.3 |
| Codestral 25.08 | 85.3 |
| Claude Fable 5 | 85.1 |
| Opus 4.8 | 85.1 |
| Gemini 3.5 Flash | 85.1 |
| Gemini 2.5 Flash | 84.9 |
| GLM-5 | 84.9 |
| Opus 4.6 | 84.7 |
| Opus 4.7 | 84.7 |
| Grok 4 | 84.7 |
| Haiku 3.5 | 84.5 |
| Opus 4.5 | 84.5 |
| Gemini 2.5 Pro | 84.5 |
| GPT-4.1 Nano | 84.5 |
| O4 Mini | 84.5 |
| DeepSeek-V4-Pro | 84.3 |
| Gemini 3.1 Flash-Lite Preview | 84.3 |
| Kimi K2 Thinking | 84.3 |
| GLM-5.1 | 83.9 |
| Llama 4 Maverick | 83.9 |
| o1-mini | 83.7 |
| GPT-5.3-Codex | 83.2 |
| GPT-5.4 | 83.2 |
| GPT-5.6 Sol | 83.2 |
| GPT-5.6 Sol Pro | 83.2 |
| GPT-5.6 Terra | 83.2 |
| GPT-5.6 Terra Pro | 83.2 |
| GPT-5.5 | 83.0 |