Ruby LLM Program Fixer — Vending Machine
Code · 2025-08-01
The Vending Machine Program Fixer task asks a model to repair a broken Ruby vending-machine implementation against 31 tests. Its composite is 90% test success plus 10% RuboCop quality, with each offense subtracting two quality points down to zero.
Top models (higher is better)
| Model | Score |
|---|---|
| GPT-5.5 | 79.1 |
| GPT-5.6 Sol Pro | 79.1 |
| GPT-5.2 | 78.9 |
| GPT-5.6 Sol | 78.9 |
| GPT-5.6 Terra | 78.9 |
| GPT-5.6 Terra Pro | 78.9 |
| Claude Opus 5 | 78.7 |
| GPT-5.3-Codex | 78.7 |
| GPT-5.4 | 78.7 |
| GPT-5.6 Luna | 78.7 |
| Kimi K2.6 | 78.7 |
| Gemini 2.5 Pro | 78.5 |
| GPT-5.6 Luna Pro | 78.5 |
| Opus 4.7 | 78.3 |
| Gemma 4 31B IT | 78.3 |
| GPT-5.3 Instant | 78.3 |
| gpt-oss-120b | 78.3 |
| Kimi K3 | 78.3 |
| Claude Fable 5 | 78.1 |
| Opus 4.8 | 78.1 |
| Sonnet 5 | 78.1 |
| DeepSeek-V4-Pro | 78.1 |
| Gemini 3.1 Flash-Lite Preview | 78.1 |
| Gemini 3.1 Pro Preview | 78.1 |
| Gemini 3.5 Flash | 78.1 |
| GPT-4.1 Nano | 78.1 |
| GPT-5.1-Codex | 78.1 |
| GPT-5.2 Instant | 78.1 |
| Grok 4.5 | 78.1 |
| Haiku 3.5 | 77.9 |
| Opus 4.6 | 77.9 |
| DeepSeek-R1 | 77.9 |
| DeepSeek-V3.2-Exp | 77.9 |
| Gemma 4 26B A4B IT | 77.9 |
| GLM 4.6 | 77.9 |
| GLM-5.2 | 77.9 |
| GPT-4 | 77.9 |
| GPT-5.1 | 77.9 |
| Qwen3.6 Plus (2026-04-02) | 77.9 |
| Opus 4.5 | 77.7 |