← All benchmarks

VibeVideoBench (overall quality, no video = 0)

Code · 2026-10-26

239 self-contained code-to-video tasks, 209 in English and 30 in Chinese, across 12 categories from data stories and music-driven pieces to classroom lessons and paper explainers. A multimodal agent runs in its native harness in a fresh sandbox (8 CPUs, no GPU) with renderers, ffmpeg and a text-to-speech tool but no image or video generation model, and must deliver a finished MP4 with sound, an editable source project and a timing sheet within one hour. Three VLM judge panels (Gemini 4 Argon; Gemini 3.8 Flash; GPT-5.6 Sol on sampled frames with Qwen3.8-Omni-Flash on audio) each rate five task-anchored quality dimensions from 1 to 5, and a run without a playable video scores 0 on every dimension. The score is the mean over tasks of the five-dimension mean, computed per panel and averaged over the three panels, on 0-5.

Top models (higher is better)