← All benchmarks

VibeVideoBench (spec compliance)

Code · 2026-10-26

Program-checked delivery compliance on the 239 VibeVideoBench code-to-video tasks: the share of tasks whose delivered MP4 passes every applicable check of the delivery spec (decodability, resolution, frame rate and duration; on the 205 core tasks also a source project, exact frame counts and audible sound where required; on lessons and papers narration coverage and language and no long black or frozen stretches), in percent. A run without a playable video fails. No model judge is involved; it measures delivery, not video quality, so it is a diagnostic.

Top models (higher is better)