Rhythm-Agent : Rhythm-Driven VLM Agent for Creative Advertising Short Videos

All videos are generated by rhythm-agent

Insight 1 · Learning from Films at Scale

Rich film-transition data taught our models distinctive video orchestration and a cinematic sense. Intelligence is a higher-level abstraction of data; rich data and simple SFT can go a long way.

YouTube videos default to 360p and can be adjusted to 720p.

Insight 2 · Everything Is a Tool — Let the Model Use It

When model capabilities fall short, a thin shell packages complex multimodal interactions as tools: video understanding, music beat extraction, editing, and even the video-understanding workflow itself.

Insight3 · Scaling works

We explored scaling Qwen-VL models from 7B to 72B, alongside larger Qwen text models at 35B and 235B.

Insight4 · Multimodality alone is not intelligence

Multimodality alone is not intelligence — thinking and reasoning are.

With a strong foundation model, image-understanding and audio-feature extraction tools, and a little guidance, the model can even use dynamic programming to orchestrate video and capture higher-level patterns in rhythm. Compute wins again.

Our proprietary data and methods remain private.

More Examples

A simple video orchestration loop A powerful reasoning foundation model calls tools, followed by a powerful open-source reasoning model, then evaluation and resampling for rhythm and style, looping back. Evaluate & Resample Does it have rhythm and style? rubric++ Large Reasoning Foundation Model Powerful Visual Foundation Model Everything Is Tools Image understanding · Beat recognition Image generation · Visual caption Cropping · Video editing · Coding
Lots of data + A simple loop work.