The Agent Writes story.json

// the problem

Agent pipelines are getting good at everything except the last step. A workflow researches a topic, drafts the script, calls a TTS voice, generates the artwork — and then the video assembly is a gamble: hand-rolled ffmpeg concat chains, drifting caption timings, or a headless-browser renderer that produces slightly different bytes every run. Worse, one changed sentence means rendering the whole thing again and re-checking all of it.

The fix is an architectural line, not a bigger script: generation ends at a manifest; rendering starts from it.

topic → script → TTS + images → story.json + assets/**   ← THE BOUNDARY
                                       │
                                       ▼
                                 m0saic make          (deterministic from here)
                                       │
                                       ▼
                                    pixels

// the m0saic program

@m0saic/story/narrated-chapters/v1 compiles a story.json — sections of narration audio, images, and knobs — into a chaptered video: title and section cards, Ken Burns scenes over the artwork, and burned captions timed to the actual speech. Ask for landscape and portrait in one manifest and a single render emits both deliverables.

The render never calls a provider. Your agent (or the example orchestrator that ships with m0saic) owns the model calls, and every generated artifact is content-addressed — the request itself is the cache key. Edit one section’s prose and exactly that section’s narration regenerates; everything else is a cache hit. And because the render is deterministic, the same story.json and assets produce byte-identical output — we verify it by hashing back-to-back renders. Same manifest, same bytes: video that behaves like code.

// how it was done before

The status quo is orchestration code that owns pixels as well as prose: a script that concatenates scene clips with ffmpeg, burns subtitles in a second pass, mixes the narration in a third — each pass another place for timing to drift — or a browser-based renderer where the output depends on the machine that ran it. Both couple your agent to the rendering internals; neither gives you a diffable artifact between “what the agent decided” and “what got rendered.”

With the manifest boundary, the agent’s entire job is a JSON file you can read, diff, version, and re-render at will. The video stops being a side effect and becomes a build artifact.

// run it yourself

One command, straight from a terminal. The CLI carries the template, the engine, and the validator — if the layout can’t render correctly, it fails before a single frame is written.

// read the source

The template behind this study is @m0saic/story/narrated-chapters/v1 — a typed, unit-tested program, not a script. The template library is headed for open release; the m0saic source lives on GitHub.

The example orchestrator — provider adapters, the content-addressed cache, and the story.json contract — ships with the project as story-studio, so the boundary is something you can read, not a diagram.