direktor

Overview

direktor is an experiment in using a language model as an orchestrator rather than as a generator. The task — turn a text file into a narrated video — is a convenient setting for that question, because it requires coordinating several models with incompatible interfaces, latencies and failure modes, and the coordination is the part that is hard.

The pipeline has six stages and each is separately resumable. A model drafts a script from the source text. BARK narrates it. Distil-Whisper transcribes the narration to recover accurate timings, which is easier and more reliable than predicting duration. A second model writes image prompts from the script. FLUX-schnell renders stills. FFmpeg composes the result into a 1920x1080 MP4 with the stills cut against the recovered timings.

Resumability is the load-bearing design decision. Every stage is slow, several call remote services that fail intermittently, and a pipeline that restarts from the beginning after a stage-five timeout is unusable in practice. Persisting each stage's output and resuming from the last good one turns an unreliable long-running job into an ordinary one, and that pattern transfers to any multi-model workflow, which is the actual research output.

The transcription step is the other transferable lesson. It would be simpler to estimate narration timing from word counts, and it does not work. Asking a second model to observe the first model's output is more accurate than asking the first model to predict itself, and that asymmetry keeps recurring in multi-model systems.

Operationally this is a demanding project: Python 3.11 or later, FFmpeg on the path, API keys for both OpenAI and Replicate, and an S3-compatible bucket for intermediate artefacts. Where it loses: a commercial text-to-video service produces a better-looking result with none of that setup. direktor is for people who want the pipeline open, inspectable and modifiable rather than for people who want a video.

The scope boundary is narrower than the category name implies. direktor produces a specific artefact — a podcast-style narrated video built from stills — and does not attempt motion synthesis, editing decisions, or anything resembling direction in the cinematic sense. Choosing a narrow output is what makes the orchestration question tractable; a general video pipeline would be a research programme rather than a project.

Within the portfolio this is the clearest example of a project whose value is the shape rather than the output. The resumable-stage pattern, the use of a second model to observe the first, and the separation of expensive remote calls from cheap local composition all recur in other multi-model systems. We keep the project public partly as a worked reference for those patterns, and partly because the failure modes of long-running generative pipelines are under-documented relative to how often people build them.