OrbitOS

The problem

Creators sit on hours of long-form video but lack the time to cut it into platform-ready shorts. The naive solution, “just use an LLM to write captions,” fails because it ignores the creator’s actual editorial taste. A creator’s voice isn’t a prompt; it’s a pattern of decisions made while editing. The real problem is building a pipeline that both produces content and learns the creator’s preferences from their behavior.

What I built

OrbitOS is an AI-native creator pipeline. Feed it a YouTube video, and it produces a week of platform-ready shorts, each with a hook, captions, and a cut, while a behavioral memory loop learns your editorial voice from how you approve, reject, and edit the output.

How it works

1. Ingest. Pull a long-form video via yt-dlp.
2. Transcribe & segment. Transcribe with Whisper and identify candidate moments.
3. Clip extraction. Keyframe-aware ffmpeg cutting that respects scene boundaries, not arbitrary timestamps.
4. Structured generation. An LLM produces structured output (via tool-use) for hooks, captions, and metadata.
5. Memory loop. Your edits and approvals become supervision signals that update a behavioral memory of your voice.
6. Cost routing. CascadeFlow routes each step to the cheapest model that meets the quality bar.

Technical decisions

• Behavioral memory loop (review-as-supervision). The system doesn’t ask you to describe your style; it infers it from how you edit. Each approval or rejection is a training signal that refines the memory. This is the differentiator. It learns your voice, not a generic “creator” voice.
• Structured LLM output via tool-use. Instead of parsing free-form JSON, the model emits structured tool calls, which is far more reliable for downstream automation.
• Keyframe-aware clip extraction. Naive timestamp cutting produces jarring cuts. The pipeline detects scene boundaries and cuts on them.
• CascadeFlow cost routing. Not every step needs a frontier model. The pipeline routes cheap steps to cheap models and reserves expensive models for the steps that matter, controlling cost at scale.

Key capabilities

• End-to-end video → shorts pipeline
• Behavioral memory that learns from editing behavior
• Structured LLM output via tool-use
• Keyframe-aware ffmpeg clip extraction
• CascadeFlow cost routing across models
• Live deployment

Honest limits

This is a creator tool, not a fully autonomous system. The memory loop improves with use but starts generic. The quality of the shorts depends heavily on the source video and the model routing. It’s a working product with a live deployment, but the “learns your voice” claim is best understood as progressive personalization, not instant cloning.