Voidscape — Teaching AI Agents to Watch Video
An AI coding agent can read images and PDFs — not video. Voidscape decomposes any video into frames plus a transcript, and prices the whole job before spending a cent or a token.
Overview
Voidscape is an open-source (MIT) Claude Code / Codex skill that gives AI agents genuine video comprehension: point it at a local file or a URL (YouTube, Loom, Vimeo…) and it extracts frames for the visual track and a transcript for the audio track — the two things an agent can actually consume. Its defining feature is the cost gate: a probe → estimate → run pipeline that prices the entire job (transcription dollars and agent-token cost) up front, defaults to free local transcription with faster-whisper, and only touches paid cloud backends after explicit approval. The engine is a single 1,300-line Python CLI built on the standard library, with an opt-in machine-readable protocol (`manifest`, `--envelope`, deterministic exit codes) added for agent callers. The public landing page is live at https://rikepilb.github.io/void-scape/.
The Problem
Agents fake video understanding by reading titles and comments. Actually watching costs real money — frames dominate agent-token spend, and cloud transcription bills by the minute — so a naive implementation surprises users with the bill after the fact. The design problem was making video comprehension both real and pre-approved: never spend before showing the price, and never let audio leave the machine without explicit consent.
Questions Addressed
- 01
What's the cheapest honest path to a transcript — and how often is it free?
- 02
Can one skill serve multiple agent harnesses (Claude Code, Codex, Gemini CLI, Copilot CLI) from a single install?
- 03
Does the skill measurably beat an agent improvising with ffmpeg on its own?
- 04
What happens when a coding agent extends the same codebase months later — does the original design hold up under real adversarial review?
Methodology
Probe → Estimate → Run, with a Cost Gate
probe inspects the input (duration, resolution, audio, existing captions); estimate computes the full cost — transcription dollars per backend and projected agent-token spend from frame count — before any work; run only executes after the user (or a zero-dollar threshold) approves. Nine transcription paths are ordered cheapest-and-most-private first: sidecar subtitles, URL captions, local faster-whisper and trx (all free) before Groq, OpenAI, OpenRouter and Gemini.
A Stdlib-Only Engine
The paid-API paths use hand-built multipart requests over urllib — no SDKs — so the free paths never pay an import cost and a missing optional dependency can never break probe or estimate. 110 pytest cases across 17 files pin down chunking, deduplication, cost estimation, frame extraction, and hardening (including an anchor fix against lookalike-domain spoofing and a subprocess-level test suite for the agent CLI contract).
Eval-Driven Skill Design
The skill was benchmarked with an eval loop against a no-skill baseline: with the skill loaded, the agent passed 14 of 15 assertions (93.3%) across visual-summary, audio-comprehension and cost-gate scenarios, versus 66.7% baseline. One install script wires it into four harnesses: Claude Code, Codex, Gemini CLI and Copilot CLI.
Build Week: Agent Protocols, Adversarial Review, Honest Security
Extended for OpenAI Build Week 2026 with Codex + GPT-5.6: adaptive local transcription tiers, GPT-5.6-native 32×32 patch cost accounting, and an opt-in agent-facing CLI protocol (`manifest`, `--envelope`/`--compact`, a deterministic exit-code taxonomy with retryability metadata). An adversarial code-review pass against the new cost/consent gate surfaced 9 findings; the 6 real defects were fixed with regression tests before shipping — including one caught only by actually running the documented commands, not by unit tests alone. A static security scanner flagged the intentional env-key-to-cloud-API data flow as CRITICAL; rather than hide it, the finding is disclosed and explained in a `SECURITY.md` the scanner's own report cross-checks against.
Key Results
Key Findings
Cost transparency is a UX feature for agents: showing the price before the work turns "the AI ran up my bill" into an informed yes/no.
Local-first ordering (captions → Whisper on-device → paid APIs) makes the free path the default path — most videos never cost a cent to read.
Evals beat vibes for skill design: a measured 93.3%-vs-66.7% gap is what separates "the skill helps" from hoping it does.
Adversarial review plus actually running the documented commands caught a real regression unit tests missed entirely: a bug fix elsewhere in the same diff silently changed what the README's own privacy-proof example demonstrated.
Conclusion
Voidscape is the most complete open-source piece in this portfolio: MIT-licensed with contribution docs, issue templates, a demo GIF, a live GitHub Pages landing page (https://rikepilb.github.io/void-scape/), 60+ commits of real iteration, and a measured eval improvement. It's also honest about scale — the eval set is small and iteration continues — but the shape is what production agent-tooling looks like: priced, tested, local-first, multi-harness, and reviewed like real software rather than shipped on vibes. Submitted to OpenAI Build Week 2026 (Developer Tools track).
Gallery
