Reef

In progress

A native Mac editor that finds the moments in hours of dive footage, and lets an agent edit the same timeline I do.

I come back from a dive with hours of GoPro footage and maybe forty good seconds. Reef finds them, keeps every cut tied to the source frames it came from, and gives an assistant the same operations the app uses. When I’m done, it hands an editable timeline to DaVinci Resolve.

Role: Solo builder, directing coding agents
Stack: SwiftUI, AppKit, Python, SQLite, GRDB, MLX, MCP
Scale: 176 MCP tools: 96 read-only and 80 that write. 145 are free, 15 ask first and 16 sit behind a hard gate.
Commits: 11,736 on this Mac since Jun 8
Links: Showcase repo
As of: Nov 1, 2026

Rebuild Put together from real Reef interface captures and Catalina footage, with a prepared conversation. It isn’t one continuous live session; a screen recording replaces it next.

Why dive footage is hard

Most highlight finders look for motion and loud audio. Underwater, those signals point the wrong way. The busiest second of a clip is usually the camera whipping around, blurred and full of bubbles. The shot I want is held still: a seal in focus, close to the middle of the frame.

On one of my clips, Reef measured that motion and sharpness were anti-correlated. The bin with the most motion was the blurriest one, a camera whip. The sharpest bin had almost no motion. So Reef splits optical flow in two: camera motion from the border of the frame, and subject motion as whatever moves differently from the camera. It adds focus and subject presence, then subtracts a penalty when the camera is moving and the frame is blurred at the same time.

Rebuild A loudness-and-motion scorer against Reef’s. The weights are Reef’s defaults, the signals are samples shaped like the measured case, and the footage lane uses stills from my own clips.

How it fits together

One rule holds it together: only the engine writes the catalog. The app reads, the agent asks, and both go through the same operations.

Mac app: SwiftUI, with AppKit where it needs it. Reads the catalog through GRDB and asks the engine for every change, so a button and an agent run the same operation.
Assistant over MCP: Claude Code, Codex or Reef’s own assistant call 176 tools: query the catalog, inspect moments, draft a storyboard, revise it.
Python engine: The only writer. Owns analysis, editing, validation, rendering and the search index, behind a library lock.
SQLite catalog: Assets, captures and their versions, moments, cuts, research. An original, its proxy and later derivatives stay linked, so a proxy never stands in for the original at export.
Analysis: Gemini-first analysis of proxies, with local MLX models on Apple silicon for motion, tracking, embeddings, detection and speech.
Render: A proxy render for review, a master render for delivery. Each attempt is bound to its source, cut and settings.
DaVinci Resolve: Builds an editable, versioned timeline through Resolve’s scripting API, after checking its project and frame rate, and keeps the link to the source frames.

Decisions

A cut ends when the animal leaves

Reef’s first rule trimmed a cut on a blend of subject presence and sharpness. On one clip the fish left at about 3.5 s, but the kelp stayed sharp, so the cut ran on to 5.25 s and 78% of it showed no fish.

Now one module places every cut. A trailing second can only be trimmed if it’s weak and the subject is gone, and a cut never runs past the subject’s exit.

Instead of one blended trim threshold

Rebuild A sample shaped like that clip, so its numbers differ from the measured 5.25 s and 78%.

The agent and I write to the same saved cut

The cut doesn’t live in the chat. It lives in Reef as saved revisions, and the agent edits it through the same engine operations as the app.

Every write carries the fingerprint of the cut it saw, expected_items_fingerprint. If I changed the cut in the meantime, the write comes back as operation.conflict and nothing is overwritten.

Instead of a timeline the agent keeps in its own chat

An agent terminal beside Reef’s Main cut at revision 1, with the cut’s fingerprint. — Rebuild The tool names and arguments are Reef’s own. The ids, the conversation and the fingerprints are samples.

The beat grid only sets the timing

Cutting on every beat looks mechanical, like a metronome. In a percussive drop, Reef finds the strongest hit and gives it a real cut, even off the grid.

It snaps nearby cuts to onsets within 120 ms, then picks cuts outward from the hit with gaps that grow 35% at each step. Shots get shorter going into the hit and longer after it.

Instead of a cut on every beat

Rebuild The onsets are a sample song. The shot frames are stills from my clips.

The region comes from the dive site, never the trip name

When Reef asks a vision model to describe a moment, it adds the dive’s location and a regional species shortlist. In the October 5 bake-off this raised correct descriptions from 38 to 44 of 52 clips I labelled myself, the biggest single accuracy gain.

I write trip names myself, and they can give the answer away, so the region comes only from the catalogued dive site. The shortlist is a hint: the prompt tells the model to report what it actually sees.

Instead of passing the trip name to the model

How it’s built and checked

Reef can run unattended across a whole library, so most of its complexity goes into not claiming more than it knows. When the evidence doesn’t hold up, it stops with an error instead of guessing.

Nothing is deleted: Re-runs and corrections flag old rows as superseded, retired or retracted. The evidence behind a removed item stays in the catalog.
Exact time: Frame rates and time bases are stored as fractions. If a file’s coded and average frame rates disagree by more than 0.5%, its timing isn’t trusted for exact mapping.
Capture isn’t container: A clip shot at 120 fps may already be conformed to 24 fps, so the file’s rate is never taken as the capture rate.
Confidence has a ceiling: Water facts use a fixed vocabulary, come from a single frame, and are capped at 0.5 confidence.
Whole-clip sweep: Detection sweeps the whole proxy at about 1 fps and links detections into per-subject tracks, so an animal crossing a quiet stretch still gets found.
Find similar: “More like this” searches the SigLIP vectors stored at ingest, with no new model call.

Where it stands

Implemented

The SwiftUI app runs over the Python engine.
Agents reach it through 176 MCP tools in three permission tiers.
Every edit goes through one writer that checks the revision first, and a stale write is refused.

In progress

Everything on this page exists in source. Client coverage and full end-to-end acceptance are still in progress.
No run from raw footage to a finished cut has been demonstrated yet.
A successful preview render isn’t a verified final delivery.
There’s no public release yet. A screen recording replaces the rebuild at the top of this page next.