Social Lab

In progress

Ask what’s working underwater, and get references you can check.

Social Lab finds real public posts for what I’m making, then answers two questions about each one separately: did it actually beat its creator’s usual, and does it actually show what I asked for. Every claim links back to the post, the frame or the number behind it.

Role: Solo builder, directing coding agents
Stack: Python 3.13, Node, a local web app, a CLI, MCP
Tests: 2,928 Python and 364 Node passed at the Oct 28 gate
Collected: 2,180 public posts, 48 playable references, 72 annotated frames, 2 carousels
Commits: 363 on this Mac since Oct 19
Links: Showcase repo
As of: Oct 28, 2026 gate

Rebuild The questions are ones I asked. The references are samples, with my own footage standing in for other creators’ posts, because the portfolio shows the method, not their numbers.

A big number isn’t a breakout

A million views sounds like a hit. But if that creator’s posts usually get two million, it underperformed. Twenty thousand views from someone who usually gets four thousand is the real breakout.

One service, three ways in

The local web app, the CLI and the MCP server all call the same research service and read the same retained evidence.

Web app: Follow-ups keep their context, and Watch with evidence plays a public video beside its sampled frames at the exact time.
MCP: social_lab_answer doesn’t secretly call a language model. It returns the supported facts and says what it couldn’t support.
Bridge to Reef: A bridge packet gives Reef six read operations: capabilities, search, answer, reference, compare and audio_references.

Decisions

Score each post against its own creator

Every post is scored against the other posts from the same creator, never against itself. Lift is plays divided by the median of those posts. Robust z is (ln plays − median ln plays) ÷ (1.4826 × MAD), so one viral outlier can’t distort the baseline. Cautious lift resamples the other posts 2,000 times with a fixed seed and divides by the 95th-percentile median, the most generous baseline the data allows.

A breakout needs all three. Under ten other posts, it refuses to score. Lift doesn’t adjust for how old a post is, and it never explains why a post did well.

Instead of comparing raw view counts

Sample data Plays on a log scale. Both feeds are synthetic, and the scoring is a port of the real scorer.

Relevance is a second question

A breakout post can still be a bad reference for a slow kelp-forest encounter. So relevance has its own evidence, and every reference says how deeply it was checked: metadata, creator comparison, provider transcript, sampled stills reviewed, then whole video and sound.

The agent can’t claim a level it hasn’t reached. Until the last level exists, nothing gets called an excellent hook.

Instead of treating popular as useful

Spending happens only when I ask

Discovery is a separate, paid tool. Everything else reads retained evidence and makes no request. A durable 30-credit allowance survives a restart. A breakout question may spend up to 12 credits and anything else up to 4, and callers can’t raise either limit.

After a lost response, the caller reuses the same request ID, so nothing retries on its own. Discovery takes public search terms only, never my footage, briefs, notes or credentials.

Instead of an agent that searches whenever it likes

My verdicts don’t train a taste model

I compare a reference’s request fit, measured performance and inspection evidence, then mark it Explore, Maybe or Pass with a note. Each save freezes that exact evidence, so reopening it after a restart shows what I saw then.

Reviewed references can change which results come first. My verdicts are kept and shown again, but they don’t train a taste model. That loop isn’t built, and the app says so.

Instead of a taste model I can’t back up

How it’s built and checked

Tests: 2,928 Python and 364 Node tests passed at the Oct 28 gate.
Cost: Indexing 2,062 captions cost $0.0415048, exactly the quote, across 21 of 21 requests.
Lossless proof: Big answers store each source document once, keyed by the SHA-256 of its canonical JSON. That’s deduplication, not summarizing.

Where it stands

Works today

Creator-relative scoring on retained evidence.
Sampled-frame review with exact-time playback.
Saved research and reference decisions that freeze their evidence.

Not yet

Whole-video motion and audio inspection. A direct-video Gemini runner passes its tests, but Google rejected generation (HTTP 402), so no model has inspected a whole video yet.
Reef’s per-chat adapter for the bridge isn’t built.