tenbrains · design spec

YouTube transcription

Let tenbrains analyze take a YouTube URL, pull the caption track for free, and run the transcript through the pipeline that already analyses tweets — same topic/summary/intent/concepts, same persistence, same --learn, search, and digest.

Draft · for team review Owner: @moeghashim Target: tenbrains 2.2.0 No new dependency No schema migration

# A transcript is just long-form content — it enters the existing pipeline
tenbrains analyze --url "https://youtube.com/watch?v=…" --summarize --learn
# → fetch captions · analyse · summarize · build a learning track — one call
# meta.source = "youtube" · search / digest / record get see them together

01 Motivation

Long-form video is a major research surface tenbrains can't touch today.
The analysis pipeline is content-agnostic — topic/summary/intent/concepts fit a transcript as cleanly as a tweet.
Cheap to add: the fetch problem is the same shape as the tweet-oEmbed path already in the codebase.

02 Goals & non-goals

Goals

Fetch a public transcript with no API key, no paid tier, no new dependency.
Reuse the whole downstream pipeline (persist → learn → search → digest → suggest).
Structured errors + a manual fallback — never crash.
Pure, fixture-tested parsers so a markup change is a one-line fix.

Non-goals (v1)

Audio transcription (Whisper / yt-dlp) for videos with no captions.
Chapter/timestamp segmentation or a transcript-viewer UI.
Playlist / channel bulk import.
Auto-translation beyond selecting an available track.

03 Fetch — free-first caption scrape

Mirrors src/x/client.ts (the tweet oEmbed path): an undocumented-but-free endpoint, pure parsers split from the network layer, structured errors.

STEP 1Watch pageGET the video's HTML

STEP 2Player responseextract ytInitialPlayerResponse

STEP 3Caption trackpick manual>ASR, lang>en>first

STEP 4Timedtextfetch baseUrl → flatten to text

No new dependency

fetch + regex/JSON parsing — the same toolkit the oEmbed path uses. Rejected: youtube-transcript (fragile, redundant), yt-dlp (external binary, breaks the self-contained stance), and the YouTube Data API captions.download (needs the owner's OAuth — the same dead end that makes X timelines a paid feature).

04 Integration — extend `analyze`

No parallel command. analyze detects a YouTube URL in --url and routes to the transcript fetcher, exactly as --thread routes differently from a plain tweet. Everything after ingestion runs unchanged.

Field	Source
`post.text`	flattened transcript
`post.externalId`	`yt:<videoId>` — distinct prefix, dedupes re-runs, no tweet-id collision
`post.url`	canonical `watch?v=<id>`
`post.authorUsername`	channel
`post.postedAt`	upload date
`post.raw`	video metadata → existing `posts.raw_json` (no migration)
`meta.source`	`"youtube"`

The analyzer prompt is tweet-tuned; pass an optional kind: "tweet" | "transcript" and swap one sentence to "You are analyzing a video transcript." Output schema untouched.

Companion actions — `--summarize` & `--learn`

Two composable follow-on actions run on the same fetched-and-analysed content in one call — pass either, both, or neither. General analyze options, but they earn their keep on long-form transcripts.

--summarize

A fuller narrative digest ({ summary, keyPoints[] }), distinct from the terse analysis summary field.
New summarizeContent() + prompt + schema + mock — offline-testable like the rest.
Returned under data.summary; meta.summarized = true; persisted in raw_json.
Doubles as the condense-first pass for long transcripts, so the expensive extraction sees a shorter document.

--learn

Builds the 7-day Feynman track from the analysis' concepts (--minutes, --ratings apply).
Already built — works for any analysed content, transcript included.
Track progress via the existing learn today / learn done.

Order in one invocation: fetch → analyse → (summarize) → (learn), all persisted against the same post.

05 Errors & fallback

Reuses the existing PROVIDER_* scaffolding. Every "no captions" path points at the manual escape hatch analyze --transcript @file, which also makes the feature testable offline.

Condition	Code	Exit
Not found / deleted / no caption tracks	NOT_FOUND	3
Private / age-restricted / region-blocked	PROVIDER_UNAUTHORIZED	5
Rate limited	PROVIDER_RATE_LIMITED	5
Network / timeout	PROVIDER_NETWORK	5
Markup changed, can't parse	PROVIDER_BAD_OUTPUT	5

06 Surface, storage, testing

analyze --url <youtube-url> # fetch + analyse captions
analyze --url <url> --lang es # prefer a caption language
analyze --transcript @file | - # supply a transcript manually (no network)
analyze --url <url> --summarize # + a fuller narrative summary
analyze --url <url> --summarize --learn # both, in one call

Storage: reuse posts + analyses; metadata + summary in raw_json — no schema change (stays v3).
Long transcripts: v1 sends as-is with a soft length warning; --summarize condenses first when set.
Testing: fixture unit tests for the three pure parsers, one offline command test via --transcript, one live smoke — the same shape as the shipped thread feature.

07 Risks

Risk	Mitigation
YouTube changes watch-page markup	Pure, fixture-tested parsers → one-line fix. Same risk class as existing oEmbed HTML parsing.
Video genuinely has no captions	Structured `NOT_FOUND` → `--transcript @file`. Audio transcription deferred.
ToS / gray-area scraping	Same posture as tweet oEmbed + archive paths — public captions, personal research; noted in docs.
Token cost on long transcripts	Soft warning in v1; `--summarize` condenses first (§4).

08 Open decisions — team input

Fold into analyzerecommended vs. a dedicated tenbrains youtube command. Folding in reuses everything and adds no surface; a separate command is only worth it if video-specific features (chapters, timestamps, a viewer) are expected to grow.

Caption-only v1recommended with --transcript @file as the manual fallback — defer audio/Whisper transcription entirely?

Ship as a minor — 2.2.0. Purely additive; no breaking changes.

Effort

Closely matches the already-shipped thread feature: a ~200-line client module (pure parsers + orchestration), a ~40-line analyze integration, fixture tests + one offline command test + a live smoke, and docs. One focused PR through the existing branch → PR → CI (DCO + typecheck/lint/test) flow.