Transcription Pipeline

Episode transcription runs as the TranscriptionWorkflow in workers/ai-processor/src/workflows/transcription.ts. Whisper (@cf/openai/whisper-large-v3-turbo) transcribes MP3 audio in chunks; a GLM pass (@cf/zai-org/glm-4.7-flash) assigns heuristic speaker labels; the result is stored as a versioned R2 artifact plus a canonical episode_transcripts row, editable on the transcript surface and publishable to the RSS feed as WebVTT.

Job creation paths

Both paths converge on createAndEnqueueAiJob() (src/lib/server/ai-jobs.ts); see the section overview for the row-then-message contract.

Manual: POST /api/ai/transcriptions/:episodeId

src/api/routes/ai/index.ts. Chain: requireAuth() then zValidator (language: ISO 639-1 or '' for auto-detect; confirmReplace) then requirePodcastRoleByResolver('member') then requireFeature('ai_transcription'). Pre-enqueue checks, in order:

Check	Failure
Audio uploaded and finalized (audio_url + audio_duration_seconds)	428 audio_not_finalized
Content type is audio/mpeg (MP3-only for this release)	422 unsupported_audio_format
Size within MAX_AUDIO_BYTES (100 MiB, roughly 2h at 128kbps)	422 audio_too_large
audio_url resolves to this episode's own managed R2 key (deriveManagedAudioKey)	422 unsupported_audio_format (externally hosted or foreign-key audio would fail at the R2 read anyway)
Credit preflight: effectiveRemaining covers ceil(minutes)	429 quota_exceeded with { needed, remaining } (UX only; the worker's hard reserve stays authoritative)
Existing transcript without confirmReplace: true	409 transcript_exists (re-running replaces the transcript, including edits)
Another active job for this episode + type	409 job_active

Success returns 201 { jobId, estimatedCredits }.

Auto-transcribe on upload

maybeAutoTranscribe() in src/api/routes/media/audio.ts runs after audio finalize when podcasts.auto_transcribe_uploads is set (20260703161337_podcast_auto_transcribe.sql). It silently skips non-MP3 uploads, over-ceiling files, unprovisioned accounts, and accounts without the ai_transcription feature. A job_active conflict is swallowed deliberately: the in-flight job owns reconciliation with the new upload (see re-runs and successors).

Workflow steps

1. validate. Loads the job row (must be transcription, status pending/running) and the episode context, then re-resolves the episode's CURRENT managed audio key from episodes.audio_url. The enqueue-time input.audioKey is informational only: a replacement upload between enqueue and start simply gets transcribed instead. MP3-only and the 100 MiB ceiling are re-checked here. Transitions the job to running.
2. plan-chunks. Streams the MP3 from R2 through a frame scanner (workers/ai-processor/src/lib/mp3-chunks.ts, rolling-window memory) and produces byte-range chunks of about AUDIO_CHUNK_BYTES (1 MiB) each. The plan's frame-derived total duration is the ONLY metering input. It deliberately runs before reserve-credits: episodes.audio_duration_seconds is client-supplied display data, and pricing the hard reserve from it would let an understated duration transcribe a long file for a few credits. The cost of a quota-refused job is one R2 stream scan, no AI spend.
3. reserve-credits. Hard pre-pay of ceil(measuredMinutes) credits via reserveTranscriptionCredits(), idempotency-keyed by the job id. Refusal is terminal: mark-skipped records a host-visible message ("Your account has no AI credits remaining this month."), notify_ai_job_result fires, and the workflow returns cleanly (a monthly quota never replenishes inside a retry window).
4. transcribe-chunk-{i} x N. Bounded-concurrency (TRANSCRIBE_CONCURRENCY, default 6) Whisper calls over R2 range reads, one durable step per chunk, so a failure resumes at the failed chunk. Live progress writes are best-effort and never fail a step.
5. diarize. Stitched segments go through sequential GLM batches (workers/ai-processor/src/lib/diarize.ts) that assign speaker letters. Progress walks 88 to 97 per batch so long episodes never look frozen. The label Map is flattened to a plain object (step outputs must be JSON-serializable).
6. persist. A staleness guard re-reads the episode's current audio key and refuses (terminal error) when it no longer matches the key this run transcribed. Otherwise the TranscriptArtifact JSON is written write-ahead to the job-versioned key raw.{jobId}.json, then upsertTranscript() commits the episode_transcripts row pointing at it (resetting edited and published state; the API required confirmReplace for exactly this reason). The superseded raw object is deleted best-effort after the commit; a missed delete is a benign orphan swept by episode deletion's prefix cleanup.
7. cleanup-stale-publish. A re-run of a previously PUBLISHED transcript leaves the episode's RSS pointers aimed at an invalidated VTT. The step is state-derived (findStaleTranscriptPointerKey reads the live episode row, never a value captured before the upsert) and ordered fail-benign: clear episodes.transcript_url/transcript_type first, then queue an RSS invalidation (transcript_replaced), then delete the VTT last and best-effort, so no failure can leave the feed pointing at a deleted object.
8. finalize. Aggregates cost telemetry into ai_jobs.cost: API-reported Workers AI neurons when present, otherwise the price-table estimate (workers/ai-processor/src/lib/pricing.ts). Marks the job complete with result_key and credits_spent.
9. notify-and-fan-out. Calls the notify_ai_job_result RPC (20260703161338; notifies owners/admins plus the requester, deduped per recipient/job/terminal-status) and sends one episode.transcribed automation event; the automation-scheduler owns rule matching from there.

Failure path

mark-failed records a user-safe error (guarded so it can never clobber a terminal success or skip), refund-credits calls the refund_ai_job_credits RPC (idempotent pool grant, see Credits & Metering), check-successor decides whether the failure traces to a replaced upload, and notify-failed runs only when no successor job covers the episode (the successor's own terminal outcome notifies instead).

Versioned artifact keys

Layout from src/lib/transcripts/keys.ts (shared by the API, cleanup paths, and the worker via relative import):

Key	Contents	Write discipline
podcasts/{p}/e/{e}/transcripts/raw.{jobId}.json	Immutable AI output (TranscriptArtifact)	Write-ahead, versioned per producing job
podcasts/{p}/e/{e}/transcripts/edited.json	Current edited state (same artifact shape)	Fixed key, overwritten in place (the writer replaces their own latest content)
podcasts/{p}/e/{e}/transcripts/transcript.{uuid}.vtt	Published WebVTT	Write-ahead, versioned per publish

The point of write-ahead versioning (PR #190 round 8, migration 20260711140000_versioned_transcript_artifact_keys.sql): the previous design wrote fixed canonical keys BEFORE the DB commit, so a re-run overwrote the last good raw.json before upsertTranscript(), and a failed upsert left the row describing the OLD transcript while its only artifact held the failed run's content. A re-publish likewise swapped the LIVE public transcript.vtt before the publish RPC could fail. Now each producer writes a fresh versioned object and only the committed row makes it live; nothing is ever overwritten in place, so a failed commit leaves the previous artifact and the previous public URL fully intact. Unversioned canonical names (raw.json, transcript.vtt) remain valid for rows created before the scheme.

Two DB-side guards back this up (20260710153000 re-created by 20260711140000): CHECK constraints pin raw_key/published_key to this row's episode-scoped transcripts/ prefix in canonical-or-versioned form (no writer can point the API at another tenant's object), and client UPDATE on episode_transcripts is column-scoped to the edit-save/publish columns.

Publish to feed

POST /api/ai/transcriptions/:episodeId/publish (admin+, per the roles matrix's Episode Operations):

1. Render the current artifact (edited_key ?? raw_key) to WebVTT with speaker voice tags (src/lib/transcripts/render.ts).
2. Write it to a fresh versionedPublishedTranscriptKey (new UUID per publish); the feed and the previously published URL are untouched until the commit.
3. Call the publish_episode_transcript RPC (20260710170000): the transcript row and the episode's Podcast 2.0 pointers (transcript_url, transcript_type = 'text/vtt') commit in ONE transaction, so the surface and the public feed can never disagree. On RPC failure the staged object is deleted best-effort and the route 500s.
4. Side effects: queue transcript_published RSS invalidation, then delete the superseded published object (after the purge, so a rebuilt feed already points at the new URL while edge/browser cache tiers age out).

DELETE .../publish mirrors this with the unpublish_episode_transcript RPC (clears both tables atomically, returns the previously published key for best-effort deletion) plus a transcript_unpublished invalidation.

DR invariant

The published VTT is a feed-referenced asset: it MUST live in the podcasterplus-media bucket under a canonical media.podcasterplus.com key that replicates to B2, or it becomes a dead link during a failover. The versioned key scheme above satisfies this; never publish transcripts to any other bucket or host. See the feed-referenced-assets invariant in .claude/rules/backend/workers.md.

Re-runs and successor jobs

Two run-time truths anchor replacement handling:

• Pending job, new upload: validate re-resolves the current key, so the job just transcribes the replacement. This is why the upload hook swallows job_active.
• Mid-run replacement: the persist guard refuses the now-stale transcript (terminal failure). On the failure path, check-successor compares the episode's current key against the key this run ACTUALLY used (ranAudioKey from the cached validate result, falling back to input.audioKey only when validate never completed; judging by the stale enqueue key would misread an ordinary failure as a replacement). When they differ AND the podcast still opts into auto-transcribe AND ai_transcription is still granted AND the new audio passes the format/size checks, insertRequeuedJob() inserts a fresh job and spawns its workflow directly. The partial unique index frees only once the failed job is terminal, which is why the upload hook could not do this. If the insert conflicts, the step ensures whatever active job exists is spawned (create-on-existing dedupes). Unrelated failures (Whisper down, invalid MP3) still point at the same audio and notify normally.

Editor surface

The transcript surface lives at src/routes/(app)/p/[slug]/e/[episodeSlug]/transcript/. It is a SUB-SURFACE of the Media tab, not a tab of its own: the shell keeps Media highlighted and the page renders a SubSurfaceHeader back link instead of chrome of its own (see SvelteKit Routing), with components in src/lib/components/transcripts/ (segment list/rows, speaker assignment, find and replace, synced audio player, promo assets panel).

A missing episode_transcripts row is a real state: the load returns transcript: null and the page renders an empty pane (grant present) or a locked upsell pane (ai_transcription not granted, resolved fail-closed by the shell layout load) instead of the pre-shell 303 redirect to the Media tab.

• GET /api/ai/transcriptions/:episodeId returns the transcript meta, the current artifact (edited over raw), the latest complete promo generation, and any in-flight jobs so the UI can resubscribe.
• PUT /api/ai/transcriptions/:episodeId saves edits: validates segments (max 20,000, per-segment text cap 5,000 chars, end >= start) and the speaker map, enforces roster integrity (every mapped episode_person_id must belong to this episode, else 400), applies edits via applySegmentEdits() (src/lib/transcripts/edit.ts, which interpolates word timings for edited segments), writes edited.json, and updates the row (edited_key, edited_at, edited_by, speakers, word_count).

Related

• AI Features overview
• Credits & Metering
• AI Processor Worker