[!TIP] Audience: people who want speech recognition + translation on their own machine, without uploading the source video anywhere. Core goal: show what separates “it runs” from “it ships”. Problem-to-solution map:

  • One line repeated 983 times → -mc 0 to disable cross-window context
  • Orphan child processes and misleading black-window errors → Windows Job Object + CREATE_NO_WINDOW
  • Line-by-line translation too slow → batches of 15 with numbered alignment

I’ve been producing bilingual subtitles for videos for a while, with one hard requirement: everything runs locally. The video never leaves the machine, recognition uses whisper.cpp, and translation goes to a local LLM. Writing the tool itself isn’t hard. What’s hard is what surfaces once it’s actually used for real work. Here are four pitfalls with one thing in common: you can’t discover any of them without running it for real.

1. The Repeating-Machine Bug: 983 Duplicates and 32 Lost Minutes

The best story. A real ~44-minute file had the same subtitle line repeated 983 times, and everything after it was lost.

My first guess was that it was random, so I only added a post-processing safety net, collapse_repetitions: collapse 4+ consecutive identical lines into one. Then I re-ran the same file, and it triggered again at the exact same position, 00:08:06. That’s a deterministic failure, not luck. The safety net only made the garbage lines look tidy — the 32 lost minutes weren’t coming back.

The root cause lives in whisper’s implementation: by default it carries the previous 30-second window’s recognized text into the next window as context (condition on previous text). Once a line enters that context, it reinforces itself, becomes increasingly sure “the next line is still this one”, and locks up until the file ends. That also explains why isolated 4-minute and 15-minute clips could never reproduce it — their context is clean.

The fix is -mc 0 (--max-context 0), which forbids carrying text context across windows. Verification ran on the same failing audio: duplicate count 983 → 0, fallbacks 5 → 0, all 44 minutes recognized in full. The cost is only about 68 seconds at the very end losing capitalization, far smaller than the original bug, and left as-is for now. I kept collapse_repetitions as defense in depth.

2. VAD: The Direction Was Backwards

The initial design enabled VAD (voice activity detection) by default, thinking it would filter silence and prevent hallucinations. Real audio from the same documentary disagreed: turning on --vad made the output lose all capitalization and punctuation — “Jeremy Wade, explorer” became “jeremy wade explorer”; off, it was completely normal, fallbacks=0.

VAD’s anti-hallucination benefit was only verified in pure-tone / long-silence scenarios (like a hallucinated “(phone ringing)”). For continuous speech — documentaries, episodes — it hurts more than it helps. So v1 ships with VAD off.

3. Child Process Governance: Orphans and the Black Window

After the main program hides its own console with windows_subsystem = "windows", a side effect appears: child processes (ffmpeg / whisper-cli) get a brand-new console window of their own. If the user closes it by accident, that sends a terminate signal to the child, and the UI reports “audio extraction failed” — it looks like a corrupt file, but really a window was closed. The fix: add CREATE_NO_WINDOW to both Commands, keep stdout/stderr on pipes, so progress parsing is unaffected.

Orphan processes were worse. It really happened once: the main program died unexpectedly (most likely the render layer panicking on “device lost” — it shares the same physical GPU with the recognition engine), and whisper-cli became an orphan on the spot, running for another half hour by itself, holding the GPU and leaving temp files nobody cleaned. remove_file-style “cleanup on exit” code is fundamentally unreliable — the moment of a crash is exactly when that code never runs. The fix is a Windows Job Object with KILL_ON_JOB_CLOSE; every child is bound into it the moment it spawns. That’s a kernel-level guarantee, independent of application code. A test that spawns a real child process locks this mechanism down.

4. Translation: Not Line-by-Line, and Not One Model for All Languages

An episode has hundreds of subtitle lines; per-line HTTP calls are too slow. Instead, send 15 lines per batch, explicitly number them in the prompt and require them preserved, then parse the reply by number and align. When the line count mismatches (missed / extra / merged lines), split the batch in half and retry to narrow the error. Lines that still won’t align keep the original text rather than blocking the whole task.

Model selection had its own trap: SakuraLLM is only trained for the Japanese-to-Chinese direction, and translating English with it is poor (outside its training distribution). So there’s a language-to-model registry: Japanese goes to Sakura, everything else to the general-purpose Qwen3-4B, chosen automatically from the detected source language.

5. Wrapping Up: Recognition Is Only Half of Readable

Getting the words right is half the job; the subtitles still need to be readable. Wrapping goes by display width — CJK / full-width counts as 2 columns, default cap 42 columns (21 Chinese characters) — with break priority space > after punctuation > hard break, never splitting an English word. Measured across four episodes, 3476 blocks total: after wrapping 0 lines exceeded 42 columns, and 38.5% of blocks got a newline inserted — matching “38.2% of English lines are longer than 42 characters”.

There’s a second-level fix: blocks longer than 6s with total width over 2× the cap are split into two by time ratio. Measured 63 blocks split (1.81%), 0 lines over the cap, timeline monotonic and non-overlapping. The surprise along the way: English lines without punctuation were the main bottleneck (34 of 69 qualifying blocks had no punctuation in the English line), so English without punctuation falls back to the nearest word boundary (a space) — still never splitting a word.

Summary

A quick perf note: on an RTX 3060 12G, large-v3 runs about 8× realtime, so a 45-minute episode produces subtitles in about 5–7 minutes. Every pitfall this pipeline hit was “thought A, measured B”: the VAD direction, the misleading closed-window error, the deterministic loop — plus a small one where Windows Notepad writes a UTF-8 BOM that breaks the first srt block’s number parsing, skipping the first subtitle. Software that ships is software that has hit all of these.