MOSS-Transcribe-Diarize · LiteRT portint4-b32 · live

Long audio is split into 90 s windows, cut at the quietest point found in the last 12 s of each window (a genuine pause, not a fixed boundary). Each window's [Sxx] tags are a LOCAL numbering the model resets every call. (The C++ Space links them across windows with CAM++; this LiteRT build does not — see the note below.)

90 s was chosen after a window-length sweep (60/90/180/300/450 s) on real 16-minute meetings in both languages, scored by WER and speaker accuracy against AMI ground truth (English) and by whether a merged segment ever crosses a genuine speaker turn rather than just combining consecutive same-speaker utterances (Chinese, which has no independent transcript). 90 s is the best point found for Chinese and is not distinguishable from any other tested length for English. Measured end-to-end on the golden 16-minute clips: 3.3× faster and about half the peak memory of the old single-pass approach, at WER and speaker accuracy matching or beating it.

Weights are the LiteRT int4 blockwise-32 decoder + dynamic-int8 encoder/embedder (0.73 GB total). Quantization was validated against the project's pinned PyTorch f32 reference, not assumed: the f32 build of this exact pipeline is byte-identical to that reference on all three golden clips, and the deployed int4 decoder scores 98.99% text fidelity (95.45% with timestamps) on the zh 90 s window and 100% on jfk — versus 96.68%/93.67% for the C++ Space's q4mix on the same metric. Each window decodes with a fresh, statically-sized KV cache (2560 entries) that lives entirely inside the runtime — prefill and decode signatures share one set of buffer-bound KV tensors.

Chinese output is converted Simplified→Traditional for display with OpenCC s2t (character-level). The more aggressive s2twp was measured and rejected: on this page's own zh example it corrupts 高端疫苗 → 高階疫苗 and 程序委員會 → 程式委員會.

Deliberately absent, same policy as the C++ Space: repetition penalty, EOS-coverage suppression, loop guards, number ITN. Also absent because they are rs.cpp-engine features this port does not implement: batched multi-window decode, audio-KV eviction, cross-window speaker linking.

Speaker linking note: cross-window CAM++ speaker linking is a C++-Space feature (it needs the rapidspeech-core native build). This LiteRT build ships WITHOUT it: [Sxx] tags are window-local — the numbering restarts at every 90 s window boundary. The engine itself is Apache-2.0/MIT throughout (LiteRT + converted Apache-2.0 weights).