Whisper large-v3 — Speech Recognition AI Model
Whisper large-v3 transcribes the speech in a video or an audio file and returns word-level timestamps rather than one block of text — which is what karaoke-style captions need. Around 99 languages, and it extracts the audio for you.
What it does
Speech Recognition takes a video or an audio file and returns a transcript of what was said, with a start and end time for every word - what captions need. The model is OpenAI's Whisper large-v3, solid on accents, noise and technical vocabulary.
If you already know what was said - an ad script, or the text you gave a TTS voice - paste it into reference_script. The captions then use your words, so a brand name Whisper mishears comes out right, while the timings still come from the audio.
When to use it
- A meeting, lecture or interview recording turned into readable text
- Subtitles and captions, built from the word-level timestamps
- Translating speech straight to English instead of transcribing it
- Audio recorded in the real world - noise, accents, uneven quality
- Captions for a TTS or dubbed voiceover, spelled exactly as the script
Details
- CategorySpeech Recognition
- Year2026
- LicenseApache-2.0
Compliance & Provenance
- ProviderOpenAI (open-source) · Specialized
- EU AI Act RiskMinimal Risk
- Art. 50 TransparencyNot applicable
Inputs & Outputs
- VideoInputvideooptional
Video clip whose speech is transcribed (audio is extracted internally). Connect this or Sound.
- SoundInputsoundoptional
Audio file to transcribe - a voiceover, TTS or dubbed take. Connect this or Video; if both are connected the sound is used.
- ScriptInputstringoptional
The exact text that was spoken (optional), e.g. the same text fed to Text to Speech. Used like the reference_script setting; when connected it wins.
- TranscriptOutputtranscript
Timed-transcript JSON: language, full text, and per-word start/end seconds.
Parameters
- languagestringdefault Autooptions: Auto, English, Korean, Japanese, Chinese
Spoken language. Auto detects it; set explicitly to force.
- taskstringdefault Transcribeoptions: Transcribe, Translate to English
Output mode. 'Transcribe' keeps the spoken language; 'Translate to English' uses Whisper's native speech-to-English mode — run the same clips twice (transcribe + translate) to get both transcripts for a bilingual subtitle stack.
- vocabularystringdefault advanced
Domain words / names to spell correctly (optional)
Optional comma-separated list of names and domain words that appear in the audio (pet names, people, places, dish names), e.g. "모카, 우유, 알파카, 축사". Whisper is biased toward the spelling of these, which fixes proper nouns and liaison mishearings it would otherwise resolve to a more common word (observed: "양이랑" heard as "냥이랑", drifting toward the far more frequent 고양이/냥이). Only takes effect on audio longer than ~30s (the sequential long-form decoding path); shorter clips ignore it. Leave blank for unbiased transcription.
- reference_scriptstringdefault advanced
The exact text that was spoken (optional). Captions use these words with the recognized timings.
Optional script that was actually spoken, e.g. an ad voiceover or the TTS input text. The recognized words are aligned to it word by word and replaced by the script words, keeping the recognized start/end times - this fixes brand names and proper nouns Whisper mishears (observed: "See-naps" heard as "Scene apps", 씨냅스 as 시넵스). Applied only when at least half of the script words and half of the heard words match; otherwise the transcript is left as recognized and `script_alignment.applied` is false (a script for a different take, or far longer than what was said). When applied, the raw recognition is kept in `asr_words`. Script words that were not heard get a short slot after the previous word. Words are split on spaces, so a Japanese or Chinese script without spaces is not applied; numbers written differently ("3" vs "three") count as mismatches. Max 5000 characters. Single video/sound input only; ignored for clip lists.
Tags
Alternatives in Audio Understanding
- Speech Recognition (Clips)
Recognize speech in each clip of a video list and convert it to timed transcripts.
- Transcribe (Gemini)
Google Gemini 3.5 Transcribe: speech to text in 85+ languages, with optional speaker labels and custom vocabulary. Needs API key.