LatentSync-1.6 — Video Generation AI Model
LatentSync redraws only the mouth area of a talking-head video so the lips follow a new soundtrack - a TTS voiceover, a translated dub or a re-recorded line - and leaves the rest of the face, the head motion and the background as they were. The voice is muxed into the result.
What it does
LatentSync 1.6 from ByteDance takes a video of someone talking and a separate voice, and regenerates the mouth region of every frame so the lips match the new audio. Everything outside the mouth is kept from the source video.
It works on one clearly visible face per frame and is strongest on frontal, well-lit faces. If the voice is longer than the video, the video plays forwards and backwards to cover it.
When to use it
- Re-voicing a presenter clip with a script read by a TTS voice
- Dubbing a talking-head video into another language
- Fixing a line in an ad without reshooting
- Matching a generated presenter to an externally produced voice
Details
- CategoryVideo Generation
- Year2025
- LicenseApache-2.0 (code); CreativeML-OpenRAIL++-M (UNet weights); MIT (VAE, Whisper)
Compliance & Provenance
- ProviderByteDance (open weights) · Specialized
- EU AI Act RiskLimited Risk
- Art. 50 TransparencyRequired — AI-generated outputs are marked
Inputs & Outputs
- VideoInputvideo
Talking-head video whose lips are re-synced (one visible face per frame)
- SoundInputsound
The new voice to sync to (up to 60 s); it is muxed into the result
- VideoOutputvideo
The video with re-synced lips and the new voice
Parameters
- stepsintegerdefault 20range 5–50
Denoising steps (5-50)
Denoising steps per 16-frame window (5-50). 20 (default) is upstream's; 50 measured only marginally sharper at 2.5x the time.
- guidance_scalefloatdefault 1.5range 1–3advanced
How strongly the audio drives the mouth (1-3)
Classifier-free guidance on the audio (1.0-3.0). 1.5 (default) is upstream's; higher exaggerates articulation.
- seedintegerdefault -1advanced
Random seed (-1 = random)
Random seed for reproducibility; -1 (default) picks a fresh one.
Tags
Alternatives in Video Generation
- Cosmos3 Nano
NVIDIA world model for text-, image- and video-to-video, with optional synced audio.
- Helios Base
On-device text-to-video, up to 60s @ 24fps. 3-10 min.
- Helios Distilled
Fast distilled text-to-video, fixed 640x384. 1-5 min.
- LTX-2.5
Lightricks LTX-2.5 distilled 22B: text, or text plus a first-frame image, to a video with a synchronized soundtrack generated alongside it.
- LTX-2.5 Audio-to-Video
Make a talking video from one picture and a voice.
- MiniMax-H3 Ref2VA
MiniMax-H3 Ref2VA (omni-reference) generates a 5-15s 24fps video with native stereo audio from a prompt plus a reference image and a reference clip, that clip's own soundtrack included. Video and its soundtrack come out of one denoising loop, so lip movement and sound land in sync. ~144GB bf16 weights — offloaded component-by-component, minutes-scale per clip.
- Wan2.2 TI2V 5B
Wan2.2 dense 5B unified text+image-to-video. Text only → 5s video from prompt; text + image → image as the first frame of the video.