Pure MLX port of LTX-2 for Apple Silicon. Three-package monorepo mirroring the reference structure — inference, pipelines, and training — running natively on Metal.
- LTX-2.5 — pipeline coverage on 2.5 packs (
--distilled,--two-stage,--two-stages-hq,--one-stage,--dfr,keyframe,a2v,retake,extend,hdr-ic-lora), auto-detected from the pack (no new flag), plus auto-predicted duration via the DurationHead (omit-f, or clamp with--auto-duration MIN:MAX). Onlyic-lora/lipdubwait on official 2.5 task LoRAs. See LTX-2.5 section. - Text-to-Video — generate video + stereo 48kHz audio from a text prompt
- Image-to-Video — animate a reference image;
--image PATH FRAME STRENGTHis repeatable for multi-anchor I2V, andFRAMEmay belastor negative (counted from the end) - Audio-to-Video — generate video conditioned on an audio track (
a2v, dev + CFG;a2v --distilledfor the fast distilled path, experimental) - Retake / Extend — edit existing videos (regenerate segments, add frames);
retake --distilled/extend --distilledrun on the distilled model, roughly 10–20× faster (experimental) - Keyframe interpolation — smooth transition between reference images
- IC-LoRA — reference video conditioning (depth/pose/edges)
- HDR IC-LoRA — single-stage ACEScct SDR-to-HDR (LTX-2.5 packs) producing an HLG BT.2020 10-bit mp4 + EXR frames
- LipDub (experimental) — lip-dub a reference video by re-syncing visuals to the source audio
- Two-stage generation — half-res → neural upscale → refine
- HQ generation — res_2s second-order sampler + CFG/STG guidance
- Prompt Relay — sequence local prompts over time within one generation (
--segment "text" [LEN]); a training-free Gaussian penalty gates each prompt's tokens to a slice of the timeline via the video→text cross-attention. Works across all generate modes; on CFG modes the mask applies to the conditional pass only. - Negative prompt (
--negative-prompt) — custom CFG negative on every CFG pipeline (--one-stage/--two-stage/--two-stages-hq/ a2v / keyframe / retake / extend); defaults to the upstreamDEFAULT_NEGATIVE_PROMPT. Distilled modes have no CFG and reject it, exceptgenerate --distilled/--dfrwith--nag(experimental Normalized Attention Guidance, applied inside the text cross-attentions). - Prompt enhancement — Gemma 3 12B rewrites short prompts into detailed descriptions
- Training — LoRA fine-tuning with flow matching (T2V and V2V strategies)
- Block streaming (
--low-ram) — stream transformer blocks from disk so q8 fits 16 GB Macs and bf16 fits 32 GB Macs (covers everygeneratemode including--distilledand--dfr, a2v / keyframe / ic-lora / hdr-ic-lora / lipdub / retake / extend; bind-time LoRA fusion supports custom distilled-lora-strength) - Modality tiling (
--tile-frames N --tile-spatial M) — split video tokens into spatial+temporal tiles to cap O(N²) attention activations. Combined with--low-ram, unblocks long / HD / 4K generations on Mac Studio (64-128 GB) that would otherwise OOM. - 6 model packs — bf16 / int8 / int4 for each of LTX-2.3 and LTX-2.5 (fits 16GB–64GB Macs)
- 3 upsamplers — spatial 2x, spatial 1.5x, temporal 2x
Production readiness: pipelines are classified as Stable / Beta / Experimental. See docs/PIPELINE_MATURITY.md before relying on a pipeline in production code. CLI subcommands flagged
[beta]or[experimental]in--helpmay have known quality limitations or pre-1.0 LoRA dependencies.
- macOS with Apple Silicon (M1/M2/M3/M4)
- Python 3.11+
- 32GB+ RAM recommended (int8) or 16GB+ with
--low-ram. 16GB minimum (int4 without streaming) - ffmpeg (for video encoding)
- HDR (
hdr-ic-loraonly): OpenEXR via thehdrextra (uv sync --extra hdrin this repo, orpip install 'ltx-pipelines-mlx[hdr]') and an ffmpeg built withlibx265
git clone https://github.com/dgrauet/ltx-2-mlx.git
cd ltx-2-mlx
uv sync --all-extrasNot sure which pipeline to reach for? See docs/PIPELINES.md § Which pipeline? for a decision tree.
# Text-to-Video — pick a pipeline mode (one of `--two-stage`, `--two-stages-hq`, `--one-stage`, `--distilled`).
# Two-stage is the upstream-recommended production default.
# NOTE: -f/--frames is required on 2.3 packs (the default model below) — it
# only defaults to an auto-predicted duration on LTX-2.5 packs. See "LTX-2.5"
# and "CLI Reference" below.
ltx-2-mlx generate --prompt "A sunset over the ocean" --two-stage -f 97 --frame-rate 24 -o sunset.mp4
# Image-to-Video (any mode supports --image)
ltx-2-mlx generate --prompt "Animate this" --image photo.jpg --two-stage -f 97 --frame-rate 24 -o animated.mp4
# HQ (res_2s sampler, highest quality)
ltx-2-mlx generate --prompt "A scene" --two-stages-hq --stage1-steps 20 -f 97 --frame-rate 24 -o hq.mp4
# Distilled two-stage (fastest, mirrors upstream DistilledPipeline)
ltx-2-mlx generate --prompt "A scene" --distilled -H 720 -W 1280 -f 97 --frame-rate 24 -o distilled.mp4
# One-stage dev + CFG (full target res, mirrors upstream TI2VidOneStagePipeline)
ltx-2-mlx generate --prompt "A scene" --one-stage -f 97 --frame-rate 24 -o one_stage.mp4
# Audio-to-Video
ltx-2-mlx a2v --prompt "Music video" --audio music.wav --frame-rate 24 -o a2v.mp4
# Retake (regenerate frames 1-3 of a video)
ltx-2-mlx retake --prompt "New action" --video source.mp4 --start 1 --end 3 -o retake.mp4
# Extend (add 2 latent frames after)
ltx-2-mlx extend --prompt "Continue the scene" --video source.mp4 --extend-frames 2 -o extended.mp4
# Distilled extend (no CFG, only the new window is denoised; experimental)
ltx-2-mlx extend --distilled --prompt "Continue the scene" --video source.mp4 --extend-frames 4 -o extended_fast.mp4
# Keyframe interpolation
ltx-2-mlx keyframe --prompt "Smooth transition" --start frame1.png --end frame2.png --frame-rate 24 -o transition.mp4
# Prompt enhancement
ltx-2-mlx enhance --prompt "a cat" --mode t2v
# Use int4 model (fits 16GB)
ltx-2-mlx generate -p "A cat" --distilled -f 97 --frame-rate 24 -o cat.mp4 --model dgrauet/ltx-2.3-mlx-q4
# Block streaming: bf16 model on 32 GB Mac
ltx-2-mlx generate -p "A cat" --two-stage -f 97 --frame-rate 24 -o cat.mp4 --model dgrauet/ltx-2.3-mlx --low-ram
# Block streaming: q8 model on 16 GB Mac
ltx-2-mlx generate -p "A cat" --distilled -f 97 --frame-rate 24 -o cat.mp4 --model dgrauet/ltx-2.3-mlx-q8 --low-ram
# Block streaming works on every generate mode + a2v / keyframe / ic-lora / hdr-ic-lora / retake / extend
ltx-2-mlx generate -p "A cat" -f 97 --frame-rate 24 -o cat.mp4 --two-stage --low-ram
ltx-2-mlx generate -p "A cat" -f 97 --frame-rate 24 -o cat.mp4 --two-stages-hq --low-ram
ltx-2-mlx a2v -p "music video" --audio music.wav --frame-rate 24 -o a2v.mp4 --low-ram
ltx-2-mlx keyframe -p "transition" --start a.png --end b.png --frame-rate 24 -o kf.mp4 --low-ram
ltx-2-mlx ic-lora -p "scene" --lora lora.safetensors 1.0 --video-conditioning depth.mp4 1.0 --frame-rate 24 --low-ram -o out.mp4
# HDR IC-LoRA — SDR mp4 -> HLG master + ACEScg EXR frames (LTX-2.5 pack; gated LoRA repo)
ltx-2-mlx hdr-ic-lora --model dgrauet/ltx-2.5-mlx-q8 --input source_sdr.mp4 \
--hdr-lora ltx-2.5-22b-ic-lora-sdr-to-hdr-1.0.safetensors \
--text-embeddings ltx-2.5-22b-ic-lora-sdr-to-hdr-scene-emb.safetensors --low-ram -o out.mp4
# Modality tiling: split video tokens for long/HD scenarios that exceed attention memory.
# Stack with --low-ram for max memory savings on big targets.
ltx-2-mlx generate -p "long scene" --two-stage --low-ram -f 97 --frame-rate 24 \
--tile-frames 2 --tile-overlap 4 -o long.mp4
ltx-2-mlx generate -p "1080p scene" --two-stages-hq --low-ram -f 97 --frame-rate 24 \
--tile-spatial 2 --tile-overlap 4 -H 1024 -W 1920 -o hd.mp4
# Model info
ltx-2-mlx info --model dgrauet/ltx-2.3-mlx-q8ltx-2-mlx generate --distilled --model /path/to/ltx-2.5-mlx-q8 \
--prompt "a heavy wooden door creaks slowly open" --frame-rate 24 -o out.mp4
# Clamp the auto-predicted duration to 2-4 seconds instead of the default [1, 20]s
ltx-2-mlx generate --distilled --model /path/to/ltx-2.5-mlx-q8 \
--prompt "a heavy wooden door creaks slowly open" --auto-duration 2:4 --frame-rate 24 -o out.mp4The 2.5 generation is auto-detected from the model pack (no new CLI
flag). A HuggingFace repo id (dgrauet/ltx-2.5-mlx-q8) or a local pack
directory both work. The pack bundles its own Gemma-4 text encoder
(text_encoder.safetensors), so no mlx-community Gemma download happens
on this path. --image (I2V) works the same as on
2.3.
-f/--frames is optional on 2.5 packs: omit it and the pack's
DurationHead predicts a clip length (seconds) from the encoded prompt
right after text encoding, snapped to the model's frame grid. Pass
--auto-duration MIN:MAX (seconds) to override the predictor's clamp
range, or pass -f explicitly to bypass prediction entirely (explicit
-f always wins). On 2.3 packs, -f stays required — there is no
DurationHead to predict from, and omitting it now raises immediately
(ValueError: ... Pass num_frames explicitly.) before any Gemma load.
Sampling: both stages of --distilled run euler-ancestral on 2.5 packs
(8 + 3 steps, SDE noise injection; stage-2 noise seeded from seed + 20000),
as upstream since v1.4.0. 2.3 packs stay deterministic Euler on both stages.
The dev model + CFG two-stage pipeline also works on 2.5 packs:
ltx-2-mlx generate --model /path/to/ltx-2.5-mlx-q8 --two-stage \
--prompt "a heavy wooden door creaks slowly open" --frame-rate 24 -o out.mp4v1 limits on 2.5 packs:
| Feature | Status |
|---|---|
--two-stage (dev + CFG) |
supported (see above) |
--two-stages-hq (res_2s + CFG) |
supported — validated e2e on 2.5 (deterministic, audio at healthy 2.3-level loudness) |
DurationHead / auto-duration (-f optional) |
supported — -f defaults to an auto-predicted duration on --one-stage/--distilled/--two-stage/--two-stages-hq/--dfr; --auto-duration MIN:MAX overrides the clamp. Not available on 2.3 packs (no DurationHead weights); -f stays required there |
keyframe |
supported — validated e2e on 2.5 (deterministic, audio -38.3 dB; requires --dev-transformer transformer-dev.safetensors) |
a2v |
supported — validated e2e on 2.5 (deterministic, conditioned audio faithfully reconstructed at -36.2 dB) |
retake, extend |
supported — validated e2e on 2.5 (retake deterministic ×2; extend +N latent frames). --low-ram wired (mirrors upstream offload_mode): 49-frame retake that OOM'd now peaks at 13.8 GB |
ic-lora, lipdub |
not yet supported (no official 2.5 task IC-LoRAs published yet) |
hdr-ic-lora |
supported, 2.5 only (single-stage ACEScct SDR-to-HDR; Experimental, validated end to end on real weights) |
enhance / --enhance-prompt |
raises a clear error (Gemma 3-only) |
--enable-teacache |
raises a clear error (not calibrated for 2.5) |
| Prompt Relay | supported — validated e2e on 2.5 --distilled and the --dfr base path; token ranges pinned against the pack's Gemma-4 tokenizer |
| Modality tiling | validated on 2.5 --distilled (--tile-frames 2, unconditioned and with an end anchor) |
| Diffusion video decoder | opt-in --video-decoder diffusion (experimental; auto-tiled above the decode budget, --diffvae-tile override); conv stays the default |
The IC-LoRA family (ic-lora / lipdub) lands once
Lightricks publishes the official 2.5 task IC-LoRAs.
For maximum detail on a 2.5 pack, generate --dfr (experimental) runs a base DFR pass with
the official detailing IC-LoRA — see the generate --dfr card in the pipelines guide.
That LoRA (Lightricks/LTX-2.5-22b-IC-LoRA-Pixel-Spatial-Upscaler) is a gated HuggingFace
repo: accept its licence on the model page once, with the account huggingface-cli login uses,
before the first --dfr run (the run fails before loading any model otherwise).
Pick the pipeline class matching your target — every public class mirrors an upstream Lightricks/LTX-2 pipeline.
Two-stage (recommended for most use cases — dev model + CFG + upscale):
from ltx_pipelines_mlx import TI2VidTwoStagesPipeline
pipe = TI2VidTwoStagesPipeline(model_dir="dgrauet/ltx-2.3-mlx-q8")
pipe.generate_and_save(
prompt="A sunset over the ocean with waves crashing",
output_path="sunset.mp4",
height=480,
width=704,
num_frames=97,
frame_rate=24.0,
seed=42,
image="photo.jpg", # optional I2V
)For other modes:
DistilledPipeline— fastest (distilled half-res + upscale).TI2VidTwoStagesHQPipeline— highest quality (res_2s + CFG + upscale).TI2VidOneStagePipeline— full-res CFG, no upscaler dependency.
Audio-to-Video:
from ltx_pipelines_mlx import A2VidPipelineTwoStage
pipe = A2VidPipelineTwoStage(model_dir="dgrauet/ltx-2.3-mlx-q8")
pipe.generate_and_save(
prompt="A musician performing",
output_path="a2v.mp4",
audio_path="music.wav",
frame_rate=24.0,
)Retake / Extend (single class — extend is folded into RetakePipeline):
from ltx_pipelines_mlx import RetakePipeline
pipe = RetakePipeline(model_dir="dgrauet/ltx-2.3-mlx-q8")
# Retake: regenerate latent frames 1-3
video_lat, audio_lat = pipe.retake_from_video(
prompt="A different scene",
video_path="source.mp4",
start_frame=1,
end_frame=3,
)
# Extend: add 2 latent frames after
video_lat, audio_lat = pipe.extend_from_video(
prompt="Continue the motion",
video_path="source.mp4",
extend_frames=2,
direction="after",
)For the full flag reference, start with docs/PIPELINES.md § Which pipeline? — a decision tree, a card per pipeline, and a flag matrix covering every CLI subcommand, its defaults, and which memory / perf flags apply where.
A generation is opaque until the final VAE decode, which for a long clip can be many
minutes away. These flags decode a short window of latent frames from the in-progress
prediction every N steps and write it as a self-contained animated WebP — a couple of
seconds of real motion at the current denoise quality, so temporal problems show up at
step 4 instead of minute 15. Available on every generating subcommand except hdr-ic-lora
(generate, a2v, retake, extend, keyframe, ic-lora, lipdub).
--stepwise-image-output-dir DIR Write per-step preview clips here
--stepwise-interval N Preview every N steps (default: 1; last step always previewed)
--stepwise-frames N Latent frames per preview (default: 8 -> 57 frames, ~2.3s)
--stepwise-frame I Latent frame the window is centred on (default: middle)
ltx-2-mlx generate -p "a cat walking" -o out.mp4 -f 97 --frame-rate 24 \
--stepwise-image-output-dir ./previewsProduces, for seed 42 on a two-stage run:
previews/
seed_42_s1_step001of030.webp # 57 frames of motion, this step
seed_42_s1_step002of030.webp
...
seed_42_s2_step003of003.webp
Each file is written once and never rewritten, so write cost stays linear in the step count. Play them back to back for the full progression: the same motion span replayed at each step, sharpening as the denoise converges.
The VAE upsamples time 8x, so N latent frames decode to 8N-7 pixel frames. A single
latent frame yields one picture while paying for the whole up-block stack — the worst
point on the cost curve. Measured on a 704x448 clip:
| latent frames | pixel frames | motion @25fps | stage 1 | stage 2 |
|---|---|---|---|---|
| 1 | 1 | — | 0.06s | 0.21s |
| 3 | 17 | 0.68s | 0.49s | 1.90s |
| 8 (default) | 57 | 2.3s | 1.55s | 6.13s |
| 16 | 121 | 4.8s | 3.23s | 13.0s |
Cost is independent of clip length — decoding 8 of 60 latent frames costs the same as 8 of 16 — so previews get proportionally cheaper the longer the clip. On a 15-minute two-stage run the default adds roughly 7%, and most of that is stage 1, which decodes at half resolution.
Notes:
- Memory: previews keep the VAE decoder resident through denoising, which the
pipelines otherwise deliberately avoid — the decoder and the transformer are live at
the same time. On a memory-tight machine that can get the process killed by the OS
with no error at all, so enabling previews always prints a warning. Combining them
with
--low-ramworks but is self-defeating, and adds a second line saying so. - The window is centred on the middle of the clip by default, because frame 0 is the clean conditioning image on image-conditioned runs and never changes.
- Multi-stage pipelines tag files
_s1/_s2; the stages run at different resolutions. - Decoding a window in isolation uses different boundary padding than the full-volume decode, so the outermost frames are approximate. Fine for judging motion, not for judging final quality.
- A preview failure never aborts a generation: the first one logs a warning and disables previews for the rest of the run.
See docs/PIPELINES.md § Speed and § Memory for the watchdog / Gemma-length / low-RAM environment variables and their trade-offs.
LTX2_COMPUTE_DTYPE=float16 runs the DiT's attention and feed-forward in float16 (opt-in; on an M1 Max with the 2.5 q8 pack a 576×1024×241 --distilled I2V render went from 701 s to 563 s).
The number of frames lives on the 8k + 1 grid (VAE temporal compression 8x); an off-grid request is floored to it with a warning (e.g. 87 → 81). Common values at 24 fps:
| Frames | Duration | Latent frames | Notes |
|---|---|---|---|
| 9 | 0.4s | 2 | Minimal, for quick tests |
| 25 | 1.0s | 4 | Short clip |
| 41 | 1.7s | 6 | |
| 49 | 2.0s | 7 | |
| 65 | 2.7s | 9 | |
| 81 | 3.4s | 11 | |
| 97 | 4.0s | 13 | Default for keyframe / a2v / ic-lora |
| 121 | 5.0s | 16 | |
| 145 | 6.0s | 19 | |
| 161 | 6.7s | 21 | |
| 193 | 8.0s | 25 | Requires 64GB+ RAM |
Higher frame counts require more RAM. With int4 on 32GB, 97 frames at 512x320 is comfortable. Reduce resolution for longer videos.
| Variant | HuggingFace | Size | RAM |
|---|---|---|---|
| bf16 | dgrauet/ltx-2.3-mlx | ~42 GB | 64 GB+, or 32 GB with --low-ram |
| int8 | dgrauet/ltx-2.3-mlx-q8 | ~26 GB | 32 GB+, or 16 GB with --low-ram |
| int4 | dgrauet/ltx-2.3-mlx-q4 | ~12 GB | 16 GB+ |
| bf16 (2.5) | dgrauet/ltx-2.5-mlx | ~120 GB | --low-ram on 32 GB |
| int8 (2.5) | dgrauet/ltx-2.5-mlx-q8 | ~75 GB | recommended for 2.5 |
| int4 (2.5) | dgrauet/ltx-2.5-mlx-q4 | ~47 GB | lower quality |
The 2.5 packs carry both the dev and the distilled transformer plus the Gemma-4 text tower, so their size is the download, not what one pipeline loads.
Weights are pre-converted to MLX format by mlx-forge.
| Package | Description |
|---|---|
ltx-core-mlx |
Model library: DiT, VAE, audio, text encoder, conditioning, guidance |
ltx-pipelines-mlx |
Generation pipelines: T2V, I2V, A2V, retake, extend, keyframe, two-stage, distilled, DFR, IC-LoRA, HDR IC-LoRA, LipDub |
ltx-trainer-mlx |
Training: LoRA fine-tuning with flow matching |
- LTX-2 — Lightricks reference (ltx-core + ltx-pipelines + ltx-trainer)
- mlx-forge — weight conversion tool
- Pre-converted weights — HuggingFace collections for LTX-2.3 and LTX-2.5
- MLX — Apple Silicon ML framework
MIT