Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4bcd205
feat(minimax-h3): add model components
lzx1413 Aug 4, 2026
a66780e
feat(minimax-h3): add generation pipeline
lzx1413 Aug 4, 2026
26226f5
test(minimax-h3): add parity validation harness
lzx1413 Aug 4, 2026
aa8fd88
docs(minimax-h3): add usage and acceptance guide
lzx1413 Aug 4, 2026
209ca87
test(minimax-h3): fix pytest module collection
lzx1413 Aug 4, 2026
1a79aa1
feat(minimax-h3): complete runnable examples
lzx1413 Aug 4, 2026
11a0313
docs(minimax-h3): document complete example workflows
lzx1413 Aug 4, 2026
1018b12
fix(minimax-h3): align probed media durations
lzx1413 Aug 4, 2026
22e1d48
refactor(minimax-h3): trim resolved request state
lzx1413 Aug 4, 2026
b9deba1
refactor(minimax-h3): separate modality VAE stages
lzx1413 Aug 4, 2026
35cb291
perf(minimax-h3): reuse denoising loop buffers
lzx1413 Aug 4, 2026
db91e93
refactor(minimax-h3): share validation hashing helpers
lzx1413 Aug 4, 2026
89f9695
refactor(minimax-h3): remove dormant VAE options
lzx1413 Aug 4, 2026
140fd9e
feat(minimax-h3): record stage execution timings
lzx1413 Aug 4, 2026
641e742
fix(minimax-h3): support direct validation scripts
lzx1413 Aug 4, 2026
610a766
feat(minimax-h3): standardize example serve entrypoints
lzx1413 Aug 4, 2026
ef6a231
docs(minimax-h3): document serve workflows
lzx1413 Aug 4, 2026
ed64263
perf(minimax-h3): optimize resident multi-gpu inference
lzx1413 Aug 5, 2026
57c6dca
perf(minimax-h3): bypass parent tensor transfers
lzx1413 Aug 5, 2026
202bb57
docs(minimax-h3): consolidate model documentation
lzx1413 Aug 5, 2026
98cd34a
fix(minimax-h3): protect transported token metadata
lzx1413 Aug 5, 2026
807730b
test(minimax-h3): add four-gpu example regression
lzx1413 Aug 5, 2026
03b3d5a
fix(ops): guard indexed modulation kernels
lzx1413 Aug 5, 2026
81e861f
test(minimax-h3): strengthen regression acceptance
lzx1413 Aug 5, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -162,4 +162,10 @@ telefuser/_version.py
!examples/data/lingbot_world_fast/image.jpg
!examples/data/lingbot_world_fast/poses.npy
!examples/data/lingbot_world_fast/intrinsics.npy
.models
# MiniMax H3 runnable example requests and frozen official reference inputs
!examples/data/minimax-h3/fl2va-reference.png
!examples/data/minimax-h3/ref2va-reference.mp4
!examples/data/minimax-h3/ref2va-voice.mp3
!examples/data/minimax-h3/ref2va.json
!examples/data/minimax-h3/provenance.json
.models
6 changes: 6 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ TeleFuser is a high-performance runtime for world model inference and multimodal

## News 📰

- ✨ **2026-08-05**: Added [**MiniMax H3**](examples/minimax_h3/README.md) T2VA, FL2VA, and Ref2VA joint
audio-video generation with standard `telefuser serve` support. On the matched 768p, five-second, 50-step T2VA
benchmark after warmup, the resident **4 x H100 80 GB** profile matched pinned local SGLang SP2+TP2 performance
while using less GPU memory; see the
[reproducible validation and performance notes](examples/minimax_h3/README.md#measured-four-gpu-profile).
- ✨ **2026-08-03**: Validated LingBot-World v2 target-side real-time generation on **4 x H100 80 GB** at
832x480 and 16 FPS. The current 77-frame gate reached **17.14 steady compute FPS**; see the
[reproducible benchmark](docs/en/benchmark_aiperf.md#current-four-h100-real-time-gate).
Expand Down Expand Up @@ -230,6 +235,7 @@ telefuser/
| `WanVideo` (Wan2.1 / Wan2.2) | T2V, I2V, FL2V | Main video generation family, including async and service examples in [examples/wan_video/README.md](examples/wan_video/README.md) |
| `HunyuanVideo` | T2V, I2V | Supported via [examples/hunyuan_video/README.md](examples/hunyuan_video/README.md) |
| `LTX Video` | I2V + Audio | Unified audio-video generation via [examples/ltx_video/README.md](examples/ltx_video/README.md) |
| `MiniMax H3` | T2VA, FL2VA, Ref2VA + Audio | Local 768p joint audio-video generation via [examples/minimax_h3/README.md](examples/minimax_h3/README.md) |
| `LongCat-Video` | T2V, I2V, VC | Long-form generation and continuation via [examples/longcat_video/README.md](examples/longcat_video/README.md) |
| **NEW** `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | Dense/MoE generation with native CFG/SP and an in-memory base-to-refiner path; see [examples/lingbot_video/README.md](examples/lingbot_video/README.md) |

Expand Down
6 changes: 6 additions & 0 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ TeleFuser 是一个面向世界模型推理与多模态生成的高性能运行

## News 📰

- ✨ **2026-08-05**:新增 [**MiniMax H3**](examples/minimax_h3/README.md) T2VA、FL2VA 和 Ref2VA 联合
音视频生成,并支持标准 `telefuser serve` 服务模式。在相同的 768p、5 秒、50 步 T2VA 请求和一次预热
条件下,常驻 **4 张 H100 80 GB** 配置的性能与固定版本的本地 SGLang SP2+TP2 对齐,同时使用更少
GPU 显存。复现条件和性能分析见
[MiniMax H3 示例文档](examples/minimax_h3/README.md#measured-four-gpu-profile)。
- ✨ **2026-08-03**:LingBot-World v2 已在 **4 张 H100 80 GB** 上通过 832x480、16 FPS 的目标侧实时生成
验证。当前 77 帧门禁达到 **17.14 steady compute FPS**,复现方法见
[基准文档](docs/zh/benchmark_aiperf.md#当前-77-帧实时计算门禁)。
Expand Down Expand Up @@ -220,6 +225,7 @@ telefuser/
| `WanVideo` (Wan2.1 / Wan2.2) | T2V, I2V, FL2V | 主力视频生成家族,含异步和服务示例,见 [examples/wan_video/README.md](examples/wan_video/README.md) |
| `HunyuanVideo` | T2V, I2V | 见 [examples/hunyuan_video/README.md](examples/hunyuan_video/README.md) |
| `LTX Video` | I2V + Audio | 统一音视频生成,见 [examples/ltx_video/README.md](examples/ltx_video/README.md) |
| `MiniMax H3` | T2VA, FL2VA, Ref2VA + Audio | 本地 768p 联合音视频生成,见 [examples/minimax_h3/README.md](examples/minimax_h3/README.md) |
| `LongCat-Video` | T2V, I2V, VC | 长视频生成与续写,见 [examples/longcat_video/README.md](examples/longcat_video/README.md) |
| **NEW** `LingBot-Video` | T2I, T2V, TI2V, MoE refiner | 支持原生 CFG/SP 的 Dense/MoE 生成与内存直传 base-to-refiner,见 [examples/lingbot_video/README.md](examples/lingbot_video/README.md) |

Expand Down
5 changes: 2 additions & 3 deletions docs/en/parallel.md
Original file line number Diff line number Diff line change
Expand Up @@ -461,9 +461,8 @@ pool.
# Device count must equal product of parallel degrees
world_size = dp * cfg * sp_ring * sp_ulysses * pp * tp

# SP and TP cannot be enabled simultaneously
if sp_degree > 1 and tp_degree > 1:
raise ValueError("SP and TP are mutually exclusive")
# SP and TP may be combined as independent mesh dimensions.
# The selected pipeline must implement and validate the requested combination.
```

## Usage Examples
Expand Down
9 changes: 8 additions & 1 deletion docs/en/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,10 @@ If not set, defaults to `"model_zoo"` (relative to working directory).
| prompt | str\|null | null | Override generation prompt |
| input_image_path | str\|null | null | Input image for I2V/edit pipelines |
| input_video_path | str\|null | null | Input video for VSR/continue pipelines |
| target_video_length | float\|null | null | Override generated duration when the example accepts it |
| use_run_with_file | bool | false | Use the standard file entrypoint to preserve container-level output such as audio tracks |
| require_audio | bool | false | Require an audio stream and compare its contract and decoded waveform |
| audio_cosine_min | float | 0.95 | Video audio: minimum decoded-waveform cosine similarity |
| ppl_config_overrides | dict | {} | Override PPL_CONFIG keys |
| psnr_min | float | 25.0 | Video: minimum PSNR vs baseline |
| ssim_min | float | 0.85 | Video: minimum SSIM vs baseline |
Expand Down Expand Up @@ -407,7 +411,8 @@ Example: `20260402_120000_wan_video__wan21_1_3b_text_to_video_h100_1gpu.log`

The runner compares outputs against baselines using:

- **Video**: PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity)
- **Video**: PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity); when `require_audio` is enabled, audio
stream presence, sample rate, channels, duration, and decoded-waveform cosine similarity
- **Image**: Mean pixel difference

#### Metrics Thresholds
Expand All @@ -417,6 +422,8 @@ Configure in YAML or per-pipeline:
```yaml
psnr_min: 25.0 # Higher = stricter
ssim_min: 0.85 # Range [0, 1], higher = stricter
require_audio: true # Require and compare the output audio stream
audio_cosine_min: 0.95 # Higher = stricter
pixel_diff_max: 0.02 # Range [0, 1], lower = stricter
```

Expand Down
5 changes: 2 additions & 3 deletions docs/zh/parallel.md
Original file line number Diff line number Diff line change
Expand Up @@ -456,9 +456,8 @@ stage 包含大量 CPU 计算时才需要提高该值;它不会修改父进程
# 设备数必须等于各并行度乘积
world_size = dp * cfg * sp_ring * sp_ulysses * pp * tp

# SP 和 TP 不能同时启用
if sp_degree > 1 and tp_degree > 1:
raise ValueError("SP and TP are mutually exclusive")
# SP 和 TP 可以作为独立的 mesh 维度组合。
# 所选 pipeline 必须实现并校验请求的组合。
```

## 使用示例
Expand Down
9 changes: 8 additions & 1 deletion docs/zh/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,10 @@ python examples/run_examples.py --all
| prompt | str\|null | null | 覆盖生成提示词 |
| input_image_path | str\|null | null | I2V/编辑 pipeline 的输入图像 |
| input_video_path | str\|null | null | VSR/续写 pipeline 的输入视频 |
| target_video_length | float\|null | null | 示例支持时覆盖生成时长 |
| use_run_with_file | bool | false | 使用标准文件入口,保留音轨等容器级输出 |
| require_audio | bool | false | 要求输出包含音轨,并比较音频契约和解码波形 |
| audio_cosine_min | float | 0.95 | 视频音频:解码波形余弦相似度下限 |
| ppl_config_overrides | dict | {} | 覆盖 PPL_CONFIG 配置 |
| psnr_min | float | 25.0 | 视频:最低 PSNR 阈值 |
| ssim_min | float | 0.85 | 视频:最低 SSIM 阈值 |
Expand Down Expand Up @@ -407,7 +411,8 @@ work_dirs/example_outputs/

Runner 使用以下指标对比 baseline:

- **视频**:PSNR(峰值信噪比)和 SSIM(结构相似度)
- **视频**:PSNR(峰值信噪比)和 SSIM(结构相似度);启用 `require_audio` 时还会校验音轨存在性、采样率、声道数、
时长和解码波形余弦相似度
- **图像**:平均像素差异

#### 指标阈值
Expand All @@ -417,6 +422,8 @@ Runner 使用以下指标对比 baseline:
```yaml
psnr_min: 25.0 # 越高越严格
ssim_min: 0.85 # 范围 [0, 1],越高越严格
require_audio: true # 要求并比较输出音轨
audio_cosine_min: 0.95 # 越高越严格
pixel_diff_max: 0.02 # 范围 [0, 1],越低越严格
```

Expand Down
Binary file added examples/data/minimax-h3/fl2va-reference.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 24 additions & 0 deletions examples/data/minimax-h3/provenance.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"schema_version": 1,
"source": "MiniMax H3 official reproducible request inputs frozen for TeleFuser parity validation",
"files": [
{
"path": "fl2va-reference.png",
"bytes": 1263485,
"sha256": "352e959e08b886406d3766cee25ebe7bbf679e6d53eeffbd1c607b8dc93e2068",
"original_uri": "https://cdn.hailuoai.com/prod/hailuo_demo/testsets/H3_AA_I2VA/gallery/sr_v17_variants_seed42_43_20260724/inputs/4a3a90bf9100_KDmcbkhzYo5sjjxr9FqcVmWVnzb.png"
},
{
"path": "ref2va-reference.mp4",
"bytes": 4396484,
"sha256": "d6edc20ac82aa980c9cbd0b05b371f60d12dc8d018492e571b42de0086819c41",
"original_uri": "https://cdn.hailuoai.com/prod/hailuo_demo/testsets/h3_promo_eval_ref2va/gallery/sr_v2p26_trio_seed42_20260724/inputs/297573323635_00_%E8%A7%86%E9%A2%911_YnyRbxEwio_video_20260525_163755_1927e9d3.mp4"
},
{
"path": "ref2va-voice.mp3",
"bytes": 204432,
"sha256": "2a48d447b343b38a2c504d5281a0c97c688db0ba87931ed9282c598bd6752454",
"original_uri": "https://cdn.hailuoai.com/prod/hailuo_demo/testsets/h3_promo_eval_ref2va/gallery/sr_v2p26_trio_seed42_20260724/inputs/f463d523c5ce_01_%E9%9F%B3%E9%A2%911_RSLcbpzJPo_6%E6%9C%885%E6%97%A5(1).mp3"
}
]
}
Binary file added examples/data/minimax-h3/ref2va-reference.mp4
Binary file not shown.
Binary file added examples/data/minimax-h3/ref2va-voice.mp3
Binary file not shown.
25 changes: 25 additions & 0 deletions examples/data/minimax-h3/ref2va.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"task": "ref2va",
"prompt": "Preserve the subject, composition, motion, and soundtrack from <Video 1>. Use the calm male voice timbre from <Audio 2> for the spoken line: Follow the wind, live free.",
"conditions": [
{
"type": "video",
"uri": "ref2va-reference.mp4",
"role": "reference"
},
{
"type": "audio",
"uri": "ref2va-voice.mp3",
"role": "reference"
}
],
"target": {
"short_edge": 768,
"aspect_ratio": "auto",
"duration_seconds": 5
},
"seed": 0,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"num_inference_steps": 50
}
17 changes: 17 additions & 0 deletions examples/example_config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,23 @@ pipelines:
ppl_config_overrides:
attn_impl: TORCH_SDPA

# ============================================================
# minimax_h3 — Joint Audio-Video Generation
# ============================================================

minimax_h3_t2va_4gpu:
script: minimax_h3/minimax_h3_fl2va_h100.py
gpu_count: 4
output_type: video
timeout_seconds: 1800
seed: 0
resolution: "768p"
target_video_length: 5
use_run_with_file: true
require_audio: true
ppl_config_overrides:
num_inference_steps: 50

# ============================================================
# lingbot_world_fast - Image to Video
# ============================================================
Expand Down
Loading
Loading