[WIP][CI][Accuracy] Add HunyuanImage3 pixel accuracy test and nightly CI#3790
Open
BLANKETusers wants to merge 12 commits into
Open
[WIP][CI][Accuracy] Add HunyuanImage3 pixel accuracy test and nightly CI#3790BLANKETusers wants to merge 12 commits into
BLANKETusers wants to merge 12 commits into
Conversation
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
… baseline Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
…eline_image comparison Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
Signed-off-by: zzh <943967662@qq.com>
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
assert_images_pixel_closehelper for full-image pixel-level comparisonwith mean/p99 absolute channel difference metrics and detailed diagnostics
test_hunyuan_image3_pixel_accuracythat generates images via offlineend2end.pyand compares output against a pre-saved baseline imageaccuracy regressions
diffusers_image→baseline_imageacross accuracy helper APIs(
assert_similarity,assert_image_sequence_similarity)Files changed
.buildkite/test-nightly.ymlvllm-omni · HunyuanImage3 · Accuracy Testtests/assets/hunyuan/hunyuan_baseline.pngtests/e2e/accuracy/helpers.pyassert_images_pixel_close; rename paramstests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.pyTest plan
HUNYUAN_IMAGE3_DEPLOY_CONFIG=../hunyuan_image3_dit_copy.yaml pytest -s -v tests/e2e/accuracy/test_hunyuan_image3_pixel_accuracy.py --run-level
Test Result
1 passed, 18 warnings in 101.49s (0:01:41)
Pixel Metrics
tencent/HunyuanImage-3.0-Instruct — (offline vs baseline)
Mismatch ratios (pixel_ratio / channel_ratio)