fix(eval): handle failed inference results without invocations#5878
fix(eval): handle failed inference results without invocations#5878he-yufeng wants to merge 1 commit into
Conversation
|
Hi @he-yufeng , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Please fix formatting errors by running autoformat.sh |
7a33065 to
0315922
Compare
|
Thanks for the review. I rebased onto the latest main, ran pyink on the touched files, and also cleaned two existing lint issues in the touched test file so the changed-file checks are clean. Local verification now passes for py_compile, the targeted eval tests, ruff, pyink --check, and git diff --check. |
0315922 to
daddfd8
Compare
|
Rebased onto current upstream/main and reran the focused eval validation. Validation: test_local_eval_service.py passed (19 passed); ruff check passed; pyink --check passed; py_compile passed; git diff --check passed. I set PYTHONPATH=src so pytest imports this worktree explicitly. |
daddfd8 to
64759da
Compare
|
Hi @ankursharmas , can you please review this. |
What changed
EvalCaseResultwhen inference failed before producing any invocations.InferenceResult(status=FAILURE, inferences=None).This prevents the eval runner from replacing the original inference error with
TypeError: object of type 'NoneType' has no len().Fixes #5876
To verify
.\.venv\Scripts\python.exe -m py_compile src\google\adk\evaluation\local_eval_service.py tests\unittests\evaluation\test_local_eval_service.py.\.venv\Scripts\python.exe -m pytest tests\unittests\evaluation\test_local_eval_service.py -k "failed_without_inferences or evaluate_single_inference_result" -q --basetemp .tmp\pytest.\.venv\Scripts\python.exe -m ruff check src\google\adk\evaluation\local_eval_service.py tests\unittests\evaluation\test_local_eval_service.py.\.venv\Scripts\python.exe -m pyink --check src\google\adk\evaluation\local_eval_service.py tests\unittests\evaluation\test_local_eval_service.pygit diff --check