Skip to content

fix(lite_llm): tolerate malformed JSON in tool call arguments#5897

Open
Ja-Crispy wants to merge 3 commits into
google:mainfrom
Ja-Crispy:fix-litellm-malformed-tool-args-json
Open

fix(lite_llm): tolerate malformed JSON in tool call arguments#5897
Ja-Crispy wants to merge 3 commits into
google:mainfrom
Ja-Crispy:fix-litellm-malformed-tool-args-json

Conversation

@Ja-Crispy
Copy link
Copy Markdown

Link to Issue or Description of Change

Problem:

When the upstream model returns malformed JSON in tool_call.function.arguments (truncated strings, unclosed objects, partial streams committed to the message), _message_to_generate_content_response in google/adk/models/lite_llm.py calls json.loads(tool_call.function.arguments or "{}") without a guard. The resulting json.JSONDecodeError propagates out of LiteLLM response parsing — upstream of any tool or callback boundary — so the entire invocation aborts and the user sees a generic streaming-error message rather than a recoverable tool-call retry.

Solution:

Wrap the json.loads call in a try / except json.JSONDecodeError. On failure, log a warning that includes the tool name, the tool-call id, and the raw arguments (so operators can debug what the model emitted), and pass an empty dict as args. The function-call Part is still constructed with the original tool name and id, so the downstream tool dispatch fires with no arguments — the resulting bad_arguments-style error from the tool layer is a recoverable signal the model can read and retry from.

This matches the expected behaviour described in #5008 ("recoverable model error path and retry/fallback") and aligns with how ADK already tolerates other model-output edge cases.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

Added tests/unittests/models/test_lite_llm_malformed_tool_args.py with four test functions covering both the new tolerant path and the existing happy path:

  • test_unterminated_string_does_not_raise — verifies the exact stack trace from [LiteLlm] JSONDecodeError on malformed tool_call.function.arguments crashes run before callbacks can recover #5008 ({"city":"unterminated) no longer raises, the resulting Part.function_call.args == {}, and a warning is logged with the raw arguments.
  • test_unclosed_object_does_not_raise — additional malformed-JSON shape ({"a": 1, "b":) for breadth.
  • test_well_formed_arguments_still_parse — regression that valid JSON still produces the parsed dict ({"city": "Paris", "units": "metric"}).
  • test_empty_arguments_become_empty_dict — regression that the existing or "{}" fallback for empty strings still works.

Local run:

$ pytest tests/unittests/models/test_lite_llm_malformed_tool_args.py -v
tests/unittests/models/test_lite_llm_malformed_tool_args.py::test_unterminated_string_does_not_raise PASSED
tests/unittests/models/test_lite_llm_malformed_tool_args.py::test_unclosed_object_does_not_raise PASSED
tests/unittests/models/test_lite_llm_malformed_tool_args.py::test_well_formed_arguments_still_parse PASSED
tests/unittests/models/test_lite_llm_malformed_tool_args.py::test_empty_arguments_become_empty_dict PASSED
4 passed in 1.50s

Full models/ suite still green to confirm no regression:

$ pytest tests/unittests/models/ -q
698 passed, 1 skipped, 10 warnings in 7.94s

Manual End-to-End (E2E) Tests:

In our deployment (which uses this fix via a local patch), the same prompt that previously crashed the invocation now produces a tool-response cycle the model recovers from — the user sees the corrected tool result instead of the generic streaming error. Happy to provide additional repro if helpful.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules.

When the upstream model returns malformed JSON in
tool_call.function.arguments (truncated string, unclosed object, ...),
_message_to_generate_content_response raised json.JSONDecodeError and
aborted the invocation before tool callbacks could recover. The crash
surfaced as an unrelated 'Something went wrong' message to the user.

Treat malformed JSON as a recoverable model error: log a warning with
the raw arguments and pass an empty dict so the downstream tool
dispatch can surface a structured error the model can retry from.
@google-cla
Copy link
Copy Markdown

google-cla Bot commented May 29, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@adk-bot adk-bot added the models [Component] Issues related to model support label May 29, 2026
@adk-bot
Copy link
Copy Markdown
Collaborator

adk-bot commented May 29, 2026

Response from ADK Triaging Agent

Hello @Ja-Crispy, thank you for your contribution and for providing such a detailed PR description and comprehensive testing plan!

It looks like the Contributor License Agreement (CLA) check is currently failing. Per our Contribution Guidelines, all contributions must be accompanied by a signed CLA before they can be reviewed and accepted.

Could you please visit https://cla.developers.google.com/ to sign the CLA? This will help the maintainers to review and merge your changes.

Thank you!

@Ja-Crispy
Copy link
Copy Markdown
Author

Ja-Crispy commented May 29, 2026

@adk-bot I signed it!

@rohityan rohityan self-assigned this May 29, 2026
@rohityan rohityan added the needs review [Status] The PR/issue is awaiting review from the maintainer label May 29, 2026
@rohityan
Copy link
Copy Markdown
Collaborator

Hi @Ja-Crispy , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share.

@rohityan rohityan requested a review from wyf7107 May 29, 2026 19:42
@rohityan
Copy link
Copy Markdown
Collaborator

Hi @wyf7107 , can you please review this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

models [Component] Issues related to model support needs review [Status] The PR/issue is awaiting review from the maintainer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[LiteLlm] JSONDecodeError on malformed tool_call.function.arguments still crashes invocation at v2.0.0 (regression of #5008)

4 participants