Skip to content

Support parallel function calls with tool_choice#1503

Open
CISC wants to merge 4 commits into
abetlen:mainfrom
CISC:parallel_tool_choice
Open

Support parallel function calls with tool_choice#1503
CISC wants to merge 4 commits into
abetlen:mainfrom
CISC:parallel_tool_choice

Conversation

@CISC

@CISC CISC commented Jun 2, 2024

Copy link
Copy Markdown
Contributor

Enable the return of parallel function calls when tool_choice is set to a specific function, previously you would only get the following response for get_current_weather and the prompt What's the weather like in Oslo and Stockholm?:

[
    {
        "id": "call__0_get_current_weather_cmpl-...",
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "arguments": "{\"location\": \"Oslo, NO\", \"unit\": \"celsius\"}"
        }
    }
]

Now you will get all the get_current_weather calls:

[
    {
        "id": "call__0_get_current_weather_cmpl-...",
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "arguments": "{\"location\": \"Oslo, NO\", \"unit\": \"celsius\"}"
        }
    },
    {
        "id": "call__1_get_current_weather_cmpl-...",
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "arguments": "{\"location\": \"Stockholm, SE\", \"unit\": \"celsius\"}"
        }
    },
]

Additionally the function call names will now be verified (and filtered if not matching), whereas before there was the possibility of picking up a different function call with similar parameters.

@lsorber

lsorber commented Dec 26, 2024

Copy link
Copy Markdown
Contributor

I opened #1884 with a number of improvements to chatml-function-calling, including streaming tool use and fixing parallel function calling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants