AutoFunctionInvocationFilter - terminating function execution

[deepinderdeol]

Is it possible to terminate function calling once an error is encountered with executing one of the actions specified in a user request?

Example:

• user asks for three actions to be completed in a single request like "Do Action 1, Action 2, Action 3"
• Action 1 is executed fine
• An error occurs during executing Action 2

Here we can call context.Terminate( ) to stop further function execution but this seems to cause issues with chat-history because Assistant message had three tool calls but only one tool message.

Terminating function calling is important because the three actions might have to be executed in that sequence and if Action 2 could not be completed then Action 3 should not be executed. Additionally we want to keep the same chat history.

Here's the error on issuing a new request after calling context.Terminate( ):

An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_3Zh 26yyqUsynQlYFBziYOYLH, call_Gm 4W 0oRWSiPo 9qdutYzt 6F 65
Status: 400 (model_error)
Content:
{
"error": {
"message": "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_3Zh 26yyqUsynQlYFBziYOYLH, call_Gm 4W 0oRWSiPo 9qdutYzt 6F 65",
"type": "invalid_request_error",
"param": "messages.[4].role",
"code": null
}
}

[dmytrostruk]

Hi @deepinderdeol , thanks for your question!

Here we can call context.Terminate( ) to stop further function execution but this seems to cause issues with chat-history because Assistant message had three tool calls but only one tool message.

Yes, if you terminate function execution, but you still want to use the same chat history for further requests, you have to manually update chat history with some response for specific tool_call_id, like in this example:

semantic-kernel/dotnet/samples/Concepts/FunctionCalling/FunctionCalling.cs

Lines 262 to 266 in 7d9f834

	// Invoking the function
	FunctionResultContent resultContent = await functionCall.InvokeAsync(kernel);
	// Adding the function result to the chat history
	chatHistory.Add(resultContent.ToChatMessage());

The other option would be to use FunctionChoiceBehavior.None(). In this case, AI model could suggest which functions to call, so you can invoke them manually, but it won't require a response for each function, which will allow you to terminate the process and still use the same chat history for further requests without modifying it.

semantic-kernel/dotnet/samples/Concepts/FunctionCalling/FunctionCalling.cs

Lines 100 to 112 in 7d9f834

	[Fact]
	public async Task RunPromptWithNoneFunctionChoiceBehaviorAdvertisingAllKernelFunctionsAsync()
	{
	Kernel kernel = CreateKernel();
	OpenAIPromptExecutionSettings settings = new() { FunctionChoiceBehavior = FunctionChoiceBehavior.None() };
	Console.WriteLine(await kernel.InvokePromptAsync("Tell me which provided functions I would need to call to get the color of the sky in Boston on a specified date.", new(settings)));
	// Expected output: "You would first call the `HelperFunctions-GetCurrentUtcDateTime` function to get the current date time in UTC. Then, you would use the `HelperFunctions-GetWeatherForCity` function,
	// passing in the city name as 'Boston' and the retrieved UTC date time. Note, however, that these functions won't directly tell you the color of the sky.
	// The `GetWeatherForCity` function would provide weather data, and you may infer the general sky condition (e.g., clear, cloudy, rainy) based on this data, but it would not specify the color of the sky."
	}

I hope this helps. Thank you!

[deepinderdeol]

Thanks @dmytrostruk