Why is the function calling not working when I use the Olive-quantized Qwen3-0.6B model?

[zhangqiang199226]

olive auto-opt --model_name_or_path D:\VS\hugingface\Qwen\Qwen3-0.6B --trust_remote_code --output_path models/4B --device cpu --provider CPUExecutionProvider --use_ort_genai --precision int4 --log_level 1 --use_model_builder

[samuel100]

Have you created an inference_model.json file? Here are the steps: https://learn.microsoft.com/en-us/azure/foundry-local/how-to/how-to-compile-hugging-face-models?tabs=Bash#step-3-create-chat-template-file

[zhangqiang199226]

{
"Name": "qwen2.5-1.5b-instruct-cpu:1",
"PromptTemplate": {
"system": "\u003C|im_start|\u003Esystem\n{Content}\u003C|im_end|\u003E",
"user": "\u003C|im_start|\u003Euser\n{Content}\u003C|im_end|\u003E",
"assistant": "\u003C|im_start|\u003Eassistant\n{Content}\u003C|im_end|\u003E",
"prompt": "\u003C|im_start|\u003Euser\n{Content}\u003C|im_end|\u003E\n\u003C|im_start|\u003Eassistant"
}
}

[zhangqiang199226]

code:
OpenAIClient openAIClient = new(new ApiKeyCredential("NO_API_KEY"), new OpenAIClientOptions
{
Endpoint = new Uri("http://127.0.0.1:63523/v1")
});
ChatClientAgent agent = openAIClient.GetChatClient("qwen2.5-1.5b-instruct-cpu:1")
.AsAIAgent(
instructions: "You know people and can change colors of the Console ('change_console_color')",
tools:
[
AIFunctionFactory.Create(personTools.GetPersons, "get_persons", "Get all the persons you know"),
AIFunctionFactory.Create(personTools.GetPerson, "get_person", "Get a specific person by name"),
AIFunctionFactory.Create(ChangeConsoleColor, "change_console_color", "Change the color of the console")
]);
result:

Change the color of the console
<tool_call>
{"name": "change_console_color", "arguments": {"color": "red"}}}
</tool_call>

[leestott]

see the example at https://github.com/retkowsky/foundry-local

[samuel100]

This issue has been automatically closed due to extended inactivity. If this is still a problem, please re-open the issue or file a new one with updated details. Thank you for your contributions!

[samuel100]

Re-opening — this issue was closed prematurely as part of a stale-issue sweep. Apologies for the noise.