b8634
chat : add Granite 4.0 chat template with correct tool_call role mapping (#20804)
- chat : add Granite 4.0 chat template with correct tool_call role mapping
Introduce LLM_CHAT_TEMPLATE_GRANITE_4_0 alongside the existing Granite
3.x template (renamed LLM_CHAT_TEMPLATE_GRANITE_3_X).
The Granite 4.0 Jinja template uses <tool_call> XML tags and maps the
assistant_tool_call role to <|start_of_role|>assistant<|end_of_role|><|tool_call|>.
Without a matching C++ handler, the fallback path emits the literal role
assistant_tool_call which the model does not recognize, breaking tool
calling when --jinja is not used.
Changes:
- Rename
LLM_CHAT_TEMPLATE_GRANITEtoLLM_CHAT_TEMPLATE_GRANITE_3_X(preserves existing 3.x behavior unchanged) - Add
LLM_CHAT_TEMPLATE_GRANITE_4_0enum, map entry, and handler - Detection:
<|start_of_role|>+ (<tool_call>or<tools>) → 4.0, otherwise → 3.x - Add production Granite 4.0 Jinja template
- Add tests for both 3.x and 4.0 template paths (C++ and Jinja)
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
Code review: follow standard format and use common logic in test-chat-template.cpp
Rename custom_conversation variable for extra_conversation to give it a more meaningful name
Co-authored-by: Claude Opus 4.6 noreply@anthropic.com
macOS/iOS:
Linux:
- Ubuntu x64 (CPU)
- Ubuntu arm64 (CPU)
- Ubuntu s390x (CPU)
- Ubuntu x64 (Vulkan)
- Ubuntu arm64 (Vulkan)
- Ubuntu x64 (ROCm 7.2)
- Ubuntu x64 (OpenVINO)
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.1 DLLs
- Windows x64 (Vulkan)
- Windows x64 (SYCL)
- Windows x64 (HIP)
openEuler: