diff --git a/CLAUDE.md b/CLAUDE.md index 868160955..7232a6c5b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -261,15 +261,17 @@ Stdlib-only OpenAI-compatible HTTP. `xtml.py` is a **port** of the release's against that file whenever `K3_DIR` is set; `regions.py` is the streaming parser that reads replies back into reasoning / content / `tool_calls`; `chatfmt.py` is the fallback for a container with no XTML markers, serving -it from the same `chat.json` the CLI reads — plain conversation only, with -tools, thinking and images refused by name rather than dropped; -`kimitools.py` is Kimi's native tool-call protocol — the five markers, the -rendering and the reply reader — which is neither of the two formats and so -gets its own module: it is carried in a container's *tokenizer* while its -`chat.json` says nothing about it, and Kimi-Linear ships those tokens with -no chat template at all. `tests/serve/test_chatfmt_upstream.py` diffs it -against K2's published one, `K2_DIR` naming the release, the way -`test_xtml` does for K3; +it from the same `chat.json` the CLI reads — plain conversation, with +thinking and images from the format and everything else refused by name +rather than dropped; `kimitools.py` and `glmtools.py` are the two native +tool-call protocols a container's *tokenizer* can carry while its +`chat.json` says nothing about them — Kimi K2's five control tokens and +GLM-5.3-Flash's `` XML grammar, each with its rendering and its +reply reader, enabled only when the whole marker set resolves. +`tests/serve/test_chatfmt_upstream.py` diffs the first against K2's +published one and `tests/serve/test_glm_upstream.py` the second against +GLM's, `K2_DIR`/`GLM_DIR` naming the release, the way `test_xtml` does for +K3; `engine.py` is the ctypes binding plus one lock held for a whole generation (a `waste_ctx` is not thread-safe). Struct layouts in `engine.py` mirror `waste.h` field for field — change one, change the other. diff --git a/README.md b/README.md index 1da83a29b..717cff532 100644 --- a/README.md +++ b/README.md @@ -443,17 +443,19 @@ It supports streaming, tools, structured output, thinking controls, and images. A GLM container is served the same way, from its own `chat.json`: plain conversation and images, with the reasoning channel returned as -`reasoning_content` beside `content`. Tools are refused by name rather than -half-rendered — four strings cannot express a tool declaration, and GLM's -tokenizer carries no protocol that could. - -Kimi-Linear's does. Since 0.7.2 a container whose tokenizer holds all five -of Kimi's native tool-call markers gets tool calling over HTTP even though -its `chat.json` describes only the ordinary turns — the format lives in -`serve/kimitools.py`, and the server says which of the three capabilities a -container has when it starts. All five or none: half of that rendering -encodes as ordinary text, so a partial set is a different protocol rather -than a smaller one. +`reasoning_content` beside `content`. Tools work here too: GLM's tokenizer +carries its own tool protocol (``, ``, ``) +as single tokens, so `serve/glmtools.py` renders a request and reads a +reply the way GLM's own `chat_template.jinja` spells them — flat XML, an +`<|observation|>` turn for results. + +Kimi-Linear's is the other one. Since 0.7.2 a container whose tokenizer +holds all five of Kimi's native tool-call markers gets tool calling over +HTTP even though its `chat.json` describes only the ordinary turns — the +format lives in `serve/kimitools.py`, and the server says which of the +three capabilities a container has when it starts. All or none, for either +protocol: half of that rendering encodes as ordinary text, so a partial set +is a different protocol rather than a smaller one. ```bash python3 -m serve ~/models/glm53.waste --port 8000 diff --git a/docs/GLM.md b/docs/GLM.md index 7028782b9..ab616fb22 100644 --- a/docs/GLM.md +++ b/docs/GLM.md @@ -331,10 +331,15 @@ advances `low` to `content_height + 1` rather than to the aligned height. release. A container converted from a release that shares a selection across layers is refused rather than produced. - **MTP.** The extra prediction layer is dropped, as above. -- **Tools.** GLM's template carries a full tool-call protocol; the - declarative `chat.json` cannot express one and refuses by name rather - than half-rendering it. The raw `.jinja` is in the container for a host - that does interpret Jinja. +- **Tools.** GLM's template carries a full tool-call protocol, and the + declarative `chat.json` cannot express one — so the server renders it + from the tokenizer instead. GLM's specials carry the whole XML grammar + (``, ``, `` and the response/observation + markers) as single tokens, `serve/glmtools.py` renders and reads it back + the way the release's own `chat_template.jinja` spells it, and + `tests/serve/test_glm_upstream.py` diffs that rendering against the + template with `GLM_DIR` naming the release. The raw `.jinja` stays in the + container for a host that does interpret Jinja. ## What is checked diff --git a/docs/SERVE.md b/docs/SERVE.md index 098ce591d..223df26d6 100644 --- a/docs/SERVE.md +++ b/docs/SERVE.md @@ -136,35 +136,47 @@ startup the server asks for the richer format first and falls back: ``` chat from ~/models/kimi-linear.waste/chat.json — plain conversation, no reasoning channel, - no images, native tools + no images, kimi tools chat from ~/models/glm53.waste/chat.json — plain conversation, a reasoning channel, - images, no tools + images, glm tools ``` The three capabilities are read from the container, never assumed: the channel and the images from `chat.json`, the tools from whether the -tokenizer carries **all five** of Kimi's native tool-call markers as single -tokens. Kimi-Linear does; GLM does not, and is refused by name. +tokenizer carries a whole native tool protocol as single tokens. There are +two of them: **all five** of Kimi K2's markers, which Kimi-Linear carries, +or **all nine** of GLM's, which GLM-5.3-Flash does — ``, +``, ``, ``, ``, ``, +``, `` and `<|observation|>`. A container +with neither is refused by name. That last one is a rendering `chat.json` itself cannot describe — four prefix/suffix strings say nothing about a tool declaration or an argument -list — so the protocol lives in `serve/kimitools.py`, its own module beside -`xtml.py`, and is enabled only when the whole marker set resolves. +list — so each protocol lives in its own module beside `xtml.py` +(`serve/kimitools.py`, `serve/glmtools.py`), and is enabled only when the +whole marker set resolves. The split is by subject rather than by size. *Whether* a container can do tools is a fact about its `chat.json` and its tokenizer, so `chatfmt.py` decides it and refuses with `ChatFormatError`. *How* a tool call is spelled -is a fact about the protocol, so `kimitools.py` owns it and a malformed one -raises `KimiToolError` — the same shape `xtml.py` has with `XTMLError`, and -`api.py` maps each to a 400. Nothing in `kimitools.py` imports `chatfmt`, -which is what lets `chatfmt` import it. **It is Kimi K2's**, and it is checked -against K2's own published `chat_template.jinja` rather than transcribed -from memory: `tests/serve/test_chatfmt_upstream.py`, which `tests/run.sh` -runs whenever `K2_DIR` names a release directory, the same discipline -`test_xtml.TestAgainstUpstream` applies to K3 with `K3_DIR`. Kimi-Linear's -own release carries the five tokens and **no chat template at all**, which -is why the grammar has to come from K2 and why an oracle for it matters -more than usual. +is a fact about the protocol, so `kimitools.py` or `glmtools.py` owns it and +a malformed one raises `KimiToolError` or `GlmToolError` — the same shape +`xtml.py` has with `XTMLError`, and `api.py` maps each to a 400. Nothing in +either imports `chatfmt`, which is what lets `chatfmt` import them. Each is +**the release's own grammar**, checked against the template that defines it +rather than transcribed from memory: +`tests/serve/test_chatfmt_upstream.py`, which `tests/run.sh` runs whenever +`K2_DIR` names a release directory, and `tests/serve/test_glm_upstream.py` +for `GLM_DIR` — the same discipline `test_xtml.TestAgainstUpstream` applies +to K3 with `K3_DIR`. Kimi-Linear's own release carries the five tokens and +**no chat template at all**, which is why the grammar has to come from K2 +and why an oracle for it matters more than usual; GLM's release ships its +template, and the two grammars differ enough that each gets its own module +and its own reader — a Kimi call is `ID<|tool_call_argument_begin|>ARGS` in +a section, a GLM call is flat XML with the name after the opening tag and +one ``/`` pair per argument, and a GLM result is an +`<|observation|>` turn wrapping `` blocks where a Kimi result +is a system turn named for the tool. One difference from that template is deliberate and asserted rather than fixed: with no system turn first, K2's template inserts Moonshot's own diff --git a/serve/__main__.py b/serve/__main__.py index 9a685d5ee..4b90ddf4c 100644 --- a/serve/__main__.py +++ b/serve/__main__.py @@ -249,7 +249,8 @@ def main(argv=None) -> int: think = ("a reasoning channel" if srv.chat_format.think else "no reasoning channel") images = "images" if srv.chat_format.image else "no images" - tools = "native tools" if srv.chat_format.tool_markers else "no tools" + protocol = srv.chat_format.tool_protocol + tools = f"{protocol} tools" if protocol else "no tools" print(f"chat from {model}/chat.json — plain conversation, " f"{think},\n {images}, {tools}") diff --git a/serve/api.py b/serve/api.py index 382e89326..74af9fb90 100644 --- a/serve/api.py +++ b/serve/api.py @@ -27,7 +27,7 @@ from pathlib import Path from typing import Any, Optional -from . import chatfmt, kimitools, xtml +from . import chatfmt, glmtools, kimitools, xtml from .engine import Engine from .regions import RegionParser @@ -385,6 +385,8 @@ def build_prompt(engine: Engine, body: dict, *, default_thinking: bool, raise APIError(str(e), param="messages") except kimitools.KimiToolError as e: raise APIError(str(e), param=e.param or "messages") + except glmtools.GlmToolError as e: + raise APIError(str(e), param=e.param or "messages") tokens = engine.tokenize_segments(segments) if n_images: diff --git a/serve/chatfmt.py b/serve/chatfmt.py index 855ad6f8e..58ecd1aab 100644 --- a/serve/chatfmt.py +++ b/serve/chatfmt.py @@ -18,10 +18,13 @@ - **Plain conversation, streaming included.** system / user / assistant turns, and a stop that comes from the format rather than from a guess. -- **No tools.** Four strings cannot express a tool declaration, an argument - list, or a result turn. K3's encoder needs 647 lines for that, and the - markup Kimi-Linear's tokenizer carries for it is not transcribed anywhere - in this repo. Refused, by name, rather than half-rendered. +- **Tools, when the tokenizer carries a protocol for them.** Four strings + cannot express a tool declaration, an argument list, or a result turn, so + neither Kimi K2's five control tokens nor GLM's `` XML grammar + live in chat.json. They live in the tokenizer, `kimitools` and `glmtools` + render whichever one a container carries, and a container whose + vocabulary has neither refuses a `tools` request by name rather than + half-rendering it. - **A reasoning channel, when the format names one.** `chat.json` may carry `think: ["", ""]`, and GLM-5.3-Flash's does: its generation prompt opens the channel and the model closes it before the @@ -57,7 +60,7 @@ from dataclasses import dataclass, field from typing import Any, Optional -from . import kimitools +from . import glmtools, kimitools from .regions import Delta, ToolCall from .xtml import Segment @@ -114,6 +117,12 @@ class ChatFormat: effort: str = "" image: str = "" tool_markers: dict[int, str] = field(default_factory=dict) + # Which native tool protocol `tool_markers` resolved to: "kimi" for + # Kimi K2's five control tokens, "glm" for GLM's `` XML + # grammar, or "" when the tokenizer carries neither. The renderer and + # the reply reader branch on this, because the two grammars differ in + # what a call, a declaration and a result turn look like. + tool_protocol: str = "" @property def markers(self) -> dict[int, str]: @@ -231,16 +240,26 @@ def load(cls, engine: Any) -> "ChatFormat": f"chat format disagree") ids[text] = got[0] - # A container may carry Kimi's native tool protocol in reserved + # A container may carry a native tool protocol in reserved # tokenizer tokens even though chat.json describes only the ordinary # turns. Whether it does is this file's question; what the protocol - # is belongs to kimitools. + # is belongs to kimitools or glmtools. Kimi first keeps the existing + # behaviour, and the two grammars are disjoint so a real container + # resolves at most one. + protocol = "" discovered = kimitools.detect(engine) + if discovered: + protocol = "kimi" + else: + discovered = glmtools.detect(engine) + if discovered: + protocol = "glm" return cls(roles=roles, opening=opening, stop_marker=stop_marker, stop_id=ids[stop_marker], prelude=prelude, think=think, think_close_id=ids[think[1]] if think else -1, - effort=effort, image=image, tool_markers=discovered) + effort=effort, image=image, tool_markers=discovered, + tool_protocol=protocol) # ---- rendering ------------------------------------------------------ @@ -283,11 +302,15 @@ def build_chat_segments(self, messages: list[Any], raise ChatFormatError( "this container is served from its chat.json, which " "cannot express tool definitions because its tokenizer " - "does not carry the Kimi K2 native tool markers", + "carries neither the Kimi K2 native tool markers nor " + "GLM's XML protocol", param="tools", ) - segments.extend(kimitools.declaration(tools)) + if self.tool_protocol == "glm": + segments.extend(glmtools.declaration(tools)) + else: + segments.extend(kimitools.declaration(tools)) for name in ("tool_choice", "response_format", "response_schema"): if kwargs.get(name) is not None: raise ChatFormatError( @@ -332,8 +355,24 @@ def build_chat_segments(self, messages: list[Any], role = _ROLE_ALIASES.get(role, role) # Kimi K2 represents a tool result as a system-style turn whose - # content begins with "## Return of ". + # content begins with "## Return of "; GLM opens an + # <|observation|> turn and wraps each result in + # …. Both are the authored side of + # the protocol, so which one lives in this file as turn framing + # and which lives in the tool module is the same split as with + # the role prefixes — the protocol only decides the body. if role == "tool": + if self.tool_protocol == "glm": + # The template groups consecutive tool results under a + # single <|observation|>, one block per result. Look back + # so a run of results shares an opener, exactly as GLM's + # template does (`loop.first or the last role != "tool"`). + if i == 0 or messages[i - 1].get("role") != "tool": + segments.append(Segment(glmtools.OBSERVATION, + markup=True)) + segments.extend(glmtools.tool_response( + _content_segments(message.get("content"), i))) + continue pair = self.roles.get("system") if pair is None: raise ChatFormatError( @@ -363,7 +402,10 @@ def build_chat_segments(self, messages: list[Any], f"messages[{i}] carries tool_calls on a non-assistant " "turn", param=f"messages[{i}].tool_calls") - segments.extend(kimitools.call_section(tool_calls, i)) + if self.tool_protocol == "glm": + segments.extend(glmtools.call_section(tool_calls, i)) + else: + segments.extend(kimitools.call_section(tool_calls, i)) segments.append(Segment(suffix, markup=True)) @@ -418,14 +460,20 @@ def _content_segments(content: Any, index: int, images: Any = None) -> list[Segm class PlainParser: - """Read a chat.json reply, including Kimi K2 native tool calls. + """Read a chat.json reply, including native tool calls. + + The reply is read back whichever tool protocol the container carries — + Kimi K2's five control tokens or GLM's `` XML grammar — by + whichever `tool_parser` the caller hands in (Kimi by default, for the + containers that were here first). Structure is recognized only from tokenizer marker ids. Marker-looking text carried by an ordinary token remains ordinary model content. """ def __init__(self, *, markers: Optional[dict[int, str]] = None, - think_close_id: int = -1, in_think: bool = False): + think_close_id: int = -1, in_think: bool = False, + tool_parser: Any = None): self._markers = dict(markers or {}) # The channel, when the format has one. `in_think` says the # generation prompt left it open — which for GLM it always does — @@ -440,7 +488,8 @@ def __init__(self, *, markers: Optional[dict[int, str]] = None, # The tool protocol reads itself; this file decides only what is # left over. `tool_calls` stays an attribute here because it is what # openai_message reports. - self._tools = kimitools.ToolParser() + self._tools = tool_parser if tool_parser is not None \ + else kimitools.ToolParser() @property def finished(self) -> bool: diff --git a/serve/glmtools.py b/serve/glmtools.py new file mode 100644 index 000000000..2c6b191fd --- /dev/null +++ b/serve/glmtools.py @@ -0,0 +1,292 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# Copyright 2026 SQLite Cloud, Inc. +"""GLM's native tool-call protocol: the XML markers, the rendering, the reader. + +`kimitools.py` holds the tool protocol a Kimi container carries in its +tokenizer. GLM-5.3-Flash is the other direction the engine ships: its +tokenizer has no XTML markers and none of Kimi's five tool tokens, but its +`specials.json` does carry an XML tool protocol of its own — ``, +``, `` — that lives in `chat_template.jinja` and that the +declarative `chat.json` says nothing about. Four prefix/suffix strings cannot +express a tool declaration, an argument list, or a result turn, so those +turns come from here, exactly as the tool turns of a Kimi container come +from `kimitools`. + +The split by subject is the same one `kimitools` makes: + +- **Whether a container can do tools at all** is decided by `chatfmt.py` + and this module's `detect()`, which asks whether *every* GLM marker is a + single token in the vocabulary. GLM's are; a container that half-carries + them carries a different protocol, and half of one renders as prose. +- **How a tool call is spelled** — the grammar — lives here, and a request + this protocol cannot spell raises `GlmToolError`, mapped to a 400 by + `api.py` the same way `KimiToolError` is. + +The grammar is not Kimi's, and the differences are exactly what the name +exists to keep honest: + +- **A call is flat XML, not a section.** Kimi nests calls inside + `<|tool_calls_section_begin|>...<|tool_calls_section_end|>` and spells the + function's name inside the id. GLM writes + `NAMEKV...` + with the name following the opening tag directly and no id at all. The + arguments are key/value pairs, not one JSON block, so the reply reader + returns a `ToolCall` whose `arguments` dict carries one entry per pair. +- **A result is an `<|observation|>` turn wrapping one or more** + **`…content…` blocks.** Kimi names the turn + for the tool and prefixes it `## Return of `; GLM opens an + `<|observation|>` turn and repeats the response block per result. The + template groups consecutive tool results under a single `<|observation|>`; + `chatfmt` knows which message starts a run and emits the opener for it, + and this module renders the individual blocks that make it up. +- **The declaration is an instruction turn, not a token literal.** GLM's + `<# Tools #>` turn is system text with the signatures as JSON inside + `…`; only the `<|system|>` opener is a control token. The + signatures are caller content, so they go out as a plain segment, in the + same way the Kimi declaration ships its JSON payload plain. + +The module imports nothing from `chatfmt`, so `chatfmt` can import it — the +dependency stays one-way, as it does for `kimitools`. +""" + +import json +from typing import Any, Optional + +from .regions import Delta, ToolCall +from .xtml import Segment + +# In protocol order. All nine or none: a container carrying only some of +# them carries a different protocol, and half of one renders as prose. The +# first six are the reply grammar a model emits; the last three are what a +# `tool` result turn is *made of*, so they must be single tokens too or a +# rendered result would show the model its own turn structure as text. +MARKERS = ( + "", + "", + "", + "", + "", + "", + "", + "", + "<|observation|>", +) + +(_TOOL_CALL, _TOOL_CALL_END, + _ARG_KEY, _ARG_KEY_END, + _ARG_VALUE, _ARG_VALUE_END, + _TOOL_RESPONSE, _TOOL_RESPONSE_END, + OBSERVATION) = MARKERS + + +class GlmToolError(ValueError): + """A tool call this protocol cannot spell. + + `param` names the request field at fault, so the 400 points at + `messages[3].tool_calls[0]` rather than at `messages`. + """ + + def __init__(self, message: str, *, param: Optional[str] = None): + super().__init__(message) + self.param = param + + +# The `<# Tools #>` system text that follows the `<|system|>` opener. Only +# the opener is markup; `` and the XML example are the template's own +# instruction prose, sent as plain text the way the template's writer wrote +# it — and those tags (``, `{...}`) are not control tokens, so sending +# them as markup would not resolve and would show as junk. +_DECL_HEADER = ( + "# Tools\n\n" + "You may call one or more functions to assist with the user query.\n\n" + "You are provided with function signatures within XML " + "tags:\n" + "" +) +_DECL_FOOTER = ( + "\n\n" + "For each function call, output the function name and arguments within " + "the following XML format:\n" + "{function-name}{arg-key-1}" + "{arg-value-1}{arg-key-2}" + "{arg-value-2}..." +) + + +def detect(engine: Any) -> dict[int, str]: + """{token id: marker} if this container carries the whole protocol. + + Defensive by construction, like `kimitools.detect`: a marker that is not + a *single* token in this vocabulary is not markup here, it is text, and + rendering it would show the model its own structure as prose. One missing + marker disqualifies the set rather than degrading it. + """ + found: list[tuple[int, str]] = [] + for text in MARKERS: + got = engine.tokenize(text, markup=True) + if len(got) != 1: + return {} + found.append((got[0], text)) + return dict(found) + + +def declaration(tools: list[dict]) -> list[Segment]: + """The `<|system|>` turn that declares the tools as JSON. + + The signatures are the caller's, so each goes out as a plain segment — + the same boundary `chatfmt` keeps for message content. The instruction + prose around them is the template's, hardcoded here the way the Kimi + declaration hardcodes its `tool_declare` opener. + """ + out = [Segment("<|system|>", markup=True), Segment(_DECL_HEADER)] + for tool in tools: + fn = tool.get("function", tool) if isinstance(tool, dict) else tool + out.append(Segment("\n" + json.dumps(fn, ensure_ascii=False) + "\n")) + out.append(Segment(_DECL_FOOTER)) + return out + + +def call_section(tool_calls: list[dict], index: int) -> list[Segment]: + """The flat `…` list of one assistant turn. + + GLM nests calls in no section and attaches no id, so each call is the + name, then one ``/`` pair per argument, directly. + The ``, `` and `` markers are markup; + the name and the argument values are the caller's and go out plain. + `arguments` may be a JSON string — the OpenAI wire shape — or a dict. + """ + out: list[Segment] = [] + for j, call in enumerate(tool_calls): + function = call.get("function") or {} + name = function.get("name") + if not name: + raise GlmToolError( + f"messages[{index}].tool_calls[{j}] has no function name", + param=f"messages[{index}].tool_calls[{j}].function.name") + arguments = function.get("arguments", {}) + if isinstance(arguments, str): + try: + arguments = json.loads(arguments) + except ValueError: + raise GlmToolError( + f"messages[{index}].tool_calls[{j}] carries JSON " + f"arguments this protocol cannot split into " + f"/ pairs", + param=f"messages[{index}].tool_calls[{j}]" + f".function.arguments") from None + if not isinstance(arguments, dict): + raise GlmToolError( + f"messages[{index}].tool_calls[{j}] arguments must be an " + "object", + param=f"messages[{index}].tool_calls[{j}].function.arguments") + out.append(Segment(_TOOL_CALL, markup=True)) + out.append(Segment(name)) + for key, value in arguments.items(): + out.append(Segment(_ARG_KEY, markup=True)) + out.append(Segment(str(key))) + out.append(Segment(_ARG_KEY_END, markup=True)) + out.append(Segment(_ARG_VALUE, markup=True)) + out.append(Segment(value if isinstance(value, str) + else json.dumps(value, ensure_ascii=False))) + out.append(Segment(_ARG_VALUE_END, markup=True)) + out.append(Segment(_TOOL_CALL_END, markup=True)) + return out + + +def tool_response(content_segments: list[Segment]) -> list[Segment]: + """One `…content…` block. + + `content_segments` is the already-rendered result content — this file + decides only the block around it, and `chatfmt` decides whether a run of + results opens with `<|observation|>`. + """ + return [Segment(_TOOL_RESPONSE, markup=True), + *content_segments, + Segment(_TOOL_RESPONSE_END, markup=True)] + + +class ToolParser: + """The reply side: which markers mean what, and where text goes. + + Owned by the reply reader rather than mixed into it — `feed_marker` and + `feed_text` each answer "did I consume this", so the reader keeps its own + rules for everything else, the reasoning channel included. A reply the + model writes for this grammar looks like: + + get_weathercityRome + ... + + Read back as one `ToolCall` per ``, its `name` from the text + that follows the opening tag and its `arguments` dict from the pairs. + There is no id in the grammar, so `to_openai()` names each call by + position, exactly as it does for a Kimi call the model failed to number. + """ + + def __init__(self) -> None: + self.calls: list[ToolCall] = [] + self._state = "content" + self._key = "" + self._value = "" + self._current: Optional[ToolCall] = None + + @property + def in_structure(self) -> bool: + """Inside a call, where ordinary text is not the reply.""" + return self._state != "content" + + def feed_marker(self, marker: str, delta: Delta) -> bool: + if marker == _TOOL_CALL: + self._current = ToolCall(name="", index=len(self.calls)) + self.calls.append(self._current) + self._state = "name" + elif marker == _ARG_KEY: + self._key = "" + self._state = "key" + if self._current is not None: + # The name is complete once the first argument begins; + # this is what announces the call to a streaming client. + delta.tool_calls.append(self._current.index) + elif marker == _ARG_KEY_END: + self._state = "between" + elif marker == _ARG_VALUE: + self._value = "" + self._state = "value" + elif marker == _ARG_VALUE_END: + self._finish_arg() + self._state = "between" + elif marker == _TOOL_CALL_END: + self._finish_call() + self._state = "content" + else: + return False + return True + + def feed_text(self, piece: str, delta: Delta) -> bool: + if self._state == "name" and self._current is not None: + self._current.name += piece + elif self._state == "key": + self._key += piece + elif self._state == "value": + self._value += piece + if self._current is not None: + delta.tool_calls.append(self._current.index) + else: + return False + return True + + def finish(self) -> None: + """Flush a call the stream ended in the middle of, and its last + argument's value.""" + if self._current is not None: + self._finish_arg() + + def _finish_arg(self) -> None: + if self._current is not None and self._key: + self._current.arguments[self._key] = self._value + self._key = "" + self._value = "" + + def _finish_call(self) -> None: + self._finish_arg() + self._current = None \ No newline at end of file diff --git a/serve/server.py b/serve/server.py index 9e1f22119..8d92db999 100644 --- a/serve/server.py +++ b/serve/server.py @@ -40,7 +40,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from typing import Optional -from . import api, xtml +from . import api, glmtools, xtml from .chatfmt import ChatFormat, ChatFormatError, PlainParser from .engine import Cancelled, Engine, EngineError from .regions import RegionParser @@ -141,10 +141,17 @@ def new_parser(self, thinking: bool): return RegionParser(in_think=thinking, in_response=not thinking, markers=self.markers) fmt = self.chat_format + # Which tool protocol the reply reader should own: a GLM container + # speaks its own `` grammar, anything else that reaches + # a PlainParser speaks Kimi K2's five control tokens. + tool_parser = None + if getattr(fmt, "tool_protocol", "") == "glm": + tool_parser = glmtools.ToolParser() return PlainParser(markers=self.markers, think_close_id=getattr(fmt, "think_close_id", -1), in_think=thinking and getattr(fmt, "think", None) - is not None) + is not None, + tool_parser=tool_parser) def handle_error(self, request, client_address): """A client hanging up is not an error worth a traceback. diff --git a/tests/run.sh b/tests/run.sh index 3a25fe782..c7dde9ebe 100755 --- a/tests/run.sh +++ b/tests/run.sh @@ -2165,5 +2165,27 @@ else fi fi +# And for the third: GLM's release ships its chat_template, and GLM's tool +# grammar (``, ``, ``) is that template's, not +# anything this repo owns. Same rule — diff the rendering against it. Needs +# the loopcontrols extension too, because the template uses `{% break %}`. +GLM_SRC="${GLM_DIR:-$HOME/models/glm53.waste}" +if [ ! -f "$GLM_SRC/chat_template.jinja" ] && [ ! -f "$GLM_SRC/tokenizer_config.json" ]; then + sk "chat.json tools vs GLM's chat_template" \ + "no template at $GLM_SRC (set GLM_DIR; only chat_template.jinja is needed)" +elif [ -n "$PY_MISS" ]; then + sk "chat.json tools vs GLM's chat_template" "$PY_MISS" +elif ! command -v uv >/dev/null 2>&1; then + sk "chat.json tools vs GLM's chat_template" "uv not installed (needs jinja2)" +else + if GLM_DIR="$GLM_SRC" run_uv run --no-project --with jinja2 \ + python -m unittest tests.serve.test_glm_upstream 2>&1 \ + | tail -3 | grep -q "^OK"; then + ok "chat.json tool rendering matches GLM's own chat_template" + else + no "chat.json tool rendering differs from GLM's chat_template" + fi +fi + printf "\n\033[1m%d passed, %d failed, %d skipped\033[0m\n" "$pass" "$fail" "$skip" [ "$fail" -eq 0 ] diff --git a/tests/serve/test_glm_upstream.py b/tests/serve/test_glm_upstream.py new file mode 100644 index 000000000..0ff6b89cb --- /dev/null +++ b/tests/serve/test_glm_upstream.py @@ -0,0 +1,160 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +# Copyright 2026 SQLite Cloud, Inc. +"""serve/glmtools.py's tool protocol against the GLM template that defines it. + +`test_chatfmt_upstream` diffs `kimitools.py`'s rendering against Kimi K2's +published chat_template. This is the same check for the GLM half, and it +exists for the same reason: chat.json cannot express a tool declaration, an +argument list, or a result turn, so the ground truth for how GLM spells those +lives in the model's own `chat_template.jinja`, not anywhere this repo owns. + +The assertion is narrower than the Kimi one because there is no single +surface to diff: the template's tool declaration is prose with embedded JSON, +and its tool/tool-response turns carry GLM's own whitespace around them, +which the declarative chat.json path spreads across segments. What is +compared is the XML the grammar is named for — the `…` +list and the `<|observation|>…` block — each +extracted from a full template render and compared exactly against what this +repo renders. + + GLM_DIR=/path/to/glm-5.3-flash python3 -m unittest \\ + tests.serve.test_glm_upstream -t . + +Only a `chat_template.jinja` is needed; no weights. Skips if the template is +not on disk. +""" + +import json +import os +import shutil +import sys +import tempfile +import unittest +from pathlib import Path + +REPO = Path(__file__).resolve().parents[2] +sys.path.insert(0, str(REPO)) + +from serve import glmtools # noqa: E402 +from serve.chatfmt import ChatFormat # noqa: E402 +from tests.serve.fake_engine import FakeEngine # noqa: E402 +from tests.serve.test_glmtools import GLM_MARKERS, TOOLS # noqa: E402 + +GLM_DIR = os.environ.get("GLM_DIR", + os.path.join(os.path.expanduser("~"), + "models", "glm53.waste")) +GLM_CHAT = REPO / "examples" / "chat-glm53.json" + +TOOL_CALL_ARGS = {"city": "Rome", "days": 3} + + +def load_template(): + """GLM's own chat template, or None if it is not on disk.""" + jinja = Path(GLM_DIR) / "chat_template.jinja" + if jinja.exists(): + return jinja.read_text(encoding="utf-8") + cfg = Path(GLM_DIR) / "tokenizer_config.json" + if cfg.exists(): + return json.loads(cfg.read_text(encoding="utf-8")).get("chat_template") + return None + + +def render_upstream(template, messages, tools): + from jinja2 import Environment + + def tojson(x, ensure_ascii=False, indent=None, separators=None, + sort_keys=False): + return json.dumps(x, ensure_ascii=ensure_ascii, indent=indent, + separators=separators, sort_keys=sort_keys) + + env = Environment(extensions=["jinja2.ext.loopcontrols"]) + env.filters["tojson"] = tojson + return env.from_string(template).render( + messages=messages, tools=tools, add_generation_prompt=True) + + +class TestAgainstGlmTemplate(unittest.TestCase): + @classmethod + def setUpClass(cls): + try: + import jinja2 # noqa: F401 + except ImportError: + raise unittest.SkipTest( + "jinja2 not installed; the template cannot be rendered") + cls.template = load_template() + if not cls.template: + raise unittest.SkipTest( + f"no chat_template.jinja at {GLM_DIR} (set GLM_DIR to a " + "GLM release directory)") + cls._tmp = tempfile.mkdtemp() + shutil.copyfile(GLM_CHAT, os.path.join(cls._tmp, "chat.json")) + eng = FakeEngine(no_markers=True, model_path=cls._tmp, + markers=dict(GLM_MARKERS)) + cls.fmt = ChatFormat.load(eng) + if cls.fmt.tool_protocol != "glm": + raise unittest.SkipTest( + "the GLM tool markers did not resolve; there is nothing " + "to compare against the template") + + @classmethod + def tearDownClass(cls): + shutil.rmtree(getattr(cls, "_tmp", ""), ignore_errors=True) + + def render_ours(self, messages, tools=None): + segs = self.fmt.build_chat_segments(messages, tools=tools, + thinking=True) + return "".join(s.text for s in segs) + + def test_tool_call_matches_the_template(self): + """Our flat `…` is byte-for-byte the call the + template writes, name and arguments in the same order.""" + import re + msgs = [{"role": "assistant", "content": "", + "tool_calls": [{ + "id": "call_1", "type": "function", + "function": {"name": "get_weather", + "arguments": dict(TOOL_CALL_ARGS)}}]}, + {"role": "tool", "tool_call_id": "call_1", + "content": "18C"}] + up = render_upstream(self.template, msgs, TOOLS) + upstream_calls = re.findall(r".*?", up, re.S) + # Drop the template's format example; keep the model's actual call. + real = [c for c in upstream_calls + if "get_weather" in c] + self.assertEqual(len(real), 1) + ours = "".join(s.text for s in + glmtools.call_section(msgs[0]["tool_calls"], 0)) + self.assertEqual(ours, real[0]) + + def test_tool_result_matches_the_template(self): + """`<|observation|>…` matches the + template's authored result turn.""" + msgs = [{"role": "assistant", "content": "", + "tool_calls": [{ + "id": "call_1", "type": "function", + "function": {"name": "get_weather", + "arguments": dict(TOOL_CALL_ARGS)}}]}, + {"role": "tool", "tool_call_id": "call_1", + "content": "18C"}, + {"role": "user", "content": "thanks"}] + up = render_upstream(self.template, msgs, TOOLS) + i = up.rfind("<|observation|>") + upstream_result = up[i:up.find("<|user|>", i)] + ours = self.render_ours(msgs, tools=TOOLS) + oi = ours.rfind("<|observation|>") + our_result = ours[oi:ours.find("<|user|>", oi)] + self.assertEqual(our_result, upstream_result) + + def test_declaration_embeds_the_same_signatures(self): + """Every signature we render also appears in the template's + `…` declaration block.""" + msgs = [{"role": "user", "content": "hi"}] + up = render_upstream(self.template, msgs, TOOLS) + ours = self.render_ours(msgs, tools=TOOLS) + self.assertIn('"name": "get_weather"', up) + self.assertIn('"name": "get_weather"', ours) + + +if __name__ == "__main__": + unittest.main() \ No newline at end of file diff --git a/tests/serve/test_glmtools.py b/tests/serve/test_glmtools.py new file mode 100644 index 000000000..7250f25be --- /dev/null +++ b/tests/serve/test_glmtools.py @@ -0,0 +1,323 @@ +# SPDX-License-Identifier: Apache-2.0 +# Copyright 2026 SQLite Cloud, Inc. +""" +test_glmtools.py — GLM-5.3-Flash's native `` tool protocol. + +chatfmt.py renders a container's chat.json. tools are the one thing four +prefix/suffix strings cannot express, so a container whose tokenizer carries +a native tool protocol gets its tool turns from `kimitools` (Kimi K2) or +`glmtools` (GLM's `` XML). This file checks the GLM half: that +the markers are discovered, that a request is rendered the way GLM's own +chat_template.jinja renders it, and that a reply the model writes is read +back into OpenAI tool_calls. + +The templates under test are the ones examples/ actually ships — GLM's own +chat-glm53.json — with the fake tokenizer carrying GLM's tool markers, the +same split test_chatfmt.py makes: the machinery is under test, not the +vocabulary, and the real specials are exercised by the converter. + + python3 tests/serve/test_glmtools.py +""" + +import io +import json +import os +import shutil +import sys +import tempfile +import unittest +from pathlib import Path + +REPO = Path(__file__).resolve().parents[2] +sys.path.insert(0, str(REPO)) + +from serve import glmtools # noqa: E402 +from serve.chatfmt import (ChatFormat, ChatFormatError, # noqa: E402 + PlainParser) +from tests.serve.fake_engine import FakeEngine # noqa: E402 + +SHIPPED = REPO / "examples" / "chat-glm53.json" + +# GLM-5.3-Flash's own specials, given the ids the fake tokenizer reserves. +# Markers 11-18 are the role/reasoning/image block chat.json names; 21-29 are +# the tool protocol `glmtools` recognizes, all single tokens here. +GLM_MARKERS = { + 11: "<|system|>", 12: "<|user|>", 13: "<|assistant|>", + 14: "", 15: "", + 16: "<|begin_of_image|>", 17: "<|image|>", 18: "<|end_of_image|>", + 21: "", 22: "", + 23: "", 24: "", + 25: "", 26: "", + 27: "", 28: "", + 29: "<|observation|>", +} + +TOOLS = [{"type": "function", "function": { + "name": "get_weather", + "description": "Current weather for a city", + "parameters": {"type": "object", + "properties": {"city": {"type": "string"}}, + "required": ["city"]}}}] + +CALL = {"role": "assistant", "content": "", "tool_calls": [{ + "id": "call_1", "type": "function", + "function": {"name": "get_weather", + "arguments": '{"city":"Rome","days":3}'}}]} + + +class Base(unittest.TestCase): + """A container directory holding GLM's own chat.json under that name.""" + + def setUp(self): + self.dir = tempfile.mkdtemp(prefix="glmtools-") + self.addCleanup(shutil.rmtree, self.dir, True) + + def engine(self, chat_json=SHIPPED, *, markers=None): + dst = os.path.join(self.dir, "chat.json") + if isinstance(chat_json, Path): + shutil.copyfile(chat_json, dst) + else: + text = (json.dumps(chat_json) if isinstance(chat_json, dict) + else chat_json) + with io.open(dst, "w", encoding="utf-8") as f: + f.write(text) + return FakeEngine(no_markers=True, model_path=self.dir, + markers=dict(markers if markers is not None + else GLM_MARKERS)) + + def load(self, chat_json=SHIPPED, **kw): + return ChatFormat.load(self.engine(chat_json, **kw)) + + +class TestLoad(Base): + def test_the_shipped_glm_chat_json_loads(self): + fmt = self.load() + self.assertEqual(fmt.stop_marker, "<|user|>") + self.assertEqual(fmt.stop_id, 12) + self.assertEqual(fmt.think_close_id, 15) + self.assertEqual(sorted(fmt.roles), ["assistant", "system", "user"]) + + def test_glm_tool_protocol_is_discovered(self): + fmt = self.load() + self.assertEqual(fmt.tool_protocol, "glm") + self.assertEqual(len(fmt.tool_markers), len(glmtools.MARKERS)) + # The observation opener and the response tags are part of the + # protocol, so they are discovered and reach the reply reader's + # marker map just like the call grammar. + self.assertEqual(fmt.markers[29], "<|observation|>") + self.assertIn("", set(fmt.tool_markers.values())) + + def test_a_partial_glm_marker_set_is_no_protocol(self): + """Half of this XML renders as ordinary text, so a container that + lacks one marker gets none — the same gate kimitools keeps.""" + for drop in glmtools.MARKERS: + partial = {k: v for k, v in GLM_MARKERS.items() if v != drop} + fmt = self.load(markers=partial) + self.assertEqual( + {}, fmt.tool_markers, + f"dropping {drop} still resolved a protocol") + self.assertEqual(fmt.tool_protocol, "") + + def test_a_no_tool_container_refuses_tools_by_name(self): + fmt = self.load(markers={k: v for k, v in GLM_MARKERS.items() + if v not in glmtools.MARKERS}) + self.assertEqual(fmt.tool_protocol, "") + with self.assertRaises(ChatFormatError) as cm: + fmt.build_chat_segments([{"role": "user", "content": "hi"}], + tools=TOOLS, thinking=True) + self.assertIn("GLM's XML protocol", str(cm.exception)) + + +class TestRender(Base): + def setUp(self): + super().setUp() + self.fmt = self.load() + + def render(self, messages, **kw): + # GLM's generation prompt always opens the reasoning channel, so a + # GLM format refuses thinking=False; the tool tests render with it on. + kw.setdefault("thinking", True) + return "".join(seg.text for seg in + self.fmt.build_chat_segments(messages, **kw)) + + def segs(self, messages, **kw): + kw.setdefault("thinking", True) + return self.fmt.build_chat_segments(messages, **kw) + + def test_tool_declaration_opens_a_system_turn(self): + """The declaration is system text with the signatures as JSON, not + the Kimi `tool_declare` token literal.""" + segs = self.segs([{"role": "user", "content": "hi"}], tools=TOOLS) + # The prelude ([gMASK]) opens the conversation, then the + # declaration's <|system|> opener. + self.assertEqual(segs[0].text, "[gMASK]") + self.assertEqual(segs[1].text, "<|system|>") + self.assertTrue(segs[1].markup) + header = "".join(s.text for s in segs[2:4]) + self.assertTrue(header.startswith("# Tools"), header) + self.assertIn("", header) + # The signature is caller JSON, so it is a plain segment. + body = "".join(s.text for s in segs) + self.assertIn('"name": "get_weather"', body) + self.assertIn("{function-name}", body) + # The caller's JSON must not be able to forge a control token. + self.assertFalse(any(s.markup for s in segs if '"name"' in s.text)) + + def test_a_tool_call_is_flat_xml(self): + """No section, no id — name, then one pair per argument.""" + segs = self.segs([{"role": "user", "content": "hi"}, CALL], tools=TOOLS) + rendered = "".join(s.text for s in segs) + self.assertIn( + "get_weathercity" + "Romedays" + "3", + rendered) + # The marker tags are their own markup segments; the caller's name and + # values are plain. The declaration's `{function-name}` + # is instruction text, so restrict the marker check to exact tags. + for s in segs: + if s.text in ("", "", "", + "", "", ""): + self.assertTrue(s.markup) + if s.text in ("get_weather", "Rome", "days", "3"): + self.assertFalse(s.markup) + + def test_two_calls_render_consecutively(self): + two = {"role": "assistant", "content": "", "tool_calls": [ + {"id": "a", "type": "function", + "function": {"name": "get_weather", + "arguments": {"city": "Rome"}}}, + {"id": "b", "type": "function", + "function": {"name": "get_time", "arguments": {"tz": "UTC"}}}]} + out = self.render([{"role": "user", "content": "hi"}, two], tools=TOOLS) + self.assertIn( + "get_weathercity" + "Rome" + "get_timetz" + "UTC", + out) + + def test_tool_calls_on_a_non_assistant_turn_are_refused(self): + with self.assertRaises(ChatFormatError) as cm: + self.fmt.build_chat_segments( + [{"role": "user", "content": "hi", "tool_calls": [ + {"id": "a", "function": {"name": "f", + "arguments": {}}}]}], + tools=TOOLS, thinking=True) + self.assertIn("non-assistant", str(cm.exception)) + + def test_a_tool_result_is_an_observation_turn(self): + msg = {"role": "tool", "tool_call_id": "call_1", "content": "18C"} + out = self.render([{"role": "user", "content": "hi"}, CALL, msg], + tools=TOOLS) + self.assertIn("<|observation|>", out) + self.assertIn("18C", out) + + def test_consecutive_results_share_one_observation(self): + """GLM groups a run of results under a single <|observation|>.""" + msgs = [{"role": "user", "content": "hi"}, CALL, + {"role": "tool", "tool_call_id": "call_1", "content": "18C"}, + {"role": "tool", "tool_call_id": "call_2", "content": "22C"}] + out = self.render(msgs, tools=TOOLS) + self.assertEqual(out.count("<|observation|>"), 1) + self.assertIn("18C" + "22C", out) + + def test_two_result_runs_get_two_observations(self): + out = self.render([ + {"role": "user", "content": "hi"}, CALL, + {"role": "tool", "tool_call_id": "call_1", "content": "18C"}, + {"role": "assistant", "content": "then"}, + {"role": "tool", "tool_call_id": "call_2", "content": "22C"}, + ], tools=TOOLS) + self.assertEqual(out.count("<|observation|>"), 2) + + +class TestGlmReplyParser(unittest.TestCase): + MARKERS = { + 1001: "<|user|>", + 21: "", 22: "", + 23: "", 24: "", + 25: "", 26: "", + } + + def parser(self): + return PlainParser(markers=self.MARKERS, in_think=False, + tool_parser=glmtools.ToolParser()) + + def feed(self, parser, items): + for token_id, piece in items: + parser.feed_token(token_id, piece) + + def test_a_glm_tool_call_is_read_back(self): + p = self.parser() + self.feed(p, [ + (2000, "I'll check the weather.\n"), + (21, ""), + (2001, "get_weather"), + (23, ""), (2002, "city"), (24, ""), + (25, ""), (2003, "Rome"), (26, ""), + (22, ""), + ]) + self.assertEqual(p.content, "I'll check the weather.\n") + self.assertEqual(len(p.tool_calls), 1) + call = p.tool_calls[0] + self.assertEqual(call.name, "get_weather") + self.assertEqual(call.arguments, {"city": "Rome"}) + msg = p.openai_message() + self.assertEqual(msg["role"], "assistant") + self.assertEqual( + msg["tool_calls"][0]["function"]["arguments"], + '{"city": "Rome"}') + + def test_a_non_string_value_is_read_back_as_json(self): + p = self.parser() + self.feed(p, [ + (21, ""), + (2001, "get_weather"), + (23, ""), (2002, "days"), (24, ""), + (25, ""), (2003, "3"), (26, ""), + (22, ""), + ]) + self.assertEqual(p.tool_calls[0].arguments, {"days": "3"}) + + def test_two_calls_are_read_back_in_order(self): + p = self.parser() + self.feed(p, [ + (21, ""), + (2001, "get_weather"), + (23, ""), (2002, "city"), (24, ""), + (25, ""), (2003, "Rome"), (26, ""), + (22, ""), + (21, ""), + (2004, "get_time"), + (23, ""), (2005, "tz"), (24, ""), + (25, ""), (2006, "UTC"), (26, ""), + (22, ""), + ]) + self.assertEqual([c.name for c in p.tool_calls], + ["get_weather", "get_time"]) + self.assertEqual(p.tool_calls[1].arguments, {"tz": "UTC"}) + + def test_markers_are_only_structure_by_id(self): + """A marker spelled out by an ordinary token is content, not markup — + the same rule PlainParser keeps for everything else.""" + p = self.parser() + self.feed(p, [(1234, " is the tag"), (1001, "<|user|>")]) + self.assertEqual(p.tool_calls, []) + self.assertIn("is the tag", p.content) + + def test_finish_flushes_a_trailing_value(self): + p = self.parser() + self.feed(p, [ + (21, ""), + (2001, "get_weather"), + (23, ""), (2002, "city"), (24, ""), + (25, ""), (2003, "Rome"), + ]) # stream ends inside , before + p.finish() + self.assertEqual(p.tool_calls[0].arguments, {"city": "Rome"}) + + +if __name__ == "__main__": + unittest.main() \ No newline at end of file diff --git a/tests/serve/test_server.py b/tests/serve/test_server.py index ee9008a7c..1967a3cb5 100644 --- a/tests/serve/test_server.py +++ b/tests/serve/test_server.py @@ -1001,3 +1001,99 @@ def test_kimi_k2_tool_call_streaming(self): self.assertEqual(reasons, ["tool_calls"]) self.assertEqual(events[-1], "[DONE]") + + +# --------------------------------------------------------------------------- +# GLM's native tool protocol over the real HTTP surface +# --------------------------------------------------------------------------- + +class TestGlmToolsFromChatJson(ServerTestCase): + """The GLM counterpart of TestKimiK2ToolsFromChatJson: a chat.json + container whose tokenizer carries GLM-5.3-Flash's XML tool grammar + instead of Kimi K2's five control tokens. + + Unlike Kimi-Linear, GLM's format always opens a reasoning channel, so it + inherits from `ServerTestCase` (which provides the chat/stream helpers) + rather than from `TestChatFromChatJson`, whose inherited cases assume + the Kimi container's semantics — thinking off by default, and tools + refused.""" + + from tests.serve.test_glmtools import GLM_MARKERS # noqa: E402 + + def setUp(self): + self.dir = tempfile.mkdtemp(prefix="serve-glm-tools-") + self.addCleanup(shutil.rmtree, self.dir, True) + shutil.copyfile(REPO / "examples" / "chat-glm53.json", + Path(self.dir) / "chat.json") + self.engine_kwargs = {"no_markers": True, "model_path": self.dir, + "markers": dict(self.GLM_MARKERS)} + # GLM's format always opens the reasoning channel, so default the + # server to thinking on, which is also the server's default. + ServerTestCase.setUp(self) + + @staticmethod + def glm_tool_reply(): + return ( + "I'll check the weather.\n" + "get_weathercity" + "Rome\n" + ) + + def test_tools_are_enabled_by_the_glm_markers(self): + """GLM's grammar, not Kimi's, enables the tool request.""" + self.engine.reply = self.glm_tool_reply() + status, body = self.chat(tools=[{ + "type": "function", + "function": { + "name": "get_weather", + "parameters": {"type": "object"}, + }, + }]) + self.assertEqual(status, 200) + self.assertEqual(body["choices"][0]["finish_reason"], "tool_calls") + calls = body["choices"][0]["message"]["tool_calls"] + self.assertEqual(len(calls), 1) + self.assertEqual(calls[0]["function"]["name"], "get_weather") + self.assertEqual(json.loads(calls[0]["function"]["arguments"]), + {"city": "Rome"}) + + def test_glm_tool_call_streaming(self): + self.engine.reply = self.glm_tool_reply() + events = self.stream(tools=[{ + "type": "function", + "function": { + "name": "get_weather", + "parameters": {"type": "object"}, + }, + }]) + name = None + arguments = "" + index = None + for event in events[:-1]: + for choice in event.get("choices", []): + for call in choice.get("delta", {}).get("tool_calls", []): + index = call.get("index", index) + fn = call.get("function", {}) + if fn.get("name"): + name = fn["name"] + arguments += fn.get("arguments", "") + self.assertEqual(index, 0) + self.assertEqual(name, "get_weather") + self.assertEqual(json.loads(arguments), {"city": "Rome"}) + reasons = [ + choice["finish_reason"] + for event in events[:-1] + if isinstance(event, dict) + for choice in event.get("choices", []) + if choice.get("finish_reason")] + self.assertEqual(reasons, ["tool_calls"]) + self.assertEqual(events[-1], "[DONE]") + + def test_a_malformed_glm_tool_call_is_a_400(self): + """A call with no function name trips GlmToolError, mapped to a 400 + naming the field — the same API surface as Kimi's KimiToolError.""" + status, body = self.chat(messages=[{ + "role": "assistant", "content": "", "tool_calls": [{}]}]) + self.assertEqual(status, 400) + self.assertEqual(body["error"]["param"], + "messages[0].tool_calls[0].function.name")