serve/glmtools.py: GLM's own tool protocol, beside kimitools

CLAUDE.md

@@ -261,15 +261,17 @@ Stdlib-only OpenAI-compatible HTTP. `xtml.py` is a **port** of the release's
 against that file whenever `K3_DIR` is set; `regions.py` is the streaming
 parser that reads replies back into reasoning / content / `tool_calls`;
 `chatfmt.py` is the fallback for a container with no XTML markers, serving
-it from the same `chat.json` the CLI reads — plain conversation only, with
-tools, thinking and images refused by name rather than dropped;
-`kimitools.py` is Kimi's native tool-call protocol — the five markers, the
-rendering and the reply reader — which is neither of the two formats and so
-gets its own module: it is carried in a container's *tokenizer* while its
-`chat.json` says nothing about it, and Kimi-Linear ships those tokens with
-no chat template at all. `tests/serve/test_chatfmt_upstream.py` diffs it
-against K2's published one, `K2_DIR` naming the release, the way
-`test_xtml` does for K3;
+it from the same `chat.json` the CLI reads — plain conversation, with
+thinking and images from the format and everything else refused by name
+rather than dropped; `kimitools.py` and `glmtools.py` are the two native
+tool-call protocols a container's *tokenizer* can carry while its
+`chat.json` says nothing about them — Kimi K2's five control tokens and
+GLM-5.3-Flash's `<tool_call>` XML grammar, each with its rendering and its
+reply reader, enabled only when the whole marker set resolves.
+`tests/serve/test_chatfmt_upstream.py` diffs the first against K2's
+published one and `tests/serve/test_glm_upstream.py` the second against
+GLM's, `K2_DIR`/`GLM_DIR` naming the release, the way `test_xtml` does for
+K3;
 `engine.py` is the ctypes binding plus one lock held for a whole generation
 (a `waste_ctx` is not thread-safe). Struct layouts in `engine.py` mirror
 `waste.h` field for field — change one, change the other.

README.md

@@ -443,17 +443,19 @@ It supports streaming, tools, structured output, thinking controls, and images.
 
 A GLM container is served the same way, from its own `chat.json`: plain
 conversation and images, with the reasoning channel returned as
-`reasoning_content` beside `content`. Tools are refused by name rather than
-half-rendered — four strings cannot express a tool declaration, and GLM's
-tokenizer carries no protocol that could.
-
-Kimi-Linear's does. Since 0.7.2 a container whose tokenizer holds all five
-of Kimi's native tool-call markers gets tool calling over HTTP even though
-its `chat.json` describes only the ordinary turns — the format lives in
-`serve/kimitools.py`, and the server says which of the three capabilities a
-container has when it starts. All five or none: half of that rendering
-encodes as ordinary text, so a partial set is a different protocol rather
-than a smaller one.
+`reasoning_content` beside `content`. Tools work here too: GLM's tokenizer
+carries its own tool protocol (`<tool_call>`, `<arg_key>`, `<arg_value>`)
+as single tokens, so `serve/glmtools.py` renders a request and reads a
+reply the way GLM's own `chat_template.jinja` spells them — flat XML, an
+`<|observation|>` turn for results.
+
+Kimi-Linear's is the other one. Since 0.7.2 a container whose tokenizer
+holds all five of Kimi's native tool-call markers gets tool calling over
+HTTP even though its `chat.json` describes only the ordinary turns — the
+format lives in `serve/kimitools.py`, and the server says which of the
+three capabilities a container has when it starts. All or none, for either
+protocol: half of that rendering encodes as ordinary text, so a partial set
+is a different protocol rather than a smaller one.
 
 ```bash
 python3 -m serve ~/models/glm53.waste --port 8000

docs/GLM.md

@@ -331,10 +331,15 @@ advances `low` to `content_height + 1` rather than to the aligned height.
   release. A container converted from a release that shares a selection
   across layers is refused rather than produced.
 - **MTP.** The extra prediction layer is dropped, as above.
-- **Tools.** GLM's template carries a full tool-call protocol; the
-  declarative `chat.json` cannot express one and refuses by name rather
-  than half-rendering it. The raw `.jinja` is in the container for a host
-  that does interpret Jinja.
+- **Tools.** GLM's template carries a full tool-call protocol, and the
+  declarative `chat.json` cannot express one — so the server renders it
+  from the tokenizer instead. GLM's specials carry the whole XML grammar
+  (`<tool_call>`, `<arg_key>`, `<arg_value>` and the response/observation
+  markers) as single tokens, `serve/glmtools.py` renders and reads it back
+  the way the release's own `chat_template.jinja` spells it, and
+  `tests/serve/test_glm_upstream.py` diffs that rendering against the
+  template with `GLM_DIR` naming the release. The raw `.jinja` stays in the
+  container for a host that does interpret Jinja.
 
 ## What is checked
 

docs/SERVE.md

@@ -136,35 +136,47 @@ startup the server asks for the richer format first and falls back:
 
 ```
 chat     from ~/models/kimi-linear.waste/chat.json — plain conversation, no reasoning channel,
-         no images, native tools
+         no images, kimi tools
 chat     from ~/models/glm53.waste/chat.json — plain conversation, a reasoning channel,
-         images, no tools
+         images, glm tools
 ```
 
 The three capabilities are read from the container, never assumed: the
 channel and the images from `chat.json`, the tools from whether the
-tokenizer carries **all five** of Kimi's native tool-call markers as single
-tokens. Kimi-Linear does; GLM does not, and is refused by name.
+tokenizer carries a whole native tool protocol as single tokens. There are
+two of them: **all five** of Kimi K2's markers, which Kimi-Linear carries,
+or **all nine** of GLM's, which GLM-5.3-Flash does — `<tool_call>`,
+`</tool_call>`, `<arg_key>`, `</arg_key>`, `<arg_value>`, `</arg_value>`,
+`<tool_response>`, `</tool_response>` and `<|observation|>`. A container
+with neither is refused by name.
 
 That last one is a rendering `chat.json` itself cannot describe — four
 prefix/suffix strings say nothing about a tool declaration or an argument
-list — so the protocol lives in `serve/kimitools.py`, its own module beside
-`xtml.py`, and is enabled only when the whole marker set resolves.
+list — so each protocol lives in its own module beside `xtml.py`
+(`serve/kimitools.py`, `serve/glmtools.py`), and is enabled only when the
+whole marker set resolves.
 
 The split is by subject rather than by size. *Whether* a container can do
 tools is a fact about its `chat.json` and its tokenizer, so `chatfmt.py`
 decides it and refuses with `ChatFormatError`. *How* a tool call is spelled
-is a fact about the protocol, so `kimitools.py` owns it and a malformed one
-raises `KimiToolError` — the same shape `xtml.py` has with `XTMLError`, and
-`api.py` maps each to a 400. Nothing in `kimitools.py` imports `chatfmt`,
-which is what lets `chatfmt` import it. **It is Kimi K2's**, and it is checked
-against K2's own published `chat_template.jinja` rather than transcribed
-from memory: `tests/serve/test_chatfmt_upstream.py`, which `tests/run.sh`
-runs whenever `K2_DIR` names a release directory, the same discipline
-`test_xtml.TestAgainstUpstream` applies to K3 with `K3_DIR`. Kimi-Linear's
-own release carries the five tokens and **no chat template at all**, which
-is why the grammar has to come from K2 and why an oracle for it matters
-more than usual.
+is a fact about the protocol, so `kimitools.py` or `glmtools.py` owns it and
+a malformed one raises `KimiToolError` or `GlmToolError` — the same shape
+`xtml.py` has with `XTMLError`, and `api.py` maps each to a 400. Nothing in
+either imports `chatfmt`, which is what lets `chatfmt` import them. Each is
+**the release's own grammar**, checked against the template that defines it
+rather than transcribed from memory:
+`tests/serve/test_chatfmt_upstream.py`, which `tests/run.sh` runs whenever
+`K2_DIR` names a release directory, and `tests/serve/test_glm_upstream.py`
+for `GLM_DIR` — the same discipline `test_xtml.TestAgainstUpstream` applies
+to K3 with `K3_DIR`. Kimi-Linear's own release carries the five tokens and
+**no chat template at all**, which is why the grammar has to come from K2
+and why an oracle for it matters more than usual; GLM's release ships its
+template, and the two grammars differ enough that each gets its own module
+and its own reader — a Kimi call is `ID<|tool_call_argument_begin|>ARGS` in
+a section, a GLM call is flat XML with the name after the opening tag and
+one `<arg_key>`/`<arg_value>` pair per argument, and a GLM result is an
+`<|observation|>` turn wrapping `<tool_response>` blocks where a Kimi result
+is a system turn named for the tool.
 
 One difference from that template is deliberate and asserted rather than
 fixed: with no system turn first, K2's template inserts Moonshot's own

serve/__main__.py

@@ -249,7 +249,8 @@ def main(argv=None) -> int:
         think = ("a reasoning channel" if srv.chat_format.think
                  else "no reasoning channel")
         images = "images" if srv.chat_format.image else "no images"
-        tools = "native tools" if srv.chat_format.tool_markers else "no tools"
+        protocol = srv.chat_format.tool_protocol
+        tools = f"{protocol} tools" if protocol else "no tools"
         print(f"chat     from {model}/chat.json — plain conversation, "
               f"{think},\n         {images}, {tools}")
 

serve/api.py

@@ -27,7 +27,7 @@
 from pathlib import Path
 from typing import Any, Optional
 
-from . import chatfmt, kimitools, xtml
+from . import chatfmt, glmtools, kimitools, xtml
 from .engine import Engine
 from .regions import RegionParser
 
@@ -385,6 +385,8 @@ def build_prompt(engine: Engine, body: dict, *, default_thinking: bool,
         raise APIError(str(e), param="messages")
     except kimitools.KimiToolError as e:
         raise APIError(str(e), param=e.param or "messages")
+    except glmtools.GlmToolError as e:
+        raise APIError(str(e), param=e.param or "messages")
 
     tokens = engine.tokenize_segments(segments)
     if n_images:

serve/chatfmt.py

@@ -18,10 +18,13 @@
 
 - **Plain conversation, streaming included.** system / user / assistant
   turns, and a stop that comes from the format rather than from a guess.
-- **No tools.** Four strings cannot express a tool declaration, an argument
-  list, or a result turn. K3's encoder needs 647 lines for that, and the
-  markup Kimi-Linear's tokenizer carries for it is not transcribed anywhere
-  in this repo. Refused, by name, rather than half-rendered.
+- **Tools, when the tokenizer carries a protocol for them.** Four strings
+  cannot express a tool declaration, an argument list, or a result turn, so
+  neither Kimi K2's five control tokens nor GLM's `<tool_call>` XML grammar
+  live in chat.json. They live in the tokenizer, `kimitools` and `glmtools`
+  render whichever one a container carries, and a container whose
+  vocabulary has neither refuses a `tools` request by name rather than
+  half-rendering it.
 - **A reasoning channel, when the format names one.** `chat.json` may
   carry `think: ["<think>", "</think>"]`, and GLM-5.3-Flash's does: its
   generation prompt opens the channel and the model closes it before the
@@ -57,7 +60,7 @@
 from dataclasses import dataclass, field
 from typing import Any, Optional
 
-from . import kimitools
+from . import glmtools, kimitools
 from .regions import Delta, ToolCall
 from .xtml import Segment
 
@@ -114,6 +117,12 @@ class ChatFormat:
     effort: str = ""
     image: str = ""
     tool_markers: dict[int, str] = field(default_factory=dict)
+    # Which native tool protocol `tool_markers` resolved to: "kimi" for
+    # Kimi K2's five control tokens, "glm" for GLM's `<tool_call>` XML
+    # grammar, or "" when the tokenizer carries neither. The renderer and
+    # the reply reader branch on this, because the two grammars differ in
+    # what a call, a declaration and a result turn look like.
+    tool_protocol: str = ""
 
     @property
     def markers(self) -> dict[int, str]:
@@ -231,16 +240,26 @@ def load(cls, engine: Any) -> "ChatFormat":
                     f"chat format disagree")
             ids[text] = got[0]
 
-        # A container may carry Kimi's native tool protocol in reserved
+        # A container may carry a native tool protocol in reserved
         # tokenizer tokens even though chat.json describes only the ordinary
         # turns. Whether it does is this file's question; what the protocol
-        # is belongs to kimitools.
+        # is belongs to kimitools or glmtools. Kimi first keeps the existing
+        # behaviour, and the two grammars are disjoint so a real container
+        # resolves at most one.
+        protocol = ""
         discovered = kimitools.detect(engine)
+        if discovered:
+            protocol = "kimi"
+        else:
+            discovered = glmtools.detect(engine)
+            if discovered:
+                protocol = "glm"
 
         return cls(roles=roles, opening=opening, stop_marker=stop_marker,
                    stop_id=ids[stop_marker], prelude=prelude, think=think,
                    think_close_id=ids[think[1]] if think else -1,
-                   effort=effort, image=image, tool_markers=discovered)
+                   effort=effort, image=image, tool_markers=discovered,
+                   tool_protocol=protocol)
 
     # ---- rendering ------------------------------------------------------
 
@@ -283,11 +302,15 @@ def build_chat_segments(self, messages: list[Any],
                 raise ChatFormatError(
                     "this container is served from its chat.json, which "
                     "cannot express tool definitions because its tokenizer "
-                    "does not carry the Kimi K2 native tool markers",
+                    "carries neither the Kimi K2 native tool markers nor "
+                    "GLM's <tool_call> XML protocol",
                     param="tools",
                 )
 
-            segments.extend(kimitools.declaration(tools))
+            if self.tool_protocol == "glm":
+                segments.extend(glmtools.declaration(tools))
+            else:
+                segments.extend(kimitools.declaration(tools))
         for name in ("tool_choice", "response_format", "response_schema"):
             if kwargs.get(name) is not None:
                 raise ChatFormatError(
@@ -332,8 +355,24 @@ def build_chat_segments(self, messages: list[Any],
             role = _ROLE_ALIASES.get(role, role)
 
             # Kimi K2 represents a tool result as a system-style turn whose
-            # content begins with "## Return of <tool_call_id>".
+            # content begins with "## Return of <tool_call_id>"; GLM opens an
+            # <|observation|> turn and wraps each result in
+            # <tool_response>…</tool_response>. Both are the authored side of
+            # the protocol, so which one lives in this file as turn framing
+            # and which lives in the tool module is the same split as with
+            # the role prefixes — the protocol only decides the body.
             if role == "tool":
+                if self.tool_protocol == "glm":
+                    # The template groups consecutive tool results under a
+                    # single <|observation|>, one block per result. Look back
+                    # so a run of results shares an opener, exactly as GLM's
+                    # template does (`loop.first or the last role != "tool"`).
+                    if i == 0 or messages[i - 1].get("role") != "tool":
+                        segments.append(Segment(glmtools.OBSERVATION,
+                                                markup=True))
+                    segments.extend(glmtools.tool_response(
+                        _content_segments(message.get("content"), i)))
+                    continue
                 pair = self.roles.get("system")
                 if pair is None:
                     raise ChatFormatError(
@@ -363,7 +402,10 @@ def build_chat_segments(self, messages: list[Any],
                         f"messages[{i}] carries tool_calls on a non-assistant "
                         "turn",
                         param=f"messages[{i}].tool_calls")
-                segments.extend(kimitools.call_section(tool_calls, i))
+                if self.tool_protocol == "glm":
+                    segments.extend(glmtools.call_section(tool_calls, i))
+                else:
+                    segments.extend(kimitools.call_section(tool_calls, i))
 
             segments.append(Segment(suffix, markup=True))
 
@@ -418,14 +460,20 @@ def _content_segments(content: Any, index: int, images: Any = None) -> list[Segm
 
 
 class PlainParser:
-    """Read a chat.json reply, including Kimi K2 native tool calls.
+    """Read a chat.json reply, including native tool calls.
+
+    The reply is read back whichever tool protocol the container carries —
+    Kimi K2's five control tokens or GLM's `<tool_call>` XML grammar — by
+    whichever `tool_parser` the caller hands in (Kimi by default, for the
+    containers that were here first).
 
     Structure is recognized only from tokenizer marker ids. Marker-looking
     text carried by an ordinary token remains ordinary model content.
     """
 
     def __init__(self, *, markers: Optional[dict[int, str]] = None,
-                 think_close_id: int = -1, in_think: bool = False):
+                 think_close_id: int = -1, in_think: bool = False,
+                 tool_parser: Any = None):
         self._markers = dict(markers or {})
         # The channel, when the format has one. `in_think` says the
         # generation prompt left it open — which for GLM it always does —
@@ -440,7 +488,8 @@ def __init__(self, *, markers: Optional[dict[int, str]] = None,
         # The tool protocol reads itself; this file decides only what is
         # left over. `tool_calls` stays an attribute here because it is what
         # openai_message reports.
-        self._tools = kimitools.ToolParser()
+        self._tools = tool_parser if tool_parser is not None \
+            else kimitools.ToolParser()
 
     @property
     def finished(self) -> bool: