The Prose Half of Function Calling

On 2 September a group at UESTC and Tsinghua posted Implicit Manipulation for Skill Selection in LLM Agents with Semantic Matching. Their attack, ISM, raises an attacker’s target-selection rate from 15.2% to 63.5% across four task domains and eight selector models. Human reviewers flag it in 2.9% of judgments; explicit steering gets flagged 91.4% of the time. It survives PPL-W, Llama Prompt Guard 2 and PIGuard — because nothing in it reads as malicious. The user’s request is benign. Every skill description is benign. Only the semantic relationship between them has been shaped. What the attack reaches is not a model, not a prompt, not a sandbox. It is a data type that was chosen in 2023 and never revisited since. It is worth tracing how that field became load-bearing, because every step was a reasonable answer to a real problem.

In 2022 tool selection was prose because the whole system was prose. MRKL (AI21, May 2022) and ReAct (Yao et al., October 2022) put tool names and one-line explanations into the context and let the model infer the rest; selection was a language judgment for want of any other mechanism. The fork came on 13 June 2023, when OpenAI shipped function calling in reaction to how brittle prose-parsing had turned out to be. It gave arguments a JSON Schema — types, enums, required fields, all validated at the boundary. It left selection as description: free text, no schema, no constraints. The field deciding which code runs stayed prose while the field deciding what gets passed to it became typed, and every layer built since has been built on the prose half.

Then the string stopped being read and started being matched. Tool catalogs outgrew the context window, and Gorilla (May 2023) and ToolLLM (July 2023, over an index of 16,464 real RapidAPI endpoints) answered with a neural retriever: embed the descriptions, take top-k. The description was no longer an input to the model’s judgment — it had become the similarity key, and the cut now happened before the model saw the candidates at all. In November 2024 MCP standardised tools/list as {name, description, inputSchema}, making the same field a cross-vendor wire format published by machines; the registry opened a year later and passed roughly 2,000 entries within months, so the descriptions competing in any given match are now mostly written by strangers. Agent Skills (October 2025, published as an open standard that December) closed the arc with progressive disclosure: only the YAML frontmatter sits in context, and the body loads only if the description wins. The template asks that field to state what the skill does and when to use it. Its sole job is to win. The Ask HN thread on managing skills files (177 points, 6 September) shows where that lands in practice — people installing skills from GitHub repos and from YouTubers, with the careful ones describing their trust boundary as “a few names I recognise.”

Four years of hardening has gone into everything around this decision and none of it into the decision. Argument schemas validate what gets passed once a tool has already been chosen. Sandboxes and VM isolation — Trail of Bits’ Coop landed on 7 September — bound the blast radius after the wrong skill has already won. Prompt-injection classifiers read text for malice, and ISM’s whole result is that a match can be benign character by character while the relationship between two benign strings does the work; a text classifier structurally cannot see that. What is missing is smaller and duller than any of those defences: a record of the selection itself. Which candidates competed. What the margin was. Who authored the winning description, and when it last changed. The sharpest comment in that HN thread is really this same move one layer up — a user pinning the model at the harness layer specifically so delegation “gets enforced at the harness instead of depending on the good will of the orchestrating model.” Take the decision out of semantic space and put it somewhere you can diff.