Loading ToolBench
A FastAPI application that exposes MCP tools over the stateless Streamable HTTP transport defined in the MCP 2026-07-28 specification. Designed to run behind Azure App Service's built-in load balancer with N instances — no sticky sessions, no in-process state, no protocol-level session at all.
Five tools with complete JSON Schema definitions and descriptions. All tools are read-only, stateless operations. Naming is clear and verb-forward (whoami, echo, lookup_fact, compute_primes, tally). Descriptions range 60 - 280 chars, meeting the 10 - 1024 baseline. Parameters are typed with descriptions. The tally tool exemplifies the 2026-07-28 stateless pattern via signed handles. Key gaps: no output schemas documented, no error handling guidance, no parameter constraints (e.g., compute_primes limit lacks min/max enforcement in schema), and descriptions lack dependency hints or recovery guidance.
CPU-bound: count primes <= limit. Capped to keep one request small.
Echo a message back, along with the serving instance ID.
Look up a static fact by topic (stateless dictionary read).
Maintain a running total across calls using an explicit handle. This is the 2026-07-28 replacement for session state. The running total lives inside the signed `handle`, not on the server — so call it repeatedly and the total accumulates correctly even though each call may land on a different App Service instance. Pass no handle to start a new tally; pass the returned handle to continue.
Return the App Service instance ID and hostname serving this request. Use it to verify load distribution.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Return the App Service instance ID') but do not specify the JSON structure, field types, or required fields. LLMs cannot plan downstream tool calls or extract data reliably without knowing response shape.
compute_primes limit parameter lacks schema constraints. Description says 'default 5000, max 50000' but JSON Schema has no minimum, maximum, or default properties. LLMs cannot validate input and may pass invalid values.
No error handling guidance. Tools lack descriptions of failure modes, recovery steps, or actionable error messages. If lookup_fact receives an invalid topic or tally receives a corrupted handle, the LLM has no guidance on what to do next.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 65 | 2026-07-28+ | v2 |
tally handle parameter lacks format specification. Description explains the concept but does not state the expected format (base64url-encoded signed JSON), making it hard for LLMs to construct or validate handles.
lookup_fact topic parameter lacks enum constraint. Description lists available topics as prose ('app-service, mcp, stateless-http, handles') but schema has no enum property. LLMs may hallucinate invalid topic values.