Loading ToolBench
A Clojure library for implementing MCP (Model Context Protocol), LSP (Language Server Protocol), and DAP (Debug Adapter Protocol) servers with a unified abstraction layer
Defport provides 24 tools with basic schemas and minimal descriptions. Most tools have 1-2 sentence descriptions (10-50 chars), well below the 50-200 char baseline for LLM-optimized descriptions. Schemas are present but sparse: many lack parameter descriptions entirely. No output schemas documented. Error handling is minimal, error-tool returns a generic error with no recovery guidance. Tool naming is inconsistent (e.g., duplicate 'add' tools with different signatures). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite having tools marked DESTRUCTIVE (trip-emergency-stop) and WRITE (write-holding-register). Composition is weak: tools like 'batch-process' and 'calculate-stats' are reasonable, but many are trivial (echo, multiply) and lack real-world utility. Parameter descriptions are almost entirely absent, LLMs cannot infer what 'query', 'pattern', or 'expression' mean without context.
Add two numbers.
Add two numbers
Process items in batch
Browse the OPC UA namespace for tags matching a prefix.
Calculate statistics
Jump to the definition of the symbol at a position.
List symbols in a document.
Echo back the input
Parameter descriptions almost entirely absent. LLMs cannot infer what 'query', 'pattern', 'expression', 'uri', 'tag', 'prefix' mean without explicit guidance. Rubric baseline: 100% of A+ tools have parameter descriptions.
Tool descriptions are too short (10-50 chars). Rubric baseline: 50-200 chars for LLM-optimized descriptions. Short descriptions lack context for tool selection and when to use vs similar tools.
No output schemas documented. Rubric baseline: 100% of A+ tools have documented return types. LLMs cannot plan downstream tool calls or extract required fields without knowing response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Always returns error
Evaluate an expression in the current frame's context.
Show hover info at a position.
Parse JSON string
List all currently active alarms.
Long operation with progress
Multiply two numbers
Read a Modbus holding register by address.
Read a named OPC UA tag from the gateway.
Find references to the symbol at a position.
Search for symbols by name pattern.
Search for code
Return a trivial one-frame stack.
List threads in the debuggee.
Trip the emergency stop. All motion halts. Requires manual reset.
Write a value to a Modbus holding register. Dangerous — real deployments should gate this behind authorization.
Destructive and write tools lack annotations. trip-emergency-stop (DESTRUCTIVE) and write-holding-register (WRITE) have no destructiveHint or readOnlyHint annotations. Agents cannot distinguish safe from dangerous operations.
Duplicate tool name 'add' with conflicting signatures (number vs int). Rubric: 'When multiple tools operate on the same resource, their names must make the distinction obvious.' This causes LLM confusion and tool selection errors.