Loading ToolBench
Deterministic, no-LLM token-usage profiler + redundancy advisor for agent conversations. Shows where tokens go (per-role), detects exact + near-duplicate payloads, reports savings potential. Public methods only (SHA-256 exact dedupe + n-gram overlap) — auditable, no proprietary semantic layer.
Two tools with clear, deterministic purposes (token profiling and duplicate detection). Both have descriptions (194 - 250 chars, within baseline 34 - 392 range) and input schemas with typed parameters. However, output schemas are not documented in the source code, only inferred from descriptions. Parameter descriptions are present but minimal (15 - 30 chars each, below baseline 72 chars). No error handling guidance, no recovery patterns, and no tool annotations (readOnlyHint, idempotentHint). Tools are read-only and deterministic, which is good for composition, but lack the structured output documentation and error classification needed for production LLM agents.
Given a list of payload strings, report which are exact or near duplicates (good for spotting repeated tool results / log tails).
Analyze a conversation/tool log for token usage + redundancy. messages: [{role, content, tool_name?, provided_tokens?}] Reports per-role token spend, exact-duplicate payloads (SHA-256), near-duplicates (n-gram), estimated savings, and cache-key suggestions (public exact hash). Deterministic, no-LLM.
Output schemas not documented. Tool descriptions mention what is returned (per-role token spend, duplicate reports) but do not specify the JSON structure, field types, or nested object shapes. LLMs cannot plan downstream tool calls or extract data reliably without knowing the response schema.
Parameter descriptions are too brief (15 - 30 chars vs. baseline 72 chars). E.g., 'Message role: system, user, assistant, or tool' lacks context on when each role appears or how it affects token counting. Descriptions should explain WHAT the parameter controls and WHY the LLM should care.
No error handling or recovery guidance. If a message object is malformed (missing 'role' or 'content'), the tool will fail silently or return a generic error. Descriptions should state: 'If a message lacks role or content, returns error_code: invalid_message_format with the offending index.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
No tool annotations. Both tools are read-only and idempotent (deterministic, no side effects), but lack readOnlyHint and idempotentHint annotations. These hints help LLMs reason about retry safety and composition.
No pagination or result limits documented. profile_tokens accepts an unbounded array of messages; duplicate_payload_report accepts an unbounded array of payloads. Large inputs could exhaust memory or timeout. Descriptions should state: 'Recommended max 1000 messages / 500 payloads per call.'