Loading ToolBench
Security-aware agent environment auditor with token cost intelligence for Claude Code
Four tools with complete input schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. receive_telemetry has good input validation logic visible in code but lacks parameter descriptions in the schema itself. global_stats, global_stats_only, and dashboard are read-only stubs with minimal descriptions. No error handling guidance, no output schemas documented, and parameter descriptions are generic or missing entirely. Naming is clear (verb_noun pattern), but composition could be tighter, global_stats and global_stats_only are near-duplicates.
Serve the dashboard UI (index.html).
Aggregate stats for the dashboard.
Aggregate stats for the dashboard (global only).
Accept an opt-in telemetry payload and store aggregate counts.
Output schemas not documented. LLMs cannot plan downstream calls or extract fields without knowing what global_stats, global_stats_only, and dashboard return. Code shows get_global_stats(), get_profile_breakdown(), get_os_breakdown(), get_daily_trend() are called but return types are opaque.
receive_telemetry parameters lack descriptions in schema. While code validates fields (e.g., 'max 32 chars', '0-10000000 range'), these constraints are not exposed in the input schema descriptions. LLMs cannot infer format or range from names alone (e.g., 'tokens_wasted' vs 'tokens_savings_potential' are ambiguous without context).
Duplicate tools: global_stats and global_stats_only are near-identical. global_stats returns {global, by_profile, by_os, trend_30d}; global_stats_only returns only global. This forces LLMs to reason about which to call, wasting tokens. Consolidate into one tool with an optional 'breakdown' parameter.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
No error handling guidance. receive_telemetry returns 202 even on database insert failure ('Still return 202, don't punish clients for server errors'). LLMs cannot distinguish success from partial failure and may retry unnecessarily or assume data was persisted when it wasn't.
dashboard tool description is vague ('Serve the dashboard UI (index.html)'). Does it return HTML, a URL, or a file path? LLMs cannot determine how to use the response. Clarify: 'Returns the dashboard HTML file (index.html) for rendering in a browser.'