Loading ToolBench
Local LLM cost & token forensics proxy with anomaly detection
Single tool 'scan' has a reasonable schema with 5 typed parameters and enums, but critical gaps undermine quality. Description is adequate (95 chars, within 10-1024 range) but lacks WHEN/WHY guidance for LLM selection. Parameter descriptions are present but generic, 'Path to JSONL/JSON request log' doesn't explain what the tool does with the log or when to call it. No output schema documented. No error handling guidance. Tool name 'scan' is generic and doesn't start with a clear action verb (should be 'analyze_logs' or 'audit_costs'). Overall definition quality is poor.
Local LLM cost & token forensics proxy with anomaly detection
Tool name 'scan' is generic and does not start with a clear action verb. Should be 'analyze_logs', 'audit_costs', or 'detect_anomalies' to signal intent to LLMs.
Tool description lacks WHEN/WHY context. It states WHAT ('Local LLM cost & token forensics proxy') but not WHEN to call it or what problem it solves. LLMs cannot determine tool selection without this guidance.
Output schema is not documented. LLMs cannot plan downstream calls or extract results without knowing the response structure (fields, types, pagination).
Tool registration is delegated to cognis_core.mcp.build_mcp_server() (not visible in source).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
No error handling guidance. If anomaly detection fails or log parsing errors occur, LLMs receive no recovery path. Should return actionable error messages like 'Invalid JSON format. Ensure logfile is valid JSONL.'