Loading ToolBench
A system that monitors LLM interactions, detects conversational ruts, and provides therapeutic interventions through the Model Context Protocol
Four tools with complete schemas and descriptions, but significant gaps in parameter documentation and error handling. Tool names follow verb_noun convention (get_, create_, evaluate_). Descriptions are present (avg ~80 chars) but lack actionable context for LLM selection. Parameters have types but many lack descriptions (e.g., 'strategy_type' and 'rut_type' in get_intervention_statistics are marked 'Optional' but lack guidance on valid values or consequences of omission). No output schemas documented. No error recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Composition is reasonable, tools are single-purpose, but chaining support unclear (e.g., does create_intervention_plan return an ID for evaluate_intervention?).
Create a plan for intervening in a conversation
Evaluate the effectiveness of an intervention
Get the history of interventions for a conversation
Get statistics about intervention effectiveness by strategy and rut type
No output schemas documented. LLMs cannot infer what fields to expect from tool responses, forcing them to guess at downstream chaining (e.g., does create_intervention_plan return intervention_id?). This breaks tool composition.
Parameter descriptions are incomplete or missing. 'strategy_type' and 'rut_type' in get_intervention_statistics lack guidance on valid values, format, or consequences of omission. 'limit' in get_intervention_history has no range (1 - 100? 1 - 1000?). LLMs cannot validate inputs without explicit constraints.
No error handling or recovery guidance. Tools lack descriptions of failure modes, retryability, or next steps. E.g., what happens if conversation_id is invalid? Should the LLM retry, search for the ID, or ask the user?
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | <=2025-11-25 | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). create_intervention_plan is marked WRITE but lacks destructiveHint annotation. Agents cannot determine if retrying is safe.
Tool descriptions lack WHEN/WHY context. 'Get statistics about intervention effectiveness' does not explain when to call this vs other tools, what problem it solves, or what the LLM should do with the result. Descriptions should be 50 - 200 chars and answer: What? When? Why?