Loading ToolBench
MCP server that evaluates policy risk and gates agent actions based on query analysis and risk assessment
Single tool with adequate naming and visible schema, but critical deficiencies in description depth, parameter descriptions, error handling guidance, and output schema documentation. The tool's description (129 chars) is within acceptable range but lacks WHEN/WHY context. Input parameters 'records' and 'query' have descriptions present, but they are generic and lack constraints or format guidance. No output schema is documented, the LLM cannot predict the structure of 'matches', 'decision', 'risk' fields or how to chain results to downstream tools. Error handling is minimal, exceptions are caught but error responses provide no recovery guidance. The implementation shows basic competence but falls well short of production standards for agentic tool design.
Evaluates policy risk by analyzing records against a query, ranking matches by relevance score, and determining whether the action requires review or can be allowed based on risk keywords
Output schema not documented. The tool returns a dict with 'matches', 'decision', 'risk', 'count', 'status' fields, but the LLM has no specification of field types, structure, or semantics. This prevents downstream tool composition and forces the LLM to infer structure.
Parameter descriptions lack format, type, or constraint guidance. The 'records' param says 'List of policy records' but does not specify: required field names, expected value types (string, int, bool?), max array size, or example structure. The 'query' param lacks minimum/maximum length, allowed characters, or example values.
Error handling provides no recovery guidance. When validation fails (e.g., 'records must be a list'), the error is returned as-is with no suggestion of what the LLM should do next. Pattern requires actionable next steps, e.g., 'Ensure records is an array of dictionaries with string keys/values.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
Tool description does not explain WHEN to use it or what it returns. The current description lists what it does but omits: when should the agent call this vs. a different analysis tool? What does each decision ('review' vs 'allow') mean in downstream workflow? This forces the LLM to guess at appropriate invocation.
No input validation constraints visible in schema. The tool accepts any array of dicts and any non-empty string. No limits on array size, record field counts, query length, or term count. An LLM could pass 10,000 records or a 100MB query without schema feedback, leading to timeout or performance issues.
Output fields lack semantic clarity. The 'rank' field is numeric but its meaning (ordinal position in sorted list?) is not documented. The 'score' field is returned as a fraction but its calculation and range (0-1?) are not specified. The 'evidence' field is truncated at 240 chars with no mention of that limit in the schema.