Loading ToolBench
Single tool 'evaluate' with critical definition gaps. Input schema is present but parameter descriptions are all null/missing, violating the baseline that 100% of A+ tools have parameter descriptions. Tool description itself is missing entirely. Schema is syntactically valid (Zod), but the absence of human-readable descriptions for 'data', 'expression', and 'bindings' parameters means LLMs cannot understand when or how to use each parameter. The 'expression' parameter lacks guidance on JSONata syntax, format, or constraints. The 'data' and 'bindings' parameters are described as JSON strings but with no indication of what structure is expected. Error handling returns raw error objects as JSON strings, which is better than stack traces but still not actionable (no 'try this instead' recovery guidance). No pagination, no structured output documentation, no security or composition considerations.
Tool 'evaluate' has no description. LLMs cannot determine when or why to select this tool vs. other JSONata evaluation methods.
All three input parameters ('data', 'expression', 'bindings') have null descriptions. The rubric baseline states 100% of A+ tools have parameter descriptions. LLMs cannot infer that 'data' and 'bindings' should be JSON strings, or what JSONata syntax is valid for 'expression'.
No constraint documentation for the 'expression' parameter. LLMs have no guidance on valid JSONata syntax, operators, or limits. The parameter description should state: 'A JSONata expression string (https://docs.jsonata.org/overview.html). Common operators: $, ., *, ?. Max length: [specify if limited].'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 34 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 26 | - | v1 |
Error responses return raw JSON objects (e.g., 'JSONata expression error: {...}') but do not guide the LLM on recovery. The rubric requires error responses to tell the agent what to do next. Instead of just returning the error object, include: 'Check JSONata syntax at https://docs.jsonata.org/' or 'Did you forget a $ accessor? Example: $.fieldName'.
No output schema documentation. The tool returns JSON.stringify(result) but LLMs cannot see what structure to expect or how to extract values for downstream processing. Document: 'Returns the JSONata evaluation result as a JSON string. If the result is an object, returns {key: value} structure. If an array, returns [item1, item2, ...]. If a primitive, returns the bare value as a string.'
No mention of 'expression' parameter accepting JSONata syntax with operators, transformations, or function calls. LLMs may not understand this is an expression language, not plain JSON. Add: 'expression: A JSONata query expression. Supports field access ($fieldName), arrays, functions ($.upper(), $.sum()), transformations ($map, $reduce), and more. See https://docs.jsonata.org/'.