Loading ToolBench
A flexible LLM orchestration framework with tool calling capabilities via MCP protocol
Jiki provides 5 basic tools with minimal but present schemas and descriptions. However, the server exhibits significant gaps in parameter documentation, lacks descriptions for individual parameters, provides no output schema documentation, and completely omits error handling guidance. Tool naming is acceptable but generic. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) are present. Parameter descriptions are entirely missing, the schemas show parameter names and types (e.g., 'a', 'b' as strings/integers) but the schema objects lack 'description' fields. This forces LLMs to infer meaning from context alone, violating pattern:tool-description. The divide tool in particular is dangerous: it accepts integer inputs but performs division without documented handling of division-by-zero, no error guidance, and no indication of output format (integer or float?). All tools are READ_ONLY by risk classification, but no tool annotations communicate this to the agent. Overall, the tools are functional for toy examples but unsuitable for production agent use.
Add two numbers
Concatenate two strings
Divide a by b
Multiply two numbers
Subtract b from a
Parameter descriptions completely missing. All tools define parameters (e.g., 'a', 'b') with types but NO description field in the schema object. LLMs cannot infer what 'a' and 'b' mean in concat vs add, is 'a' an operand, an array, a message? Violates pattern:tool-description hard requirement: 'Every parameter needs a description.'
No output schema documentation. Tools return untyped results. LLMs cannot predict the structure of responses: does add(2, 3) return '5' (string), 5 (integer), {result: 5} (object)? Without documented output types, agents cannot chain tools or extract required fields. Violates pattern:tool: 'Document the output schema.'
Dangerous divide tool lacks error handling documentation. Integer division by zero is undefined behavior. No error recovery guidance. Tool description does not mention: (1) what happens on division-by-zero, (2) whether result is integer or float, (3) whether rounding occurs. Violates pattern:recovery-guide: 'Error responses must tell the LLM what to do next.'
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 5 tools are marked READ_ONLY by risk classification but this is NOT exposed in tool annotations. Agents cannot detect idempotency or safety properties from the tool schema itself. Modern MCP spec (2026-07-28) requires annotation support for full clarity.
Generic descriptions offer minimal guidance. concat is described as 'Concatenate two strings' and add as 'Add two numbers', these state WHAT but not WHEN to use vs similar tools, no context on prerequisites, no guidance on parameter constraints. Violates pattern:tool-description guidance: 'Write descriptions as if prompt-engineering. State WHAT the tool does, WHEN to use it, and any prerequisites.'
Parameter naming lacks type suffixes. 'a' and 'b' in all tools are ambiguous, no _type suffix, no clarity on whether they are system IDs, values, or indices. Pattern baseline: 'When a parameter could be an ID, name, email, or position, suffix it with the type: user_id, user_name.' Parameternames should be 'first_number', 'second_number' or 'left_operand', 'right_operand' for clarity.