Loading ToolBench
A collection of MCP server examples implemented in Java using jbang and Quarkus MCP framework
This server exhibits significant definition quality gaps. While 4 tools are present with basic input schemas and descriptions, they are minimal and lack depth required for LLM selection and reasoning. Tool names lack strong action verbs (operation, getWeather, start, stop are weak/generic). Descriptions are short (10-40 chars) and do not answer WHAT, WHEN, or WHY. Parameters exist but lack sufficient description detail. No output schemas are documented. No error handling guidance. The 'operation' tool is particularly problematic, it accepts free-form string values instead of an enum constraint, inviting hallucinated operations. Descriptions do not state state-modification semantics. Schemas lack detail on return types and structure.
A tool to get the current weather a given city and country code
Performs basic arithmetic operations (add, subtract, multiply, divide)
Starts the stopwatch
Stops the stopwatch
Tool names lack strong action verbs. 'operation', 'start', 'stop' are generic or lack clear intent. 'operation' is particularly vague, LLMs cannot infer whether it adds, subtracts, multiplies, or divides without reading the full schema. Rename to 'perform_arithmetic' or similar.
Descriptions are critically short (10-40 chars) and lack WHAT, WHEN, WHY context. Baseline is 194 chars avg. Tool 'operation' description ('Performs basic arithmetic operations (add, subtract, multiply, divide)') is 64 chars but does not explain WHEN to use it or what it returns. 'start' and 'stop' are 24 chars each.
'operation' tool accepts 'operation' as a free-form string parameter. This invites hallucinated values. Should be an enum: {"type": "string", "enum": ["add", "subtract", "multiply", "divide"]}. Baseline: 100% of A+ tools use enums for constrained inputs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 25 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 27 | - | v1 |
No output/return schemas documented for any tool. LLMs need to know what fields to expect. For 'operation', does it return {"result": 42} or {"result": 42, "expression": "10 + 32"}? For 'getWeather', what fields are returned? Baseline: 100% of A+ tools have documented return types.
Parameter descriptions are minimal. 'operation' param is 'The operation to perform - add, subtract, multiply, divide' (72 chars baseline avg). Parameter 'a' is 'The first operand' (18 chars), not enough detail on format or range. Baseline: avg param annotation 72 chars, 100% of A+ params have descriptions.
No error handling guidance. If 'operation' receives 'divide' with b=0, what error is returned? If 'getWeather' gets an invalid city, what guidance does the LLM receive? Baseline: errors must tell LLM what to do next, not just return raw error codes.
'start' and 'stop' tools have no input parameters but lack documentation explaining what state they manage. Is there a global stopwatch or per-user stopwatch? What does 'stop' return, elapsed time? Should these be 'start_timer' and 'stop_timer' to clarify intent?
No idempotency semantics documented. If 'start' is called twice, what happens? If 'getWeather' is called with the same parameters, does it cache or fetch fresh data? Agents retry on failure and need to know which tools are safe to retry.