Loading ToolBench
A rule as one MCP tool: a server for agents that call business rules generated from rulec rule files. The server exposes a single rule as an MCP tool with inputs, outputs, and trace information.
Four tools with complete JSON schemas and descriptions. Naming is clear and action-oriented (baggage_fee, cancellation_verdict, compensation_amount, pension_premium). Descriptions are adequate (100-200 chars) and explain WHAT each tool does. However, parameter descriptions lack depth, they state the field name and type but omit guidance on when to use the tool vs. alternatives, dependencies between parameters, or error recovery paths. Output schemas are documented in code comments but not exposed to the LLM. Error handling is present (ValueError raised with messages) but not structured for agent recovery. The pension_premium tool uses Japanese parameter names (報酬月額, 料率) which may confuse LLMs trained primarily on English; no enum constraints visible for categorical inputs like membership/cabin.
Compute the checked baggage fee for a reservation from the policy's allowance table. Use it instead of working the allowance out yourself.
Decide from the policy whether a reservation can be cancelled. Use it instead of applying the cancellation rules yourself.
Compute the compensation certificate amount the policy allows for a complaint, in dollars. 0 means the policy allows none. Use it instead of applying the compensation rules yourself.
Rule pension_premium v1: Decides outputs from inputs. Every number is an integer in the unit its description states; a rate is a count of its steps; a date is YYYY-MM-DD; an enum is one of the names listed. The result is one record: in, observed (the outputs), trace (the table rows that decided it).
Parameter descriptions lack actionable guidance. 'The cabin class of the reservation: basic_economy, economy or business' states the enum but does not explain when to use this tool vs. alternatives or what happens if the cabin is invalid.
Enum values are documented in descriptions as free text ('regular', 'silver', 'gold') rather than declared as JSON Schema enums. LLMs cannot parse these constraints programmatically and may hallucinate invalid values.
pension_premium uses Japanese parameter names (報酬月額, 料率) without English aliases. LLMs trained on English will struggle to map these to user intent or compose them with other tools.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2025-06-18+ | v2 |
Output schema is documented in Rust code comments (TRACE_SCHEMA constant) but not exposed in the MCP tool definition. LLMs cannot see what fields to expect in the response and cannot plan downstream tool calls.
Error handling returns ValueError with messages like 'membership must be one of...' but does not guide the LLM on recovery (e.g., 'Call search_memberships() to list valid options'). Errors are not categorized as retryable or user-fixable.