Loading ToolBench
MCP server: convert Japanese between romaji, hiragana, and katakana.
Three simple tools with minimal but present schemas and descriptions. All tools have input schemas with proper type definitions (object with required 'text' string parameter). No parameter descriptions exist, the 'text' parameter has no explanation of what text format is expected (romaji? mixed?). No output schema documentation. No error handling guidance. No tool annotations (readOnlyHint, etc.). Tools are well-named with clear verb_noun structure (to_hiragana, to_katakana, to_romaji), but lack the context LLMs need to select them confidently.
Convert romaji (or katakana) to hiragana.
Convert romaji (or hiragana) to katakana.
Convert hiragana or katakana to Hepburn romaji.
Parameter 'text' lacks description. LLMs cannot infer whether to pass romaji, hiragana, katakana, or mixed input. Add description: 'The text to convert (accepts romaji, hiragana, or katakana depending on target).'
Tool descriptions are 10-14 characters, far below the 34-char p10 baseline. Expand to 50-150 chars explaining WHAT the tool does, WHEN to use it, and what it returns. E.g., 'Convert romaji or katakana text to hiragana syllabary. Use when normalizing Japanese input for indexing or display.'
No output schema documented. LLMs cannot plan downstream operations or validate results. Add explicit documentation: 'Returns a string containing the converted text.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 50 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All three tools are read-only and idempotent, annotate them to help agents reason about safety and retry behavior.
Error handling returns raw exception messages ('kana failed: ...') without recovery guidance. When input is invalid, tell the LLM what to do next: 'Invalid input: text must be a non-empty string. Ensure input is valid Japanese text or romaji.'