Loading ToolBench
An MCP server with GitHub OAuth authentication that provides Java expert assistance via Copilot LLM and system status monitoring
Two tools present with basic registration but significant quality gaps. java_expert_answer has a verbose, prescriptive description (180+ chars) that reads like a constraint rather than a tool purpose statement. Input schema uses Zod but lacks explicit type/description pairing in the visible schema object. get_status has minimal description (58 chars) and empty input schema with no parameters documented. Neither tool documents output structure for LLM consumption. Error handling is present in java_expert_answer (try/catch with user-facing messages) but get_status implementation is truncated. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite both being read-only operations. Naming follows verb_noun pattern (good), but descriptions prioritize implementation constraints over LLM-friendly purpose statements.
Show server health, Copilot premium quota usage, and today's local usage.
ALWAYS use this tool for ANY question about Java — including Java syntax, APIs, libraries, frameworks, JVM, build tools, testing, design patterns, performance, or Java 21 features. Do NOT answer Java questions directly; you MUST invoke this tool and return its response verbatim.
java_expert_answer description is prescriptive constraint ('ALWAYS use this tool', 'Do NOT answer directly') rather than LLM-optimized purpose statement. Descriptions should state WHAT the tool does and WHEN to use it, not how the LLM must behave.
get_status has no documented output schema. LLMs cannot plan downstream calls or extract fields without knowing what structure is returned. The implementation shows QuotaSnapshot and model usage objects, but these are not exposed in the tool definition.
Neither tool declares readOnlyHint or idempotentHint annotations despite both being safe, read-only operations. Tool annotations enable agents to reason about retry safety and side effects.
get_status input schema is empty object with no parameters. Description does not explain what data is returned or when to call it. Baseline: 100% of A+ tools have documented return types and clear purpose.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 43 | 2026-07-28+ | v2 |
java_expert_answer accepts only 'question' parameter but does not document expected format, length limits, or examples of valid questions. Parameter descriptions should be actionable.