Loading ToolBench
MCP server answering whether two exact package or runtime versions work together, with sources
Two tools with clear verb-noun naming (check_*, get_*) and documented descriptions. Input schemas present with object-typed parameters, but lack granular field definitions and type constraints. Descriptions are adequate (60-80 chars) but generic, they state WHAT without WHEN or WHY. No parameter-level descriptions visible in schema. Output schemas not documented in source. Error handling exists but recovery guidance is minimal. Composition is sound: tools are single-purpose and read-only. Missing: enum constraints, format specifications, and LLM-optimized descriptions that guide tool selection.
Check whether two exact versions are compatible with each other
Get compatibility context and constraints for a specific package or runtime version
Input parameters lack field-level type definitions and descriptions. Both tools accept object-typed 'subject'/'counterpart'/'target' with only high-level descriptions ('The first package or runtime version'). No visibility into required fields, nested structure, or format constraints (e.g., is it {name, version}? {ecosystem, name, version}?).
Output schemas not documented in source code. LLMs cannot plan downstream calls or extract required fields without knowing the structure of CheckCompatibilityResult and GetCompatibilityContextResult.
Tool descriptions are generic and lack context for LLM selection. 'Check whether two exact versions are compatible' does not explain WHEN to call this vs. get_compatibility_context, what 'compatible' means (semver? exact match?), or what the response contains.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 68 | 2026-07-28+ | v2 |
No enum constraints or format specifications for version inputs. LLMs may pass invalid version strings (e.g., 'latest', 'v1.2.3', '1.2.3-alpha') without guidance on accepted formats.
Error handling exists but recovery guidance is minimal. Code references InputError and InvariantViolation but source does not show actionable error messages (e.g., 'Version format invalid. Expected: {ecosystem}@{name}@{version}').