Loading ToolBench
Secure shell server for the Model Context Protocol. Integrates with Claude Desktop.
This server exhibits severe definition quality issues across all three tools. All tools have parameter descriptions that are generic and lack actionable constraints. Input schemas are present but critically incomplete: parameters lack proper type definitions (all hardcoded as 'string' despite semantic differences), lack validation constraints (enums, min/max), and lack dependency documentation. Tool descriptions are present but minimal (10-30 chars) and fail to explain WHEN to call each tool or WHAT distinguishes them. No output schemas are documented. There is no error handling guidance. The server demonstrates no awareness of composition patterns, the three tools serve a narrow NYC legislative domain but lack the explanatory scaffolding that would let an LLM use them confidently.
Get committee positions for a specific council member
Get council members
get matters (aka legislation, intros, bills, matters before the council).
Parameter types are all hardcoded as 'string' without semantic distinction. 'top', 'skip', and 'year' should not all be strings, 'top' and 'skip' are numeric pagination parameters, 'year' is a date filter. This forces the LLM to guess the format and type-convert at runtime.
No parameter descriptions at all. 'top' and 'skip' lack explanation of their pagination semantics (what is 'top'? maximum results? first N? a cursor?). 'activeOnly' is a boolean concept but has no description explaining what 'active' means in council member context. LLM cannot infer meaning from names alone.
Tool descriptions are under 20 characters and provide zero actionable context. 'get matters (aka legislation, intros, bills, matters before the council).' is explanatory but does not state WHEN to call it, what the return structure is, or how it differs from similar legislative APIs. 'Get council members' and 'Get committee positions for a specific council member' are bare statements with no context.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 39 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 25 | - | v1 |
No output schemas documented. The tools return text/json with mimeType hints, but the LLM has no schema of what fields to expect. A response might contain 'id', 'name', 'status', 'url', the LLM cannot plan downstream calls without knowing the structure. Pattern baseline: 100% of A+ tools document return types.
Parameter dependencies and constraints are undocumented. 'top' and 'skip' imply pagination, is there a max page size? Does 'skip' accept 0? Are these mutually exclusive with a cursor? The code lacks validation and the descriptions lack constraints.
'personId' parameter in get_council_member_positions is opaque, is this a numeric ID, a UUID, or a slug? The description says 'the ID of the council member' but does not specify format. Requires LLM to infer from prior tool outputs or fail. Pattern baseline: accept human-friendly identifiers (name, email) alongside system IDs, or document format strictly.
No error handling or recovery guidance. The requestHandler catches no errors and returns no guidance if a query fails. What if 'personId' is invalid? What if the year filter returns no results? The LLM receives no actionable error message.
No idempotency or side-effect clarity. Tools appear read-only (marked READ_ONLY in risk), but this is not stated in descriptions. Agents do not know if they can retry without duplication. Pattern: 'If the tool modifies state, the description must say so.'