Loading ToolBench
MCP server for managing user preferences and cafe/hotel lookups
Three tools present with basic verb_noun naming (get_*), but descriptions lack depth and clarity for LLM selection. All three tools have functional input schemas with type definitions, but parameter descriptions are minimal (6-12 chars). No output schemas documented. Error handling is absent, tools will fail silently on network errors or invalid responses. No guidance on when to use each tool or how they chain together. Security: tools call external services hardcoded to localhost with 2-second timeouts; no production URL configuration visible. The commented-out composite tool (get_id_for_email) suggests missing tool composition and fallback logic that should be explicit. Median tool score: 42/100.
Return the cafe id for a provided email address
Return the hotel user id for a provided email address
Return the current communication preferences for a provided cafe id
No output schema documentation. Tools return raw API responses (e.g., response.json()) without specifying what fields LLMs should expect. This forces agents to guess at response structure and blocks downstream tool chaining.
Minimal parameter descriptions (6-12 chars). 'cafe id to look up', 'email address to look up' provide no context for LLMs. Should explain: what identifies a valid cafe_id? What format? What happens if not found?
No error handling or recovery guidance. Tools call external services with 2-second timeouts and no try-catch. Network failures, 404s, 500s will raise unhandled exceptions. LLM receives no actionable error message or next steps.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
Tool descriptions do not clarify when to use each tool or how they relate. An LLM seeing 'get_cafe_id_for_email' and 'get_hotel_user_id_for_email' cannot distinguish which to call without trial-and-error. Missing: 'Use this when you have the user's email and need their cafe system ID' vs 'Use this for hotel system lookups.'
Hardcoded external service URLs (localhost:7070, localhost:8080, localhost:9090) prevent production deployment. No environment variable or configuration file visible for service discovery.
Potential tool composition issue: get_cafe_id_for_email and get_hotel_user_id_for_email both accept email and return 'id' (or similar), but the response field names appear to differ (get_cafe_id_for_email returns list[dict] vs get_hotel_user_id_for_email). Without documented output schemas, agents cannot reliably chain these tools or distinguish between 'cafe_id' and 'hotel_user_id' in responses.
No input validation. Tools accept any string for cafe_id or email. No format checking, no feedback if the value is malformed (e.g., invalid email format). LLMs will pass garbage and get cryptic errors.