Loading ToolBench
Planning Center Demo is a minimal MCP server with 4 tools. While all tools have basic descriptions and input schemas are visible, the implementation has significant gaps: (1) Tool naming is inconsistent, 'lookup_pco_service_types' and 'lookup_pco_next_service' use 'lookup' instead of action verbs like 'get' or 'list'; (2) Parameter descriptions are generic or missing context (e.g., 'service_type_id' lacks guidance on how to obtain valid IDs); (3) Output schemas are not documented, return types are inferred from code but LLMs cannot verify structure; (4) No error handling guidance, API calls could fail (network, invalid IDs, auth) with no recovery path for the agent; (5) Dependencies between tools are not documented (must call lookup_pco_service_types first to get valid service_type_id for other tools); (6) Tools mix concerns, generate_random_number is a utility unrelated to PCO functionality. Average tool quality is C grade.
Generate a random number between min and max.
Gets the order items for the next service in Planning Center Online.
Gets information about the next plan from Planning Center Online.
Look up service types from Planning Center Online.
Tool naming inconsistency: 'lookup_pco_*' does not follow verb_noun convention. 'lookup' is non-standard; should be 'get_', 'list_', or 'search_'. This violates the naming baseline where 90% of A+ tools start with action verbs (get, list, create, search, update). LLMs may struggle to recognize these as retrieval tools.
Output schemas are not documented. None of the 4 tools have an explicit return schema definition visible in the code or in tool registration. LLMs cannot verify the structure of responses (e.g., list_dict vs dict, field names) and must infer structure from descriptions alone. This violates the baseline: 100% of A+ tools have documented return types.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 51 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Parameter descriptions lack context. 'service_type_id' appears in 3 tools with description 'The service type ID' but does not explain: (1) how to obtain a valid ID (must call lookup_pco_service_types first); (2) expected range/format; (3) what happens if invalid ID is passed. This violates the rule: 'Describe the expected format, range, and allowed values directly in the parameter description.' (pattern:constrained-input)
No error handling guidance. Tools call external PCO API (pco.iterate()) which can fail due to network, invalid service_type_id, authentication, or rate limits. No try/catch, no error descriptions, no recovery paths. An agent that receives an unhandled exception has no guidance on what to do next. This violates pattern:recovery-guide.
Tool dependencies not documented. lookup_pco_next_service and get_pco_next_service_order_items both require a valid service_type_id from lookup_pco_service_types, but no description states 'Call lookup_pco_service_types first to obtain valid service_type_id values'. Without this hint, agents may pass invalid IDs and fail. This violates the rule: 'Include dependency hints' (pattern:tool-description).
Mixed concerns. Tool 1 (generate_random_number) is a standalone utility unrelated to Planning Center Online functionality. Mixing unrelated tools in one MCP server reduces cohesion and forces agents to reason about irrelevant options. Each MCP server should serve a clear purpose.
Schema parameter descriptions are generic. Input parameters (e.g., 'min', 'max' for generate_random_number) have minimal descriptions ('Minimum value (default: 0)') that do not explain use case or constraints beyond the default. Parameters should guide the LLM on when and why to use them.