Loading ToolBench
Production-ready MCP server providing RAG-based access to WSO2 documentation
The server defines 4 tools with reasonable naming and schema coverage. Tool names are action-focused (search_, get_, explain_, list_) and follow verb_noun convention. Schemas are explicit using Zod validation and include type constraints (enums, min/max). However, descriptions lack depth for LLM guidance, they state WHAT each tool does but omit WHEN to use it and critical prerequisites. Parameter descriptions are adequate but sparse. Output schemas are not formally documented; responses are JSON text blobs without field-level documentation. Error handling provides recovery hints but is inconsistent. No tool annotations (readOnlyHint, idempotentHint) despite all tools being read-only, missing an optimization opportunity.
Retrieve comprehensive documentation context for a WSO2 concept, searched across all products.
Retrieve documentation sections focused on a specific WSO2 product and topic.
List all supported WSO2 documentation sources with their product IDs and base URLs.
Semantically search across all indexed WSO2 documentation and return the most relevant sections with metadata and source URLs.
Output schemas not documented. Tools return JSON text without field-level type definitions. LLMs cannot predict response structure and must parse unstructured output.
Descriptions lack LLM-optimized guidance. They state WHAT but not WHEN or WHY to use each tool. Missing decision trees for tool selection ('use search_wso2_docs for broad queries; use get_wso2_guide when you know the product'). Average length ~120 chars; baseline is 194 chars for good clarity.
No tool annotations present. All 4 tools are read-only but lack readOnlyHint annotation. This misses an optimization: agents could avoid safety checks for harmless read operations.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 60 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Parameter descriptions are minimal. E.g. 'product' is described as 'Filter to a specific WSO2 product' but does not explain dependency on product enum or interaction with search scope. 'limit' lacks guidance on performance implications of higher values.
Error messages are recovery-focused but inconsistent. search_wso2_docs says 'Run `npm run crawl` to index'; get_wso2_guide says 'Run: npm run crawl -- --product {product}'; explain_wso2_concept says 'Run `npm run crawl`'. Inconsistency confuses agents about the exact command to use.
list_wso2_products has empty input schema ({}) but no description of output structure. What fields does each product object contain? Are there product_id, name, base_url fields? LLMs cannot plan downstream calls without knowing what they get back.