Loading ToolBench
MCP server for searching Magento 2 REST API documentation
The server provides 5 well-intentioned tools for Magento API exploration, but suffers from significant gaps in schema documentation, parameter validation, and error handling. Tool naming is clear and verb-driven (search_*, get_*, list_), and descriptions are present but generic. However, parameter schemas lack critical details (no enums for HTTP methods, no constraints on query lengths despite the 'SHORT keyword' guidance in the description). Output is returned as unstructured strings rather than structured JSON objects, forcing LLMs to parse plain text. Error handling is minimal, there are basic 'not found' messages but no actionable recovery guidance. The server reads from a local SQLite database and does not expose any secrets, which is good, but the tool composition is weak: search_endpoints and search_schemas are near-duplicates, and there is no clear dependency chain between discovery (list_tags, search_endpoints) and detail retrieval (get_endpoint_details, get_schema). Pagination is stubbed (DB_TOP_K = 5) but not exposed as a parameter, and results are truncated silently without indicating whether more items exist.
Get complete documentation for a specific Magento 2 REST API endpoint. Provide the exact path and optionally the HTTP method to get full details including parameters, request/response schemas.
Get the complete definition of a Magento 2 data schema/model. Schemas define the structure of request/response data objects (e.g., quote-data-cart-interface, customer-data-customer-interface).
List all available API category tags in the Magento 2 REST API. Tags group related endpoints together (e.g., carts, customers, products, orders).
Search for Magento 2 REST API endpoints using keywords. Use SHORT keyword queries (1-3 words) to find endpoints by path, method, category, or description. You can optionally filter by HTTP method or category tag.
Search for Magento 2 data schemas by keyword. Use this to find data structure definitions by name or description (e.g., 'cart', 'customer', 'product').
No output schemas documented. All tools return unstructured text (plain strings or markdown), not structured JSON objects with typed fields. LLMs must parse text, which is error-prone and wastes tokens.
Parameter enums missing for constrained inputs. 'filter_by_method' accepts 'GET, POST, PUT, DELETE, PATCH' but defines them only in the description, not as JSON Schema enum constraint. Similarly, 'method' in get_endpoint_details lists allowed values in description rather than as enum.
Pagination not exposed to the LLM. Results are silently truncated at DB_TOP_K=5 with no indication in the tool output whether more results exist. Users and agents cannot iterate to get complete result sets.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Weak error handling. Functions return bare strings like 'No matching endpoints found.' or 'Endpoint not found: GET /path'. No actionable recovery guidance (e.g., 'Try a broader search' or 'Call list_tags() to see available categories').
Duplicate tool intent. search_endpoints and search_schemas both search a database by keyword using identical patterns. The distinction is unclear to an LLM, consider unifying into a single search_api tool with an optional 'search_in' parameter or clarify which is preferred for common tasks.
Parameter format constraints missing. 'path' in get_endpoint_details should specify it must match /V1/.* pattern. 'query' in search_schemas should enforce a maximum length (currently unbounded). 'schema_name' in get_schema should document the lowercase-hyphen convention.
Descriptions lack actionable context. 'Use SHORT keyword queries' is stated but not enforced (no parameter constraint). 'You can optionally filter' does not explain the precedence or behavior when both filters are applied. Descriptions read like API docs, not LLM prompt engineering.