Loading ToolBench
MediaWiki MCP has clear verb-based naming (create_, edit_, delete_, get_, search_) and complete Zod schemas with type definitions. All 5 tools have descriptions and input parameters are typed. However, descriptions are generic (avg ~80 chars) and lack LLM-optimized guidance on WHEN to use each tool vs alternatives. Output schemas are not documented, responses are inferred from code (text content + isError flag). Error handling returns plain error messages without recovery guidance. No parameter constraints (enums, ranges) despite several parameters accepting free-form strings. Missing idempotency hints for write operations.
Create a new article on the MediaWiki instance. Will fail if the article already exists.
Delete an article from the MediaWiki instance. Requires appropriate permissions.
Edit an existing article on the MediaWiki instance. Creates the article if it does not already exist.
Get the current content of an article from the MediaWiki instance.
Search for articles on the MediaWiki instance.
Output schemas not documented. Code shows responses return {content: [{type: 'text', text: string}], isError?: boolean}, but LLMs cannot infer this structure. Agents cannot plan downstream calls or extract structured data.
Error responses lack recovery guidance. Errors return plain text (e.g. 'Error: Login failed: unknown error') without suggesting next steps. LLMs cannot self-correct or know whether to retry.
Descriptions are generic and lack LLM-optimized guidance. E.g. 'Create a new article' (24 chars) does not explain WHEN to use create_article vs edit_article (which also creates if missing). Baseline for A+ tools is 50-200 chars with clear differentiation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
No parameter constraints. 'title' and 'query' accept free-form strings with no length limits, format validation, or enum constraints. LLMs may pass invalid MediaWiki titles (e.g. with forbidden characters).
Destructive operations (delete_article) lack confirmation or dry-run support. Agents can irreversibly delete articles without a safety gate. No idempotency hints for write operations.