Loading ToolBench
A simple Model Context Protocol (MCP) server that integrates with Notion's API to manage my personal todo list.
The notion-mcp server has 3 tools with basic definitions but significant quality gaps. Tool names are action-oriented (add_todo, show_all_todos, complete_todo) which is positive. However, descriptions are very short (9-30 characters), falling well below the 10-1024 character baseline and the critical 20-character minimum for actionable descriptions. Input schemas are present but parameter descriptions are minimal or missing. The server lacks documentation of output schemas, error guidance, and recovery instructions. Tool definitions appear directly visible in src/server.py, so no inference penalty applies. The implementation uses dotenv for secrets (good practice) but the descriptions do not convey prerequisites or failure modes.
Add a new todo item
Mark a todo item as completed.
Show all todo items from Notion.
Tool descriptions are critically short (9-30 characters). 'Add a new todo item', 'Show all todo items from Notion.', 'Mark a todo item as completed.', all under 30 chars. LLMs need 50-200 character descriptions to disambiguate when/why to select a tool.
No output schemas documented. The LLM cannot predict what add_todo returns, whether show_all_todos yields a list or string, or what fields complete_todo exposes. Without documented output types, agents cannot plan downstream calls or extract data reliably.
Parameter 'task' in add_todo has a one-line description ('The todo task description') but no constraints, length limits, format hints, or guidance on special characters. Parameter 'task_id' in complete_todo similarly lacks detail, is it a UUID? An integer? A slug? The description must specify the format explicitly.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 42 | - | v1 |
No error handling or recovery guidance. When a Notion API call fails (invalid token, database not found, rate limit), the handler catches httpx.HTTPError and returns a generic error message. LLMs receive 'Error adding todo: <raw exception>' with no actionable next steps. Should include 'Check your NOTION_TOKEN and DATABASE_ID in .env', 'Retry in 60 seconds', or 'Call show_all_todos to verify access'.
show_all_todos has no input parameters, so its schema is effectively empty. The tool description doesn't mention prerequisites (e.g., 'Requires PAGE_ID env var set', 'May be slow for large page with 1000+ todos'). Agents have no context for when/why to call it.
add_todo and complete_todo are WRITE operations (risk=WRITE) but their descriptions do not declare this explicitly. 'Mark a todo item as completed' implies a side effect, but 'Add a new todo item' should say 'Creates a new todo in Notion' to be clear. Agents need to know which calls are idempotent and which can be retried.
No pagination or result limits documented. show_all_todos fetches all todos on a page without pagination or count limits. If a user has 1000+ todos on a single Notion page, the LLM receives a massive response that wastes tokens and may exhaust context. Should implement limit/offset parameters and document them.