Loading ToolBench
An MCP server that integrates LangGraph agents with Notion workspace management, enabling multi-agent orchestration for searching, creating, updating, and managing Notion pages, databases, comments, and user information.
This MCP server exposes 13 Notion tools with reasonable naming conventions (verb_noun style) and basic descriptions. However, there are significant quality gaps: (1) Parameters lack detailed type information and constraints, most accept generic strings or objects without enum constraints, format specifications, or validation guidance. (2) Output schemas are not documented in the source code, we cannot verify what fields tools return or how to chain them together. (3) Descriptions, while present, are often generic (e.g., 'Fetch and retrieve content from a Notion page' is redundant) and do not explain WHEN to use each tool or what downstream operations are enabled. (4) Error handling is not visible in the tool definitions, no recovery guidance, retry policies, or actionable error messages. (5) The server appears to be a custom agent orchestrator wrapping LangChain + LangGraph + Notion API, not a proper MCP server with formal tool registration. Tool definitions are inferred from mcp/mcp_client.py, which we cannot fully audit here. Without seeing explicit MCP protocol registration (e.g., ToolDefinition objects with inputSchema and description fields in valid JSON Schema), per-tool scores are capped at 50. The overall Definition Quality score reflects the average gap: reasonable naming and basic presence of descriptions, but pervasive absence of detailed parameter types, output schemas, and error guidance.
Create a comment on a Notion page
Create a new Notion database with specified properties
Create a new page in Notion with properties and optional child blocks
Duplicate an existing Notion page (async operation)
Fetch and retrieve content from a Notion page
Retrieve all comments from a Notion page
Retrieve bot user and workspace information
Output schemas are not documented. Tools like 'search', 'fetch', 'create-pages' return data without visible schema definitions, making it impossible for LLMs to plan downstream tool chains or extract required fields (e.g., page_id from search results to pass to fetch or update-page).
Parameters 'properties' and 'children' in create-pages and update-page accept generic objects without structure, field names, or examples. No enum constraints, format specs, or validation rules. Causes LLMs to guess or hallucinate Notion API structure.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 38 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 22 | - | v1 |
Retrieve information for a specific user by ID
Retrieve all users in the Notion workspace
Move a Notion page to a new parent location
Search across Notion workspace and integrated tools
Update an existing Notion database's title and properties
Update an existing Notion page's properties and/or append child blocks
No error handling guidance visible. Tools do not document what errors can occur, whether they are retryable, or what recovery steps an LLM should take. For example, 'move-pages' could fail if the parent page is invalid, but the tool definition provides no guidance.
Descriptions are generic and redundant. E.g., 'Fetch and retrieve content from a Notion page' repeats 'fetch' and 'retrieve'. Descriptions do not explain WHEN to use a tool (vs a similar one) or what downstream operations are enabled.
Parameters like 'filter' (in search) and 'parent' (in create-pages, create-database) accept both string and object types without clear guidance on when to use each. This ambiguity forces LLMs to reason about type choice, increasing errors.
Tool definitions are inferred from code (mcp/mcp_client.py) and not directly visible in a formal MCP protocol registration file. Without explicit ToolDefinition objects in the MCP spec format, we cannot verify compliance with JSON Schema, proper description fields, or protocol-level metadata.
No pagination support visible. Tools like 'get-users' and 'get-comments' do not advertise limit, offset, or cursor parameters, risking context window exhaustion if the Notion workspace has many users or comments.
Destructive tools like 'duplicate-page' (async operation), 'move-pages', 'update-page', and 'create-pages' do not declare idempotency or provide confirmation/dry-run mechanisms. If an agent retries a failed create-pages call, it may create duplicate pages.