Loading ToolBench
A Model Context Protocol server providing tools to interact with Roam Research's functionality programmatically via LLMs
The server has 9 tools with generally clear naming and adequate descriptions, but exhibits inconsistent schema completeness and several parameter description gaps. Most tools follow verb_noun convention (get_page, create_block, search_by_text), which is positive. However, 3 of 9 tools have significant parameter documentation gaps, particularly semantic_search, which has 8 optional parameters with minimal constraint guidance. Descriptions range from adequate (44-185 chars) to good (194 chars), but several lack detail about when to use a tool vs. alternatives (e.g., search_by_text vs. semantic_search distinction is unclear). Error handling is mentioned but not visible in schema definitions. No input validation constraints are documented in parameter descriptions (e.g., 'days: range 1-30' is stated in schema but not in description). Security patterns (secret injection, audit logging) are not evident in the provided code. Overall, this is a competent mid-tier server that would benefit from tighter parameter documentation and clarified error recovery paths.
Create a new block in a Roam page.
Get the last N days of daily notes with their linked references for context.
Get all blocks that reference a specific page.
Get the context around a specific block, including parent and sibling blocks.
Retrieve a page's content in clean markdown format. This uses the Roam API to fetch the page content and converts it to a well-formatted markdown representation, suitable for display or further processing.
Execute a raw Roam datalog query directly.
create_block description lacks clarity on required parameters and state-changing semantics. Description says 'Create a new block in a Roam page' (44 chars) but does not explain what happens, when to use it, or prerequisites (page must exist, etc.). Violates pattern:tool-description requirement for 10-1024 char descriptions with WHAT/WHEN/WHY.
semantic_search has 8 optional parameters (include_context, include_children, children_limit, include_backlink_count, include_siblings, sibling_count) with no constraint descriptions. Parameter 'children_limit' defaults to 3 but lacks guidance on valid range. Parameter 'sibling_count' defaults to 1 with no upper bound documented. Violates pattern:constrained-input and review:param-validation-rules.
No distinction between search_by_text and semantic_search in their descriptions. Both search for content, but descriptions do not explain when to use text search vs. semantic (vector) search. LLM cannot disambiguate without this guidance. Violates pattern:tool-chain clarity.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 53 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 21 | - | v1 |
Search for blocks containing specific text in Roam.
Search for semantically similar content in the Roam graph using vector embeddings.
Build or update the vector index for semantic search.
raw_query tool is dangerously generic. Description says 'Execute a raw Roam datalog query directly' (43 chars) but does not warn that this is a power-user tool, does not require authorization checks, and has no documented error recovery if the query is malformed. Violates pattern:secret-injection and pattern:permission-gate.
No output schema documentation visible for any tool. Rubric requires 'Document the output schema. LLMs need to know what fields to expect.' For example, semantic_search likely returns blocks with uid, content, metadata, but the response structure is not documented. Violates pattern:tool and pattern:response-shaper.
daily_context accepts 'days' parameter with constraint '1-30' visible in schema but NOT documented in the parameter description. Rubric requires 'state format, range, and allowed values directly in the parameter description.' This forces LLMs to infer the constraint from schema inspection rather than reading the description. Violates review:param-validation-rules.
get_backlinks has 'limit' parameter defaulting to 20, but no upper bound documented. Rubric requires 'limit parameters 1-100 ranges.' No guidance on what happens if limit is omitted or very large. Violates review:param-validation-rules.
No error handling guidance in any tool description. Rubric requires error responses to 'tell the LLM what to do next.' No mention of retry strategies, user-fixable errors, or fatal conditions. Violates pattern:recovery-guide.
sync_index tool does not document what fields are indexed, what vectors are created, or what the return value looks like. 'Build or update the vector index' is vague. Does this return a count of indexed items? A success status? This breaks pattern:response-shaper.
No indication of idempotency for any tool. Rubric requires 'Make tools produce the same result on repeated calls.' create_block is clearly NOT idempotent (creates duplicates on retry), but this is not documented. Violates pattern:idempotent-operation.