Loading ToolBench
Working memory for AI companions — mutable markdown notes via MCP
The folio-mcp server defines 2 tools with reasonable parameter schemas and descriptions, but several quality issues prevent a higher score. The 'folio' tool has a comprehensive parameter list with proper type annotations and detailed descriptions covering multiple action modes. The 'search' tool is simpler but well-structured. However, neither tool explicitly documents output schemas, error handling is minimal, and descriptions lack actionable guidance for LLM selection. The long, example-heavy descriptions for 'folio' risk LLMs reusing literal example values. Tool naming is acceptable but not verb-prefixed in the strict sense (e.g., 'folio' alone lacks a clear action verb).
Markdown notes in folders with tags and versioning. Use prepend for running logs, section to refresh one heading body, replace to rewrite. Examples: Create: action='create', path='journal/2026-02-23.md', title='Sunday', tags=['journal'], content='## Morning\nCycled in -6°C...\n## Evening\n...' Read section: action='read', path='plans/cabin-weekend.md', section='Weekend Cabin Trip' Table of Contents: action='toc', path='huge-note.md' (Returns headings and their sizes) Append: action='update', path='watching/watchlist.md', mode='append', content='\n- The Terror S1 — slow-burn arctic horror' Section: action='update', path='projects/companion.md', mode='section', target='Status', content='Folio MCP complete. Testing phase.' Retag: action='update', path='shows/dark.md', tags=['favorite', 'pinned'] Move: action='move', path='notes/pizza.md', destination='food/pizza.md' List: action='list', folder='journal', page=2
Full-text search across all notes with fuzzy matching, filtering by tags/folder, and relevance ranking.
Tool naming lacks clear action verbs. 'folio' and 'search' are nouns/generic verbs; LLMs benefit from explicit verb_noun conventions like 'manage_note' or 'search_notes'. Current names are ambiguous about what action is performed.
Output schemas not documented. Both tools return structured dicts/responses but no formal schema is visible in docstrings or comments. LLMs cannot plan downstream calls without knowing what fields to expect. For example, 'folio' with action='create' returns a response dict with 'status', 'path', 'title', etc., but this is not documented in the tool description.
Error handling is bare minimum. The folio tool returns dict with 'error' keys (e.g., 'path is required for create'), but does not guide recovery. No actionable error messages like 'Use action=list to discover available notes.' Exception handling is present but responses lack next-step guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 41 | - | v1 |
folio tool description includes detailed examples with literal values (e.g., 'journal/2026-02-23.md', 'Sunday', '## Morning\nCycled in -6°C'). LLMs tend to reuse example values directly rather than adapting them to the actual context, leading to incorrect calls.
Parameter 'tags' in folio accepts 'any' type with description 'Can be a list or comma-separated string'. This type ambiguity is error-prone. Code normalizes strings to lists internally, but the schema should declare a union of string|array, not 'any'.
No explicit documentation of parameter relationships. For example, 'action' parameter determines which other parameters are required/valid (path required for read/create/delete/move, folder required for list, etc.). This dependency is not stated in parameter descriptions, forcing LLMs to infer from context.
folio action='undo' is mentioned in the action description but not documented in the docstring examples or behavior. Undocumented features confuse LLMs about what is actually available.
search tool does not document pagination behavior clearly. 'limit' defaults to 10 and 'offset' defaults to 0, but it is not stated whether the tool returns a total count or next_cursor for chaining results.