Loading ToolBench
MCP server exposing Shared Context Memory Store (SCMS) as tools via FastMCP with HTTP transport and optional OAuth authentication
Cairn exhibits significant definition quality gaps. Tool naming is generally verb-based (search, store, create, list, update, archive, queue, check, get, test) but several tools are duplicated with inconsistent schemas (scms_search appears twice with different parameter types; scms_store appears twice; create_project and update_project appear twice with conflicting input schemas). Parameter descriptions are present but often generic. Many tools lack output schema documentation. Error handling is minimal, no recovery guidance or actionable error messages visible in the code. The metatool.py excerpt shows basic validation but no structured error classification. Overall, the server prioritizes functionality over LLM-friendly definition patterns.
Archive (soft-delete) a project. Sets status to 'archived'. The project and all linked memories/decisions are preserved but hidden from active views. Reversible via update_project(name, status='active').
Archive (soft-delete) a project. Sets status to 'archived'. The project and all linked memories/decisions are preserved. Reversible via update_project(name, status='active').
Check the task queue status and list tasks.
Create a new project in the SCMS.
Create a new project in the SCMS knowledge base.
Create a new tool by writing Python code, testing it in the Docker sandbox, and submitting it for human approval.
Duplicate tool definitions with inconsistent schemas. scms_search, scms_store, create_project, and update_project each appear twice with different parameter types and descriptions. This creates ambiguity for LLMs and violates the single-responsibility principle.
Missing output schema documentation. No tool explicitly documents what fields are returned, their types, or structure. LLMs cannot plan downstream calls or extract required data without knowing response shape.
Inconsistent parameter types across duplicate tools. scms_store v1 uses array for tags; v2 uses comma-separated string. create_project v1 uses object for metadata; v2 uses JSON string. This forces LLMs to guess which variant to use.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 61 | <=2025-11-25 | v2 |
Get full context for a project: metadata, recent memories, and decisions.
Get the result of a specific task by ID.
List all custom tools that are pending human approval. Shows tool name, description, creation date, and test results.
List all projects in the SCMS.
Add a task to the cairn agent's task queue.
Search the Shared Context Memory Store for relevant memories and context.
Search memories in the SCMS knowledge base using semantic similarity.
Store a new memory in the Shared Context Memory Store.
Store a new memory in the SCMS knowledge base.
Test an existing custom tool with new test code in the Docker sandbox.
Update an existing project's fields. Only provided fields are changed.
Update an existing project's fields. Only non-empty fields are changed.
No error handling guidance. Code shows basic validation (e.g., _validate_tool_name) but returns generic error strings. No recovery hints, categorization of retryable vs fatal errors, or actionable next steps for LLMs.
list_projects has minimal description (50 chars) and no documented output structure. LLMs cannot determine what fields are returned or how to filter/sort results.