Loading ToolBench
Self-hosted MCP server for searchable documentation (SQLite default, PostgreSQL optional)
gnosis-mcp demonstrates solid definition quality with 11 well-named, read-heavy tools supplemented by 3 write tools. Tool names follow verb_noun convention (get_context, search_docs, list_docs, etc.), and descriptions are substantive (avg 180+ chars), meeting the 10 - 1024 character baseline. All tools have input schemas with type definitions. However, several parameters lack explicit descriptions, and output schemas are not documented in the source. Error handling is present but lacks actionable recovery guidance (ToolError raised without recovery hints). Parameters on write tools are reasonably constrained but missing enums for 'relation_type' in get_related. The server properly gates write tools behind GNOSIS_MCP_WRITABLE environment variable, demonstrating good permission-based design. Overall strong naming and descriptions, but schema completeness and error guidance need improvement.
Delete a document and its chunks (write tool, requires GNOSIS_MCP_WRITABLE=true). Returns counts of chunks and links deleted.
Get corpus context: most-accessed documents, statistics, and optional topic-guided orientation. Returns most-accessed docs, document count, chunk count, categories, and corpus statistics.
Retrieve a complete document by file path. Returns all chunks ordered by position, with title, category, audience, and tags metadata.
Report corpus topology and link graph statistics. Returns node counts, edge counts by relation type, hubs (highly connected documents), and orphans (isolated documents).
Walk the link graph to find related documents. Returns documents connected via frontmatter relations, body links, or git co-change history. Supports depth-first traversal up to 3 hops.
List all document categories with the count of documents in each.
Output schemas not documented in source. Tool descriptions state what is returned (e.g., 'Returns ranked document previews with file paths, scores, and content highlights') but the actual JSON response structure is not formally specified. LLMs cannot reliably extract nested fields or plan downstream calls without explicit output schema documentation.
get_related parameter 'relation_type' accepts free-form strings ('relates_to', 'content_link', etc.) without an explicit enum. This invites hallucinated values. Should be constrained to ['relates_to', 'content_link', 'co_change'] or similar.
Error handling raises ToolError without recovery guidance. When a document is not found or git history fails, the error message should suggest alternative actions (e.g., 'Document not found. Try search_docs() to locate it by keyword.').
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 74 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 59 | - | v1 |
List all indexed documents with title, category, and chunk count metadata.
Search documentation by keyword or semantic query. Returns ranked document previews with file paths, scores, and content highlights. Supports BM25 keyword search, vector semantic search, and hybrid search with MMR reranking.
Search git commit history to answer 'when and why did this change' questions. Returns commits that touch the given file path or query-related patterns.
Update metadata on all chunks of a document (write tool, requires GNOSIS_MCP_WRITABLE=true). Updates title, category, audience, and tags. Returns rows affected.
Insert or replace a document's chunks (write tool, requires GNOSIS_MCP_WRITABLE=true). Returns number of chunks written.
Write tools (upsert_doc, delete_doc, update_metadata) lack confirmation/dry-run support. Destructive operations like delete_doc should offer a dry-run mode or require explicit confirmation to prevent accidental data loss.
search_docs 'limit' parameter lacks bounds documentation. Should specify min/max (e.g., 'integer, 1 - 100, default 5') to prevent LLMs from requesting thousands of results.