Loading ToolBench
Shared team memory for AI coding agents — MCP server and CLI for persistent, queryable knowledge across sessions and teammates
Knowit demonstrates solid definition quality with well-structured schemas, clear naming conventions, and comprehensive parameter descriptions. All 9 tools are explicitly registered with Zod schemas in src/server/tools.ts. Tool names follow verb_noun patterns (list_sources, search_knowledge, store_knowledge, etc.). Descriptions are present and informative (average ~150 chars, within 10-1024 baseline). However, there are gaps in output schema documentation, responses are returned as text JSON rather than being formally documented in the tool definitions. Error handling lacks recovery guidance (no actionable error messages visible). Confidence score parameter (0-1 range) is well-constrained. Security consideration: credentials appear to be handled server-side (no visible secret parameters), which is correct. Tool composition is strong, tools are single-purpose and chainable (search_knowledge → get_knowledge two-phase retrieval pattern). Some parameter descriptions could be more LLM-optimized (e.g., resolve_context says 'Resolve implementation context' but doesn't explain when to use it vs search_knowledge).
Batch store multiple knowledge items from this coding session. Existing entries with the same title, type, scope, repo, and domain are updated rather than duplicated. Call this at the end of a session to persist decisions, patterns, and conventions discovered during the session.
Connect a first-class source provider such as local or notion. This is the preferred product-facing source onboarding flow.
Fetch full content for one or more knowledge entries by ID. Use after search_knowledge or resolve_context to retrieve the complete content of relevant entries (Phase 2 of tiered retrieval).
List Knowit sources, including local and external MCP-backed sources.
Register an external MCP server as a Knowit source using explicit tool mappings.
Resolve implementation context across one or more Knowit sources. Returns title, summary, and metadata — not full content. Call get_knowledge with the returned IDs to fetch full content for relevant entries.
Output schemas not formally documented in tool definitions. Responses are serialized to text JSON (asTextContent helper) rather than returning structured ToolResultBlockParam with explicit result schema. This forces LLMs to parse unstructured responses and makes chaining less reliable.
Error handling lacks actionable recovery guidance. No visible try-catch blocks or recovery hints in tool definitions. Example: if search_knowledge returns 0 results, the LLM has no guidance to widen filters or call resolve_context instead. Raw errors provide no 'what to do next' context.
resolve_context description is ambiguous. 'Resolve implementation context across one or more Knowit sources' does not explain when to use it vs search_knowledge. LLMs cannot distinguish between semantic search and context resolution without clearer guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 75 | <=2025-11-25 | v2 |
Determine whether Knowit should handle a read/write directly or route the agent to an external provider MCP next. Use this when the user mentions Knowit but not the downstream provider.
Search knowledge across one or more Knowit sources. Returns title, summary, and metadata — not full content. Call get_knowledge with the returned IDs to fetch full content for relevant entries.
Store knowledge in the selected Knowit source. This can target the local store or an external MCP-backed source.
resolve_source_action tool name lacks clarity. 'resolve_source_action' does not self-document what action is being resolved (read vs write routing decision). Consider rename to route_to_source or determine_source_handler for LLM clarity.
No confirmation/dry-run support for destructive writes. store_knowledge and capture_session_learnings modify state but have no preview or confirmation step. An agent mistake could overwrite entries or create duplicates.
Chaining context missing. search_knowledge returns 'title, summary, and metadata, not full content' and instructs 'Call get_knowledge with the returned IDs' but does not document what fields get_knowledge expects. Response must include ID structure that get_knowledge.ids array accepts.