Loading ToolBench
Knowledge base management system with MCP server and CLI - Security frameworks ready for enterprise deployment
KB-MCP has 28 tools with significant quality gaps. While schemas are present for most tools, there are critical issues: (1) Tool name collisions, kb_read, kb_list, kb_update, kb_delete, kb_search, kb_create appear multiple times with different behaviors, violating the single-responsibility principle and creating LLM confusion. (2) Inconsistent schema rigor, some tools like kb_read have regex patterns and helpful descriptions, while others like kb_help lack meaningful parameter validation. (3) Parameter descriptions vary widely in quality; many are under 50 chars and lack guidance on constraints or expected formats. (4) Output schemas are completely undocumented, no tool shows what fields are returned, making it impossible for LLMs to plan downstream calls or extract specific data. (5) Error handling is minimal; no tool provides recovery guidance or categorizes errors. (6) Several code analysis tools (analyze_codebase, find_function_calls, get_class_hierarchy, impact_analysis, find_similar_code) have sparse parameter descriptions that lack type constraints or format guidance. Transport is STDIO-only, which is a hard architectural limitation regardless of definition quality.
Analyze a codebase to extract entities, relationships, and generate insights
Find all functions that call a specific function and trace call relationships
Find code patterns similar to a given code snippet using semantic search
Get the inheritance hierarchy for a class, including parents and children
Analyze the impact of changing a code entity (function, class, etc.)
Query audit logs
Check the health status of the current storage backend
Tool name collisions, kb_read, kb_list, kb_update, kb_delete, kb_search, kb_create defined twice with different behaviors. Violates single-responsibility principle. LLMs will conflate these tools and invoke wrong variants.
Output schemas are completely missing, no tool documents what fields are returned. LLMs cannot plan downstream calls, extract specific data, or validate responses. Violates pattern:response-shaper.
Parameter descriptions lack specificity, many are under 50 characters, missing constraints, format guidance, and examples. E.g., kb_info has empty properties object with no parameter guidance. Violates pattern:tool-description.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 56 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 28 | - | v1 |
Get information about the current storage backend and available options
Switch between storage backends (filesystem or graph)
Create a backup of the knowledge base
Create a new file in the knowledge base (alias for kb_update)
Create a new file in the knowledge base
Delete a file from the Script knowledge base
Delete a file from the knowledge base
Execute a custom graph query using Cypher syntax (graph backend only)
Get help and usage information
Get information about the knowledge base
Get the current known issues in the Script language implementation
List files and directories in the Script knowledge base
List files and directories in the knowledge base
Read a file from the knowledge base
Read a file from the Script language knowledge base
Search for content in Script knowledge base files
Search for content in knowledge base files
Perform semantic search using vector embeddings (graph backend only)
Get the current implementation status of the Script language
Create or update a file in the Script knowledge base
Update an existing file in the knowledge base
Code analysis tools (analyze_codebase, find_function_calls, get_class_hierarchy, impact_analysis, find_similar_code) have sparse parameter descriptions. 'languages' array, 'maxDepth' numeric, and 'threshold' float parameters lack constraints, valid ranges, or units. Violates pattern:constrained-input.
No error handling or recovery guidance, tools provide no recovery hints, error categorization, or actionable next steps. Violates pattern:recovery-guide.
Destructive operations (kb_delete, kb_backend_switch with migrate_data=true) lack confirmation or dry-run support. Agents can trigger irreversible data loss without safeguards. Violates pattern:confirmation-request.
Duplicate tool semantics, kb_create and kb_update with merge=false both write files. No canonical 'create' behavior. Design should offer create_file (new), update_file (existing), upsert_file (either) to clarify intent.
Pattern constraints in schemas not documented in descriptions, kb_read, kb_list, kb_create, kb_update all use regex patterns (^[a-zA-Z0-9\-_/]+\.(md|markdown)$) but descriptions do not explicitly state file extension and character restrictions. LLMs cannot read JSON Schema patterns, they rely on text.
Code analysis tools accept path and filePath parameters inconsistently. Some use 'path', others 'filePath'. If downstream tools consume these, naming mismatch forces LLMs to reason about field mappings, increasing error risk. Violates pattern:tool-chain.
kb_help tool only provides generic enum values (getting-started, security, search, markdown, permissions) with no explanation of when or why to call it. Violates pattern:tool-description.