Loading ToolBench
Knowledge base MCP server providing semantic search, structured queries, rule-set access, data ingestion, evaluation/feedback and snapshots for enterprise AI
Server has 9 tools with basic schemas and descriptions, but significant gaps prevent higher scoring. Most tool descriptions are present (10 - 100 chars) but lack actionable context for LLM selection. Parameter schemas are partially defined (types present, but many lack descriptions or constraints). No output schemas documented. Error handling is absent, no recovery guidance, categorization, or actionable error messages. Security concerns: PII token handling in search_knowledge is under-specified; manage_policies lacks permission gates. Composition is reasonable (single-responsibility tools), but missing pagination, idempotency hints, and chaining IDs in responses.
Create a snapshot of the current knowledge base state
Retrieve evaluation statistics and feedback metrics
Query the knowledge graph with actions like find_node, get_neighbors, get_path, list_nodes
Ingest documents into the knowledge base with PII scanning and classification
List all available snapshots
Manage OPA policies for role-based access control and data filtering
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does search_knowledge return result_id, score, text, metadata?). Violates pattern:tool and pattern:response-shaper.
graph_query and manage_policies accept free-form 'action' and 'policy_action' strings without enums. LLMs will hallucinate invalid actions (e.g., 'find_node_recursive', 'update_policy_with_merge'). Should declare enums: action ∈ {find_node, get_neighbors, get_path, list_nodes}; policy_action ∈ {get, set, delete, list}.
No error handling guidance. Tools lack recovery hints, error categorization, or actionable messages. E.g., if ingest_data fails due to PII detection, does the LLM retry, ask the user, or abort? No guidance. Violates pattern:recovery-guide and pattern:error-classification.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Query structured data from datasets with optional filtering and limits
Semantic search across the knowledge base with optional PII vault access and result summarization
Submit feedback on search results for evaluation and model improvement
Security: manage_policies (set, delete actions) lacks permission gates. No indication that destructive policy changes require authorization. Violates pattern:permission-gate. Also, pii_access_token in search_knowledge is under-specified, no description of token format, validation, or scope.
Parameter descriptions are sparse or missing. E.g., query_data 'conditions' param has no format guidance (SQL WHERE clause? JSON filter? key-value pairs?). ingest_data 'classification' has no enum or valid values. Violates pattern:tool-description.