Loading ToolBench
An MCP server providing knowledge management and DLX orchestration capabilities with OneDrive integration, vector search, and dynamic tool creation
Knowledge-MCP server presents mixed quality. All 13 tools have explicit definitions with names, descriptions, and input schemas visible in src/handlers/dlx/tools.ts and src/handlers/knowledge/tools.ts. However, the server suffers from several systematic issues: (1) output schemas are NOT documented anywhere, tools return results but the response structure is never formally specified, forcing LLMs to reason about response shapes without guidance; (2) parameter descriptions, while present, are often terse and lack actionable constraints (no min/max for numeric fields, no examples of valid enum values, no guidance on when parameters are required vs optional beyond JSON Schema); (3) error handling is not visible in the tool definitions, no recovery guidance, no indication of retryable vs fatal errors; (4) three tools (use-orchestration, use-connection, use-knowledge-source) create NEW tools dynamically, which is architectural pattern that makes composition and validation hard to reason about; (5) destructive operations (delete-knowledge-source) lack confirmation/dry-run patterns; (6) tool naming is generally clear (verb_noun), but some tools are generic (search, use-*). The annotations field (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) is present on dlx tools but MISSING from knowledge tools, inconsistent coverage. Schema quality is moderate: properties are typed and described, but required fields are not always explicit in all tools.
Add a new knowledge source to the system
Delete a knowledge source and all its data
List all connections
List available knowledge sources, optionally filtered by name.
List all orchestrations
Refresh a knowledge source by re-ingesting its original content from the source (e.g., OneDrive).
Output schemas are entirely undocumented. No tool in the codebase declares what fields are in the response, their types, or structure. LLMs cannot plan downstream tool calls or extract specific data without guessing response shapes.
Inconsistent annotation coverage. DLX tools (list-orchestrations, use-orchestration, list-connections, use-connection) have readOnlyHint, destructiveHint, idempotentHint, openWorldHint annotations, but knowledge tools (add-knowledge, search, use-knowledge-source, etc.) have annotations on some but missing on search-onedrive-files and retrieve-onedrive-file. Tool annotations are production-critical for agent decision-making.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | D | 50 | - | v1 |
Retrieve and extract text from a file in the user's OneDrive account.
Search across all knowledge sources for relevant information
Search for files in the user's OneDrive account.
Share a knowledge source with another user
Create tools for a specific connection based on its capabilities
Create a specialized tool for a specific knowledge source. To use this, first call the list-knowledge-sources tool to retrieve available sources and their details. Then, provide a user-friendly toolName (e.g., 'search-acme-co') and a toolDescription (e.g., 'Use this tool to find information on Acme Co.') that clearly describe the tool's purpose.
Create a tool for triggering a specific orchestration
Three tools (use-orchestration, use-connection, use-knowledge-source) create new tools dynamically by returning handler configs. This makes composition and static validation impossible, the LLM cannot know what tools exist until runtime. No discovery mechanism is documented.
Destructive operations (delete-knowledge-source) lack confirmation or dry-run pattern. The tool accepts only 'name' parameter, no safety guard prevents accidental deletion. No recovery guidance documented.
Parameter descriptions lack actionable constraints. 'limit' in list-knowledge-sources defaults to 10 but has no min/max documented. 'query' in search has no length guidance. 'accessLevel' in share-knowledge-source declares enum values in schema but not in description text (LLMs rely on descriptions, not schema inspection).
No error handling guidance visible. Tools do not document recovery paths (e.g., 'If search returns no results, try broadening the query' or 'If delete-knowledge-source fails due to permissions, contact admin'). Agents cannot self-recover.
Generic tool names reduce clarity. 'search' is ambiguous, it could search knowledge, OneDrive files, or orchestrations. 'use-orchestration', 'use-connection', 'use-knowledge-source' all follow the same pattern but signal different intent. Names should reflect the resource type (search-knowledge, search-onedrive, trigger-orchestration).
Pagination not documented for list operations. list-knowledge-sources accepts 'limit' but does not specify if there is a 'next_cursor' field in responses, or how to fetch the next page. Without pagination docs, agents cannot reliably iterate large result sets.
Parameter 'dataSchema' in use-orchestration is type 'object' with vague description. No schema validation rules are documented. LLMs will struggle to construct valid dataSchema payloads without examples or formal constraints.