Loading ToolBench
Autonomous Repository Intelligence engine with web UI and MCP server. Unified semantic code understanding, local RAG, and agent working memory.
cntx-ui presents a mixed profile: several tools have minimal descriptions (under 20 chars), incomplete schemas, and missing parameter annotations. While 13 tools are registered and visible in lib/mcp-server.ts and lib/agent-tools.ts, most lack the depth required for confident LLM reasoning. Tool names are reasonably verb-forward (agent/discover, agent/query, list_bundles, read_file), but descriptions are often vague or incomplete. Parameter schemas exist for most tools but many lack type information on nested properties and descriptions for individual parameters. The artifacts/* tools (list, get_openapi, get_navigation, summarize) have particularly sparse definitions. executeCommand is an IRREVERSIBLE operation with no dry-run or confirmation mechanism, violating pattern:confirmation-request. Error handling is minimal across the board, no recovery guidance, no actionable messages, no error classification.
Discovery Mode: Comprehensive architectural overview.
Investigation Mode: Suggest integration points.
Query Mode: Answer technical questions.
Get Navigation artifact payload (summary + parsed manifest).
Get OpenAPI artifact payload (summary + parsed content when JSON).
List normalized project artifacts (OpenAPI and Navigation manifests).
Get compact summaries for OpenAPI and Navigation artifacts.
Descriptions under 20 characters on 7 tools (artifacts/list, artifacts/get_openapi, artifacts/get_navigation, artifacts/summarize, list_bundles), LLMs cannot infer selection intent from 'List normalized project artifacts' or '25-char summaries.
Parameter descriptions missing across all 13 tools. E.g., 'scope' in agent/discover has no description; 'dirPath' in listFiles has no description; 'options' in searchChunks is an object with no property descriptions. LLMs cannot reason about what values to pass.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Execute a shell command within the project context
Get technical metadata for a file
List files in a directory, respecting ignore patterns
List all project bundles.
Read a file.
Search for code chunks based on semantic similarity or text matching
executeCommand is IRREVERSIBLE but has no confirmation step, dry-run, or undo mechanism. Agents can destructively run 'rm -rf /' or equivalent without warning. Violates pattern:confirmation-request.
Output schemas are not documented. Tools return unstructured responses (e.g., searchChunks returns SemanticChunk[] but no schema is defined for LLM consumption). LLMs cannot plan downstream calls without knowing response structure.
No error handling guidance. Error responses return generic { error: error.message } without recovery hints. E.g., 'File not found' does not suggest searching or listing alternatives. Violates pattern:recovery-guide.
Parameter constraints are missing. E.g., searchChunks accepts maxResults but no min/max bounds are documented; executeCommand accepts any string command with no validation or sandboxing hints.
Nested parameter schema incomplete. artifacts/* tools return { options: { maxResults, type } } but neither maxResults nor type are documented as to type, range, or valid enum values.
No pagination support or result limits documented. searchChunks defaults to maxResults=10 but no upper bound is enforced; large codebase searches could return thousands of chunks, exhausting context.
Tool composition is unclear. Tools like agent/discover and agent/query are vague about what they return and when to use one vs another. Descriptions do not distinguish their roles or suggest a calling sequence.