Loading ToolBench
Local-first knowledge graph that watches your project files, extracts entities and relationships via LLMs, and lets you query across projects in natural language.
Cortex MCP Server has 11 tools with reasonable descriptions and basic schema structure, but suffers from significant gaps in parameter documentation, output schemas, error handling guidance, and adherence to composition patterns. Naming is verb-based but three tools are marked deprecated, suggesting API surface design issues. Parameter descriptions exist but lack specificity around constraints, formats, and ranges. Output schemas are not documented in the visible code. Error handling is minimal, no recovery guidance or actionable error messages evident. The server prioritizes breadth (11 tools) over depth of quality, landing squarely in the fair-to-poor range (C/D territory).
Register a new project directory for Cortex to watch and index. The path must exist and be a directory.
Ask the Cortex knowledge graph any question in natural language. Returns an LLM-generated answer plus matched entities (with types, relationships, source files), relevant contradictions, and deduplicated file paths — all in one response. This is the primary way to query Cortex. Use it for questions like "What functions depend on the User model?", "What tech decisions have been made?", or "What contradictions exist in the auth flow?"
[Deprecated — use cortex_ask instead] Look up a specific entity by name or UUID. Returns entity details and optionally its relationships to other entities.
List contradictions detected in the knowledge graph. Contradictions occur when two entities make conflicting claims (e.g., different tech choices for the same thing). Returns both entities with summaries so you can understand and help resolve them.
Get current status of the Cortex knowledge graph: entity count, relationship count, file count, and whether the graph has data. Check this first to verify Cortex is populated.
Three tools (find_entity, query_cortex, search_entities) are marked deprecated with guidance to use cortex_ask instead, but all remain callable. This creates API surface confusion and forces LLMs to reason about tool selection when a simpler canonical interface should exist. Deprecated tools should be removed or clearly gated.
Output schemas are not documented in tool definitions. LLMs cannot plan downstream actions or extract structured data when response format is unknown. Every tool must document what fields it returns, their types, and their purpose.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 50 | - | v1 |
Trigger ingestion of a single file into the knowledge graph. Extracts entities and relationships using LLMs. The file must belong to a registered project. If projectId is omitted, Cortex auto-detects the project from the file path.
List all projects registered in Cortex with their file and entity counts. Use the project id to scope query_cortex to a specific project.
[Deprecated — use cortex_ask instead] Answer a natural language question using the knowledge graph. Returns an LLM-generated answer with cited entities.
Unregister a project from Cortex. Removes it from the project registry but preserves all extracted entities in the knowledge graph.
Resolve a contradiction by choosing an action: supersede (entity A replaces B), keep_old (keep B, discard A), dismiss (not a real contradiction), both_valid (both are correct in context). Get contradiction IDs from get_contradictions first.
[Deprecated — use cortex_ask instead] Full-text search across entities. Returns matching entities ranked by relevance.
Parameters lack constraint documentation. Examples: 'status' in get_contradictions omits allowed values (active/resolved/dismissed); 'action' in resolve_contradiction lists enum values in description but not in schema; 'privacyLevel' in add_project mentions 'standard/sensitive/restricted' but no validation rules. Constraints must be in both description AND schema enum.
No error handling guidance visible. Tools lack actionable error messages or recovery hints. When ingest_file fails (file not found, file not registered to project), LLMs need explicit guidance: e.g., 'File not found. Verify path is absolute and file exists.' or 'File path not in any project. Call add_project first or use projectId parameter.'
Missing pagination and result limits in list/search tools. list_projects and get_contradictions do not document max result counts or offer pagination. Large knowledge graphs could return thousands of contradictions, overwhelming context windows. Add limit caps and next_cursor support.
Ambiguous parameter types and missing range validation. 'limit' in get_contradictions and search_entities is a number with no min/max. Should be capped (e.g., 1-100). 'expand' in find_entity is boolean but implications unclear, does it return relationships, neighbor entities, both? Descriptions must specify what data gets added.
Tool composition gaps: No tool returns IDs or references needed by downstream tools. Example: add_project description does not state it returns a project_id, but list_projects returns project objects that include IDs. LLMs cannot chain add_project → list_projects → cortex_ask without explicit ID visibility.
resolve_contradiction accepts 'action' as free-form string despite listing valid values (supersede, keep_old, dismiss, both_valid) in description only. Schema must declare this as an enum so LLMs see valid options and validation prevents typos.