A native macOS state runtime with high-density MCP tools for AI development
Critical failures across all dimensions. Tool definitions are inferred from benchmark test files rather than visible in actual MCP server code (Sources/AIShellMCP/MCPServer.swift shows only factory_diagnostics with empty schema {}). Nine of ten tools lack verifiable schemas, descriptions are vague/generic, and no parameter documentation exists. Tool names lack clear verb-noun structure (e.g., 'run_check', 'run_observe' are ambiguous). No evidence of error handling, output schemas, or composition patterns. This server appears to be a test harness, not a production MCP implementation.
Three SHA-guarded edits were applied atomically and the resulting diff was retained
The retained logs were compared around the root error and the comparison is complete
Affected symbols, dependencies, and tests were identified with evidence
Provides factory diagnostics for the AIShell runtime
The requested files and symbol ranges were read under the shared byte budget
Repository check completed and complete diagnostics were retained
The existing managed run was observed and the requested wait or cancellation completed
No input schemas visible in source code. factory_diagnostics shows empty schema {}; nine tools have no verifiable schema definitions.
Tool definitions inferred from benchmark test files (test-tool-discovery-evaluator.mjs), not from actual MCP server registration code. Nine of ten tools violate this.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 29 | 2025-06-18+ | v2 |
All requested regex and fixed-string searches completed with changed files and tests prioritized
Workspace, Git and branch/worktree differences, and project commands were summarized completely
A file change was observed after the bound cursor without polling
Descriptions are generic and under 20 characters for most tools (e.g., 'Repository check completed...' is vague about what the tool actually does).
Tool names lack clear verb-noun structure. 'run_check', 'run_observe', 'workspace_wait' are ambiguous about what action occurs. LLMs cannot infer intent from these names alone. Baseline: 90% of A+ tools start with action verbs like get_, create_, search_, update_.
No parameter descriptions visible. Even factory_diagnostics with empty input {} provides no guidance on what parameters are accepted or required. Baseline: 100% of A+ tools have parameter descriptions.