HTTP proxy and retry loop server for the Malloy test UI. Handles natural language questions, generates Malloy queries via Claude, executes them against a Publisher, retries on failure, analyzes results, and proposes/applies model enhancements.
This server exposes 11 tools, but only 3 are intended as public MCP endpoints (ask, propose, apply). The remaining 8 are internal functions not registered as MCP tools. Of the 3 public tools, descriptions are present but lack LLM-optimized structure. Input schemas are visible but incomplete: parameters lack type definitions (e.g., 'history' is described as array but no item schema; 'change' object has no property definitions). No output schemas are documented. Error handling is minimal, no recovery guidance or error classification. Tool naming is reasonable (verb_noun pattern) but descriptions are verbose and lack the concise, actionable format LLMs need. The 'apply' tool performs destructive operations (file backup, git commit) with no confirmation step or dry-run option, violating the confirmation-request pattern.
Internal function. Analyzes query results using Claude to generate insights, analysis text, optional chart specification, and caveats.
POST /api/apply endpoint. Applies a proposed model change by backing up the source file, splicing the snippet into the Malloy source, mirroring the change to the Publisher's working copy, triggering a reload, and committing to git.
Internal function (model-enhancer.js). Applies a proposed model change by backing up the source file, splicing the snippet into the Malloy source, mirroring to Publisher's copy, triggering reload, and committing to git.
POST /api/ask endpoint. Takes a natural language question and optional conversation history, generates a Malloy query via Claude, executes it against the Publisher with retry logic (up to MAX_RETRIES attempts), and returns the final result rows or error along with analysis.
Internal function. Executes a Malloy query against the Publisher API and returns result rows or error.
Internal function. Generates a Malloy query from a natural language question using Claude, with optional error context for retry scenarios and conversation history for context.
Only 3 of 11 tools are public MCP endpoints; 8 are internal functions with no MCP registration, descriptions, or schemas visible. Per HARD SCORING RULES, tools without visible definitions must be capped at 50.
Input schemas lack type definitions and nested object schemas. 'history' parameter described as array but no item schema; 'change' object has no property definitions. Per HARD SCORING RULES, parameters without type definitions cap schema score at 30.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields without knowing response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 30 | 2026-07-28+ | v2 |
Internal function (model-enhancer.js). Retrieves git history of commits that touched the Malloy source files.
Internal function (model-enhancer.js). Heuristic check to determine if an error message indicates a missing field that could be enhanced.
POST /api/propose endpoint. Takes a failed query, error message, and question; asks Claude to propose an additive Malloy model change (new dimension, measure, or view) to fix the issue.
Internal function (model-enhancer.js). Proposes an additive Malloy model change by analyzing a failed query and error, then asking Claude to suggest a new dimension, measure, or view to add.
Internal function (model-enhancer.js). Reverts the most recent git commit that touched the Malloy source files.
Destructive operations (apply, applyChange, rollbackLastChange) lack confirmation step or dry-run option. Agents can trigger file backups, git commits, and model changes without user approval.
Error responses lack recovery guidance. extractError() returns raw API text; no classification (retryable vs fatal) or actionable next steps for LLM.