Analyzes workflow failure records and diagnoses issues through ranked search matching and failure extraction
This server has a single tool 'run' with critical definition and design flaws. The tool name violates naming conventions (generic 'run' instead of verb_noun like 'search_failures' or 'analyze_failures'), the description is vague about actual capabilities, parameter descriptions lack essential context, and the schema is minimally structured. Input validation is present but error messages are not actionable. Output structure is underdocumented. The server treats a complex multi-faceted operation (search + ranking + failure extraction + replay evaluation) as a single monolithic tool, violating the single-responsibility principle.
Analyzes workflow failure records by searching for query terms and identifying failed records. Returns ranked matches with scores, extracted failures, and replay availability.
Tool name 'run' is generic and violates verb_noun convention. Should be 'search_workflow_failures', 'analyze_workflow_failures', or 'diagnose_failure_records' to clearly indicate action and subject.
Tool description (165 chars) is vague about what 'analyzes' means and does not explain WHEN to use this tool vs alternatives, what failures are 'extracted', or what 'replay availability' entails. Should be 50-200 chars with clear WHAT/WHEN/RETURNS structure.
'records' parameter lacks actionable description. Source says 'List of workflow failure records (dictionaries) to analyze' but does not specify required record fields (e.g. must have 'status', 'title', 'body'?), expected data types per field, or example structure.
'query' parameter description 'Search query string containing terms to match against record content' is incomplete. Does not specify: min/max length, special characters allowed, case sensitivity, boolean operators support, regex support.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 32 | 2026-07-28+ | v2 |
Output schema is undocumented. Caller cannot know the structure of 'matches', 'failures', or what fields each contains (e.g. 'rank', 'score', 'evidence' are returned but not declared in any schema spec). LLM must guess or reverse-engineer from code.
Tool combines five responsibilities: (1) search/filter by query, (2) rank results by relevance, (3) extract failures by status, (4) evaluate replayability, (5) return metadata. Should be split into focused tools: 'search_workflow_records', 'list_failed_workflows', etc.
Error handling is minimal. ValueError exceptions are caught and wrapped in {'status': 'error', 'error': str(exc)} but error messages like 'records must be a list and query must be non-empty' do not guide recovery. E.g. should say 'records must be a list of dicts; got {type}. Each dict should have status, title, body fields.'
Output field naming is inconsistent and undocumented: 'rank' (integer), 'record_index' (integer), 'score' (float), 'evidence' (string), but 'matches' is a list of these objects, and 'failures' is a list of raw records. No schema explains structure or purpose of each field.
Term extraction uses regex r'[a-z0-9]+' which loses meaning: multi-word phrases, punctuation, and capitalization are stripped. Query 'API key' becomes ['api', 'key']; 'user-id' becomes ['user', 'id']. Algorithm is fragile and LLM cannot predict behavior.