Model Context Protocol server for Honeybadger integration
Honeybadger MCP has complete, well-structured tool definitions with proper naming conventions and clear descriptions. All four tools follow verb_noun naming (get_*, list_, analyze_) and include detailed inputSchemas with typed parameters. However, there are significant gaps: (1) No output schemas documented, LLMs cannot plan downstream calls or extract fields reliably; (2) Parameter descriptions lack critical format/constraint details (e.g., 'limit' max values stated but not validated); (3) Analyze tool description is vague about what 'comprehensive analysis' actually returns; (4) No error handling guidance, tools throw generic McpError with insufficient context for LLM recovery; (5) Tool responses are not shaped, raw API data likely included verbatim, wasting tokens and diluting signal. Average parameter quality is good (descriptions present, types clear), but missing output documentation and error recovery patterns significantly impact production readiness.
Comprehensive analysis of a Honeybadger issue with fix suggestions
Fetch a specific fault/error from Honeybadger by ID
Fetch notices (occurrences) for a specific fault
List recent faults from Honeybadger
No output schemas documented. LLMs cannot infer what fields each tool returns, forcing them to guess at response structure and breaking composition chains. Critical for getFault, getNotices, listFaults, analyzeIssue.
analyze_honeybadger_issue description is vague ('Comprehensive analysis...with fix suggestions'). Does not explain WHAT analysis is performed, WHEN to call it vs other tools, or WHAT structure the response has. LLM cannot determine appropriateness.
Error handling is generic ('Tool execution failed: [error]'). No recovery guidance for common failures (invalid fault_id, auth failure, rate limit). LLMs cannot self-correct.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 45 | - | v1 |
list_honeybadger_faults and get_honeybadger_notices accept 'limit' parameter with default and max documented (10-100, 20-100) but no validation shown. If LLM passes limit=500, tool silently caps or errors without guidance on valid range.
Parameter 'environment' in list_honeybadger_faults has no enum constraint. Description says 'e.g., production, staging', LLM may infer these are only examples and pass invalid values like 'prod' or 'dev'. Should declare valid enum values.
Tool responses are not shaped. Code returns raw API data via response.data without filtering, pagination controls, or metadata stripping. HoneybadgerFault and HoneybadgerNotice types suggest verbose nested structures (backtrace arrays, context objects, cgi_data), these waste tokens in LLM context.
API key exposed as environment variable without validation. If HONEYBADGER_API_KEY is missing, error is generic ('environment variable is required'). Should validate at startup and fail early with actionable message.