Loading ToolBench
MCP server for retrieving GitHub Dependabot alerts
Single tool with minimal definition quality. Tool name 'get_dependabot_alerts' follows verb_noun convention (+), but description is extremely terse (24 chars, below 34-char p10 baseline). Input parameters lack descriptions entirely, 'repo_owner' and 'repo_name' have no guidance on format, constraints, or examples. No output schema documented. Error handling returns empty list silently instead of actionable recovery guidance. No pagination support despite tool likely returning variable-length alert lists. Tool is directly visible in source code (FastMCP decorator), so no inference penalty applies.
Get Github Dependabot alerts
Parameter descriptions missing entirely. 'repo_owner' and 'repo_name' have no descriptions in schema, LLMs cannot infer whether to pass 'octocat' or '[email protected]' or 'O12345'. Violates pattern:tool-description baseline (100% of A+ tools have param descriptions).
Tool description is 24 characters ('Get Github Dependabot alerts'), below p10 baseline of 34 chars. Lacks WHAT/WHEN/WHY context. Does not explain when to call this vs other security tools, what the response contains, or prerequisites (e.g., 'requires GitHub token with repo:read scope'). Violates pattern:tool-description.
No output schema documented. Tool returns 'list[dict]' but LLMs cannot plan downstream actions without knowing field names (e.g., does response include 'alert_id', 'severity', 'package_name'?). Violates pattern:tool baseline (100% of A+ tools have documented return types).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | <=2025-11-25 | v2 |
Error handling returns empty list silently on all failures (HTTP errors, network errors, auth failures). LLM cannot distinguish 'no alerts' from 'permission denied' from 'network timeout'. Violates pattern:recovery-guide (errors must tell LLM what to do next).
No pagination support. Tool accepts no limit/offset/page parameters. If a repo has 500+ alerts, response will be truncated by GitHub API (default 30 items) with no way for agent to fetch remaining results. Violates pattern:paginated-result.