Loading ToolBench
MCP server for monitoring Apache Airflow DAGs and their execution history
Three tools with basic verb-noun naming (list_, get_) and functional descriptions (50-100 chars each). All tools have input schemas with typed parameters and descriptions. However, output schemas are undocumented, LLMs cannot predict response structure for downstream chaining. Error handling returns generic status/message pairs without recovery guidance. No parameter constraints (enums, ranges, patterns) despite opportunities (e.g., 'state' field in get_recent_runs could be constrained to success|failed|running). Missing idempotency hints and destructive operation warnings. Tool descriptions lack WHEN-to-use context and dependency hints.
Get execution history summary for a DAG over the past N days. Returns success/failure counts and average duration.
Get recent runs for a specific DAG. Returns run status, start/end times, and duration.
List all active DAGs in Airflow. Returns DAG ID, description, owner, and schedule interval.
Output schemas undocumented. Tools return dicts with fields like 'dags', 'runs', 'success_count' but LLMs cannot predict structure for chaining. E.g., list_active_dags returns {status, count, dags:[{dag_id, description, owner, schedule_interval, is_paused, next_dagrun}]}, not declared anywhere.
No parameter constraints. 'limit' in get_recent_runs accepts any integer, should be bounded (1-100). 'days' in get_execution_history unbounded, should be 1-365. 'state' field in responses could be an enum (success|failed|running) but is free-form string.
Error handling lacks recovery guidance. All errors return {status: 'error', message: '...'} with no actionable next steps. E.g., 'Failed to list DAGs: Connection refused' tells LLM nothing, should suggest 'Check AIRFLOW_URL and credentials' or 'Retry in 30s'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
Descriptions lack WHEN-to-use context and dependency hints. E.g., get_recent_runs description does not explain when to call it vs get_execution_history, or that dag_id must exist (call list_active_dags first if unknown).
No tool annotations. Tools are read-only but lack readOnlyHint. No idempotentHint declared, though all three tools are safe to retry. Agents cannot optimize retry logic without these hints.