Loading ToolBench
An MCP server for Apache Airflow that enables non-technical users to interact with Apache Airflow workflows through natural language.
The Airflow MCP server has 11 tools with visible definitions in mcp_airflow.py. Tool naming follows verb-noun conventions (get_*, trigger_dag, change_dags_pause_status) which is good. However, there are critical gaps: (1) No input parameter descriptions are visible in the source code for ANY tool except get_all_the_runs, which has detailed descriptions. (2) No output schemas are documented, the code calls Airflow REST APIs and returns raw responses, but there's no structured schema definition for what fields agents should expect. (3) Several descriptions are vague or misleading (e.g., 'Get all the runs for a specific run' is confusing, it should say 'Get all runs for a specific DAG'). (4) Error handling is minimal, the make_airflow_request function returns raw Exception objects, not actionable guidance. (5) Parameters that should be enums (e.g., states in get_all_the_runs) are free-form comma-separated strings, inviting invalid values. (6) No documentation of parameter constraints or format expectations in most tools. This server is functional but lacks the rigor needed for production agent interaction.
Change the pause status for one or more DAGs via the Airflow API.
Retrieve filtered DAG runs across multiple DAG IDs with various criteria.
Get all the runs for a specific run
Fetch all available Airflow connections via the Airflow REST API
Get a simplified view of the DAG that retains all essential details
Fetch all available Airflow DAGs and return the list of them
No output schemas documented. Tools return raw Airflow API responses without structure definition. LLMs cannot plan downstream calls or extract required fields with confidence.
Most input parameters lack descriptions. Only get_all_the_runs provides parameter documentation. This prevents LLMs from understanding what values to pass or the format/constraints for each parameter.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 43 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 54 | - | v1 |
Retrieve the source code of a specific DAG via the Airflow API.
Get a simplified view of the task that retains all essential details
Get all tasks for a DAG.
Retrieve the next scheduled or triggered run metadata of a specific DAG via the Airflow API.
Trigger specific dag
Error handling is minimal. make_airflow_request() returns raw Exception objects instead of actionable guidance. When an API call fails, the LLM receives no direction on what to do next (retry, check parameters, etc.).
Tool description for get_all_the_runs_for_dag says 'Get all the runs for a specific run' which is contradictory and misleading. Should clarify it returns DAG run history for a given DAG ID.
Parameters that should be enums (e.g., states in get_all_the_runs: 'failed', 'success', 'running') are passed as comma-separated strings. No enum constraint forces LLMs to guess valid values and invites typos.
No pagination guidance for tools returning lists (get_dags, get_connections, get_dags_tasks, get_all_the_runs_for_dag). Large result sets will blow context window. No limit parameter or documentation of result capping.
Naming inconsistency: get_all_the_runs_for_dag uses 'all_the_runs' phrasing which is verbose; conventional name would be get_dag_runs. Naming does not follow production baseline patterns (18 char avg, verb_noun style).
No documentation of which tools are idempotent vs stateful. trigger_dag and change_dags_pause_status have side effects, but agents need explicit warning that these are not safe to retry blindly.
Parameter format constraints not documented. For example, get_all_the_runs accepts ISO 8601 timestamps, but there's no hint in descriptions for dag_id, start_date_gte, start_date_lte about expected format or validation rules.