Loading ToolBench
Model Context Protocol server for integrating with Datadog, providing tools to query dashboards, incidents, metrics, monitors, and logs
Datadog MCP server has 11 well-structured tools with consistent naming conventions and solid parameter schemas. All tools use clear verb_noun naming (list_*, get_*, query_*, search_*). Descriptions are present and generally informative (average ~120 chars). Input schemas are explicit with type information and parameter descriptions. However, there are notable gaps: output schemas are not documented (tools serialize to JSON but no schema specification is provided), error handling is minimal (generic catch-all returns), and several patterns are missing (no pagination cursor guidance in output, no per-item success/failure in batch operations, no tool annotations). The implementation shows competent baseline quality but lacks production-grade polish in error recovery guidance and response structure documentation.
Get detailed information about a specific dashboard including widgets and configuration.
Get detailed information about a specific incident by ID.
Get metadata for a specific metric including type, description, and unit.
Get detailed information about a specific monitor by ID. Use this to see full monitor configuration, thresholds, and current state.
List all dashboards in Datadog. Use this to discover available dashboards and their IDs.
List incidents from Datadog. Use this to get an overview of ongoing and past incidents.
No output schemas documented. Tools return JSON serialized responses but no schema specification exists for LLMs to understand returned field types, structure, or presence guarantees. LLMs cannot plan downstream operations or validate responses without schema documentation.
Error handling returns generic JSON error objects without recovery guidance. Catch blocks return {"error": "<message>"} with no categorization (retryable/user-fixable/fatal) or suggested next steps. LLMs receive no actionable direction on how to proceed after failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 66 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
List available metric names in Datadog. Use this to discover what metrics are available.
List monitors with full metadata. Use list_monitors_summary for lighter queries.
List monitors with minimal data (id, name, state only). Use this for quick overview and navigation. For full details, use get_monitor with a specific ID.
Query metrics from Datadog. Use this to retrieve time-series data for specific metrics.
Search logs in Datadog. Use this to find log entries matching specific queries, services, hosts, or time ranges.
Pagination responses lack next_cursor or clear continuation guidance. List tools return total/page/pageSize but no cursor or explicit 'has_more' flag to guide agents on whether to fetch the next page. Response includes pageSize but not whether more results exist.
query_metrics time parameter descriptions accept both ISO 8601 and relative formats (e.g., '1h', '1d') but the schema type is just 'string' with no enum or pattern constraint. LLMs may invent invalid formats like '2 hours' or '1 week'.
Tool annotations (readOnlyHint, destructiveHint, idempotentHint) are absent. All 11 tools are READ_ONLY per the risk metadata, but this is not declared in the MCP tool registration via @readOnlyHint. Agents cannot programmatically distinguish safe from unsafe tools.
Response fields strip away context needed for tool chaining. Example: list_dashboards returns dashboard.id but get_dashboard requires dashboardId as parameter name. Monitor tools refer to 'monitorId' (integer) vs 'id' (string) inconsistently. Naming mismatches between response fields and parameter names force agents to manually map values.
Parameter descriptions lack explicit constraints on numeric ranges. pageSize parameters clamp internally to 1-50 but descriptions say '(1-50)' informally. Agents should see formal min/max constraints in schema or as enum alternatives.