Loading ToolBench
MCP server for querying Datadog error traces and providing diagnostic capabilities
This Java MCP server exposes 5 tools for Datadog log and trace inspection. Strengths: all tools have descriptions, most parameters are typed with enums where appropriate, and the schema structure is visible. Significant weaknesses: descriptions are often generic or instruction-heavy rather than concise; parameter descriptions lack detail about formats and constraints; output schemas are not documented; error handling guidance is absent; tool naming uses dot notation (filter.configure, log.correlate) which is unconventional and may confuse LLM parsing; no evidence of pagination for list-returning tools; descriptions contain implementation details ('action=status') that should be internalized. The server lands in the C/D range, functional but with noticeable gaps that would require refinement before production use.
Configure stack trace filters for log searches. IMPORTANT: Before searching logs, check if filters are configured using action='status'. If not configured, ask the user if they want to: (1) set global filters, (2) set project-specific filters, or (3) show full stack traces without filtering.
Correlate logs and traces by trace ID
Search logs for a service within a time window
Extract a structured test scenario from a trace for debugging and unit test generation
Inspect a specific error trace and generate diagnostic workflow
Naming convention uses dot notation (filter.configure, log.correlate) instead of verb_noun style (configure_filter, correlate_logs). Dot notation is atypical for LLM tool parsing and may reduce clarity. LLMs expect get_, create_, search_, update_ prefixes to infer intent.
Descriptions are instruction-heavy (e.g., filter.configure contains 'IMPORTANT: Before searching logs, check if filters are configured...') rather than concise statements of what the tool does. These read as prompts for the LLM rather than tool documentation. Descriptions should state WHAT the tool does, not HOW the LLM should use it.
Parameter descriptions lack actionable format and constraint information. For example, 'ISO-8601 start timestamp' in log.correlate, log.search_logs, and trace.extract_scenario is present but inconsistent across tools. Many parameters (e.g., 'service', 'env') lack descriptions entirely or are minimal. No mention of valid enum ranges, length limits, or what values trigger errors.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 54 | - | v1 |
No output schemas documented. The source code shows parameter input schemas but provides no documentation of what each tool returns. LLMs cannot plan downstream calls or extract the correct data without knowing the response structure (e.g., does log.correlate return trace_data, log_lines, correlation_metadata?).
No error handling guidance. Tools do not specify what errors they may return, how to recover, or what the LLM should do if a trace is not found, Datadog API is unreachable, or authentication fails. This forces LLMs to guess recovery strategies.
Tool log.search_logs accepts 'limit' parameter defaulting to 100 but no pagination guidance. Description does not explain what happens if results exceed limit, whether pagination is available, or how to retrieve next pages. LLMs may not know how to handle large result sets.
Parameters such as 'stackTraceDetail' in log.correlate and log.search_logs use enums (full, relevant, minimal) but lack context in descriptions. What differentiates 'relevant' from 'minimal'? The description 'Stack trace detail level' is vague, a fuller explanation tied to the use case would help LLMs choose correctly.
filter.configure tool has a WRITE risk designation but no documentation of what state changes occur, whether they are reversible, or what side effects persist. LLMs need to know: is this idempotent? Can it be called multiple times safely? What happens if called with conflicting actions?