Loading ToolBench
MCP server providing integration with multiple data sources and services including Grafana Loki, S3, Sentry, Swagger/OpenAPI, databases, and file systems
dev-mcp exposes a single tool (loki_query) with a clear purpose and reasonable schema. The tool has a proper description, enumerable parameters with types, and explicit input validation. However, there are significant gaps in output documentation, error handling guidance, and composition patterns. The tool is READ_ONLY and well-scoped, but lacks the contextual richness and recovery guidance expected of production-grade agent tools. The mock implementation (returning hardcoded results rather than calling the actual Loki client) raises concerns about real-world reliability.
Query Grafana Loki logs using LogQL
Mock implementation returns hardcoded results instead of calling actual Loki client. Code comment states 'For demonstration purposes, return a mock result. In a real implementation, you would call client.Query(args.Query)'. This prevents the tool from functioning in production.
Output schema is not documented. The tool returns a JSON structure with 'status', 'data.resultType', 'data.result[].stream', 'data.result[].values' fields, but LLMs have no schema definition to parse the response. Agents cannot reliably extract fields from the response.
Error messages lack recovery guidance. The tool returns 'Invalid arguments: ...' and 'query parameter is required' but provides no suggestions for what the LLM should do next (e.g., 'Try calling with a valid LogQL query like {count(some_metric)}'). Agents cannot self-correct.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 43 | - | v1 |
Missing pagination support. The tool accepts a 'limit' parameter but returns results in an unbounded array. Large result sets could exhaust context window. No documentation of maximum results or guidance on pagination strategy.
Parameter descriptions are minimal. 'query' is described as 'LogQL query to execute' with no guidance on valid syntax, examples of query patterns, or constraints. 'limit' has no bounds documented (e.g., min 1, max 10000).
Tool name 'loki_query' is vendor-specific and does not follow verb_noun convention. Should be named based on action (e.g., 'search_logs', 'query_logs', 'retrieve_logs') to make the intent clearer to LLMs.