Loading ToolBench
Universal MCP server for querying GraphQL and REST APIs using natural language
API Agent provides 5 tools with complete input schemas and generally adequate descriptions. Tools are well-named with action verbs (graphql_query, sql_query, rest_call, poll_until_done, search_schema). Descriptions range from 100-250 chars, explaining purpose and parameters. However, several gaps prevent a higher score: (1) Output schemas are not documented, responses return 'JSON string' but field structure is not specified; (2) No error handling guidance, tools say 'errors still processed by LLM' but don't specify what errors look like or how to recover; (3) Parameter descriptions lack constraints, e.g., sql_query's 'sql' param has no DuckDB syntax hints, rest_call's 'method' says 'GET recommended, others may be blocked' but doesn't enumerate allowed methods or explain blocking logic; (4) No pagination hints despite sql_query and rest_call potentially returning large results; (5) return_directly parameter across multiple tools lacks clarity on when to use it, description says 'skip LLM processing' but doesn't explain the use case or when the LLM should be skipped. Tools are functional but lack the LLM-optimization and constraint clarity expected of production-grade agent tools.
Execute GraphQL query and store result for sql_query. Args: query: GraphQL query string name: Table name for sql_query (default: "data") return_directly: Skip LLM processing, return data directly to client. Only applies on success. Errors still processed by LLM. Returns: JSON string with query results
Poll async API until done_field equals done_value. Args: method: HTTP method path: API path done_field: dot-path (e.g., "status", "data.0.complete", "trips.0.isCompleted") done_value: target value as string ("true", "COMPLETED") body: JSON string for request body (optional) name: Table name for sql_query (default: "data") delay_ms: ms between polls (default configured, max configured) Returns: JSON string with final API response after polling completes
Execute REST API call and store result for sql_query. Args: method: HTTP method (GET recommended, others may be blocked) path: API path (e.g., /users/{id}) path_params: JSON string for path values (e.g., '{"id": "123"}') query_params: JSON string for query params (e.g., '{"limit": 10}') body: JSON string for request body (e.g., '{"name": "John"}') name: Table name for sql_query (default: "data") return_directly: Skip LLM processing, return data directly to client. Only applies on success. Errors still processed by LLM. Returns: JSON string with API response
Grep-like search on schema. Output: "line_num:match" or "line_num-context". Args: pattern: Regex pattern (case-insensitive) context: Lines around each match (default 10) before: Lines before match (overrides context) after: Lines after match (overrides context) offset: Number of matches to skip (for pagination)
Output schemas not documented for any tool. Responses described as 'JSON string with results' but field structure, data types, and nesting are not specified. LLMs cannot plan downstream tool chains or extract data without knowing output shape.
Error handling guidance missing. Tools mention 'errors still processed by LLM' or don't address errors at all. No indication of what error responses look like, what causes them, or how agents should recover.
rest_call 'method' parameter says 'GET recommended, others may be blocked' but does not enumerate allowed methods. 'Blocked' behavior is undocumented, does it error? Return 405? Silently fail? LLMs will guess and fail.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 64 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 57 | - | v1 |
Run DuckDB SQL on stored REST API results. Tables available = names from rest_call calls + auto-extracted top-level keys. Args: sql: DuckDB SQL query return_directly: Skip LLM processing, return query results directly
No pagination guidance for tools returning lists. rest_call and sql_query can return large result sets but no limit, offset, or total_count parameters visible. Description says 'Tables available = names from rest_call calls' implying dynamic schema, but no discovery mechanism documented.
Parameter 'return_directly' across graphql_query and rest_call is under-explained. Description says 'skip LLM processing' but use case, side effects, and when to use it are unclear. Does it bypass safety checks? Change output format? Affects logging?
SQL injection and command injection prevention not documented. rest_call and graphql_query accept arbitrary user input (query, method, path, body). No mention of how sanitization or parameterization is handled.
No authentication or authorization documentation. Which APIs require auth? How are credentials passed? Are tools gated by permission? No audit trail or scope declaration mentioned.