Loading ToolBench
Cross-check answers from multiple public LLM APIs given a prompt. Shows a list of answers from different LLMs.
Single tool with critically incomplete definition. The tool `cross_check` has a description present but severely lacks actionable detail for LLM selection. Input schema exists but is minimal. No output schema documentation. No error handling guidance. No parameter constraints or validation hints. The docstring describes what the tool returns at a high level, but provides no guidance on WHEN to call it, WHAT the output structure actually is (field names, types), or HOW to handle failures from individual LLM APIs. Per the rubric baseline, A+ tools document output schemas and error recovery paths, this server has neither.
Cross-check answers from multiple public LLM APIs given a prompt. To goal is to show a list of answers from different LLMs. Arguments: prompt (str): The input prompt to send to the LLMs. Returns: dict: A dictionary containing the responses from each LLM. Each key in the dictionary corresponds to the name of an LLM, and its value is either: - The response from the LLM in JSON format (e.g., containing generated text or completions). - An error message if the request to the LLM failed.
No output schema documentation. The tool description states it 'returns a dict' but does not specify field names, types, or structure. Downstream tool calls or LLM reasoning cannot plan on the response shape.
Description lacks WHEN/WHY context. At 140 characters, the description says WHAT (cross-check from multiple LLMs) but does not explain WHEN an LLM should invoke this vs. querying a single LLM directly, or what problem it solves. LLMs struggle to select tools without clear use-case guidance.
No error handling or recovery guidance. The docstring mentions 'error message if the request failed' but provides no guidance to the LLM on whether errors are retryable, how to interpret them, or what to do next. A bare error dict gives the agent no path forward.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 41 | - | v1 |
No parameter constraints or validation hints. The 'prompt' parameter lacks min/max length, format guidance, or examples of what constitutes a good prompt for LLM cross-checking. LLMs will pass arbitrary strings without guidance.
API credentials exposed in code and via environment variables without secret injection abstraction. The tool directly reads OPENAI_API_KEY, ANTHROPIC_API_KEY, etc. from environment at tool invocation time. If these tools are traced or logged, secrets leak into agent logs.
No rate limiting or timeout protection. External API calls in query_llm() have no explicit timeout (httpx defaults apply, but not declared). An agent in a retry loop could hammer external APIs without bounds.
Incomplete error handling in query_llm(). Generic 'Exception' catch returns {'error': '...'} but does not categorize errors (retryable vs. fatal). An LLM cannot distinguish a temporary network timeout from a permanent auth failure.