Loading ToolBench
Multi-agent planning and execution server with support for code execution, document retrieval, web search, and AWS service operations
This server presents significant definition quality issues across most tools. Tool naming is inconsistent and sometimes ambiguous (e.g., 'agentcore_coder' vs 'agentcore_drawer' lack action verbs; 'use_aws' is vague). Descriptions are present but often generic and lack actionable guidance. Parameter schemas are partially defined but lack proper constraints and type information. Output schemas are entirely undocumented. Error handling guidance is absent. The 'use_aws' tool is particularly problematic, it accepts arbitrary service/operation pairs with an unvalidated 'parameters' object, creating a security and usability nightmare. Code execution tools (agentcore_coder, agentcore_drawer) lack safety guards and timeout documentation. The retrieve tool's description is incomplete (cut off mid-sentence in source). Tavily tools have better descriptions but still lack output schema documentation.
Use this to execute python code and do math. If you want to see the output of a value, you should print it out with `print(...)`. This is visible to the user. code: the Python code was written in English
Execute a Python script for draw a graph. Since Python runtime cannot use external APIs, necessary data must be included in the code. The graph should use English exclusively for all textual elements. Do not save pictures locally bacause the runtime does not have filesystem. When a comparison is made, all arrays must be of the same length. code: the Python code was written in English return: the url of graph
Query the keyword using RAG based on the knowledge base. keyword: the keyword to query return: the result of query
Performs a web search using Tavily's AI search engine and generates a direct answer to the query, along with supporting search results.
Searches recent news articles using Tavily's specialized news search functionality.
Tool naming lacks action verbs. 'agentcore_coder', 'agentcore_drawer', and 'use_aws' do not follow verb_noun convention (e.g., 'execute_code', 'draw_graph', 'invoke_aws_service'). LLMs cannot infer intent from these names alone.
Output schemas are entirely undocumented. No tool declares what fields or structure the LLM should expect in responses. LLMs cannot plan downstream tool calls or extract relevant data without knowing response structure.
'use_aws' tool accepts arbitrary 'service_name', 'operation_name', and 'parameters' without validation. This is a dangerous anti-pattern: (1) LLMs can invoke any AWS operation, including destructive ones without proper guards; (2) unvalidated JSON object 'parameters' invites injection attacks; (3) no output schema means LLM cannot interpret results; (4) error messages promise 'helpful' guidance but mechanism is unclear.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 41 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 39 | - | v1 |
Performs a comprehensive web search using Tavily's AI-powered search engine. Excels at extracting and summarizing relevant content from web pages, making it ideal for research, fact-finding, and gathering detailed information.
Execute AWS service operations using boto3 with comprehensive error handling and validation. This tool provides a universal interface to AWS services, allowing you to execute any operation supported by boto3. It handles authentication, parameter validation, response formatting, and provides helpful error messages with schema recommendations when invalid parameters are provided.
Code execution tools (agentcore_coder, agentcore_drawer) lack critical documentation: no timeout specifications, no sandboxing guarantees, no resource limits, no safety warnings about code injection risk. Descriptions do not clarify whether code runs in a restricted environment or has full system access.
'retrieve' tool description is incomplete/truncated in source ('Query the keyword u[...]'). Parameter 'keyword' lacks context on what knowledge base is queried, what format results take, and when to use this vs web search.
Parameter descriptions often lack format constraints, ranges, and examples. E.g., 'search_depth' is enum (basic|advanced) but description does not explain the difference or when to choose each. 'max_results' lacks min/max bounds. LLMs must guess at valid ranges.
No error handling guidance across any tool. Descriptions do not explain what errors to expect, when to retry, or what to do if a call fails. LLMs have no recovery path on failure.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) present. 'use_aws' can execute destructive operations (e.g., delete-*) but tools are not marked as such. Agents cannot distinguish safe reads from irreversible mutations.