Loading ToolBench
A collection of Python example implementations demonstrating context engineering patterns: ReAct agents with memory, context optimization, MCP servers, multi-agent orchestration, and RAG pipelines.
Server has 7 tools with basic descriptions and input schemas, but significant quality gaps. Tool names lack consistent verb-noun structure (e.g., 'calculator' instead of 'evaluate_expression'). Descriptions are present but generic (avg ~80 chars, below 194-char baseline). Parameter descriptions exist but lack constraints, ranges, and format guidance. No output schemas documented. Error handling is minimal, tools return raw error strings without recovery guidance. No tool annotations (readOnlyHint/destructiveHint). The 'create_event' tool modifies state but lacks confirmation/dry-run pattern. Overall, definitions are functional but would not pass production code review.
Evaluate a safe mathematical expression. Supports: +, -, *, /, **, sqrt, log, sin, cos, pi, e
Create a new calendar event.
Fetch a URL and return a truncated excerpt of its text content.
Search notes by keyword and optional tag. Use this to look up research notes, meeting notes, or ideas.
Get calendar events for a date range. Use this to answer questions like "what's on my calendar today?" or "what do I have scheduled this week?".
List recent emails.
Search the web for information. Mock implementation returns plausible snippets for demo queries.
Tool 'calculator' uses generic noun instead of action verb. Should be 'evaluate_expression' or 'compute_math' to clarify intent.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract structured data. E.g., 'search' returns a string snippet, but what fields does 'get_events' return? Are there event_id, start_time, end_time fields?
Destructive tool 'create_event' lacks confirmation/dry-run pattern. No way to preview before committing. Agents can accidentally create duplicate events.
Parameter descriptions lack constraints and format guidance. E.g., 'date' param in get_events says 'YYYY-MM-DD' but no validation or error message if agent passes '2024/01/15'. 'days_ahead' has no min/max bounds.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 51 | 2026-07-28+ | v2 |
Error handling is minimal. Tools return raw exception strings (e.g., 'Error: <exc>') with no recovery guidance. LLM cannot determine if error is retryable, user-fixable, or fatal.