Loading ToolBench
Server has 5 tools with significant quality gaps. Tool names lack consistent verb prefixes (add_note, brave_search_results, fetch_weather are acceptable; read_notes and add are weak). Descriptions are present but minimal (10-50 chars), below the 50-200 char baseline for LLM optimization. Input schemas are visible and typed, but parameter descriptions are sparse or missing. No output schemas documented. Error handling is absent, tools return raw API responses or generic strings without recovery guidance. The add_note tool exposes file I/O without validation. brave_search_results has a critical bug (params={'q': {news}} creates a set instead of string). No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Resources and prompts are defined but not integrated with tools.
Adds two numbers together
Append a new note to the note file.
Fetch search results from brave search engine
Fetch current weather for a city
Read and return all notes from the note file.
brave_search_results has a critical bug: params={'q': {news}} creates a Python set instead of passing the string. This will cause API failures. Should be params={'q': news}.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract structured data. Tools return raw strings (response.text) instead of parsed JSON objects with typed fields.
Descriptions are too short (10-50 chars, baseline 50-200). 'Fetch search results from brave search engine' and 'Fetch current weather for a city' lack context on when to use them vs similar tools, what fields are returned, or error handling.
API keys (WEATHER_API_KEY, BRAVE_API_KEY) are injected via environment variables (correct), but brave_api_test.py contains a hardcoded API key 'BSAKX9fLyuevUrUhSygp_7RCgJWQcsh' in source control, creating a security leak.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 41 | 2026-07-28+ | v2 |
No error handling or recovery guidance. Tools return raw HTTP responses or generic strings ('Note saved!') without telling LLMs what to do on failure (retry, ask user, abort). Missing actionable error messages.