Loading ToolBench
A RAG-based chat server for Il Post articles with WebSocket streaming and MCP tool integration
Single tool 'search_articles' has a clear verb-noun name and reasonable description (194 chars, within baseline 34-392 range). Input schema is present with proper types (string, integer) and descriptions. However, output schema is only documented in the docstring as free-form text ('List of chunks ranked by relevance, each with: chunk_id, score, text, url, title, pub_date, categories') rather than as a formal JSON Schema structure. No pagination parameters despite the tool returning lists. No error handling guidance. No tool annotations (readOnlyHint, etc.). The description adequately explains WHAT and WHEN but lacks recovery guidance for failure cases.
Search the ilpost.it article database for chunks relevant to the query. Args: query: The search query in natural language (Italian or English). top_k: Number of top results to return (default 6). Returns: List of chunks ranked by relevance, each with: chunk_id, score, text, url, title, pub_date, categories.
Output schema not formally documented. Docstring lists fields as prose ('chunk_id, score, text, url, title, pub_date, categories') rather than JSON Schema. LLMs cannot reliably parse unstructured output descriptions.
No pagination support. Tool accepts top_k but no offset/page parameter or next_cursor. Large result sets risk context window exhaustion.
No error handling guidance. Description does not explain what happens on invalid query, database errors, or empty results. LLM has no recovery path.
Missing tool annotations. No readOnlyHint, destructiveHint, or idempotentHint declared. Agents cannot infer safety properties.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
top_k parameter lacks bounds. Description does not specify min/max (e.g., 1 - 100). LLM could pass absurd values.