Loading ToolBench
A local MCP server for retrieving paragraphs from one Jekyll-style blog.
Single tool 'retrieve_paragraphs' has a clear verb-noun name and structured output enabled. Description is adequate (94 chars, within 10-1024 range). Input schema is present with two parameters: 'query' (string, described) and 'top_k' (integer with default=5, described). However, the schema lacks explicit type constraints (min/max for top_k), and the output schema is not documented in the tool definition, only inferred from code (asdict conversion). No error handling guidance, no parameter validation rules stated, and no documentation of what fields the response contains. The tool is READ_ONLY (good security posture) but lacks depth in parameter constraints and output documentation expected of production tools.
Retrieve structured paragraph results from the loaded blog index for one query.
Output schema not documented. Tool returns list[dict[str, object]] but LLM cannot infer field names, types, or structure. Missing documentation of response fields (e.g., paragraph_id, text, score, source, etc.) forces LLM to guess downstream field names.
Parameter 'top_k' lacks numeric constraints. No min/max bounds stated in description or schema. LLM could pass top_k=0, top_k=-1, or top_k=999999, but code only validates top_k > 0 at runtime. Should declare 'top_k: integer, 1-100' in description.
No error recovery guidance. If query returns zero results or top_k is invalid, error message does not guide LLM on next steps. Should include 'If no results, try a broader query' or 'top_k must be 1-100'.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | <=2025-11-25 | v2 |
No result limit enforcement stated. Tool description does not mention if results are capped or paginated. If index is large, returning all top_k results could exceed context window. Should state 'Returns up to top_k results' and document typical response size.
Parameter descriptions are minimal. 'query' description is 'The search query to retrieve paragraphs for' (47 chars), adequate but lacks guidance on format (free text? keywords? phrases?). 'top_k' description lacks constraint info.