Loading ToolBench
A CLI tool for searching and indexing Reddit posts across subreddits with RAG (Retrieval-Augmented Generation) integration
Three tools with complete input schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. Tool names follow verb_noun convention (search_, index_, deep) but descriptions lack actionable context for LLM selection. Parameters have types and enums where appropriate, but descriptions are sparse (avg ~40 chars, below 72-char baseline). No documented output schemas, error handling guidance, or recovery paths. Schemas are visible and properly typed, preventing a lower score, but lack the depth needed for production use.
Single-subreddit deep-dive: 50 most relevant posts for a query.
Fan-out search across N subreddits and index results into RAG.
Find subreddit communities by topic.
Output schemas not documented. LLMs cannot predict what fields to expect from tool responses, forcing them to guess at downstream field names and risk broken tool chains.
Parameter descriptions are minimal (avg 40 chars vs 72-char baseline). 'skip_index' lacks explanation of what 'inspect output before committing' means. 'sort_by' enum values ('relevance', 'hot') lack context on when to use each.
Tool descriptions lack WHEN/WHY context. 'Fan-out search across N subreddits' does not explain when to use index_subreddits vs deep, or what 'RAG indexing' means to an LLM unfamiliar with the system.
No error handling or recovery guidance. Code shows subprocess calls to external rag-cli tool (stage3_index, stage4_search) with minimal error handling. If rag-cli fails, LLM receives no actionable error message.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 47 | <=2025-11-25 | v2 |
Tool 'deep' name is ambiguous. Does it mean 'deep dive', 'deep search', or 'deep learning'? Verb_noun convention violated. Should be 'search_subreddit_deep' or 'deep_search_subreddit' for clarity.