Loading ToolBench
Two tools with basic descriptions and partial schemas. Both tools have input schemas with types and descriptions, meeting minimum requirements. However, descriptions lack actionable context for LLM selection (no WHEN to use, no prerequisites, no error guidance). Output schemas are completely undocumented, LLMs cannot plan downstream calls or extract structured data. No error handling guidance; exceptions return raw error strings. Tool names are verb-noun but descriptions are generic. Parameters lack constraints (enums, ranges, formats). No pagination support despite tools returning lists. Baseline: avg tool description 194 chars; these are ~150 chars but lack specificity. No per-tool error recovery hints.
Fetch hot threads from a subreddit Args: subreddit: Name of the subreddit limit: Number of posts to fetch (default: 10) Returns: Human readable string containing list of post information
Fetch detailed content of a specific post Args: post_id: Reddit post ID comment_limit: Number of top level comments to fetch comment_depth: Maximum depth of comment tree to traverse Returns: Human readable string containing post content and comments tree
No output schema documentation. LLMs cannot infer what fields are returned or plan downstream calls. Both tools return unstructured strings instead of structured JSON.
Descriptions lack actionable context. No WHEN to use, no prerequisites, no error guidance. Generic docstrings like 'Fetch hot threads' do not help LLMs decide between similar tools or understand failure modes.
No pagination support. fetch_reddit_hot_threads returns up to 10 posts (default) with no offset/cursor. Large result sets will blow context window. No total count or next_cursor returned.
Error handling returns raw exception strings ('An error occurred: {str(e)}') with no recovery guidance. LLMs cannot determine if error is retryable, user-fixable, or fatal.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
Parameters lack constraints. 'limit' and 'comment_limit' are unbounded integers, LLMs could pass 10000, causing API timeouts or rate limits. No min/max documented.