Loading ToolBench
Server defines 4 tools with complete input schemas and descriptions. All tools follow verb_noun naming (get_, search_, list_). Descriptions are concise (60-180 chars) and explain WHAT the tool does. However, output schemas are undocumented, LLMs cannot predict return structure. Parameter descriptions lack format constraints (e.g., no guidance on valid package names, search query syntax). Error handling returns structured {error, message} but lacks recovery guidance. No tool annotations (readOnlyHint, idempotentHint). Missing pagination details in search_packages despite returning 'next' field.
Get metadata for a Dart/Flutter package on pub.dev: latest version, description, homepage, repository, supported SDK environment, and publish date (from the pubspec).
Get the pub.dev quality score for a Dart/Flutter package: granted/max pub points, like count, popularity, 30-day download count, and classification tags (platforms, SDKs, license).
List all published versions of a Dart/Flutter package on pub.dev, with each version string and its publish date, plus the current latest version.
Search pub.dev for Dart & Flutter packages by keyword. Returns matching package names and a next-page URL.
Output schemas are not documented. LLMs cannot predict return structure (e.g., what fields are in the response, their types, whether fields can be null). This forces agents to guess and risks parsing errors.
Parameter descriptions lack format constraints. 'name' parameter has no guidance on valid package name format, length, or character restrictions. 'query' in search_packages has no syntax hints. This invites invalid input from LLMs.
Error responses lack recovery guidance. When pubGet() returns {error, message}, the LLM receives no hint on whether to retry, call a different tool, or ask the user. E.g., 'Package not found. Try search_packages() to find similar names.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
search_packages returns 'next' field for pagination but tool description does not explain pagination behavior or how to use 'next'. LLMs may not understand how to fetch subsequent pages.
No tool annotations (readOnlyHint, idempotentHint). All tools are read-only and idempotent, but this is not declared. Agents cannot infer safety properties and may apply unnecessary retry logic or caution.