Loading ToolBench
Explore package health, security advisories, adoption, dependencies, releases, and ecosystem signals across supported registries.
Five tools with clear, well-structured schemas and descriptions. All tools have proper input parameter definitions with types and descriptions. Tool names follow verb_noun convention (inspect_*, search_*, check_*, compare_*). Descriptions are substantive (100-200 chars), explaining what each tool does and when to use it. However, output schemas are not documented in the source, only inferred from example JSON files. No error handling guidance, no recovery patterns, and no tool annotations (readOnlyHint, idempotentHint). Parameter constraints are present but could be more explicit (e.g., registry enum values are mentioned in descriptions but not formalized as JSON Schema enums). All tools are READ_ONLY, which is good for safety, but this is not declared via annotations.
Evaluate a package against minimum health and repository CI policies.
Compare two or more package choices using health, adoption momentum, community, release recency, and data completeness. Returns advisory recommendations with evidence and confidence; it never installs packages.
Fetch normalized package intelligence. Without a registry, resolve exact matches across supported ecosystems; with a registry, inspect that specific package source.
Load one project from secchi.toml and return its combined registry report.
Search supported package registries and return ranked normalized matches.
Output schemas not documented in source code. Only example JSON files visible; no formal schema definitions for tool responses. LLMs cannot plan downstream operations without knowing return structure.
Registry parameter accepts free-form strings instead of enum. Description lists valid values (pypi, crates, npm, homebrew, go, cran, pub) but no JSON Schema enum constraint. LLMs may hallucinate invalid registry names.
No error handling guidance. Tools do not document what happens on invalid input (e.g., nonexistent package, malformed registry name, network failure). No recovery patterns or actionable error messages.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Tool annotations missing. All tools are READ_ONLY but lack idempotentHint or readOnlyHint declarations. Agents cannot determine retry safety or side-effect risk from tool metadata.
search_packages limit parameter lacks explicit bounds. Description says '1-50' but no JSON Schema minItems/maxItems. LLMs may pass 0, negative, or >50 values.