Loading ToolBench
Standardized Agent Runtime — preset-based AI agent gateway with stable HTTP API
This server exposes 7 tools for agent recommendation and repo-to-agent discovery. All tools have descriptions and input schemas are visible in code. However, critical gaps undermine quality: (1) Parameter descriptions are minimal or absent, most inputs lack field-level documentation explaining what each parameter controls or expects. (2) Output schemas are not documented, callers cannot see what fields to expect from responses. (3) Tool names lack action verbs (e.g., 'recommend_tools_for_agent' is acceptable, but 'discover_tools_from_repo', 'get_paths_to_inspect_for_code_tools', 'wrap_discovered_tools' are vague about what 'discover' and 'wrap' mean in context). (4) No error handling guidance, tools do not indicate what errors are retryable, user-fixable, or fatal. (5) No security annotations, tools accept complex nested objects without validation hints. Baseline: 194 chars avg description, 72 chars avg param annotation. Most tools fall short on param annotation depth.
Classify risk level of a discovered tool from name, command, and source path.
Recommend bundle_id and additional tool_ids from repo scout + architect output.
Produce a conservative list of likely source file paths to fetch for code tool detection.
Return (file_paths, folder_paths) to fetch for tool discovery.
True only for low-risk tools. Medium/high risk must be surfaced but not auto-enabled.
Deterministic recommendation over catalog bundles and tools (layered pipeline).
Output schemas not documented. Callers cannot see what fields responses contain, forcing them to guess or inspect code. This violates pattern:tool and pattern:response-shaper.
Parameter descriptions are minimal or absent. Input objects like 'scout', 'arch', 'discovered_tools' lack field-level documentation. LLMs cannot infer what nested fields mean (e.g., does 'scout' contain 'repo_summary' or 'summary'?). Violates pattern:tool-description.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Convert discovered repo tools into structured wrapped tools (metadata only; no execution).
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal. No recovery hints (e.g., 'If repo not found, try search_repos() first'). Violates pattern:recovery-guide.
Vague tool names. 'wrap_discovered_tools' and 'discover_tools_from_repo' do not clearly convey what happens. 'wrap' is ambiguous, does it serialize, validate, or transform? 'discover' is generic. Violates pattern:tool naming guidance (verb_noun, clarity).
No input validation hints. Complex nested objects (scout, arch, discovered_tools) lack constraints, format specs, or required field lists. LLMs cannot validate inputs before calling. Violates pattern:constrained-input.