Loading ToolBench
MCP server for skill creation, evaluation, and optimization. Provides tools for running evaluations, improving skill descriptions, generating reports, and packaging skills.
This server has 7 tools with descriptions and basic parameter schemas, but significant quality gaps prevent higher scoring. Most tools lack output schema documentation (critical for agent planning). Parameter descriptions are present but often generic. Tool names are reasonably clear (run_eval, improve_description, generate_report) but lack action-verb consistency. No error handling guidance, no security considerations documented, and no tool annotations (readOnlyHint/destructiveHint). The generate_review tool modifies state (writes feedback.json) but this is not clearly signaled in the description. Average tool score: 42/100.
Aggregate individual run results into benchmark summary statistics. Reads grading.json files from run directories and produces run_summary with mean, stddev, min, max for each metric and delta between configurations.
Generate an HTML report from run_loop.py output. Takes the JSON output from run_loop.py and generates a visual HTML report showing each description attempt with check/x for each test case.
Generate and serve a review page for eval results. Reads the workspace directory, discovers runs (directories with outputs/), embeds all output data into a self-contained HTML page, and serves it via a tiny HTTP server. Feedback auto-saves to feedback.json in the workspace.
Improve a skill description based on eval results. Takes eval results (from run_eval.py) and generates an improved description by calling `claude -p` as a subprocess.
Creates a distributable .skill file of a skill folder. Validates the skill and packages it as a zip file with excluded build artifacts.
Run trigger evaluation for a skill description. Tests whether a skill's description causes Claude to trigger (read the skill) for a set of queries. Outputs results as JSON.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does run_eval return? JSON structure?). This violates the response-shaper pattern and forces agents to guess at output structure.
generate_review modifies state (writes feedback.json, serves HTTP) but description does not signal this. Agents need to know which calls are safe to retry and which have side effects. Missing destructiveHint/idempotentHint annotations.
Parameter descriptions are generic and lack constraints. E.g., 'num_workers' has no min/max bounds (default 10, but what's the valid range?). 'timeout' lacks units clarity (seconds assumed but not stated). 'trigger_threshold' is a number but no range given (0.0 - 1.0 assumed).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Quick validation of a skill. Checks SKILL.md exists, validates YAML frontmatter, verifies required fields (name, description), and enforces naming conventions and character limits.
No error handling guidance. Tools call external processes (claude -p, HTTP server) but descriptions do not explain failure modes or recovery steps. E.g., what if claude -p is not installed? What if the port is already in use?
Tool names lack consistent action verbs. 'run_eval' and 'improve_description' are clear, but 'generate_report', 'aggregate_benchmark', 'generate_review', 'package_skill', 'validate_skill' use different verb patterns (generate, aggregate, package, validate). Inconsistency increases LLM disambiguation overhead.