Loading ToolBench
A dependency-free MVP for an OpenCLAW MCP experiment assistant. Manages experiment configurations, runs mock training loops, reads metrics, and generates diagnostic reports.
Server has 10 tools with basic schemas and descriptions, but significant gaps prevent production use. Only 6 of 10 tools have complete schema definitions in TOOL_SCHEMAS (validate_config, get_training_status, read_metrics, compare_runs missing). Descriptions are present but generic (avg ~80 chars, below 194-char baseline). Parameter descriptions are sparse or absent, most params lack explanation of format, constraints, or purpose. No output schemas documented. Error handling is minimal (generic exception catch). Tool names follow verb_noun convention but lack clarity on state-modifying operations. No tool annotations (readOnlyHint/destructiveHint) despite clear risk distinctions (WRITE vs READ_ONLY marked in metadata but not in schema).
Check one runtime integration by name, such as openclaw or hermas.
Compare metrics across multiple training runs.
Create a validated experiment config snapshot.
Get the current status of a training run.
List known OpenCLAW experiment runs and their current status.
List configured OpenClaw/Hermas runtime integrations and reachability.
Read metrics from a completed training run.
4 of 10 tools missing from TOOL_SCHEMAS registry (validate_config, get_training_status, read_metrics, compare_runs). Schema definitions incomplete, only 6 tools registered. LLM cannot discover or call unregistered tools.
No parameter descriptions for any tool. 'config' param in validate_config, 'run_ids' in compare_runs, 'name' in check_runtime_integration lack explanation of format, constraints, or purpose. LLMs cannot infer parameter meaning from names alone.
No output schemas documented. Tools return dicts but LLM has no visibility into response structure (fields, types, required keys). Prevents downstream tool chaining and forces LLM to guess field names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Run the mock training loop for an experiment.
Generate a training diagnosis report from metrics.
Validate experiment configuration and estimate training hours.
Tool annotations missing. Risk metadata (WRITE vs READ_ONLY) exists in server metadata but not in schema as readOnlyHint/destructiveHint/idempotentHint. LLM cannot distinguish safe from destructive operations.
Generic error handling. Exception caught and returned as string ('error': str(exc)). No recovery guidance, categorization (retryable vs fatal), or actionable next steps for LLM.