Loading ToolBench
Turn plain-English requirements into production-ready multi-agent AI projects (CrewAI, LangGraph, WatsonX Orchestrate). Exposes the local-first build loop as live MCP tools over STDIO transport.
The agent-generator MCP server defines 6 tools with matrix_* naming convention, but exhibits significant gaps across naming clarity, parameter descriptions, schema completeness, and error handling. Tool names (matrix_plan_batch, matrix_prompt, matrix_check, matrix_repair, matrix_commit, matrix_publish) use a prefix rather than action verbs (plan_, generate_, validate_, create_repair_, finalize_, distribute_). Descriptions are present but generic and lack WHEN/WHY guidance. Parameter schemas are visible but descriptions are inconsistent, some parameters (like 'goal', 'batch_id', 'coder') lack clarity on format/constraints. No output schemas are documented. The server is STDIO-only transport with no toolAnnotations (readOnlyHint/destructiveHint/idempotentHint), limiting agent understanding of side effects and idempotency. Error handling is not visible in the provided code.
Validate the coder's changes against quality rules. Runs the validation engine and reports violations with severity levels and remediation guidance.
Commit a successful batch and transition the project state. Marks validation as passed and increments commit counters.
Plan a new batch of work for the controlled development loop. Generates a batch plan with structured goal, change type, and ordinal tracking.
Generate prompt and helper files for the coder to execute the batch plan. Writes .gitpilotrules and coder-specific prompts to the project.
Publish the completed project to Hugging Face Spaces or other distribution targets. Bundles the project and uploads it.
Generate repair prompts for validation issues. Provides coder-specific guidance to fix violations from a failed validation report.
CRITICAL: No tool annotations present. Tools with state-modifying side effects (matrix_plan_batch, matrix_prompt, matrix_commit, matrix_publish, all WRITE risk) lack destructiveHint=true, and read-only tools (matrix_check, matrix_repair) lack readOnlyHint=true. This prevents agents from reasoning about idempotency and retry safety.
CRITICAL: No output schemas documented for any tool. Agents cannot know what fields to expect (e.g., batch_id returned by matrix_plan_batch, violation list structure from matrix_check). This breaks tool chaining and forces agents to guess field names.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 44 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
HIGH: Action verbs missing or domain-specific. Tool names (matrix_plan_batch, matrix_check, matrix_repair, matrix_publish) use 'matrix_' prefix instead of action verbs (create_, validate_, generate_, distribute_). LLMs depend on verb_noun naming to infer intent.
HIGH: Critical parameter constraints are missing or under-specified. 'target' in matrix_publish is a free-form string but the description lists only 'huggingface_space' and 'github', should be an enum. 'commit_ref' lacks format guidance (SHA vs ref name). 'goal' in matrix_plan_batch has no length/format constraints.
HIGH: Credential leak risk in matrix_publish. The 'config' parameter is a generic object expected to contain 'HF token, repo name, etc.' per the description. Tokens/secrets must never be passed as tool parameters, use server-side environment variable injection instead.
MEDIUM: Optional parameters have unclear precedence. In matrix_check, 'changed_files' (array) and 'result_json_path' (string) are both optional. Unclear which takes precedence or what happens if both are provided. In matrix_repair, 'batch_id', 'validation_id', and 'issue' have complex optional relationships undocumented.
MEDIUM: No dry-run or confirmation pattern for irreversible operations. matrix_commit marks a batch as permanent (transitions project state), matrix_publish sends code to internet, no mention of preview, rollback, or confirmation steps to prevent agent mistakes.
MEDIUM: Descriptions are too brief or generic. Average description length ~50 chars, below the 194-char baseline. 'Validate the coder's changes against quality rules' and 'Commit a successful batch and transition the project state' lack context on WHEN to call, what structure is returned, or error recovery guidance.
MEDIUM: No error handling or recovery guidance documented. If matrix_plan_batch dry_run shows conflicts, what should the agent do? If matrix_commit fails, is it retryable? No error messages visible in source or in tool descriptions.