Loading ToolBench
FastAPI-based server that converts React code to Next.js 13+ using Claude AI. Supports single file migration and bulk repository migration with MCP protocol integration.
Server has 5 tools with basic descriptions and visible input schemas, but significant quality gaps. Tool names lack clear verb-noun structure (e.g., 'list-react-files' uses hyphens inconsistently). Descriptions are present but generic (avg ~80 chars, below 194-char baseline). Parameter descriptions exist but lack constraints, formats, and validation guidance. No output schemas documented. Error handling is minimal, errors return raw strings without recovery guidance. No tool annotations (readOnlyHint/destructiveHint). Security concerns: tools accept repo URLs and file paths without sanitization, risking path traversal and arbitrary code execution via Claude API prompts.
Download the migrated repository as a zip file
Fetch a file from the local React demo repository
Clone a GitHub repository and list all React/TypeScript files found in it
Convert a plain React component to a Next.js 13+ component using the App Router and TypeScript
Migrate all React/TypeScript files in a cloned repository to Next.js 13+ and return a downloadable zip file
No output schemas documented. LLMs cannot predict response structure, forcing them to parse unstructured text. migrate-repo returns a download_url string; migrate returns convertedCode without type info. Agents cannot chain tools reliably.
Path traversal vulnerability: fetch accepts arbitrary 'file' parameter without validation. An attacker could pass '../../../etc/passwd' to read outside the intended repo. No input sanitization or allowlist.
No error recovery guidance. Errors return raw strings ('error': str(e)) without actionable next steps. E.g., 'Failed to migrate file.py: [Errno 2] No such file or directory' tells the LLM nothing about what to do.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Parameter descriptions lack constraints and formats. 'repo_url' has no validation hint (must be valid GitHub URL). 'file' has no format guidance (relative path only, no absolute paths). 'code' has no length limit or language hint.
Tool names use inconsistent conventions. 'list-react-files' and 'migrate-repo' use hyphens; 'fetch' and 'migrate' do not. Naming should follow verb_noun pattern (e.g., list_react_files, migrate_repository). Inconsistency confuses LLM tool selection.