Loading ToolBench
An AI-powered multi-agent crew system for end-to-end software development, from design through deployment. Orchestrates specialized agents (System Architect, Software Designer, Unit Testing Engineer, Software Developer, QA Automation Engineer, DevOps Engineer) using CrewAI with MCP tool integrations.
DevCrew exposes only 2 tools with minimal schema documentation and descriptions that lack actionable guidance for LLM selection. The 'clone github project' tool has a basic 1-parameter schema with a short description (23 chars). The 'run local command' tool has 2 parameters but descriptions are equally sparse. Neither tool provides error handling guidance, recovery paths, or context about side effects. The descriptions fail to explain WHEN to use these tools, WHAT they return, or WHY an LLM should select them over alternatives. No output schemas are documented. No parameter constraints (ranges, enums, patterns) are visible. The implementations appear to exist in Tools/execute_command.py but lack the structured registration and metadata expected of production-grade MCP tools. The server itself is a CrewAI wrapper that delegates to MCP providers (GitHub, Render, Playwright, Google Docs, Jira), but exposes only these 2 core execution tools. This is a significant usability and safety gap, agents have no guidance on validation, retry logic, or what data they should expect.
Clone a github project to output directory.
Prints the command, asks the user for confirmation, and runs it if approved.
Tool descriptions are under 50 characters and lack WHEN/WHY context. 'Clone a github project to output directory.' tells what happens but not when to use it, how it differs from alternatives, or what side effects occur. LLMs cannot reliably select these tools without richer guidance.
No output schemas are documented. 'clone github project' presumably returns success/failure and a cloned path, but agents cannot plan downstream tool calls without knowing the response structure. 'run local command' returns command output, but character limit and structured field names are unknown.
No error handling or recovery guidance. If 'clone github project' fails (invalid URL, network timeout, permission denied), agents get no instruction on what to do next. Pattern: recovery-guide requires 'Try search_users() first' style guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 31 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 20 | - | v1 |
'run local command' is marked DESTRUCTIVE but has no confirmation-request or dry-run capability. Agents can invoke destructive shell commands (rm -rf, dropdb, etc.) without safeguards. This violates the confirmation-request pattern for irreversible operations.
Parameters lack descriptions and constraints. 'github_url' and 'command' have descriptions, but 'directory' (in run local command) is underdescribed. No mention of required format (absolute vs relative paths), length limits, or character restrictions. LLMs may pass invalid paths without validation guidance.
'run local command' tool description says 'Prints the command, asks the user for confirmation, and runs it if approved.' This is a runtime behavior claim, not a tool definition. The description should explain WHAT the tool does, not HOW the internal implementation works. No mention of return value, error handling, or timeout behavior.
Naming: 'clone github project' is vague. Better names following verb_noun convention: 'clone_repository', 'git_clone', or 'clone_from_github'. 'run local command' is similarly generic, what type of command? Shell? Container? Script? No prefix hints the action scope.
No parameter typing or schema clarity visible in source. The input schema shows {'github_url': {'type': 'string', ...}} but lacks minLength, pattern, or enum constraints. For security, 'github_url' should validate URL format; 'directory' should validate path traversal (no '..' allowed). LLMs cannot self-validate without explicit constraints.
Security: 'run local command' accepts arbitrary shell commands and runs them after user confirmation. No mention of sandboxing, allowed command categories, or injection prevention. A malicious agent could invoke 'rm -rf /', 'curl <exfil-server>', or other destructive payloads. Requires scope declaration and permission gating.