A collection of MCP server examples and implementations demonstrating integration with CrewAI, including ComfyUI image generation, financial analysis, marketing strategy, autonomous software agencies, and various utility servers.
CrewAI-MCP-Lab is a collection of example agents with 16 tools scattered across multiple files. Tool definitions are visible in source code but lack rigor: descriptions are present but generic (avg ~50 chars, well below the 194-char baseline); input schemas exist but are minimal (no enums, no constraints, no output schemas documented); no error handling guidance; no tool annotations (readOnlyHint/destructiveHint); no pagination support for list operations. The 'Execute Shell Command' tool is particularly dangerous, accepts arbitrary commands with no validation, sanitization, or permission gates. File I/O tools lack path traversal protection. Overall, definitions are functional for demo purposes but fall short of production quality.
Create a new directory.
Execute a shell command. Use this to run tests, install dependencies, etc.
List files and directories in a given path.
Read content from a file.
Reads a file's content.
Retrieve a specific memory by its key.
Run tests using pytest and return structured JSON results.
Runs pytest on the specified directory or file. Returns a structured summary of the results.
Execute Shell Command tool accepts arbitrary commands with no validation, sanitization, or permission gates. Agents can invoke destructive commands (rm -rf, format disk) without confirmation or audit trail.
File I/O tools (Write File, Read File, List Files) lack path traversal protection. Relative paths are resolved via environment variables, but no validation prevents '../../../etc/passwd' attacks or escaping the workspace root.
No output schemas documented for any tool. LLMs cannot predict response structure, forcing them to parse unstructured output or make assumptions about field names and types.
Duplicate tool definitions: 'Run Tests' appears twice (tools 6 and 9) with slightly different parameter names (test_dir vs test_path). LLMs will be confused about which to use.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 58 | 2026-07-28+ | v2 |
Save a piece of information to the persistent memory. Useful for storing architectural decisions, bug reports, or project status.
Writes code to a file.
Write content to a file. Overwrites if exists.
Pretend to generate an image and write a 1x1 PNG to the requested path.
List the workflows this mock server can run.
Pretend to run an img2img workflow and write a different 1x1 PNG.
Pretend to remove the background and write a transparent PNG.
Pretend to upscale an image and write a slightly different PNG.
No error handling guidance in any tool description. Agents do not know what to do if a file is not found, a command times out, or a directory already exists. Descriptions lack recovery hints.