Loading ToolBench
An MCP server for the codetoprompt library, enabling integration with LLM agents.
The server provides three well-intentioned tools for code analysis and context extraction with proper Pydantic schema definitions and clear descriptions. However, there are notable gaps: (1) output schemas are not documented, LLMs don't know what fields to expect from responses; (2) parameter descriptions are present but lack specificity around formats, ranges, and constraints (e.g., what constitutes a valid 'glob pattern'?); (3) tool names use hyphens (ctp-get-context) which is non-standard and makes verb-based parsing harder; (4) error handling guidance is absent, no recovery hints when a path doesn't exist or gitignore parsing fails; (5) no permission or scope declarations despite these being file I/O operations. The Pydantic models are well-structured and schemas are syntactically correct, which prevents lower scores, but the lack of output documentation and actionable error messaging keeps this at the C+/B- boundary.
Provides a detailed statistical analysis of a codebase, including token counts, line counts, and breakdowns by file type.
Generates a comprehensive, context-rich prompt from an entire codebase directory, applying filters and formatting options.
Retrieves the content of a specific list of files from the project, formatted into a prompt.
Output schemas are not documented. None of the three tools declare what fields/structure the response contains. LLMs cannot plan multi-step operations or extract required IDs when they don't know what the response looks like.
Tool names use hyphens (ctp-get-context) rather than underscores (ctp_get_context). This non-standard convention complicates LLM verb parsing and conflicts with most production MCP tools which use snake_case or camelCase. LLMs may struggle to recognize these as action verbs.
No error recovery guidance. When root_path doesn't exist, gitignore parsing fails, or files are unreadable, the tools provide no actionable next steps. Error responses should guide the LLM (e.g., 'Path not found. Verify the project root with list_directory() tool.', but no such tool exists either).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 38 | - | v1 |
Parameter constraints and formats are underdocumented. 'include_patterns' accepts 'Comma-separated glob patterns' but no examples, format regex, or constraints are provided. What happens if you pass invalid glob syntax? Can you pass Windows paths on Linux? No guidance for LLM input validation.
No scope or permission declarations. These tools perform file I/O on the server's filesystem. No indication of what permissions they require (read:filesystem, read:project, etc.) or audit logging to trace which agent accessed which files. Security-critical for multi-tenant deployments.
Path traversal risk. The 'paths' parameter in ctp-get-files accepts relative paths from root_path but doesn't explicitly validate against '..' or absolute paths. An LLM could be tricked into passing '../../../etc/passwd' and leak sensitive files.
No pagination or result limits. ctp-get-context and ctp-analyse-project can return entire projects, potentially megabytes of tokens. For large codebases, this exhausts context windows and wastes cost. No limit parameter, no pagination cursor, no warning.
Inconsistent parameter naming. 'root_path' vs 'root_dir' (used in CodeToPrompt init). 'paths' is generic; 'file_paths' would be clearer. Field naming should match downstream API to avoid LLM confusion.