An MCP server for architectural scanning, security auditing, ML asset tracking, and code metrics analysis of projects.
CodeOracle has 4 tools with basic verb-noun naming (audit_*, check_*, get_*), but critical gaps undermine quality. All tools have descriptions (10-50 chars), but they are too brief and lack context on WHEN to use each tool or what distinguishes them. Input schemas are present but minimal: each tool accepts only a single 'directory_path' string parameter with a description. No output schemas are documented, responses are JSON strings with ad-hoc structures (e.g., {"Stack": [...], "Files": [...]}) that LLMs must parse without guidance. Error handling is minimal: tools return plain strings like 'Error: Path not found.' with no recovery guidance. No parameter validation beyond existence checks, no pagination, no rate limiting. Tools lack idempotency guarantees and do not document side effects or permissions. Security: tools perform filesystem traversal on user-supplied paths without sanitization, risking directory traversal attacks.
ML model asset discovery and size reporting.
Fast architectural scan of the tech stack.
Surgical security scan for hardcoded secrets.
Quantifies the scale of the project (Lines of Code).
No output schemas documented. Tools return JSON strings with ad-hoc field names (e.g., 'Stack', 'Files', 'Models Found', 'Line Count Breakdown'). LLMs cannot plan downstream operations or extract structured data without knowing the response format.
Descriptions are too brief (20-50 chars) and lack context. 'Fast architectural scan of the tech stack' does not explain WHEN to call audit_project vs get_code_stats, what tech stacks it detects, or what the output contains. Descriptions should be 50-200 chars with WHAT, WHEN, and output hints.
Path traversal vulnerability: tools accept 'directory_path' without validation. An LLM could pass '../../../etc/passwd' or absolute paths outside the intended scope. Sanitize paths, enforce a base directory, and reject '..' sequences.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 46 | 2026-07-28+ | v2 |
Error handling is non-actionable. 'Error: Path not found.' gives the LLM no recovery path. Errors should suggest next steps: 'Path not found. Verify the directory exists and is readable. Try listing parent directories with get_code_stats().'
No pagination or result limits documented. Tools walk entire directory trees and return all results. Large projects could return thousands of files/models, exhausting context windows. Implement limit parameters (default 50) and document in descriptions.