AI-assisted code security scanner with LangChain and LangGraph runtime.
CodeScan has 4 tools with complete input schemas and descriptions. All tools follow verb_noun naming (scan_*). Descriptions are adequate (60-90 chars) but lack actionable context for LLM selection. Parameter descriptions are present but minimal. Output schemas are documented via Pydantic models (MCPScanResult, MCPIssue) with typed fields. Error handling is basic, no recovery guidance or categorization. No tool annotations (readOnlyHint, destructiveHint). Missing dependency hints and prerequisites.
Scan a directory and return a structured repository security report.
Scan a single source file and return a structured security report.
Scan files changed between the current HEAD and a base Git branch or ref.
Clone a Git repository URL to a temporary directory and scan it.
Parameter descriptions are minimal (5-15 chars). 'Path to the file to scan' lacks format, constraints, or prerequisites.
Tool descriptions lack WHEN-to-use context and prerequisites. 'Scan a single source file...' does not explain when to prefer scan_file over scan_directory, or what file types are supported. LLMs cannot disambiguate without explicit guidance.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are read-only and idempotent, but this is not declared. Agents cannot infer safety properties without explicit hints.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | <=2025-11-25 | v2 |
Error handling is minimal. Validation errors (e.g., 'File does not exist') are raised as ValueError but not categorized as retryable or user-fixable. No recovery guidance (e.g., 'Try scan_directory instead' or 'Check the path and retry').
Output schema (MCPScanResult) includes 'stats' and 'project_info' as generic dicts. Verbose responses waste tokens and dilute signal.