Scans AI-generated code for invisible Unicode, Trojan Source, and supply-chain threats.
Five tools with clear security-focused purpose, but significant gaps in parameter descriptions and output schema documentation. All tools have descriptions (10-150 chars), but most lack detail on expected return formats. Input schemas present but sparse: scan_file and scan_directory lack descriptions for their parameters; ai_analyze's 'code' param is minimally described. No enum constraints, no pagination guidance, no error recovery hints. Naming is verb-forward (scan_*, check_*, ai_analyze) but descriptions don't explain WHEN to use each tool or how they differ. Output schemas are entirely undocumented, LLMs cannot plan downstream actions or extract structured results.
Deep AI analysis of code using the trained CodeBERT model. Classifies code chunks as malicious or benign with confidence scores. Detects obfuscation and complex attack patterns.
Check package.json for typosquatting, suspicious install scripts, and dependency risks
Recursively scan a directory for hidden malicious code patterns across all source files
Scan a single file for hidden malicious code patterns (invisible chars, BiDi, homoglyphs, steganography, obfuscation, etc.)
Scan an AI configuration/rules file for prompt injection and Rules File Backdoor attacks
No output schemas documented. LLMs cannot infer what scan_file, scan_directory, scan_rules_file, or check_dependencies return. Are results arrays? Objects? What fields? This blocks downstream tool composition and forces LLMs to guess.
Parameter descriptions missing or minimal. scan_file's 'file_path' and scan_directory's 'dir_path' and 'extensions' lack guidance on format, constraints, or examples. ai_analyze's 'code' param has only 'Code snippet to analyze', no length limits, language hints, or expected structure.
No error handling guidance. Tools lack descriptions of failure modes (file not found, permission denied, model inference timeout) or recovery hints. LLMs cannot self-correct or retry intelligently.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | 2026-07-28+ | v2 |
Tool selection ambiguity. scan_file, scan_directory, and scan_rules_file all scan for threats but descriptions don't clarify when to use each. scan_rules_file targets 'prompt injection and Rules File Backdoor attacks', but scan_file also scans for 'hidden malicious code patterns'. Overlap is not explained.
No constraints on numeric/array parameters. scan_directory's 'extensions' array has no length limit, format guidance, or examples. ai_analyze's 'code' param has no max length, LLMs could pass megabyte-sized inputs, causing timeouts or OOM.