AI-native Python toolkit for maintaining codebase integrity with MCP server support for graph queries, RBAC, sessions, and monitoring
SIOF has 8 tools with basic schemas and descriptions, but significant quality gaps prevent production use. All tools have descriptions (10-80 chars, mostly under the 194-char baseline), and input schemas are present with type definitions. However, descriptions are generic and lack LLM-optimized guidance on WHEN to use each tool or WHAT happens next. Parameter descriptions are minimal (e.g., 'Scope to analyze' for get_dead_paths lacks format/range guidance). No output schemas are documented, LLMs cannot plan downstream calls. Error handling is absent: no recovery guidance, no categorization of retryable vs fatal errors. The apply_patch_to_file tool is destructive but has no confirmation pattern, dry-run option, or audit trail documentation. Tool composition is weak: find_data_lineage and impact_of_change overlap semantically but lack clear disambiguation. No pagination support visible despite tools returning potentially large result sets (dead_paths, unhandled_exceptions). Security: no evidence of permission gates, audit logging, or secret injection patterns despite enterprise claims.
Apply a patch to a file
Find data lineage for a node or symbol
Find unhandled exceptions in the codebase
Get dead code paths in the codebase
Get developer intent history for a symbol or area
Get energy consumption metrics for a run
Analyze impact of changes to a file or symbol
No output schemas documented. LLMs cannot infer what fields to expect from tool responses, preventing downstream tool chaining and forcing wasteful discovery calls.
Destructive tool (apply_patch_to_file) lacks confirmation pattern, dry-run option, or audit trail documentation. No guidance on error recovery or rollback.
Parameter descriptions are generic and lack actionable constraints. 'Scope to analyze' (get_dead_paths, find_unhandled_exceptions) does not specify format, valid values, or examples. LLMs cannot self-correct invalid input.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 48 | <=2025-11-25 | v2 |
Validate relationship between source and target nodes
No error handling guidance. Tools do not document what errors are retryable, what user action is needed, or how to recover. LLMs receive no direction on next steps after failure.
Semantic overlap between find_data_lineage and impact_of_change. Both analyze code relationships but lack clear disambiguation in descriptions. LLMs will struggle to choose the right tool.