Code Intelligence OS for AI Agents — persistent memory, GitHub intelligence, structural search
Critical failures across naming, descriptions, and schemas. Most tools (30/36) lack visible input schemas in source code, only 6 tools show explicit schema definitions (forge_discover, forge_describe, github.search_repos, github.search_code, github.repo_overview, github.repo_file, github.repo_tree, import.plan, import.validate, import.adapt_style, evidence.query, evidence.add). The remaining 24 tools are inferred from dynamic-tools.ts comments without explicit registration or schema visibility. Descriptions are present but generic (10-50 chars), failing the 50-200 char LLM-optimized baseline. No parameter descriptions visible for most tools. No output schemas documented. Tool names lack consistent verb_noun pattern (e.g., 'genius.hunt', 'code.reach', 'memory.recall' are vague). Dynamic tool pattern reduces token cost but sacrifices schema clarity and LLM discoverability.
Repository structure map with architecture detection
Find symbol with callers, callees, and deps
List exported symbols in a scope
Trace call chain between two functions
Compressed module analysis with purpose detection
Add a node to the evidence graph.
Query the evidence graph across repos, blobs, symbols, patterns, and archetypes.
24 of 36 tools lack visible input schemas in source code. Only inferred from comments in dynamic-tools.ts. Per HARD SCORING RULE, tools without visible schemas must score 0 on schema dimension.
Tool descriptions are generic and under 50 characters (baseline 50-200 chars for LLM optimization). Examples: 'Cross-source code search with structural ranking' (49 chars), 'Find single best implementation of a pattern' (43 chars), 'Score breakdown for a code result' (32 chars). Descriptions lack WHEN to use, prerequisites, or return value hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 40 | 2026-07-28+ | v2 |
Get full input schema and usage example for a ForgeMCP tool. Call forge_discover first to find tool names.
Search ForgeMCP capabilities by intent. Returns matching tool names with one-line descriptions. Example: forge_discover('find rate limiter code') → genius.hunt, genius.find_best
Score breakdown for a code result
Find single best implementation of a pattern
Cross-source code search with structural ranking
Fetch repository actions/workflow information.
Fetch contributor information, preferring GraphQL hydration when available.
Fetch the content of a repository file.
Fetch repository issues.
Fetch a language breakdown for a repo.
Fetch a repository overview with stars, license, activity, and quality hints.
Fetch repository pull requests.
Fetch repository releases.
Fetch a repository tree listing.
Search GitHub code with the dedicated code-search budget.
Search GitHub repositories with rate-budget awareness.
Adapt imported code to match local quotes, semicolons, and indentation.
Extract code from GitHub with provenance tracking
Resolve a provenance-aware import plan with dependencies, policy, and conflict analysis.
Validate that an import plan is internally consistent and dependency-complete.
Version a pattern while preserving lineage
Search code memory for patterns from past sessions
Memory health and confidence distribution
Store code pattern with confidence score
Add evidence to ongoing investigation
Trace code evolution via git history
Conclude research with validated findings
Multi-repo quality comparison
Begin multi-step research investigation
Tool names lack consistent verb_noun pattern. 'genius.hunt', 'code.reach', 'memory.recall', 'research.archaeology' are vague and do not clearly signal action. LLMs cannot infer intent from names alone. Should be: search_code_patterns, find_symbol_references, retrieve_pattern_memory, trace_code_history.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., does genius.hunt return symbol_id, file_path, line_number?). Baseline: 100% of A+ tools have documented return types.
No error handling guidance visible. Tools do not document what errors are retryable, user-fixable, or fatal. No recovery hints (e.g., 'If symbol not found, try search_code() first'). Baseline: A+ tools include recovery guides.