Loading ToolBench
MCP server that extracts complete knowledge from any codebase — architecture, patterns, dependencies, API surface
Ferret has 8 tools with clear action verbs (scan, analyze, extract) and reasonable descriptions (100-250 chars). However, critical gaps undermine production readiness: (1) Input schemas are visible but minimal, only 'path' (string) for most tools, with no enum constraints, min/max bounds, or validation hints. (2) Parameter descriptions are present but generic ('Absolute path to the repository root directory'), they lack format guidance, error recovery hints, or examples of valid input. (3) Output schemas are completely undocumented, LLMs cannot predict what fields to expect from scan(), dependencies(), or architecture(). (4) No error handling guidance, tools that depend on external LLM APIs (deep, ask) have no documented failure modes, retry logic, or recovery paths. (5) Tools 7-8 (deep, ask) require environment variables (FERRET_LLM_API_KEY, FERRET_LLM_PROVIDER) but do not document what happens if they are missing. (6) No tool annotations (readOnlyHint, idempotentHint) despite all tools being read-only. Baseline: 94% of A+ tools have full param descriptions; 100% document return types. Ferret has neither.
Extract the complete API surface — REST endpoints, MCP tools, CLI commands, GraphQL, gRPC, public exports.
Analyze repository architecture — layers, patterns (MVC, hexagonal, pipeline, etc.), module breakdown.
Ask a specific question about a repository, answered by AI with full codebase context. The LLM receives all static analysis data + key file contents, then answers your question based on that evidence. Requires FERRET_LLM_API_KEY (for Anthropic) or a local LLM server.
AI-powered deep analysis — produces a comprehensive Knowledge Extraction Report. Uses an LLM (Anthropic or local) to analyze all static extraction data plus key file contents, producing an expert-level architectural analysis with insights about design decisions, data flow, strengths, risks, and learning path. Requires FERRET_LLM_API_KEY (for Anthropic) or a local LLM server. Configure via env vars: FERRET_LLM_PROVIDER, FERRET_LLM_MODEL, FERRET_LLM_BASE_URL.
Extract all dependencies — external packages and internal import graph with core modules.
No output schemas documented. LLMs cannot predict return types (e.g., does scan() return a dict, string, or structured object?). This violates pattern:tool and forces LLMs to guess field names for downstream processing.
Parameter descriptions are generic and lack validation guidance. 'Absolute path to the repository root directory' does not explain: must it exist? What happens if it doesn't? Should it be relative or absolute? This violates pattern:tool-description and pattern:constrained-input.
Tools deep() and ask() depend on environment variables (FERRET_LLM_API_KEY, FERRET_LLM_PROVIDER, FERRET_LLM_MODEL, FERRET_LLM_BASE_URL) but do not document failure modes. What error does the LLM see if FERRET_LLM_API_KEY is missing? This violates pattern:recovery-guide.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Run ALL analyses and return a comprehensive knowledge extraction report. Combines: scan, dependencies, architecture, patterns, and API surface into a single document. Use this for complete repo understanding.
Detect code patterns and conventions — design patterns, naming, testing, error handling, config.
Scan a repository and return an overview: languages, file structure, entry points, config files.
No tool annotations despite all 8 tools being read-only. Missing readOnlyHint annotation prevents agents from understanding which tools are safe to call without side effects. This violates current MCP spec (2026-07-28) tool annotation requirement.
No pagination or result limits documented. Tools like scan() and full_extraction() may return very large outputs (entire repo structure, all dependencies). No mention of max result size, truncation, or pagination guidance. This violates pattern:paginated-result and mxe:enforce-result-limits.