Loading ToolBench
MCP server that builds a SPARQL-queryable RDF knowledge graph of your codebase
Beret provides 16 well-intentioned tools for codebase analysis with consistent naming (verb_noun pattern: query_codebase, find_symbol, list_structures, describe_*). Descriptions are present and moderately detailed (avg ~120 chars), explaining WHAT each tool does. However, critical gaps emerge: (1) Input schemas lack formal type definitions and constraints, parameters are documented in descriptions but not in JSON Schema with proper 'type', 'enum', or 'minLength' declarations. (2) Output schemas are entirely undocumented, LLMs cannot predict what fields to expect from query_codebase or find_dead_code. (3) No error handling guidance, tools return generic Result<Value, String> with no recovery hints. (4) Parameter descriptions lack actionable constraints (e.g., exclude array format, query syntax rules). (5) No pagination or result-limiting guidance despite tools that could return large datasets (list_structures, find_dependencies). Naming is strong; descriptions are adequate but not LLM-optimized; schemas are the critical weakness.
Describe git activity: contributors, active files, recent commits, and development velocity.
Describe codebase architecture: detected layers, structure counts, monorepo detection, and architectural patterns.
Describe dependency management: package managers, dependency counts, and dependency graph insights.
Describe all detected engineering practices in the codebase, including CI/CD platforms, test frameworks, linters, formatters, build tools, containerization, package managers, type checking, architecture layers, documentation, conventions, deployment platforms, code analysis tools, packaging formats, and configuration management.
Consolidated project analysis combining practices, testing, architecture, dependencies, and activity.
Describe testing practices: frameworks used, test file count, test-to-code ratio, test dependencies, and test coverage if available.
Output schemas completely undocumented. Tools return Value (JSON) with no declared field structure. LLMs cannot predict what fields exist (e.g., does query_codebase return 'results' or 'bindings'? Does find_dead_code return 'func' or 'function'?). This forces LLMs to guess and wastes tokens on exploratory calls.
Input schemas lack formal JSON Schema type definitions. Parameters documented in descriptions (e.g., 'exclude: array of paths') but not declared with 'type': 'array', 'items': {'type': 'string'}, or constraints. LLMs cannot validate inputs before calling; servers receive malformed data.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 56 | 2026-07-28+ | v2 |
Get statistics on structure types in the codebase. Returns counts grouped by type.
Find all functions called by a given function. Optionally filter by caller name.
Find all functions that call a given function. Optionally filter by callee name.
Find functions that are never called (dead code analysis).
Find all file-level dependencies in the codebase.
Find likely entry points in the codebase (main, index, app, server, cli, cmd functions/files).
Find symbols (functions, classes, etc.) by name substring. Returns matching symbols with their types.
Generate a LikeC4 architecture diagram of the codebase based on detected layers and structure.
List all structures (functions, classes, etc.) in the codebase. Optionally filter by path or kind.
Execute a SPARQL SELECT or ASK query against the codebase knowledge graph. The graph uses the repo: prefix (http://repo.example.org/). Triple patterns: ?s <repo:a> ?type (structure types), ?s <repo:calls> ?t (call relationships), ?s <repo:dependsOn> ?t (dependencies), ?s <repo:inFile> ?f (file locations), ?s <repo:hasLine> ?l (line numbers).
No error handling guidance. Tools return generic Err(String) with no recovery hints. If query_codebase fails due to invalid SPARQL syntax, the LLM receives only the error message with no suggestion to validate syntax or try a simpler query. Agents cannot self-correct.
No pagination or result-limiting guidance. Tools like list_structures and find_dependencies could return hundreds of results, exhausting context windows. No mention of limits, pagination parameters, or result caps in descriptions.
Parameter descriptions lack actionable constraints. 'exclude: array of paths' does not specify format (gitignore patterns? regex? exact paths?). 'query: SPARQL SELECT or ASK query string' does not explain valid triple patterns or namespace prefixes. LLMs must infer rules from examples.