Loading ToolBench
MCP server providing access to code knowledge through semantic search, keyword search, visual search, git grep, file listing, and enrichment-based code discovery tools. Integrates with repositories, commits, enrichments, and includes wiki and documentation retrieval capabilities.
Kodit provides 15 read-only tools for repository exploration with consistent naming (kodit_* prefix) and mostly complete schemas. All tools have descriptions and input parameters with type definitions. However, descriptions are often generic and lack depth about use cases and composition flow. Output schemas are not documented. Error handling guidance is missing. No tool annotations (readOnlyHint, idempotentHint) despite all tools being read-only. Parameter descriptions tend to be minimal. The semantic/visual/keyword search tools lack pagination and result limits documentation. No batch operation variants or natural-identifier support (e.g., accepting human-friendly repo names instead of requiring URLs). Overall structure is solid but lacks the depth and LLM-optimization seen in A-grade tools.
Interface documentation for a repository
High-level structure and design documentation for a repository
Recent changes and context from commits
Complete usage examples for a repository
Data models and database schema documentation
Search file contents using git grep with regex patterns (returns resource URIs)
Find files matching keywords using BM25 search (returns resource URIs)
List files matching a glob pattern in a repository
Output schemas not documented. LLMs cannot infer what fields search tools return (e.g., does semantic_search return file paths, snippet previews, relevance scores?). This forces agents to guess and wastes tokens on failed parsing.
Search tools (semantic_search, keyword_search, visual_search) lack pagination and result-limit documentation. Descriptions mention 'top_k' and 'limit' parameters but do not state whether results are sorted by relevance, capped at a server maximum, or how to handle large result sets. This risks context window exhaustion.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | B | 74 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 26 | - | v1 |
Read file content from a resource URI returned by search tools. Supports line ranges, line numbers, and document page rendering (raster/text mode)
Lists all tracked repositories. Discover available repositories (call this first!)
Find files matching a natural language query using semantic similarity (returns resource URIs)
Returns the kodit server version
Find document pages (PDFs, etc.) matching a text query using visual similarity
Get the table of contents for a repository's wiki
Get the content of a specific wiki page by slug
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All 15 tools are read-only, but clients cannot see this from the tool definition. Agents may assume tools can have side effects and plan unnecessarily conservatively.
Repository parameter is a URL string (e.g., repository_url). Typical users know repo names ('myproject'), not full Git URLs. Tool should accept both human-friendly repo names and URLs, with server-side resolution or clear lookup documentation.
Parameter descriptions are minimal (10-30 chars). E.g., kodit_semantic_search 'query' is described as 'Natural language search query' but does not explain best practices (short vs. long queries, example topics, when to use semantic vs. keyword search). Descriptions lack context to guide LLM decision-making.
No error handling guidance. Tools lack descriptions of failure modes (e.g., 'If repository is not found, what should the agent do?', 'Are search results paginated or capped?', 'What happens if a file is binary?'). LLMs cannot plan recovery without this context.
kodit_read_resource accepts multiple optional parameters (lines, page, mode) with undocumented interdependencies. E.g., does 'page' apply only when 'mode' is 'raster'? Can 'lines' and 'page' both be specified? Parameter relationship descriptions are missing.
kodit_read_resource 'mode' parameter is free-form string (documented as 'raster' or 'text') but not declared as an enum in the schema. LLMs may pass invalid values like 'image' or 'html'. Schema should declare mode as enum: ['raster', 'text'].
No composition guidance. Tools exist but their recommended calling order is unclear. E.g., should agents call kodit_repositories first? Should they chain semantic_search → read_resource? Should wiki_page always come after wiki? Tool descriptions lack these hints.