Loading ToolBench
FastAPI-based MCP server for indexing and searching JVM microservice repositories (Java/Kotlin). Provides vector search over code chunks, file retrieval, and discovery of classes, methods, Kafka listeners, and REST endpoints.
This HTTP-based MCP server exposes 6 code search and analysis tools for Java/Kotlin microservices. Tool names follow verb_noun convention (search_code, get_file, find_class, find_method, find_kafka_listeners, find_rest_endpoints) which is good. However, definitions have significant gaps: descriptions are present but generic (averaging ~80-120 chars, below the 194-char baseline); parameter descriptions are often missing or minimal; input schemas exist in Pydantic models but are not consistently documented in docstrings; output schemas are partially structured but lack detail in most tool docstrings; error handling is present (HTTPException for 404/500) but does not provide recovery guidance; no tool annotations (readOnlyHint, etc.) despite all tools being read-only. The server is READ_ONLY by design (safe for agents), but this safety property is not declared in tool metadata. Scoring is conservative: tool definitions are functional but below production baseline for agent guidance.
Find class definitions by name within a specific microservice.
Return all Kafka consumers (@KafkaListener) for a given microservice.
Find method definitions by name across a single microservice.
Return REST endpoints based on Spring annotations for a given microservice.
Return file contents for a given path.
Search the indexed codebase and return relevant chunks from a specific microservice.
Parameter descriptions are missing or minimal in tool docstrings. 'service_name' appears in 5 tools but is never explained in docstrings, LLMs must infer from context what a 'microservice name' means or how to obtain it. The 'query' parameter in search_code lacks guidance on format (natural language? code snippets? keywords?).
Tool docstrings do not document output schemas. E.g. search_code returns SearchResponse, find_class returns List[CodeChunk], but docstrings do not explain what fields CodeChunk contains (file_path, start_line, end_line, method_name, etc.). LLMs cannot plan downstream tool calls without knowing response structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
No tool annotations despite all tools being read-only. MCP spec (2026-07-28) supports readOnlyHint, destructiveHint, idempotentHint. These tools should be marked readOnlyHint=true to signal to agents that they are safe to call and retry without side effects.
Error handling does not provide recovery guidance. HTTPException(404, 'File not found: <path>') tells the agent a file is missing, but not what to do next (search_code first? try a different path?). HTTPException(404, 'Service not indexed...') is better (includes hint to run indexer), but inconsistent.
get_file description is under 30 characters ('Return file contents for a given path.'). Current description offers no guidance on when to call get_file vs search_code, or whether paths should be absolute/relative.
'service_name' parameter is required in 5 tools but there is no discovery tool to list available services. An agent has no way to know what service names are valid without prior knowledge. Should add list_services() or document the naming convention (repo folder names inferred by indexer).
No pagination support in search_code. The tool accepts top_k to limit results, but does not support offset/cursor for retrieving results beyond top_k. Large codebases could return unbounded result sets, bloating agent context.