Loading ToolBench
MCP server for OpenGrok code search (AOSP-scale codebases)
Server has 21 well-named tools with consistent verb_noun patterns (search_*, get_*, list_*). All tools have descriptions and input schemas with typed parameters. However, descriptions are uniformly terse (10-30 chars), falling well below the 50-200 char LLM-optimized baseline. Parameter descriptions exist but lack actionable detail: no format constraints, no examples of valid values, no guidance on when to use related tools. Output schemas are not documented, LLMs cannot predict response structure for chaining. Error handling is absent from visible code. Tool composition is sound (single responsibility, clear naming), but descriptions need substantial expansion to meet production standards.
Advanced search with multiple field options
Retrieve file annotation (blame information)
Retrieve file content
Retrieve definitions within a file
Retrieve file genre (type classification)
List projects within a group (including subgroups)
Retrieve file revision history
All tool descriptions are under 30 characters, far below the 50-200 char LLM-optimized baseline. Descriptions like 'Full-text search query (Lucene syntax)' and 'Retrieve file content' lack context for when to use each tool vs. similar ones, and do not explain what the tool returns or any prerequisites.
Output schemas are not documented in visible code. LLMs cannot predict response structure, field names, or types, making it impossible to chain tools or extract specific data. E.g., does search_code return file paths, line numbers, snippets, or all three?
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | <=2025-11-25 | v2 |
Get a per-project property value
Get a repository property value
List all configured projects
List directory contents
List all configured project groups
List indexed projects
List files in a project directory
List repository paths for a project
Full-text search query (Lucene syntax)
Symbol name to find definitions for
File path glob pattern (Lucene syntax)
History search query
Symbol name to find references for
Autocomplete suggestions for search fields
Parameter descriptions lack actionable constraints. E.g., 'max_results' has no min/max bounds; 'sort' in advanced_search lists valid values in description but not as an enum; 'field' in suggest lists options but no enum constraint. LLMs cannot validate inputs without formal constraints.
No error handling guidance visible. If a search returns no results, or a file path is invalid, or a project does not exist, the LLM has no recovery path. Error responses should guide the agent: 'Project not found. Call list_indexed_projects() to see available projects.'
Parameter 'project' is required in most search tools but descriptions do not explain how to discover valid project names. Should reference list_indexed_projects() or list_all_projects() as discovery tools, or accept empty string to search all projects (already documented for search_code but not others).