Loading ToolBench
An MCP server that interacts with Sourcegraph to search for code, retrieve file contents, and analyze React component prop usage
Server has 4 tools with explicit schemas and descriptions. Naming follows verb_noun convention (test, search, get). Descriptions are present but generic, most lack guidance on WHEN to use each tool vs. alternatives. Schemas use Zod with type constraints (min/max for limits), but output schemas are undocumented. Error handling returns isError flag but lacks recovery guidance. No tool annotations (readOnlyHint, idempotentHint). Parameter descriptions are adequate but could specify format/constraints more explicitly (e.g., 'repository format: github.com/owner/repo').
Search for code in Sourcegraph. Use this tool for general searches. Do not prioritize it over more specific search tools. Do not use this tool for retrieving the full source code of a specific file. Always present the information by highlighting the repository, file name and a snippet of the matching code.
Returns the contents of a specific file in GitHub via the Sourcegraph API. Useful for gathering more context on a particular file or for when the user requests the source code of a specific file. If the file is not found, do not attempt to retrieve the file by another method.
A more specific type of Sourcegraph search. Use this tool when the user asks about the usage of a specific prop of a React component. This could be helpful for library maintainers when they need to understand the usage of their components. Both a prop name and the component name must be specified in the user prompt.
Test the connection to Sourcegraph and return version info
Output schemas undocumented. Tools return formatted text via formatters.js but LLMs cannot see the structure of returned data (fields, types, pagination). This forces LLMs to parse unstructured text and prevents chaining.
Descriptions lack WHEN-to-use guidance. generalSearch says 'Use this tool for general searches' but does not explain when to prefer it over searchReactPropUsage. LLMs must infer intent from names alone.
Error responses lack recovery guidance. All tools return isError=true with a message, but do not tell the LLM what to do next (retry, try a different tool, ask user). Per pattern:recovery-guide, errors should guide the agent's next step.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
Parameter 'repository' in getFileContent lacks format specification. Description says 'e.g., github.com/user/repo' but does not enforce the format or explain what happens if format is wrong. LLMs may pass invalid values.
No tool annotations (readOnlyHint, idempotentHint). All tools are read-only and idempotent, but this is not declared. LLMs cannot infer safety properties and may over-caution or misplan retries.