Loading ToolBench
The server has 5 tools with reasonable naming and basic input schemas, but significant gaps in documentation and output schema definition. Tool names follow verb-noun conventions (reverse-dependencies, build, test, sources, deps), which is good for LLM interpretation. However, descriptions are present but brief (10-50 chars for most), and critically, NO output schemas are documented anywhere in the source code. Parameter descriptions exist but are minimal. The server lacks error handling guidance, output structure documentation, and context-aware parameter descriptions. STDIO-only transport is a hard cap at 50, but definition quality alone sits at ~48.
Builds a given Bazel target.
Finds the dependencies of a given Bazel target.
Given a bazel target, or file path, find all other bazel targets that depend on it.
Finds the direct source files associated with a given Bazel target.
Runs tests for a given Bazel target.
No documented output schemas for any tool. Tools return string output (bazel command results) but no structured response format is defined. LLMs cannot plan downstream tool calls or extract specific data from results.
Tool descriptions are too brief (10-50 characters). The rubric baseline is 194 chars. Descriptions like 'Builds a given Bazel target' lack context on WHEN to use the tool vs similar tools, what prerequisites exist, and what the output contains.
Parameter 'max_depth' in reverse-dependencies uses type 'number' but semantics require integers. Description says '-1 for indefinite' but does not specify valid range (e.g., -1 or 0+). Similarly, 'depth' parameter in deps defaults to 1 but constraint documentation is incomplete.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 45 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 49 | - | v1 |
No error handling guidance. Tools execute bazel commands via ExecuteCommand() which returns combined stdout/stderr on failure, but there is no documentation on how LLMs should interpret or recover from errors (e.g., 'target not found', 'build failed', 'permission denied').
Parameter descriptions lack format and constraint details. 'The target to find reverse dependencies for' does not explain the accepted format (bare name vs full //path:target vs file path). The ResolveInputToBazelTarget() function in target.go shows the code handles multiple formats, but this is not communicated in the tool schema.
No idempotency or confirmation guidance. The 'build' tool modifies state (compiles artifacts) but the description does not indicate whether repeated calls are safe. No dry-run or confirmation pattern is offered for potentially expensive operations.
Parameter 'project_path' is required on all tools but descriptions only say 'Where MODULE.bazel or WORKSPACE is located.' This does not clarify: must it be absolute? Can it be relative to the working directory? What if neither file exists?