Loading ToolBench
Extract database schema from PostgreSQL migration files using pg_dump or native extraction
Two tools with basic structure but significant quality gaps. Both tools have adequate names and descriptions (90-110 chars each), but parameter documentation is sparse and output schemas are not visible in the source code. The extract_schema tool has a complex input schema with enums and optional parameters well-defined in code (mcp.WithString, mcp.Enum, mcp.Required), but the validate_migrations tool lacks detailed parameter descriptions. Neither tool documents its output structure, which is critical for LLM reasoning about downstream tool chains. Error handling exists (mcp.NewToolResultError) but lacks actionable guidance. No pagination, structured output documentation, or idempotent operation markers are evident.
Extract database schema from PostgreSQL migration files using pg_dump
Validate migration files in directory without running them
Output schemas are not documented. Tools return free-text responses (mcp.NewToolResultText) without declaring what fields or structure the LLM should expect. This forces LLMs to parse unstructured output, increasing errors and token waste.
Parameter descriptions are incomplete or missing. 'postgres_image' in both tools lacks guidance on format, valid ranges, or when to use non-default images. 'format' enum is defined in code but the description does not explain what each format produces or when to use it.
Error messages lack actionable recovery guidance. mcp.NewToolResultError() is called with error strings, but errors do not tell the LLM what to do next (retry with different params, check prerequisites, etc.). A Docker image not found error should suggest checking Docker installation.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 54 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 42 | - | v1 |
Tool descriptions do not clarify side effects or state modifications. extract_schema spins up a PostgreSQL container via testcontainers-go and runs migrations, these are non-trivial operations with latency and resource costs. The description should state 'This tool runs actual database migrations in a temporary container (may take 10-30 seconds).'
No evidence of pagination or result limits. If migration files produce large schemas, the extract_schema tool may return thousands of lines of SQL. The tool description and response should cap results and offer pagination.
validate_migrations tool does not document what validation checks are performed or what constitutes a 'valid' migration. Does it check syntax, schema conflicts, ordering? The description is too generic ('Validate migration files') for LLM reasoning.