Loading ToolBench
Python MCP server exposing PostgreSQL tools + Ollama NL integration
Server has 9 tools with consistent naming (verb_noun pattern: get_, list_, execute_, aggregate_, ask_) and reasonable descriptions (avg ~150 chars). All tools are read-only, which is good for safety. However, parameter descriptions are sparse or missing in several tools. Input schemas are present but lack enum constraints for constrained values (e.g., 'function' in aggregate_table accepts only COUNT|SUM|AVG|MIN|MAX but is declared as free-form string). Output schemas are not documented, LLMs cannot predict response structure. Error handling returns plain text rather than structured guidance. No tool annotations (readOnlyHint, etc.) despite all being read-only.
Run an aggregate function (COUNT, SUM, AVG, MIN, MAX) on a table column. Identifiers are strictly validated — no raw user input is injected into SQL.
Ask a natural language question about the PostgreSQL database. Pipeline: 1. Loads the full DB schema as context. 2. Asks Ollama to generate a PostgreSQL SELECT query. 3. Executes the query (read-only, guarded). 4. Feeds results back to Ollama for a plain-English summary.
Execute a read-only SELECT query against the PostgreSQL database. Returns results as a Markdown table. Only SELECT statements are permitted — any DDL, DML, or dangerous functions are blocked by the security layer.
Get the total row count for a table using COUNT(*).
Get full schema for a specific table — columns, types, nullability, PK/FK.
Retrieve the top N rows from a table — useful for previewing data.
No enum constraints on constrained parameters. 'function' in aggregate_table accepts only COUNT|SUM|AVG|MIN|MAX but is declared as free-form string. 'order_direction' in get_top_rows accepts only ASC|DESC but lacks enum. LLMs will hallucinate invalid values.
Output schemas not documented. Tools return Markdown strings, but LLMs cannot predict structure (table format, field names, error format). Prevents reliable downstream parsing and chaining.
Error handling returns plain text ('**Error:** {exc}') without recovery guidance. LLMs cannot determine if error is retryable, user-fixable, or fatal. No actionable next steps.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
List all non-template databases on the PostgreSQL instance.
List all locally available Ollama models.
List all tables in the PostgreSQL database. Returns table names, schemas, column info (name, data type, nullable, PK/FK flags), and approximate row counts from pg_class.
No tool annotations despite all tools being read-only. Missing readOnlyHint would help clients optimize caching and retry logic. FastMCP supports @mcp.tool(readOnlyHint=True).
Parameter descriptions incomplete or missing. 'schema' param in multiple tools has description 'Schema name (default: 'public')' but no guidance on valid schema names or how to discover them. 'top_n' in get_top_rows states range '1 - 100' in description but no validation enforces it.