Loading ToolBench
NestJS framework for building Model Context Protocol servers with tools, resources, and prompts support
The server defines 18 tools across multiple example applications with varying quality. All tools have descriptions and most have well-defined input schemas with enums and parameter descriptions. However, there are systematic gaps: (1) Output schemas are never documented, LLMs cannot predict what fields to expect in responses; (2) Error handling lacks recovery guidance; (3) Some tools combine multiple concerns or have unclear naming distinctions (e.g., lookup_product vs check_stock); (4) No pagination guidance for list-like tools; (5) Security considerations are largely absent from descriptions. Tool naming is generally good (verb-first, clear intent), but composition could be stronger. Most descriptions are adequate (50-150 chars) but lack strategic guidance on when to call each tool vs. alternatives.
Reset server cache (requires API key)
Get server statistics (admin only)
Install Playwright Chromium browser binary. Call this if you get an error about the browser not being installed.
Perform basic arithmetic operations
Check stock availability for a product by SKU
Convert temperature between Celsius and Fahrenheit
Get the full structure of a table: columns with data types and defaults, primary keys, unique and foreign key constraints, and indexes.
Output schemas are completely undocumented. No tool describes what fields its response contains, what types they are, or how they chain to other tools. LLMs cannot predict the structure of results and must guess or hallucinate field names.
Error handling is absent from tool descriptions. No guidance on what errors can occur, whether they are retryable, or what the LLM should do next. E.g., browser_install says 'if you get an error' but does not explain how the agent should respond to specific error codes.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 62 | 2026-07-28+ | v2 |
| 2026-03-09 | C | 67 | 1.10.0+ | v1 |
Execute a SQL query against the PostgreSQL database. When POSTGRES_READONLY=true only SELECT, WITH, EXPLAIN, SHOW, TABLE, and VALUES queries are allowed. Returns rows as JSON along with field metadata and row count.
Run EXPLAIN (ANALYZE, BUFFERS) on a SQL query to show the execution plan with actual timing, row counts, and buffer usage. Essential for diagnosing slow queries.
Retrieve web page content from a specified URL using a headless Chromium browser.
Retrieve web page content from multiple URLs in parallel using a shared headless Chromium browser context.
Generate a PDF from a web page URL using a headless Chromium browser.
Get analytics data for a metric (registered via forFeature)
Get overall database statistics: size, connection count, cache hit ratio, transaction stats, and top tables by size.
List all indexes for a table with their definitions, sizes, and usage statistics (scan count, tuples read/fetched) from pg_stat_user_indexes.
List all schemas in the PostgreSQL database with table/view counts and total size. System schemas (pg_catalog, information_schema, etc.) are excluded by default.
List all tables, views, and materialized views in a schema with estimated row counts and sizes.
Look up product details by SKU or search by name
lookup_product combines two distinct concerns: SKU lookup AND name search. LLMs must decide which parameter to use. Split into lookup_product_by_sku and search_products_by_name for single responsibility and clarity.
No documentation of pagination or result limits. list_schemas, list_tables, list_indexes do not mention whether results are paginated, what the max result count is, or how to specify page size. Large result sets could blow context windows.
admin_stats, admin_reset_cache, and admin_stats lack descriptions explaining permission requirements or when to use them. admin_reset_cache says 'requires API key' in the description but does not specify if this is in a header, parameter, or environment variable.
execute_sql description mentions POSTGRES_READONLY=true but does not explain how an agent would know this flag is set or how to handle the case when it is not. Missing context on permissions and constraint enforcement.
fetch_url and fetch_urls accept a 'debug' parameter that 'overrides --debug flag', but no description of what visibility/behavior this enables or how the agent should use it. Underdocumented parameter behavior.