Loading ToolBench
Lint and benchmark MCP tool contracts before ambiguous descriptions break LLM tool selection.
MCPLint is a linting tool, not a production MCP server. It contains 19 example tools across three intentionally flawed servers (ambiguous_customer_server, bad_server, good_server). Most tools lack adequate descriptions (7 under 20 chars), parameter descriptions are sparse or missing entirely, and schemas are minimal. Tools 1-2 have detailed descriptions but tools 3-4 are duplicates with vague descriptions. Tools 7-9 have no or trivial descriptions. Tool 18 has an excessively long description (500+ chars). Duplicate tool names (get_customer, search_customers appear twice) indicate schema conflicts. Only tool annotations are present; no resources, prompts, or error handling patterns visible.
Count records matching a limit.
Deletes all records for a customer from the database.
Delete a customer.
Does stuff.
Look up a record by its identifier in the internal store.
Retrieve a widget by search criteria.
Retrieve a single customer record by its exact customer identifier. Returns the customer's profile fields as JSON. Raises a not-found error if the identifier does not exist. Use search_customers instead when you don't already have the exact identifier.
Duplicate tool names (get_customer, search_customers appear twice with conflicting schemas and descriptions). LLMs cannot disambiguate; tool selection will fail or be random.
7 tools have missing or trivial descriptions (<20 chars): ping, get_status, do_thing, get_report, summarize_report.
Tool 18 (summarize_report) has excessively long description (500+ chars, violates 10-1024 char guideline). Wastes tokens and buries key details.
Ambiguous tool pairs lack clear distinction: get_widget vs find_widget both accept widget_id and return similar structures. Descriptions do not explain when to use each.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 49 | 2026-07-28+ | v2 |
Get a customer record by identifier.
Get the compliance report for an account.
Get status
Retrieve a widget by its ID.
List items filtered by category.
Look up an account's current AUM and risk tier.
Search for customer records matching a company name and status filter. Returns a list of zero or more matching customers as JSON. Raises an error if the status filter is invalid. Use get_customer instead when you already know the exact customer identifier.
Search a customer record by identifier.
Summarizes a compliance report for a given account over a configurable time window, aggregating transaction history, risk flags, prior audit findings, related-party relationships, sanctions screening results, politically-exposed-person status, source-of-funds notes, beneficial ownership chains, historical name changes, address history, linked accounts across business units, previously filed suspicious activity reports, watchlist hits and their resolutions, know-your-customer refresh dates, risk rating history and rationale, relationship manager notes, and any open remediation items, then produces a single natural language narrative intended for a compliance analyst to review before an account periodic review meeting, including a recommended next action and a confidence indicator for that recommendation based on how complete the underlying records were at generation time, which may vary considerably between accounts depending on how long the relationship has existed and how many systems of record contributed data to this particular summary generation run.
Update a customer.
Updates the notification settings for a customer.
Many parameters lack descriptions or have type mismatches (e.g., count_records limit is string but description says 'number of records'). Schema-description conflicts confuse LLMs.