Loading ToolBench

Reading the current corpus…

ai-coding-toolkit MCP server · D · ToolBench

← Browse

ai-coding-toolkit

github.com/plaid/ai-coding-toolkit

Model Context Protocol (MCP) server to build fast integration testing with Plaid Sandbox data.

<=2025-11-25STDIO4 tools

54/100

Definition Quality56

Protocol Readiness32

Supportability64

Spec posture

Inferred effective spec <=2025-11-25·current baseline 2026-07-28

Static source inference · medium confidence · detected: Logging

Deprecated protocol patterns detected

Summary

This Plaid MCP server has significant definition quality gaps. Of the 4 tools, only 2 have complete input schemas visible in the source code. Tool descriptions are present but suffer from vague language, embedded procedural instructions mixed with technical documentation, and lack of proper WHAT/WHEN/WHAT-RETURNS structure. Parameters lack consistent type declarations and validation guidance. Output schemas are not documented. The tool definitions appear partially inferred from file paths rather than explicit schema registration in the main server code. STDIO transport limits practical deployment.

Tools (4)

get_mock_data_promptread onlysource verified50/100

Get the mock data prompt for the AI agent to generate mock data for testing purposes. <important> - You must have a real time generation display of the mock data to the end user. - You must ask use if they would like to persist the mock data to a local file, and if they do, you MUST write the mock data to a file named `mock_data.json` in the current working directory. </important>

get_sandbox_access_tokenwriteauthsource verified53/100

Get a workable access token for the Plaid sandbox environment. So that we can test the integration with sandbox mocked data. For a success response, the tool will return both access_token and item_id. <important> BEFORE you call this tool: - You MUST ask the user if they would like to provide the webhook url to listen to events update. - You MUST ask the user if they would like to provide the customized account data to be associated with the access token. If they do, you MUST use the tool `get_mock_data_prompt` to generate the mock data with the format we accept. </important>

search_documentationread onlysource verified62/100

Search Plaid documentation for relevant information. <important> - You MUST use this tool when the user asks questions about Plaid's products, or API endpoints. - You MUST use this tool when you run into any coding issues or errors you cannot resolve and need more information about the API endpoints or products. </important>

simulate_webhookwriteauthsource verified57/100

Simulate a Plaid webhook event in the sandbox environment to test your application's webhook handling. This tool triggers specific webhook events like transaction updates, account status changes, or transfer notifications. Specify the access_token for the item and the webhook_code representing the event type you want to trigger. <important> - Unless user specifies the webhook_code and webhook_type, you MUST use the tool `search_documentation` to find the right webhook_code and webhook_type. </important>

Issues (6)

CRITICALtool ↗

Output schemas completely undocumented. No tool returns a documented response structure.

get_mock_data_promptsimulate_webhookget_sandbox_access_tokensearch_documentation

HIGHtool-description ↗

Descriptions embed procedural agent instructions instead of technical specifications.

get_mock_data_promptsimulate_webhookget_sandbox_access_tokensearch_documentation

HIGHconstrained-input ↗

Constrained parameters missing enum declarations and validation rules.

simulate_webhooksearch_documentation

HIGHtool-description ↗

Parameters lack type declarations and min/max constraints in visible schema.

get_sandbox_access_tokenget_mock_data_prompt

MEDIUMrecovery-guide ↗

Recommendations

Add complete input/output schemas for all 4 tools. For each param: declare type (string|integer|boolean|object|array), min/max, regex pattern if applicable, and enum values for constrained inputs. Example: 'num_of_transactions: integer, min=1, max=1000, description="Number of transactions to generate per account."'
Rewrite tool descriptions to follow the WHAT/WHEN/RETURNS template. Remove embedded agent instructions (<important> tags). Example for get_mock_data_prompt: 'Returns a structured prompt template for AI to generate realistic mock banking data. Use this to create test datasets before calling get_sandbox_access_token. Returns template with account types, transaction patterns, and formatting guidelines.'
Convert webhook_code and webhook_type to enum constraints. Document all valid options (DEFAULT_UPDATE, SYNC_UPDATES_AVAILABLE, TRANSACTIONS_REMOVED, etc.) in schema, not description.
Add error handling section to each tool definition: what errors can occur, what they mean, and what the LLM should do. Example: 'Error: invalid_access_token (401). Action: Verify token was obtained from get_sandbox_access_token and has not expired. If needed, call get_sandbox_access_token again.'
Document output schemas explicitly. For get_sandbox_access_token, specify: returns {access_token: string, item_id: string, request_id?: string}. For search_documentation, specify: returns {results: [{title: string, url: string, excerpt: string, relevance_score: float}], total_count: integer}.
Add parameter validation rules to descriptions. Example for initial_products: 'Comma-separated list of product names: auth, identity, transactions, investments, liabilities (do not include balance). Max 5 products.'

Spec posture evidence

Inferred effective spec: <=2025-11-25.

Relies on Logging (deprecated) - log to stderr or use OpenTelemetry

Score history

Overall score trend

↑ 54 points across a rubric change (v1 → v2)

54/100

Scored	Grade	Overall	Spec posture	Rubric
2026-09-22	D	54	<=2025-11-25	v2
2026-03-09	F	0	-	v1