Loading ToolBench
Latch CLI - MCP Guard Proxy for AI Agents. A security proxy that intercepts MCP tool calls, classifies them by action class and risk level, and routes them through an approval/policy engine.
This is a demo/test MCP server with significant definition quality gaps. The server exposes 14 tools across two files (packages/demo-mcp-server/src/index.ts and scripts/mock-mcp-server.ts), many with high-risk operations (shell execution, payments, file writes). Tool descriptions are extremely brief (5-20 chars), parameter descriptions are minimal, schemas are present but sparse, and there is no documented output structure or error handling guidance. The tool names follow basic verb_noun convention but lack clarity around intent. This appears to be a test harness rather than a production-grade tool definition. Per the rubric calibration, this falls into the D-F range due to missing descriptions (hard cap), incomplete parameter documentation, and lack of error guidance.
Test helper: get invocation count
Test helper: reset invocation log
Send an email (SEND action class)
Execute a shell command (EXECUTE action)
Write content to a file (WRITE action class)
Submit a form to a URL (SUBMIT action class)
Read a note by ID (READ action class)
Descriptions are critically brief (5-20 characters). Keep descriptions between 10 - 1024 characters.' Examples: 'Read a note by ID (READ action class)' (39 chars), 'Write content to a file (WRITE action class)' (44 chars). These provide minimal context for LLM tool selection.
Parameter descriptions are missing or one-liner labels. The 'command' param in shell_exec has description 'Shell command to execute', technically present but provides zero detail on constraints (injection risk, allowed commands, timeout, cwd scope). The 'to' param in payment_send has 'Recipient identifier' but does not explain format (email? ID? Username?).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 32 | 2024-11-05+ | v1 |
Send a payment (TRANSFER_VALUE action class)
Read a file (READ action)
Send an email (SEND action)
Execute a shell command (EXECUTE action class)
Submit a form (SUBMIT action)
Transfer money (TRANSFER_VALUE action)
Write to a file (WRITE action)
Destructive and irreversible operations lack confirmation or dry-run support. shell_exec (DESTRUCTIVE), payment_send (IRREVERSIBLE), and transfer_money (IRREVERSIBLE) can cause harm without agent ability to preview or confirm. Agents make mistakes, a confirm_before_execute pattern prevents catastrophic errors.'
No documented output schemas. The source code shows input schemas (properties/required) but no indication of what these tools return. LLMs need to know what fields to expect so they can plan downstream tool calls.' Without output schemas, agents cannot chain tool results effectively.
Error handling and recovery guidance absent. No indication of what errors each tool can return or how an agent should respond.
Test tools (_test_getInvocations, _test_reset) pollute the production interface. Names starting with underscore suggest internal helpers but are exposed as callable tools. These should be hidden from the public tool list or documented as test-only.
Duplicate tools across two files (demo-mcp-server and mock-mcp-server). Both define read_file, write_file, send_email, execute_command, submit_form, and transfer_money with identical signatures. If search_users and find_users both exist, the LLM wastes reasoning cycles deciding between them.' This creates ambiguity for tool selection.
No input validation constraints documented. The 'amount' param in payment_send is type 'number' but has no minimum, maximum, or format guidance. The 'data' object in form_submit has no schema definition.