Loading ToolBench
Multi-turn security benchmark for MCP agents — measures behavioral degradation under tool poisoning
MCPDrift is a security benchmark tool with 4 mock tools. All tools have basic descriptions (10-50 chars) and minimal input schemas. Naming is clear (file_read, file_write, email_send, get_time) but descriptions lack depth on WHEN to use tools, prerequisites, or return value structure. No parameter descriptions visible in schema. No output schemas documented. Error handling is absent, mock implementations return generic strings. Tool composition is reasonable (single responsibility) but lacks idempotent guarantees and recovery guidance. This is a research/testing tool, not production-grade.
Send an email to a recipient.
Read a file from the filesystem.
Write content to a file on the filesystem.
Get the current time.
No parameter descriptions in input schemas. Schema shows 'path' and 'content' but does not explain what format, constraints, or valid values are expected.
Tool descriptions are too brief (10-50 chars) and lack context on WHEN to use each tool, what it returns, or prerequisites. LLMs cannot reliably select tools without richer descriptions.
No output schemas documented. Callers cannot know what fields to expect from responses, forcing LLMs to guess or parse unstructured text.
Mock implementations return generic success strings (e.g. 'Successfully wrote X bytes to path'). No error classification, recovery guidance, or actionable error messages. Agents cannot distinguish retryable from fatal failures.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
No idempotency guarantees documented. file_write and email_send are destructive operations but lack confirmation steps or dry-run modes. Agents cannot safely retry without risking duplicate side effects.