Loading ToolBench
A Model Context Protocol (MCP) server for mail-utils (mu) email indexing and search tool.
mu_mcp is a STDIO-only server with 4 tools for email search and viewing via the `mu` command-line utility. Tool definitions are visible and properly registered via FastMCP decorators. However, the server has severe definition quality gaps: input schemas exist but are minimal (only type and description, no constraints or validation guidance); descriptions lack critical context about prerequisites, error handling, and when to use each tool; parameter descriptions are generic and provide no guidance on format or constraints; output schemas are undocumented (tools return raw strings with no structure); no error recovery guidance; no tool annotations; and no explanation of what makes tools idempotent or safe for retry. The health_check tool is trivial. The query, view, and get_attachment tools wrap `mu` subprocess calls but expose implementation details (e.g., 'paths' parameter expects space-separated strings, 'command' parameter expects a specific format without the mu extract prefix) that would require LLM trial-and-error. The get_attachment tool is marked WRITE but has minimal error handling. By the rubric, this server is 'Poor' to 'Fair', definitions exist but lack the depth and structure needed for reliable agentic use.
Open attachments in email by providing the email path. The tool downloads the attachment into a temp dir and open it. The `command` includes the paths and the pattern of attachment files. The prefix `mu extract --target-dir /tmp --overwrite --play` SHOULD NOT appear in `command`. See the man page for `mu extract`.
Health check for the MCP server.
Query `mu` by providing a valid query to be sent in the following way. Syntax: ``` mu find $query ``` Here is the syntax guide for mu queries.
View emails using `mu`, by providing their paths. ``` mu view $paths ``` Paths can be extracted using the following: ``` mu find --fields "l" SOME_QUERY ```
Input schemas lack format constraints and validation guidance. The 'query' parameter says 'A valid mu query string' but provides no examples of validity (syntax rules, allowed operators, length limits). The 'paths' parameter says 'Space-separated paths to email files' but doesn't document path format or what happens if a path is invalid. The 'command' parameter says 'Paths and pattern of attachment files (without the mu extract prefix)' but this is vague, what is the exact format? What patterns are allowed? LLMs will guess and fail.
Output schemas are completely undocumented. All four tools return raw strings with no documented structure. The 'query' tool returns 'result.stdout.strip()' but what fields does mu find output? Is it one email path per line? Are there headers? The 'view' tool returns raw email content as a string, is this markdown, plaintext, or raw RFC822? The 'get_attachment' tool returns 'result.stdout.strip()', but what does mu extract emit? Without documented output schemas, LLMs cannot plan downstream operations or extract relevant fields.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 46 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 36 | - | v1 |
Descriptions are too brief and lack actionable context. The 'query' description references mu query syntax with a code block but then says 'Here is the syntax guide for mu queries.', yet the actual guide is appended from 'mu-query.txt' at runtime, which is not visible in the source provided. The description does not explain: When should I use query vs view? What does query return? What are valid query operators? The 'view' description explains how to extract paths but doesn't explain what the output looks like or how to parse it. The 'get_attachment' description says 'downloads the attachment into a temp dir and open it', but does 'open it' mean display it, save it, or return its contents? This is ambiguous.
No error recovery guidance. All tools return 'Error: {e.stderr.strip()}' on subprocess failure, which is a raw mu error message. These messages are opaque to LLMs, e.g., 'Error: parse error in query' tells the LLM nothing about how to fix it or what to try next. The pattern recommends: 'Error responses must tell the LLM what to do next.' For example, if query fails with a syntax error, the response should guide the LLM to try a simpler query or list available operators.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The 'Risk' metadata in the spec lists 'query' and 'view' as READ_ONLY and 'get_attachment' as WRITE, but these annotations are not visible in the FastMCP code. The MCP tool decorator has no annotations, so LLMs cannot infer which tools are safe to retry, which modify state, or which require confirmation. This is a protocol-level issue, FastMCP should support tool annotations per the current spec.
Parameter naming exposes subprocess implementation details. The 'query' tool accepts a 'query' string that is split and passed to 'mu find', the description says 'Syntax: mu find $query' but does not explain that spaces in the query need escaping or that the string will be split on whitespace. The 'paths' parameter in 'view' is space-separated but what if a path contains spaces? The 'command' parameter in 'get_attachment' is described as 'Paths and pattern of attachment files' but the code splits on whitespace, this will break on paths with spaces. The rubric recommends accepting human-friendly identifiers and abstracting API complexity; here, the tool interface leaks subprocess-specific details.
No documented prerequisites or dependencies. The 'query', 'view', and 'get_attachment' tools all depend on the 'mu' command being installed and configured. The description should state: 'Requires the mu email indexing tool to be installed and initialized (see mu handbook). If mu is not found or the index is stale, this tool will fail.' Without this, LLMs will invoke the tool and receive cryptic 'command not found' errors with no guidance on how to proceed.
The 'health_check' tool is trivial and adds no value. It always returns 'ok' regardless of whether mu is installed or the email index is healthy. A more useful health check would verify that the 'mu' command is accessible and the email index exists, returning structured output like {"status": "ok", "mu_version": "...", "index_size": ...}. The current implementation is a no-op that wastes a tool slot.