Loading ToolBench
A simple MCP server that reads files and fetches docs via Context7
Server has 5 tools with explicitly registered names, descriptions, and Zod schemas. However, significant quality gaps reduce the overall score. Naming is generally verb-based (read_, list_, get_, count_, latest_) which is good. Descriptions exist but are inconsistent in quality, some are detailed (read_file: 120+ chars), others are vague or contain typos (latest_song has 'lastest' instead of 'latest'; description is 95 chars but lacks guidance). Parameter descriptions are present but often generic. No output schemas are documented. Error handling returns isError flag but lacks recovery guidance. No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite tools being read-only. The inputSchema objects use Zod but are not always complete, optional parameters are declared correctly, but constraints (ranges, patterns, enums) are sparse. Tool composition is acceptable (5 separate concerns), but several tools (count_notes, latest_song) are OS-specific and will fail on non-macOS systems without clear fallback guidance.
Returns the total number of notes stored in the macOS Notes app. Use this when the user asks how many notes they have.
Fetch the latest, version-accurate documentation for any programming library or framework using Context7. Always call this before answering questions about a specific library so you have current docs instead of stale training data.
gets the latest song played on the macOS Spotify application. Use this when the user asks for the lastest streamed song
List all files and sub-folders inside a directory. Useful to explore which context/skill files are available before reading them.
Read the full text content of a single file from the local file system. Use this to load context files, skill documents, notes, configs, etc.
Typo in description: 'lastest' instead of 'latest' in latest_song tool. LLMs may misread or fail to parse intent.
No output schema documentation. Callers cannot predict response structure. Responses return { content: [...], isError?: boolean }, but this contract is not declared to clients.
Insufficient error recovery guidance. Errors return isError=true with a message, but do not guide the LLM on next steps (e.g., 'Try with an absolute path' for read_file, 'Ensure Spotify is running' for latest_song).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 51 | - | v1 |
count_notes and latest_song are macOS-specific (osascript calls to Notes and Spotify apps). Will fail silently or with cryptic errors on Linux/Windows. No platform detection or graceful fallback.
count_notes and latest_song have empty inputSchema ({}). While this is valid, they lack tool annotations (readOnlyHint) that would signal to clients and LLMs that these are safe read-only operations.
read_file tool accepts arbitrary absolute paths. No validation against path traversal attacks. An agent could be tricked into reading /etc/passwd or other sensitive files. Should validate against a whitelist or allowed directory prefix.
get_library_docs relies on external Context7 API. No timeout specified in fetch calls. If Context7 hangs, the server blocks indefinitely. Should set explicit timeouts and return actionable timeout errors.
list_directory will reveal directory structure including potentially sensitive files (e.g., .env, .ssh, config files with secrets). No filtering or permission checks. Should exclude hidden files or respect a safe-list.