Loading ToolBench
An MCP server that enables MCP clients like Claude Desktop to interact with data from protocols.io.
This server has 5 tools with clear verb-noun naming and a reasonable overall structure. However, there are significant gaps: (1) Parameter descriptions are sparse or missing critical detail. For example, 'create_protocol' accepts a 'steps' array but provides no schema for what each step object should contain, the description just says 'List of protocol steps' without specifying the ProtocolStepInput structure. (2) Output schemas are not documented in tool definitions, forcing LLMs to infer response structure. The code shows Pydantic models (User, Material, ProtocolStep, ProtocolStepInput) but these are not exposed in the tool schema declarations. (3) Error handling is minimal, no guidance on retryable vs. fatal errors, no recovery hints. (4) Tool descriptions are moderately detailed (100-250 chars) but lack dependency ordering and prerequisites. The search_public_protocols tool has unusually verbose guidance embedded in its description (160+ chars of recommendations), which is helpful but not typical structured guidance. (5) Input validation constraints (enums, ranges, formats) are absent from most parameters. (6) The 'steps' parameter in both create_protocol and update_protocol lacks type definition visibility, it's declared as 'array' with minimal detail.
Create a new protocol with specified title, description, and steps.
Retrieve basic information for all protocols belonging to the current user. To get detailed protocol steps, use get_protocol_steps.
Get detailed protocol steps for a given protocol ID.
Search for public protocols on protocols.io using a keyword. Results are sorted by protocol popularity and paginated with 3 protocols per page (use the page parameter to navigate, default is 1). When searching for reference protocols to create a new protocol: - Avoid referencing protocols from before 2015 as they may be outdated. - If the found protocols have topics that are not closely related to your needs, ask the user for clearer direction before proceeding. - If the found protocols are highly relevant, use get_protocol_steps to examine at least 2 protocols' detailed steps and integrate insights from different approaches to ensure more reliable protocol development.
Update an existing protocol with new title, description, and/or steps.
Array parameters ('steps' in create_protocol and update_protocol) lack input schema, declared as 'array' type with minimal description ('List of protocol steps') but the internal object structure (ProtocolStepInput with fields: description, materials, reference_protocol_ids) is not exposed in tool schemas. LLMs cannot construct valid step objects without this schema.
No output schemas documented for any tool. Agents do not know what fields to expect in responses, forcing them to infer structure and making downstream tool composition fragile. For example, does get_protocol_steps return protocol_id in each step? Does search_public_protocols return next_cursor or total_count for pagination?
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
Parameter descriptions lack validation constraints. 'keyword' in search_public_protocols has no length bounds. 'page' has no min/max. 'protocol_id' in get_protocol_steps and update_protocol has no error guidance (what happens if ID doesn't exist?). 'title' and 'description' in create/update have no length limits or required flags.
Tool descriptions are brief and lack dependency guidance. get_protocol_steps (67 chars) and create_protocol (71 chars) fall below the 100-char minimum recommended baseline. More critically, descriptions do not explain when to use each tool relative to others, agents must reason this out instead of being told 'Call search_public_protocols first, then get_protocol_steps to examine results.'
Error handling is absent. No guidance on what errors can occur, whether they are retryable, or how to recover. For example, if create_protocol fails because the title is too long, there is no error message telling the agent to shorten it. If get_protocol_steps fails with 'protocol not found', there is no suggestion to search for the protocol first.
Destructive operations (create_protocol, update_protocol) lack confirmation/dry-run support. Agents could accidentally overwrite or create unwanted protocols without a safety mechanism to preview or confirm changes first.
update_protocol description does not clarify whether it performs a full replacement or partial merge. This ambiguity affects idempotency and error handling, agents cannot safely retry if they don't know whether omitted fields will be cleared.