Loading ToolBench
OSF (Open Science Framework, by the Center for Open Science) MCP. Keyless public-data API (JSON:API) for searching and retrieving research projects and preprints.
Three tools with complete schemas and reasonable descriptions. All tools follow verb_noun naming (search_*, get_*). Descriptions are adequate (120-180 chars) but lack actionable guidance on when to use each tool vs. alternatives. Parameters are typed and described, but lack constraints (enums, ranges). Output schemas are not documented, LLMs cannot predict response structure. Error handling is minimal: generic 'error' field with no recovery guidance. No tool annotations (readOnlyHint, idempotentHint). Missing dependency hints (e.g., 'search_projects first to get project IDs for get_project').
OSF: fetch metadata for a single public research project (node) by its OSF id (e.g. "abc12"). Returns title, description, category, dates, tags, and public flag.
OSF: search public preprints by title (Open Science Framework, by the Center for Open Science). Returns id, title, description, publication date, DOI, and tags. Keyless public data; the API can be slow.
OSF (Open Science Framework, by the Center for Open Science): search public research projects ("nodes") by title. Returns id, title, description, category, creation date, and tags. Keyless public data; the API can be slow.
Output schemas not documented. LLMs cannot predict response structure (fields, types, nesting). Responses include count, total, and arrays of objects, but this is not declared anywhere.
Parameter 'limit' lacks constraints. No min/max declared; description says 'default 15, max 50' but schema has no maximum property. LLMs may pass invalid values.
Error handling is generic. callTool() catches all errors and returns {error: message}, but provides no recovery guidance. LLMs cannot determine if error is retryable, user-fixable, or fatal.
No tool annotations. Tools are read-only but lack readOnlyHint annotation. This prevents agents from understanding safety properties and planning accordingly.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Missing dependency hints. Descriptions do not explain that search_projects returns IDs usable by get_project, or when to call each tool. LLMs must infer composition.