Loading ToolBench
Model Context Protocol server for Thoth AI Research Assistant - provides 60+ research tools for academic paper processing, knowledge management, and autonomous research workflows
The Thoth MCP server exhibits critical gaps across definition quality dimensions. All three tools have extremely brief, generic descriptions (14-19 characters) that fail to explain WHAT the tools do, WHEN to use them, or what they return. Per the rubric baseline (avg 194 chars; p10=34), these fall far below minimum threshold. Input schemas are present but minimally documented, parameters lack descriptions explaining what 'agent_id', 'message', and 'question' actually control or what format they expect. No output schemas are documented anywhere in the visible code, violating the requirement that tools document what LLMs should expect back. Tool composition violates the single-responsibility principle: 'send_message_to_agent_and_wait_for_reply' and 'send_message_to_agent_async' both send messages to agents, these should be unified with a single 'send_message_to_agent' tool accepting an optional 'wait_for_reply' boolean, not two separate tools. Error handling is absent, no guidance on what happens if an agent_id doesn't exist, network fails, or a research question is malformed. No evidence of input validation, parameter constraints (enums, ranges), or recovery steps. The tool names themselves are awkwardly long and compound ('send_message_to_agent_and_wait_for_reply' mixes action and outcome); shorter, clearer names like 'send_agent_message' + a 'wait' parameter would be more idiomatic. Security: no visible secret injection, permission checks, or audit logging. Naming does follow verb-first convention, which is a small positive.
Agentic research question tool - processes research questions through the autonomous research agent system
Synchronous agent communication tool - sends a message to another agent and waits for their response
Asynchronous agent communication tool - sends a message to another agent without waiting for immediate response
All three tools have descriptions under 20 characters ('Synchronous agent communication tool' = 36 chars is borderline, but 'Asynchronous agent communication tool' = 37 chars, and 'Agentic research question tool' = 30 chars are all below the minimum context needed for LLM selection).
No output schemas documented for any tool. LLMs cannot infer what fields to expect in responses, preventing downstream tool chaining and forcing manual data extraction.
Input parameter descriptions are completely absent. The 'agent_id' parameter has description 'The ID of the agent to communicate with' (acceptable), but 'message' is just 'The message to send to the agent' (generic, no format guidance). 'question' lacks any explanation of expected format, length, or scope.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 27 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 21 | - | v1 |
Violates single-responsibility principle (pattern:tool). Both 'send_message_to_agent_and_wait_for_reply' and 'send_message_to_agent_async' are separate tools that differ only in whether they block. This should be a single 'send_message_to_agent' tool with a boolean 'wait_for_reply' parameter (defaulting to false). Keeping separate tools forces LLMs to reason about which to call for the same logical action.
No error handling or recovery guidance. No description of what happens if agent_id doesn't exist, message delivery fails, or research question is invalid. Per pattern:recovery-guide, errors must tell the LLM what to do next.
Tool names are compound/awkward. 'send_message_to_agent_and_wait_for_reply' is 41 characters; baseline median is 18 chars. 'agentic_research_question' mixes adjective + noun. Shorter, action-first names ('send_agent_message', 'research') would be clearer.
No input validation or constraints visible. Parameters accept arbitrary strings with no enum, pattern, length, or format constraints. An LLM could pass 'agent_id' = 'invalid-xyz' without validation.
No permission checks or security declarations visible. Tools that send messages or trigger research agents lack scope declarations (e.g., 'requires: write:agent_messages'). No audit trail or access control visible.