Loading ToolBench
A multi-agent framework using Agent-to-Agent (A2A) messaging via a broker, MCP Registry for document storage, and Groq LLM for analysis. Includes Researcher, Analyst, and Frontend services orchestrating document analysis workflows.
This server exhibits significant quality gaps across naming, descriptions, and schema documentation. While 22 tools are defined with basic Pydantic models, most lack LLM-optimized descriptions and parameter-level documentation. Tool names are inconsistent (mix of verb_noun and compound forms like 'pull_process_and_reply', 'produce_note_and_enqueue'). Parameter descriptions are sparse or missing entirely. Output schemas are not formally documented. Error handling is minimal, most tools return generic JSON without recovery guidance. The codebase shows functional HTTP transport and basic FastAPI structure, but falls short of production-grade agent tool standards.
Ask the Analyst to run /pull_process_and_reply once. Analyst will poll the Broker and, if a message is available, process it and send back a reply to Researcher.
Allows an agent to poll its queue for pending messages. If queue empty → returns {"status": "empty"}. If queue has messages → pops and returns the first one.
GET from Broker to check if a message exists for this agent.
Enqueues a message for the receiver. The message is appended to the receiver's queue.
POST message to Broker (enqueue).
Returns the number of pending messages in a receiver's queue.
Call the Groq chat completion API with a system+user prompt. If GROQ_API_KEY is not set, return a canned warning message.
Compound tool names violate single-responsibility principle. 'pull_process_and_reply', 'produce_note_and_enqueue' combine multiple actions (pull+process+reply, produce+enqueue). LLMs cannot decompose these into reusable steps. Split into separate tools: pull_message → process_message → send_reply.
Parameter descriptions are missing or generic across all tools. E.g., 'pointer' in mcp_get lacks format guidance ('MCP pointer in format ctx://note/<rid>' is in tool description, not param description). 'agent' in broker_poll has no validation hint (valid values: Analyst, Researcher?). LLMs cannot infer constraints from names alone.
Output schemas are not documented. Tools return JSON (e.g., broker_poll returns {status, message}, mcp_fetch returns {rid, data, owner, meta}) but these structures are not formally declared in tool definitions. Agents cannot plan downstream calls without knowing what fields to expect.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Debug endpoint: show messages this Analyst has recently pulled.
Debug endpoint: clear local inbox.
Clears all messages from the Researcher's inbox.
Returns all messages currently in the Researcher's inbox.
Stores a document (data) sent by an agent. Returns a unique pointer (ctx://note/<rid>) that can be shared. Requires header: Authorization: Bearer <REGISTRY_TOKEN>
Retrieves the stored document by its resource ID (rid). Requires header: Authorization: Bearer <REGISTRY_TOKEN>
Fetch a stored document from MCP using a pointer like 'ctx://note/<rid>'. We only need the final <rid> for the fetch API.
Lists all stored resources for debugging/inspection. Only shows rid, owner, and metadata — not the full document. Requires header: Authorization: Bearer <REGISTRY_TOKEN>
Store text in MCP Registry and return a pointer (ctx://note/uuid).
Sends a POST request to MCP to store text data. Returns a pointer like ctx://note/<uuid> which other agents can fetch later.
Creates a small example business note (e.g., quarterly EdTech report), saves it to MCP Registry and gets a pointer, constructs an A2A message for Analyst with that pointer, and sends it via Broker.
End-to-end Analyst step: Poll Broker for a message addressed to 'Analyst', fetch document from MCP if pointer exists, analyze with Groq (KPIs + ~120-word summary as JSON), send a 'confirm' reply back to Researcher via Broker (same conv_id). Returns a small status payload useful for debugging.
Polls the Broker for messages addressed to 'Researcher'. If found, stores them in local INBOX.
Display simple HTML form for the A2A+MCP Frontend.
Orchestrates the full A2A workflow: stores user text in MCP, sends A2A message to Analyst, triggers Analyst to process, polls Broker until reply is received, displays the final analysis result on webpage.
Error handling is minimal. Tools return generic HTTP errors or empty status fields without recovery guidance. E.g., 'broker_poll' returns {status: empty} but does not suggest retry timing or alternative actions. 'mcp_get' calls raise_for_status() but does not catch/translate errors into actionable messages for LLMs.
Vague tool names reduce discoverability. 'root' (returns HTML form), 'run_pipeline' (orchestrates full workflow), 'analyst_process_once' (triggers Analyst to process one message) lack clarity. LLMs struggle to distinguish when to call 'run_pipeline' vs 'analyst_process_once' vs 'pull_process_and_reply'. Rename to: 'get_frontend_form', 'orchestrate_analysis_workflow', 'trigger_analyst_single_step'.