Loading ToolBench
Two Claude agents collaborating on one machine over A2A (agent-to-agent) and MCP (tools), with Claude on Vertex AI.
Six tools with complete schemas and descriptions, but significant quality gaps. Tool names lack action verbs (summarize, score_relevance, extract_entities are acceptable; web_search, fetch_document, research_and_analyze are weak). Descriptions are present but generic, most under 100 chars, lacking WHEN/WHY context. Parameters have types and descriptions, but no enums for constrained inputs. Output schemas are undocumented, LLMs cannot predict return structure. Error handling is minimal; no recovery guidance. The research_and_analyze tool conflates two concerns (research + analyze) and lacks parameter validation. Overall: functional but below production baseline.
Extract likely named entities (capitalized multi-word spans).
Fetch the full text of a document by its ``doc_id``.
Research a topic and return an analyzed answer produced by two collaborating Claude agents. A Researcher agent gathers sources with MCP tools, delegates to an Analyst agent over A2A, and the Analyst summarizes, scores relevance, and extracts entities. Use this when you want a researched, cross-checked answer rather than a single-model response.
Score 0-100 how relevant ``text`` is to ``query`` (deterministic).
Extractive summary: the highest-signal sentences from ``text``.
Search the document corpus for a query and return ranked hits. Each line includes a ``doc_id:`` that can be passed to ``fetch_document``.
research_and_analyze violates single-responsibility principle: combines research + analysis in one tool. Should split into separate tools so agents can compose them independently.
No output schemas documented. LLMs cannot predict return structure (e.g., does web_search return JSON or plaintext? What fields?). All tools return unstructured strings.
Descriptions lack WHEN/WHY context. 'Extractive summary' does not explain when to use summarize vs. other text processing. Descriptions should guide tool selection.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 55 | 2026-07-28+ | v2 |
web_search and fetch_document lack error recovery guidance. If search returns no results or doc_id is invalid, LLM has no next step. Should include 'Try a broader query' or 'Available doc_ids: ...'
No enums for constrained inputs. limit parameters accept any integer; max_sentences accepts any integer. Should declare min/max bounds and valid ranges in schema.