Loading ToolBench
Multi-agent RAG system for Amazon Seller Central queries using Strands Agents + LangGraph with MCP server exposing seller knowledge base
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation, output schema clarity, and error handling. search_seller_knowledge has a reasonable description (127 chars, within baseline 34-392) and proper input schema with types. score_answer lacks output schema documentation and has minimal guidance on error cases. Neither tool includes error recovery hints, validation rules, or actionable error messages. Parameters lack constraints (enums, ranges, patterns). No tool annotations (readOnlyHint, idempotentHint). Output format for search_seller_knowledge is inferred from code (JSON array of objects with content/source/relevance_score), not formally documented in schema. score_answer has no documented return type at all.
Score the faithfulness and relevance of an answer against retrieved context.
Search the Amazon Seller Central knowledge base for information about listings, fees, FBA, account health, returns, shipping, brand registry, inventory management, and policies.
score_answer has no documented output schema. LLMs cannot plan downstream actions or extract validation scores without knowing return type and field names.
No error handling guidance. Tools do not indicate what errors are retryable, user-fixable, or fatal. No recovery hints (e.g., 'If query returns no results, try a broader search term').
Parameter 'k' in search_seller_knowledge lacks constraints. No min/max bounds documented. LLMs could pass absurd values (k=10000) causing performance issues.
score_answer parameter descriptions are minimal (answer: 'The generated answer to evaluate', context: 'The retrieved context used to generate the answer'). No guidance on expected format, length, or structure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
No tool annotations (readOnlyHint, idempotentHint, destructiveHint). Both tools are read-only but this is not formally declared in the schema, forcing LLMs to infer safety.