Loading ToolBench
An AWS Labs Model Context Protocol (MCP) server for AWS Documentation that provides tools to access public AWS documentation, search for content, and get recommendations.
AWS Documentation MCP server has clear tool names and detailed descriptions, but exhibits significant schema and composition issues. Tools are well-named and descriptions are verbose (exceeding the 10-1024 char baseline), but critical gaps exist: no input schemas visible in the source code provided, duplicate tool naming (two 'read_documentation' tools for different partitions), missing parameter type definitions, and no documented output schemas. Error handling is absent from the visible source. The server follows a reasonable composition pattern (separate tools for search, read, recommend) but the schema quality is demonstrably below production grade.
Fetch available services from AWS China documentation. ## Usage Available services in AWS China are different from global AWS services. This tool retrieves a list of available services and their documentation URLs. ## Output Format The output is formatted as markdown text with: - Preserved headings and structure - Code blocks for examples - Lists and tables converted to markdown format
Fetch and convert an AWS documentation page to markdown format. ## Usage This tool retrieves the content of an AWS documentation page and converts it to markdown format. For long documents, you can make multiple calls with different start_index values to retrieve the entire content in chunks. ## URL Requirements - Must be from the docs.aws.amazon.com domain - Must end with .html ## Example URLs - https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html - https://docs.aws.amazon.com/lambda/latest/dg/lambda-invocation.html ## Output Format The output is formatted as markdown text with: - Preserved headings and structure - Code blocks for examples - Lists and tables converted to markdown format ## Handling Long Documents If the response indicates the document was truncated, you have several options: 1. **Continue Reading**: Make another call with start_index set to the end of the previous response 2. **Stop Early**: For very long documents (>30,000 characters), if you've already found the specific information needed, you can stop reading
Fetch and convert an AWS China documentation page to markdown format. ## Usage This tool retrieves the content of an AWS China documentation page and converts it to markdown format. For long documents, you can make multiple calls with different start_index values to retrieve the entire content in chunks. ## URL Requirements - Must be from the docs.amazonaws.cn domain - Must end with .html ## Example URLs - https://docs.amazonaws.cn/en_us/AmazonS3/latest/userguide/bucketnamingrules.html - https://docs.amazonaws.cn/en_us/lambda/latest/dg/lambda-invocation.html ## Output Format The output is formatted as markdown text with: - Preserved headings and structure - Code blocks for examples - Lists and tables converted to markdown format ## Handling Long Documents If the response indicates the document was truncated, you have several options: 1. **Continue Reading**: Make another call with start_index set to the end of the previous response 2. **Stop Early**: For very long documents (>30,000 characters), if you've already found the specific information needed, you can stop reading
Duplicate tool names: two tools both named 'read_documentation' (AWS and AWS China variants). This violates single-responsibility and composition principles. LLMs cannot reliably distinguish between them based on naming alone.
No input schemas visible in source code. The rubric requires explicit JSON Schema definitions with type declarations for every parameter. Visible parameter descriptions show 'url' (string), 'max_length' (integer with defaults and bounds), 'start_index' (integer with defaults and bounds), and 'search_phrase' (string), 'limit' (integer), but no formal schema registration code is shown. Cannot verify fastmcp tool registration mechanism.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 49 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 44 | - | v1 |
Get recommendations for related AWS documentation based on a documentation URL.
Search AWS documentation using the official AWS Documentation Search API. ## Usage This tool searches across all AWS documentation for pages matching your search phrase. Use it to find relevant documentation when you don't have a specific URL. ## Search Tips - Use specific technical terms rather than general phrases - Include service names to narrow results (e.g., "S3 bucket versioning" instead of just "versioning") - Use quotes for exact phrase matching (e.g., "AWS Lambda function URLs") - Include abbreviations and alternative terms to improve results ## Result Interpretation Each result includes: - rank_order: The relevance ranking (lower is more relevant) - url: The documentation page URL - title: The page title - context: A brief excerpt or summary (if available)
No documented output schemas. Tool descriptions detail input behavior (truncation, chunking for read_documentation; search result fields for search_documentation) but nowhere in the source code is a return type or output schema defined. LLMs cannot plan downstream tool chains without knowing what fields to expect.
'recommend' tool has minimal description ('Get recommendations for related AWS documentation based on a documentation URL.'). At 68 characters, it barely meets the 10-char minimum and lacks critical context: When should the LLM call this vs. search_documentation? What does 'recommendations' return? How many? In what order? This violates the pattern requirement that descriptions answer 'What, When, What returns?'
No error handling visible. Tool descriptions (especially read_documentation and recommend) lack actionable error guidance. What happens if a URL is invalid, unreachable, or outside docs.aws.amazon.com? What should the LLM do on a search failure? No recovery patterns, classifications (retryable vs. fatal), or suggestions provided.
get_available_services accepts zero parameters but no description explains what 'available services' means. Does it list all AWS services? Only China-region services? Does it return URLs, docs, SKUs? The description states output 'formatted as markdown text with headings, code blocks, lists, tables' but does not clarify structure or use cases.
Missing parameter descriptions for 'recommend' tool. The 'url' parameter has a description, but the tool description does not explain: What kind of URL? (docs.aws.amazon.com only, or any AWS documentation?) What constraints on the URL? This forces LLMs to infer from the tool name alone.
No pagination or result limits documented for search_documentation output. The tool accepts a 'limit' parameter (1-50, default 10), but the description does not clarify: What is returned if limit is 10 but there are 500 matching docs? Is there a next_cursor or offset? Can you paginate? This violates the pattern that tools returning lists should support pagination.
read_documentation tools accept 'max_length' (default 5000, max 999999) and 'start_index' for chunking, but no guidance on total document size or chunking strategy. How large are typical AWS docs? At 5000 char default, how many chunks? Should LLMs request the entire doc or stop when they find relevant info? No dependency hints or multi-step workflow guidance.