Loading ToolBench
AWS MCP Server - A client implementation that connects to AWS Documentation and Git MCP servers, with LLM agent capabilities
This server exhibits significant quality gaps across naming, descriptions, schemas, and error handling. While tool names follow a verb_noun pattern (search_documentation, read_documentation, git_status, etc.), many tool descriptions are generic and under-optimized for LLM selection. Critical issues: (1) Duplicate/near-duplicate tools without clear differentiation (search_documentation vs search_aws_documentation; read_documentation vs read_aws_documentation). (2) Input schemas are present but minimally described, parameters lack guidance on format, constraints, or prerequisites. (3) No output schemas documented anywhere; LLMs cannot reason about what these tools return. (4) No error handling guidance, no retry hints, recovery paths, or actionable error messages visible in the code. (5) Parameter descriptions are trivial (e.g., 'The search query for AWS documentation', does not explain when to use vs read_documentation, expected format, or result limits). (6) Git tools (git_commit, git_add) lack confirmation/dry-run patterns despite being destructive. (7) No pagination hints despite documentation tools potentially returning large result sets. The code sample shows a client implementation rather than explicit tool definitions with rich metadata.
Stage files for commit
Create a git commit
Get the git diff showing changes in the working directory or staged changes
Get the git commit history
Get the current git status of the repository, showing modified, staged, and untracked files
Read a specific AWS documentation page given its URL
Read a specific AWS documentation page
Duplicate and near-duplicate tools without differentiation. Both 'search_documentation' and 'search_aws_documentation' exist, as do 'read_documentation' and 'read_aws_documentation'. LLM will waste reasoning cycles deciding between them. No clear documentation of when to use one vs the other.
No output schemas documented. Tool descriptions do not state what fields are returned, data types, or structure. LLMs cannot plan downstream operations or extract required data (e.g., does search_documentation return URLs, summaries, or both?).
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 42 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 30 | 2024-11-05+ | v1 |
Search AWS documentation for information about AWS services, features, and best practices
Search AWS documentation
Parameter descriptions lack depth. 'The search query for AWS documentation' does not explain expected format, character limits, or query syntax. 'The URL of the AWS documentation page to read' does not explain URL format validation or maximum content length implications. No dependency hints (e.g., 'Call search_documentation first if you do not have a direct URL').
Destructive tools (git_commit, git_add) lack confirmation/dry-run patterns. No evidence of dry-run mode or user confirmation steps. Agents can commit and stage files without verification, risking unintended repository mutations.
No error handling guidance. No visible retry hints, recovery paths, or actionable error messages in code. If search_documentation returns no results, the LLM has no guidance on what to try next. If git_commit fails due to uncommitted changes, there is no suggestion to call git_add first.
git_log has integer parameter 'max_count' with no min/max bounds specified. Description says 'default: 10' but does not state upper limit, risking unbounded result sets that exhaust context windows. No pagination guidance.
read_aws_documentation has optional 'max_length' parameter with no guidance on behavior if content exceeds limit. Is it truncated? Paginated? Streamed? Ambiguity forces LLM to guess.
git_add parameter 'files' is an array with no schema details. Are paths relative or absolute? Do they support globs? What happens if a file does not exist? No validation rules stated.
Tool definitions appear to be in aws_mcp_client.py and git_mcp_client.py as client implementations, not as explicit server-side tool registrations with full metadata. The actual MCP server implementation (main entry point, tool registration, schema export) is not visible in provided code. Cannot verify complete tool definitions or registration mechanism.