Loading ToolBench
MCP server for reading and querying data from S3 bucket data lakes
The S3 Data Lake MCP server has 5 tools with reasonable naming and decent descriptions. All tools are properly registered with the @mcp.tool() decorator and include docstrings. However, there are notable gaps in schema completeness, parameter descriptions, and error handling guidance. The tool names follow verb_noun convention (list_, read_) which is good. Descriptions are present but inconsistent in quality and some lack detail about WHEN to use the tool versus alternatives. Output is returned as JSON strings rather than structured objects, reducing downstream composability. Error handling returns JSON blobs but does not provide actionable recovery guidance. No tool annotations (readOnlyHint, etc.) are present despite tools being READ_ONLY.
List all S3 buckets accessible to the current AWS credentials.
List objects in an S3 bucket with optional prefix filtering.
Read a CSV file from S3 and return its contents as JSON.
Read a JSON file from S3 and return its contents.
Read a Parquet file from S3 and return its contents as JSON.
Output returned as JSON strings rather than structured response objects. Tools return json.dumps() of dicts, forcing clients to parse strings and losing type safety. Should return structured TextContent with parsed JSON or direct object serialization.
Missing tool annotations. All 5 tools are READ_ONLY but lack readOnlyHint annotation. This prevents clients from understanding safety properties and optimizing caching/retry behavior.
Error responses do not provide actionable recovery guidance. Errors return a status/message JSON but do not tell the LLM what to try next, whether the error is retryable, or how to fix it. Example: 'AWS error listing objects' should suggest 'Verify bucket exists (call list_s3_buckets), or check IAM permissions.'
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | C | 63 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No pagination support for list_s3_objects despite accepting max_keys. Response includes is_truncated boolean but no continuation token or next_cursor, forcing agents to re-call with a different prefix or offset logic that is not exposed. Large bucket listings will be incomplete.
Parameter descriptions lack detail about valid ranges and constraints. 'max_rows': 'Maximum number of rows to return (default: 1000)' does not specify minimum, maximum, or what happens if the file is smaller. Should state: 'Maximum rows to read (1 - 1000000; returns fewer if file is smaller).'
Output schema not documented. Tools return 'status', 'message'/'error' fields but response structure is not formally specified. LLMs cannot plan downstream operations or extract fields reliably. Should document: 'Returns {status: "success"|"error", buckets?: Array<{name, creation_date}>, message?: string}'
No mention of data size/performance limits. read_csv_from_s3, read_json_from_s3, read_parquet_from_s3 accept large max_rows values but don't document memory constraints or what happens if the file is multi-GB. Could cause OOM errors or timeouts without clear messaging.
Global s3_client initialized once and reused. If AWS credentials expire or rotate, the global reference becomes stale. initialize_s3_client() is called on-demand but not idempotent, repeated calls overwrite the client, risking race conditions in concurrent requests.