Loading ToolBench
MCP server for AWS S3 bucket access via Britive JIT (Just-In-Time) access management. Enables checkout/checkin of S3 profiles with programmatic access and token retrieval.
This server has significant definition quality issues. While tool names follow verb_noun convention (list_, checkout_, get_, summarize_, checkin_), parameter descriptions are minimal or entirely absent. Most critically, output schemas are not documented anywhere, the tools return unstructured dicts with arbitrary field names. Parameters like 'profile_id', 'environment_id', and 'tx_id' lack descriptions explaining what these identifiers represent or how to obtain them. The tool descriptions are extremely terse (10-50 chars), providing almost no guidance on when to use each tool, what prerequisites exist, or how they compose. Error handling returns bare exception strings without actionable recovery paths. The summarize_s3 tool accepts AWS credentials as parameters, violating the secret-injection pattern, credentials should never be exposed in tool parameters where they enter logs and traces.
Check in the given profile at end of session using the transaction ID (txID).
Checkout the given profile/environment and return the tx_id.
Get AWS tokens for a checked-out profile using the tx_id.
List all profiles/environments that support programmatic S3 access.
List and summarize S3 buckets using the provided AWS credentials (boto3).
AWS credentials passed as tool parameters (access_key, secret_key, session_token in summarize_s3). Credentials must use server-side secret injection via environment variables or vault. Exposing secrets as parameters logs them into agent traces and prompt history, creating a critical security leak.
Output schemas completely undocumented. Tools return unstructured dicts (e.g., {'aws_profiles': [...], 'error': '...'}) without documenting what fields agents should expect. LLMs cannot plan downstream tool calls or extract the right data without knowing the structure.
Parameter descriptions almost entirely missing. 'profile_id' and 'environment_id' have minimal descriptions ('The profile ID to checkout'). No explanation of what these IDs are, how to obtain them, or their format. Users/agents cannot populate these without trial and error.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 32 | - | v1 |
Tool descriptions are extremely terse (10-50 chars, well below 194-char baseline). No guidance on WHEN to use each tool, prerequisites, or composition. Description does not answer: What does it do? When should the LLM call it instead of a similar tool? What does it return?
Error handling returns bare exception strings without actionable recovery guidance. E.g., 'Checkout failed: Connection timeout' tells the LLM nothing, should it retry? Call a different tool? The error response must guide the agent's next step.
Tool composition dependencies undocumented. The workflow is clearly: list_s3_profiles → checkout_s3_profile → get_s3_tokens → summarize_s3 → checkin_s3_profile_when_session_done. But no tool description hints at this sequence. Agents cannot infer the workflow without explicit dependency documentation.
checkin_s3_profile_when_session_done has a confusing name containing 'and'. The phrase 'when_session_done' is a temporal qualifier, not a separate action. Better names: checkin_s3_profile, release_s3_profile, or return_s3_profile.
No mention of permission checks or scope declarations. What permissions does an agent need to call these tools? Are they read-only or write? checkout_s3_profile and checkin_s3_profile are destructive (checkout/release resources), but no scope declarations or audit logging is evident.