Loading ToolBench
Controlled excavation tools for agents on Amazon Bedrock AgentCore
Campus-Claws exhibits significant definition quality gaps. While 12 of 14 tools have brief descriptions (10-50 chars), most lack depth and actionable guidance. Only 'discover' and 'excavate' show partial input schemas; 'probe', 'plan', 'refine', 'watch', 'watches', 'team_plans', 'share_plan', 'remember', 'recall' have NO visible input schemas in the source. Tool names are action-oriented (discover, excavate, refine) but descriptions are minimal and do not explain WHEN to use each tool or what it returns. No output schemas documented. Error handling, parameter constraints, and recovery guidance are absent. The two internal tools (approve_plan, audit_export) have schemas but are not registered as AgentCore tools, creating confusion about their role.
Internal Lambda for IRB plan approval - NOT an AgentCore tool
Internal Lambda for compliance audit record export - NOT an AgentCore tool
Find data sources matching a topic
Execute a concrete query from a plan
Export query results to external destinations
Generate a concrete query plan from a natural language objective
Probe a data source to understand its structure and content
10 of 14 tools lack visible input schemas. Only 'discover', 'excavate', 'approve_plan', and 'audit_export' show parameter definitions. Tools 'probe', 'plan', 'refine', 'watch', 'watches', 'team_plans', 'share_plan', 'remember', 'recall' have no documented input structure, making it impossible for LLMs to construct valid calls.
Descriptions are uniformly brief (10-50 chars) and lack actionable guidance. 'Probe a data source to understand its structure and content' does not explain WHEN to call probe vs discover, what parameters it accepts, or what it returns. Baseline for A+ tools is 50-200 chars with clear WHAT/WHEN/RETURN guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Retrieve previously stored information from memory
Refine a query based on results and feedback
Store information in memory for later recall
Share a plan with other team members
List and manage team collaboration plans
Create a watch on a data source for continuous monitoring
List and manage active watches
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields without knowing what each tool returns. 'discover' should document that it returns a list of source_id, name, type, and description fields.
No error handling or recovery guidance. Tools do not document what errors can occur, whether they are retryable, or what the LLM should do next. E.g., 'excavate' with an invalid query_type should return 'Invalid query_type: got "invalid_sql", must be one of: athena_sql, dynamodb_partiql, opensearch_dsl, s3_select_sql, mcp_tool'.
Internal tools 'approve_plan' and 'audit_export' are marked 'NOT an AgentCore tool' but are deployed in the same stack and have schemas. This creates ambiguity: are they callable by agents or not? If not, remove them from the tool list. If yes, document their role and permissions clearly.