Loading ToolBench
A bonded red team simulator that attacks live AgentGate infrastructure and reports findings
This red-team simulator exposes 24 attack/utility tools with significant definition gaps. While tool names are action-oriented (signedPost, attack4_1, etc.), descriptions lack LLM-optimization guidance. Most critically: 18 of 24 tools have minimal descriptions (<50 chars); parameter descriptions are sparse or missing context about constraints, ranges, and error conditions. Input schemas are present but lack depth, many parameters accept 'object' or 'string' without enum constraints or format specifications. No output schemas are documented. Error handling is absent from descriptions. The toolkit is designed for red-team testing (legitimate use case), but the definitions do not guide an LLM to use these tools safely or correctly.
Reputation pumping — tests whether AgentGate can be manipulated via cheap successful actions
Sybil campaign chain — tests whether AgentGate can be overwhelmed by many identities
Over-commit exposure — tests whether AgentGate enforces bond capacity limits
Double-resolve — tests whether AgentGate prevents resolving an action twice
Act on expired bond — tests whether AgentGate enforces bond TTL expiration
Admin endpoint without admin key — tests whether AgentGate enforces admin access control
Resolve another identity's action — tests cross-identity authorization boundaries
Attack tools (attack2_*, attack4_*, attack5_*, attack11_*) use generic names that do not clearly convey intent. 'attack4_1' is opaque; LLMs cannot infer what authorization boundary is being tested without reading the description. Rename to 'test_admin_access_control', 'test_cross_identity_authorization', etc.
18 of 24 tools have descriptions under 50 characters, violating the 10 - 1024 character baseline (median 194 chars). Descriptions like 'Fetch the reputation score for an identity' lack context about when to call this tool, what it returns, and how it fits into the attack workflow. Expand to 100 - 200 chars with actionable guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 39 | 2026-07-28+ | v2 |
Execute action on another identity's bond — tests bond-to-identity binding
Oversized payload — tests whether AgentGate rejects payloads exceeding size limits
TTL exceeding cap — tests whether AgentGate enforces maximum TTL limits
Negative bond amount — tests whether AgentGate rejects negative monetary amounts
Type coercion — tests whether AgentGate properly validates numeric fields against string inputs
String fields at max-length boundary — tests whether AgentGate enforces string length limits
Payload at 4096-byte boundary with multi-byte UTF-8 — tests payload size validation with UTF-8 encoding
Generate a fresh Ed25519 keypair and register a new identity with AgentGate
Fetch the status of an action by ID
Fetch the status of a bond by ID
Fetch the reputation score for an identity
Send an unsigned GET request
Send an unsigned POST request with custom headers
Send a properly signed POST request to AgentGate with Ed25519 signature
Send a signed POST request using a specific identity's keypair
Send a signed POST request using the resolver identity's keypair
Send a signed POST request with custom nonce and timestamp control
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., identityId from createIdentity, status from getActionStatus). Add explicit return type documentation with field names, types, and examples.
Parameter constraints are missing or vague. 'apiPath' accepts any string starting with '/' but lacks guidance on valid endpoints. 'payload_bytes' in attack5_1 has no min/max bounds. 'ttl_seconds' in attack2_3 lacks range specification. Add explicit constraints (enums, min/max, regex patterns) to every parameter description.
No error handling guidance in any tool description. If signedPost fails with 401, what should the LLM do? If createIdentity hits the 3-identity cap, how does the agent recover? Add error classification (retryable, user-fixable, fatal) and recovery hints to every tool.