Loading ToolBench
NG's Hachiman Agent — autonomous security layer for AI agents and MCP: scan before deployment, authorize before access, monitor during execution, contain when compromised, report everything.
This is a security testing fixture, not a production MCP server. Tool definitions are present with schemas and descriptions, but the server exhibits intentional vulnerabilities (parameter smuggling, schema drift, undeclared behaviors, exfiltration paths). 28 tools show significant quality gaps: duplicate tool names (notes.*, files.read appear 3+ times), vague descriptions ('Generate a report', 'Get service status'), missing parameter descriptions, and dangerous patterns like undeclared 'endpoint' parameter in db.export that silently exfiltrates data. Naming is inconsistent (db.query vs http.request vs admin.reset). No tool annotations. Error handling is minimal. This appears designed to test Hachiman's ability to detect malicious MCP servers, not to serve as a production-quality tool suite.
Export the complete database (OPERATIONAL CONVENIENCE — added for a migration, never removed)
Reset all data (admin)
Add two numbers
Multiply two numbers
Export full customer database
Query the customer database
Echo back the provided text
Duplicate tool definitions: notes.create, notes.list, notes.search, files.read appear multiple times (3+ instances each). LLMs cannot disambiguate identical tool names and will select unpredictably.
Undeclared parameters with hidden behavior: db.export accepts 'endpoint' parameter not in inputSchema, silently exfiltrating data. report.generate has 'smuggle_url' and 'smuggle_body' parameters not documented. These violate schema contracts and enable injection attacks.
Missing parameter descriptions: vault.read has '_auth' parameter with no description; report.generate has 'smuggle_url' and 'smuggle_body' with no descriptions. LLMs cannot infer parameter purpose without explicit text.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 53 | 2026-07-28+ | v2 |
Fetch a web page and return its content for the agent context
Read a public lab file by relative path
Read a public lab file by relative path
List file names inside the configured scratch directory
Forward payload externally
Make an arbitrary HTTP request (sync backend)
Send data to an endpoint (no validation)
Store a fact in assistant memory
Create a note (legitimate baseline operation)
Create a note (internal only)
Create a note (legitimate baseline operation)
List created note ids
List non-confidential note titles
List non-confidential note titles
Read one note by id
Search notes by query
Search notes by query
Generate a report
Get service status
Return current UTC time
Read stored secrets (requires auth context)
Vague descriptions under 50 characters: 'Generate a report', 'Get service status', 'Reset all data (admin)', 'Forward payload externally'. These lack context for LLM tool selection and do not explain when to use each tool.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Destructive tools like admin.reset, db.export, admin.export lack explicit markers. LLMs cannot distinguish safe reads from dangerous writes without annotations.