Loading ToolBench
Open-source firewall and safety sandbox for autonomous AI agents. Provides runtime security gateway that intercepts and validates agent tool calls, detects command injection attacks, sanitizes credentials, and enforces zero-trust security policies.
OpenAgentShield is a security firewall, not an MCP server. The codebase defines 17 tools with destructive/sensitive operations (command execution, file I/O, database drops). However, NO MCP protocol implementation is visible. Tools are defined as Python functions in agent_shield/ modules, not registered via MCP's tool registration mechanism. Input schemas are partially visible in app.py demo but NOT formally declared in MCP schema format. Descriptions exist but are generic security-focused strings, not LLM-optimized. No output schemas documented. This is a standalone Python library, not an MCP server ready for integration.
Android Debug Bridge shell command execution with security inspection for injection and dangerous operations
Bash shell execution with security inspection for command injection and destructive payloads
Read file contents (cat command) with sensitive file pattern blocking
Delete all records (requires human approval per policy)
Drop database (requires human approval per policy)
Generic command execution with security inspection for injection risks and dangerous operations
Fetch file contents with sensitive file pattern blocking
NOT an MCP server. No MCP protocol implementation detected. Tools are Python functions in a standalone library, not registered via MCP's tool registration mechanism. Cannot be used as an MCP server without a transport layer and protocol adapter.
Input schemas not formally declared in MCP JSON Schema format. Schemas inferred from demo code (app.py) but not visible in tool definitions.
Descriptions are generic and security-focused, not LLM-optimized. Average ~50 chars, below baseline 194 chars. Missing WHEN to use, WHAT it returns, and dependency hints. E.g., 'Executes shell commands with security inspection' does not explain when an LLM should call run_command vs bash vs execute_command.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 36 | 2026-07-28+ | v2 |
Force push to Git repository (requires human approval per policy)
Launch Android application with package name validation to prevent command injection (Artemis Issue #55 mitigation)
Read file contents with sensitive file pattern blocking (.env, SSH keys, AWS credentials, /etc/shadow, /etc/passwd)
Replace file content with credential redaction
Executes shell commands with security inspection for command injection, destructive payloads, and dangerous binary execution
Stop Android application with package name validation
Terminal command execution with security inspection
Terminate Android application with package name validation
View file contents with sensitive file pattern blocking
Write content to file with credential redaction to prevent secret leakage
No output schemas documented. Tools return EvaluationResult (verdict, risk_score, reasons, sanitized_arguments) but LLMs cannot see this structure. Agents cannot plan downstream calls or extract fields reliably.
Duplicate/overlapping tool names. run_command, bash, execute_command, terminal all execute shell commands with identical security logic. LLMs waste reasoning cycles deciding between them. Consolidate to one canonical tool.