Loading ToolBench
End-to-end pipeline for fetching, validating, and storing OpenAQ air quality data with MCP tool integration for permission checks, Docker control, and test execution
This MCP server has critical definition quality gaps across nearly all dimensions. Of 7 tools, only 2 have input schemas visible in the source code (permission_check and docker_check_permissions). The remaining 5 tools (requests_check_permissions, postgres_query_check_permissions, postgres_backup_check_permissions, command_start_all_containers, command_run_all_tests) have empty input objects {} with no parameters defined. Tool descriptions exist but are generic and do not follow the 10-1024 character guideline or provide sufficient context for LLM selection. No output schemas are documented anywhere. The code snippet cuts off mid-function (execute_command is incomplete), preventing full assessment of error handling and response structure. Tool naming partially follows verb_noun convention but lacks clarity about what distinguishes the permission check tools from each other. No parameter descriptions are present except for permission_check which has a single 'permission' parameter with a description. Security concerns: command_start_all_containers executes arbitrary shell commands (via execute_command function) but provides no input validation, error handling, or confirmation gate for a destructive operation. No evidence of rate limiting, audit logging, or permission enforcement beyond basic env var checks.
Runs all tests using the 'Python3 all_tests.py' command.
Starts all containers using the 'make up' command.
Checks the permissions for Docker-related operations.
Its main purpose is to check whether the PERMISSION section in the .env file is True or False: PERMISSION_DOCKER_CONTROL, PERMISSION_POSTGRES_QUERY, PERMISSION_POSTGRES_BACKUP, PERMISSION_REQUESTS
Checks the permissions for PostgreSQL backup-related operations.
Checks the permissions for PostgreSQL query-related operations.
5 of 7 tools have no input schema (empty {} objects). Per HARD SCORING RULES, tools with NO input schema must have schema score of 0. This violates pattern:constrained-input and makes it impossible for LLMs to understand what parameters are valid.
No output schemas documented for any tool. The execute_command function (incomplete in provided code) is called by command_* tools but its return structure is never defined in the tool definitions. LLMs cannot plan downstream calls or extract required fields without knowing what fields are returned.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 36 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 6 | - | v1 |
Checks the permissions for Requests-related operations.
command_start_all_containers is marked WRITE risk but has no confirmation gate, dry-run option, or input validation. The function signature shows it accepts arbitrary shell commands via execute_command() but provides no protection against injection or accidental destructive operations. Violates pattern:confirmation-request.
Permission check tools (docker_check_permissions, requests_check_permissions, postgres_query_check_permissions, postgres_backup_check_permissions) are 4 separate tools that all do the same thing: call permission_check() with a hardcoded permission string. This violates pattern:tool (one responsibility per tool) and wastes LLM reasoning cycles deciding between nearly identical tools. Should be a single parameterized tool or exposed as variants via a single discovery endpoint.
Tool descriptions are generic and under 50 characters for most tools, providing minimal context. E.g., 'Checks the permissions for Docker-related operations.' does not explain WHEN to call it, WHAT it returns, or HOW to act on the result.
No error handling guidance. The execute_command function catches no exceptions and provides no recovery paths. If 'make up' fails, the agent receives no actionable error (e.g., 'Container already running' vs 'Docker daemon not accessible' vs 'Compose file syntax error'). Per pattern:recovery-guide, errors must tell the LLM what to do next.
execute_command() uses shlex.split() and subprocess but provides no input validation against command injection. An LLM could be tricked into passing 'make up; rm -rf /' or similar payloads. Per pattern:tool-gateway, all agent-provided input must be sanitized.
Tool naming is inconsistent. permission_check uses verb_noun, but the other tools use check_permissions (verb then object, not noun first). Additionally, 'docker_check_permissions' vs 'permission_check' creates ambiguity, does one check Docker permissions while the other checks generic permissions? Per pattern:tool, clear naming avoids LLM confusion.