Loading ToolBench
OpenBao MCP exhibits critical definition quality gaps. All 4 tools have descriptions, but they are generic and lack actionable guidance (10-50 chars). Tool names lack clear verb-noun structure (e.g., 'openbao_mcp_auth' vs 'authenticate_user'). Input schemas are present but severely underspecified: the 'action' parameter accepts free-form strings instead of enums, forcing LLMs to guess valid values. The 'params_json' parameter is a string escape hatch that bypasses schema validation entirely, this is a major anti-pattern. No output schemas are documented. Parameter descriptions are minimal (e.g., 'JSON string of parameters' provides no guidance on what parameters are valid). No error handling guidance. No security considerations documented despite WRITE-risk tools managing secrets and SSH keys.
Manage OpenBao auth operations.
Manage OpenBao secrets operations.
Manage OpenBao SSH operations.
Manage OpenBao system operations.
Action parameter uses free-form string instead of enum. LLMs cannot determine valid actions without trial-and-error. E.g., 'openbao_mcp_auth' accepts 'login', 'mfa_login', 'mfa_validate', 'token_create', etc., but schema does not constrain these.
params_json parameter is a string escape hatch that bypasses schema validation. This anti-pattern forces LLMs to construct JSON strings, invites injection attacks, and prevents validation. All 4 tools use this.
Tool names lack verb-noun structure. 'openbao_mcp_auth' does not convey action (authenticate? authorize? manage?). Names should start with action verbs: 'authenticate_user', 'create_token', 'read_secret', etc.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 31 | <=2025-11-25 | v2 |
No output schemas documented. LLMs cannot plan downstream calls or extract required fields. E.g., does 'token_create' return a token string, an object with token+ttl, or a reference ID?
Descriptions are generic and lack context. E.g., 'Manage OpenBao auth operations' (30 chars) does not explain WHEN to use this tool, what it returns, or what prerequisites exist. Baseline: 194 chars average for A+ tools.