Loading ToolBench
A Quarkus-based OAuth 2.0 authorization server implementation with support for authorization code flow, PKCE, refresh tokens, and client registration. Includes consent flow and cleanup operations.
This EMP OAuth server exhibits severe definition quality gaps across all dimensions. While it is an HTTP-based server with 4 tools, the implementation lacks fundamental requirements for agent tooling: NO visible input parameter schemas for any tool, NO output schemas documented, minimal and incomplete descriptions, and missing parameter documentation. Tool definitions appear to be inferred from JAX-RS annotations rather than explicit MCP tool registration. The server implements OAuth 2.0 endpoints but does not expose them as proper MCP tools with structured schemas. Descriptions exist but are vague (e.g., 'Simple greeting endpoint') and do not follow LLM-optimized patterns. Parameters like 'state' and 'decision' have minimal context. No error handling guidance is visible, and no tool annotations (destructiveHint, idempotentHint, readOnlyHint) are present despite destructive operations.
Simple greeting endpoint that returns a test message
Display the consent page for a user to approve or deny an authorization request. Requires a state parameter referencing a valid, non-expired auth request.
Cleanup endpoint that removes expired authorization requests, authorization codes, refresh tokens, and inactive clients based on configured TTLs and inactivity periods
Handle user consent decision. If approved, issues an authorization code and redirects to the client's redirect URI. If denied, returns an access_denied error.
NO input parameter schemas visible for ANY tool. Tool 1 (cleanup) has `Input: {}` (empty object). Tool 2 (consent GET) and Tool 3 (consent POST) show minimal schema with 'state' and 'decision' as bare strings lacking type definitions beyond 'string'.
NO output schemas documented for any tool. LLMs cannot infer the response structure, field types, or what data is available for downstream tool calls or agent reasoning. This breaks the composition chain and forces agents to guess.
Descriptions are vague and sub-LLM-optimal. 'Simple greeting endpoint that returns a test message' (GET /hello) is 60 chars but trivial and uninformative. 'Display the consent page...' (GET /oauth/consent) lacks WHEN to call it or what prerequisites exist. Pattern baseline expects 50-200 character, LLM-optimized descriptions that explain WHAT, WHEN, and WHAT TO EXPECT.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 28 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 48 | - | v1 |
Parameters lack descriptions or are minimally described. 'state' (GET/POST /oauth/consent) is described as 'Authorization request state identifier' but does NOT explain: What format? How long? What happens if invalid? 'decision' (POST /oauth/consent) only mentions values 'approve' or 'deny' but lacks a proper enum constraint and does not explain edge cases.
POST /oauth/cleanup is flagged DESTRUCTIVE (removes expired data) but has no confirmation pattern, dry-run option, or error recovery guidance. A dangerous operation with no safety guardrails violates pattern:confirmation-request.
NO tool annotations (readOnlyHint, destructiveHint, idempotentHint) visible in tool definitions. GET /oauth/consent should be marked readOnly. POST /oauth/cleanup should be marked destructive. These hints are essential for agents to reason about side effects and safety.
Tool definitions appear INFERRED from JAX-RS path/method annotations rather than explicitly registered with MCP-compliant schemas. Source shows @Path and @POST/@GET annotations but no visible explicit tool registration with names, descriptions, inputSchema, outputSchema, and annotations.
No visible error handling guidance. If GET /oauth/consent is called with an invalid or expired 'state', what happens? Is it a 400? 404? How should an agent retry or recover? Error responses must tell the LLM what to do next (pattern:recovery-guide).
Naming convention issues. 'GET /oauth/consent' and 'POST /oauth/consent' are the same resource with different methods but do not follow verb_noun pattern (e.g., 'get_consent_page' + 'approve_authorization'). REST conventions are transparent to agents, MCP tools should use explicit action verbs.