Loading ToolBench
Convert API repositories into CLI tools automatically. Platform with Core service (auth, user management), TPS service (third-party app integration, OAuth, token management), Temporal worker, and Next.js frontend.
Critical failures across naming, descriptions, and schemas. 24 tools listed but many are duplicates (list_users, get_user, create_user appear 3+ times each). Tool definitions appear to be inferred from test fixtures rather than explicit MCP registrations. Most tools lack parameter descriptions entirely. No output schemas documented. Descriptions are minimal (10-50 chars) and many tools have none. Parameter types are present but descriptions are absent for most params (e.g., 'authorization' param has description in some tools but not others). No error handling guidance. Security concern: 'authorization' token passed as parameter rather than server-side injection.
Create a new project.
Create a new user.
Create a new user.
Delete a user.
GET /apps/{identifier} — polymorphic lookup by app_name or app_code.
Duplicate tool definitions: list_users, get_user, create_user, delete_user appear 3+ times with inconsistent schemas and descriptions. This violates single-responsibility and creates ambiguity for LLM tool selection.
Authorization token passed as tool parameter instead of server-side secret injection. Tokens will appear in agent logs and traces, creating a security leak.
Most parameter descriptions are missing or minimal. 'authorization' param has description in some tools but not others. Parameters like 'limit', 'offset', 'user_id' lack descriptions explaining constraints, ranges, or expected formats.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | 2026-07-28+ | v2 |
Get current user from token.
Get a project by ID.
Get the current user's session info including workspace.
Get a user by ID.
Get a user by ID.
Health check endpoint.
GET /apps — list all active apps, optionally filtered by category.
List all projects.
List all users.
List all users.
List all users.
Login and get a token.
Log out by deleting the session.
Send a magic link to the user's email. Always returns 200 regardless of whether the email exists (prevents enumeration). Uses Resend for email delivery. Falls back to console logging if no API key.
Root endpoint.
Verify a magic link token, create/find user, set session cookie.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., user_id from create_user response). This breaks tool composition.
Tool 'root' has non-verb name and minimal description (15 chars). Does not follow verb_noun naming convention. Unclear what this endpoint does or when to call it.