Loading ToolBench
An MCP server gateway that proxies tool calls through OAuth-protected endpoints, with a demo API backend providing sample tools (echo, notes, CRM) for testing user identity scoping and OAuth token verification.
The server defines 10 tools with explicit schemas and descriptions. Most tools have reasonable naming conventions (verb_noun style: addNote, listMyNotes, seedMyNotes, crmGetSalesSummary, crmAttemptCrossUserRead) and clear descriptions (average ~120-150 chars, within baseline range of 34-392). However, several tools lack meaningful input validation guidance, some descriptions are vague about scope/side effects, and output schemas are not documented. The CRM-prefixed tools create minor naming ambiguity (crmGetSalesSummary vs crmExplainAccess, both operate on CRM data but names don't clearly distinguish the operation). Parameter descriptions are minimal; most are 1-2 sentences without format/constraint guidance. Error handling is mentioned in tool names (crmAttemptCrossUserRead) but error recovery guidance is absent from descriptions. Security-aware naming (whoami, crmAttemptCrossUserRead) shows some pattern awareness but responses are not shown, cannot verify output schemas or secret leakage.
Add a note for the current user (scoped by OAuth sub).
Demonstrate that cross-user CRM access is denied. Provide a target user label (e.g. from another login) and the tool will refuse and explain why.
Explain what CRM data the current authenticated user can access and what they cannot. Emphasizes user-scoped isolation and that CRM stores no PII. Also returns suggested demo commands.
Return mock CRM sales totals for a given quarter/year for the current authenticated user. Example: {year: 2025, quarter: 2}.
Initialize a mock CRM for the current user (no PII stored). Seeds dummy deals if this user is new and returns a welcome message plus database counts and suggested commands.
Reset and reseed the current user's mock CRM data. Useful for demos and clean replays.
Output schemas not documented. Tool descriptions explain what is returned in prose (e.g. 'returns a welcome message plus database counts') but no structured output schema is visible. This forces LLMs to infer output structure, risking failed downstream tool chains.
Parameter descriptions lack format/constraint guidance. The 'year' parameter in crmGetSalesSummary says 'e.g. 2025' but does not specify valid range (e.g. 2000-2100) or format. The 'quarter' param says '1-4' but is not marked as an enum. This invites LLM hallucination of invalid values.
Missing error handling guidance. Tool descriptions do not explain what errors may occur, how to recover, or whether operations are retryable. E.g., crmResetMyData (DESTRUCTIVE risk) has no confirmation step or dry-run mode documented. crmAttemptCrossUserRead lacks actionable failure guidance.
Inferred effective spec: 2025-06-18+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 59 | 2025-06-18+ | v2 |
| 2026-03-09 | D | 50 | 2024-11-05+ | v1 |
Echo back the input (starter demo tool).
List the current user's notes (scoped by OAuth sub).
Create a few demo notes for the current user (stored by OAuth sub).
Return an OAuth-verified identity proof panel (user + issuer/audience/scopes/expiry). Use to validate user-scoped auth end-to-end.
CRM tool naming creates ambiguity. 'crmGetSalesSummary', 'crmExplainAccess', and 'crmInit' all operate on CRM data but the verb/action distinction is unclear. 'Explain' and 'Init' are not standard API action verbs. Consider rename: 'get_crm_sales_summary', 'describe_crm_permissions', 'initialize_crm'.
No pagination declared for 'listMyNotes', despite the name suggesting it returns a list. If user has 1000 notes, is the entire list returned or paginated? Description says 'List the current user's notes' but omits limit/offset parameters or total count in output.
Example value in parameter description. 'targetUser' in crmAttemptCrossUserRead says 'e.g. "user-abc123"', this invites LLMs to reuse the example literal. Replace with constraint description: 'A user label string matching the pattern user-[a-z0-9]+.'