Loading ToolBench
Multi-Tenant, OAuth-Enabled MCP Server Framework with tenant isolation, API key management, and external OAuth provider support
MCP Plexus defines 11 tools with mixed quality. 10/11 tools have descriptions (good baseline), but descriptions are vague and lack context on WHEN to use each tool or what the agent should do next. Parameter descriptions are mostly absent, tools like get_admin_info, get_entity_info, get_global_info have ZERO input parameters, making them non-composable and hard to chain. Naming is inconsistent: verb-noun is mostly present (use_alpha_service_key_tool, fetch_secure_external_data, manage_plexus_session_tool) but some names are vague (get_entity_info, get_admin_info). No input schemas are visible in the provided source for most tools, only manage_plexus_session_tool has a documented schema. Output schemas are completely undocumented across all tools. Error handling guidance is absent. Security model (API key injection, secret management) is declared but not shown in practice. Tenant/scope isolation is a notable feature but adds complexity without clear benefit documentation.
Administrative task tool restricted to tenant A and categorized under admin tasks.
Fetches external data using authenticated GitHub client.
Reporting tool categorized under the 'reporting' tool set.
Retrieves administrative information, restricted to specific tenants and in admin tool set.
Retrieves entity and session information from the Plexus context.
Global tool available to all tenants without restrictions.
Shared resource tool accessible by both tenant A and tenant C.
9 out of 11 tools have NO visible input schemas in the provided source code. Only manage_plexus_session_tool has a documented schema with action, key, and value parameters. This violates the critical schema requirement that every tool parameter needs a type definition and description.
Tool descriptions are generic and lack context. Examples: 'Retrieves administrative information, restricted to specific tenants and in admin tool set' (get_admin_info) and 'Retrieves entity and session information from the Plexus context' (get_entity_info) do not explain WHEN an agent should call them, what distinguishes them from other get_* tools, or what the response structure is. Baseline for A+ tools is 50-200 chars with WHAT, WHEN, and WHAT-IT-RETURNS context. These descriptions miss the WHEN and WHAT-IT-RETURNS components entirely.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 47 | - | v1 |
Tool restricted to tenant A for accessing tenant-specific data.
Exclusive feature tool available only to tenant B.
Manages session data with get, set, and delete operations.
A tool that requires an API key for 'my_test_service_alpha' and returns it.
No output schemas documented for any tool. When a tool returns results, the LLM needs to know the structure (fields, types, whether paginated, etc.) to chain calls downstream. Example: manage_plexus_session_tool with action='get', what fields does it return? Is there a 'value' field? A 'found' boolean? Undocumented outputs force LLMs to guess and risk parsing failures.
Tenant/scope isolation logic is complex and not clearly documented in tool descriptions. Tools reference 'restricted to specific tenants', 'exclusive to tenant B', 'shared by tenant A and C', but there's no guidance in the descriptions on how to know WHICH tenant is calling or how to determine if a tool is available for your use case. This adds cognitive load on agents and risks misconfiguration.
No error handling guidance visible. If a tool fails (permission denied, data not found, service unavailable), the descriptions do not advise the LLM on recovery steps. E.g., if fetch_secure_external_data fails with a 404, should the agent retry, fall back to an alternative tool, or ask the user? Absence of guidance forces agents to stop or retry blindly.