Loading ToolBench
PoC MCP server for SAP BTP/Cloud Foundry management — arc-1 auth model, read-only by default.
Server demonstrates solid tool design with explicit schemas, clear descriptions, and thoughtful safety annotations. All 5 tools have complete input schemas with typed properties and descriptions. Tool names follow verb_noun convention (CFInspect, CFApps, BTPInspect, BTPServices, whoami). Descriptions are substantive (100-250 chars) and explain WHAT, WHEN, and prerequisites. However, parameter descriptions are generic/reused across tools (PARAM_DESC dict), and output schemas are not documented. Error handling is present but recovery guidance is minimal. Security model is well-designed (scope-based access control, no secrets in params), but composition could be tighter, some tools bundle multiple actions (e.g., CFInspect has 8 actions) rather than splitting into single-responsibility tools.
READ an SAP BTP account via the btp CLI account APIs, as the logged-in user (or a shared read-only technical user); works on a free/local subaccount. Read-only. Reads need the matching read-only role (Global Account Viewer, or Subaccount Viewer for subaccount-scoped reads) — HTTP 403 otherwise. Actions: global_account; subaccount; subaccount_entitlements; entitlements.
SAP BTP service-instance WRITES (Service Manager) — these EXECUTE as the acting identity. The target subaccount must be allowlisted; provisioning/deletion may complete asynchronously (verify with BTPInspect). Actions: create_service; delete_service.
Cloud Foundry app-lifecycle WRITES — these EXECUTE against the Cloud Controller. Address the app by 'guid'; the server resolves the app's real space and refuses if it is not allowlisted. Actions: app_restart; app_stop; app_start.
READ Cloud Foundry via the Cloud Controller v3 API, as the logged-in user (or a shared CF token). Read-only. Reports the backend unavailable if CF is not configured. Actions: org_spaces; org_quota; org_usage_summary; app_summary; app_logs; app_env; service_instance_parameters.
Diagnostic: show the caller's resolved scopes, plus raw token claims (origin/scope) when the token is a readable JWT — sealed-JWE and API-key tokens are opaque. Always available, even with no scopes.
Multi-action tools violate single-responsibility principle. CFInspect bundles 8 read actions (org_spaces, org_quota, org_usage_summary, app_summary, app_logs, app_env, service_instance_parameters) into one tool. BTPServices bundles create_service and delete_service. LLMs reason better with focused tools, split into get_cf_org_spaces, get_cf_app_summary, etc.
Output schemas are not documented. Tool descriptions state WHAT is returned (e.g., 'Reports the backend unavailable if CF is not configured') but do not specify the response structure, field names, or types. LLMs cannot plan downstream tool calls or extract data reliably without knowing response shape.
Parameter descriptions are generic and reused. The 'guid' parameter uses identical description across CFInspect, CFApps, and BTPServices, but the GUID type differs (APP vs ORG vs SERVICE-INSTANCE). Descriptions should clarify which resource type is expected for each tool.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 67 | 2026-07-28+ | v2 |
Error handling lacks recovery guidance. Tool descriptions mention 'Reports the backend unavailable if CF is not configured' but do not tell the LLM what to do next (e.g., 'Check CF_API_URL environment variable' or 'Call whoami to verify credentials'). Errors should guide the agent toward resolution.
Destructive operations lack confirmation step. CFApps (app_restart, app_stop) and BTPServices (delete_service) modify state but do not offer dry-run or confirmation. Agents can accidentally stop production apps or delete services. Implement a confirm_before_execute pattern.