Loading ToolBench
ARC-1 edition that delegates all ABAP/ADT work to SAP's embedded adt-ls language server.
arc-1-lsp demonstrates solid tool design with consistent naming (verb_noun pattern), comprehensive descriptions (avg 180 chars), and well-structured schemas using Zod. All 12 tools are explicitly registered with descriptions and input schemas. However, output schemas are not documented, responses are text-wrapped without formal structure declarations. Error handling is present but generic ('No ABAP destination is connected'). Missing tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Parameter descriptions are good but some lack format/constraint details (e.g., 'pattern' in search_objects could specify wildcard syntax).
Create an ABAP object (mutating — requires ARC1_ALLOW_WRITES; package must be in the allowlist).
Get the input schema for a specific object generator (use `generatorId` from list_generators). Read-only. Object-referencing generators (e.g. OData UI / Web-API service) also need `referencedObjectType` + `referencedObjectName`.
Describe the fields required to create an object of a given ADT type (e.g. "CLAS/OC") — read-only creation metadata.
Inspect a published OData service binding (SRVB): binding type, OData version, published state, and service URIs. Find binding names via `search_objects` with types ["SRVB/SVB"].
Report arc-1-lsp and embedded adt-ls health (adt-ls version, MCP port, up state). No SAP connection required.
Output schemas not documented. Tools return text-wrapped responses without formal structure declarations. LLMs cannot plan downstream calls or extract typed fields reliably.
Tool annotations missing. Despite clear risk classifications (READ_ONLY vs WRITE), tools lack readOnlyHint/destructiveHint/idempotentHint metadata. Agents cannot distinguish safe from destructive operations.
Inferred effective spec: 2026-07-28+.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 60 | 2026-07-28+ | v2 |
List the ABAP object types creatable on a connected system (federated from adt-ls). Uses the startup-connected destination when `destination` is omitted. Exercises a real backend call.
List the available ABAP system destination names (ids only), federated from the embedded adt-ls.
List the object generators available on the connected system (e.g. RAP "OData UI Service") — what `generate_objects` can produce.
List inactive (draft) objects on the connected system — those edited but not yet activated.
List ABAP user master records readable on the connected system (id + display name). An empty result means none are visible to your SAP user (it does not necessarily mean the system has no users).
Read the source of an ABAP object by name + type (modern types: class/interface/CDS/…). Classic types (program/table/…) are not served by adt-ls headless — use main ARC-1 for those.
Search ABAP repository objects on the connected system by name pattern (e.g. "CL_ABAP*"). Returns name, type, description, and ADT uri for each hit. The first search right after startup may hit a cold index — this auto-retries once, so an empty result means no match (not a warm-up miss).
Parameter constraints underspecified. 'pattern' in search_objects lacks wildcard syntax docs. 'maxResults' has max=200 but no min stated. 'include' enum is present but not all parameters use enums where appropriate.
Error messages generic and non-actionable. 'No ABAP destination is connected' repeated across 5 tools but lacks recovery guidance (e.g., 'Configure ARC1_SAP_HOST/PORT/USER/PASSWORD or call list_destinations first').
create_object lacks confirmation/dry-run pattern. Mutating tool with WRITE risk has no pre-execution validation or confirmation step. Agents could accidentally create objects in wrong packages.