Loading ToolBench
Server has 22 tools with complete schemas and descriptions. Naming follows verb_noun convention well (assign_, create_, delete_, execute_, get_, list_, revoke_). Descriptions are present and adequate (avg ~100 chars), meeting the 10-1024 char baseline. All parameters have type definitions and descriptions. However, several critical gaps reduce quality: (1) No output schemas documented, LLMs cannot predict response structure for chaining; (2) No error handling guidance, tools return raw errors without recovery hints; (3) No pagination support on list tools despite potentially large result sets; (4) Destructive tools (delete_role, kill_process) lack confirmation/dry-run patterns; (5) Permission/security context not declared per-tool. Parameter constraints are well-defined (enums for actions/resource_types), but descriptions could be more prescriptive about format and constraints.
Assign a permission to a specific role.
Assign a permission to a specific user.
Assign a role to a specific user.
Create a new role in the Stardog server.
Delete a role from the Stardog server. Optionally force delete the role, deleting the role while it is assigned to other users.
Execute a SPARQL read query against a Stardog database. SELECT, ASK, CONSTRUCT, and DESCRIBE queries are supported.
No output schemas documented. LLMs cannot predict response structure, making tool chaining and result extraction error-prone. Each tool should declare what fields it returns (e.g., list_users returns array of {username, enabled, superuser, permissions}).
Destructive tools (delete_role, kill_process) lack confirmation or dry-run patterns. Agents can irreversibly delete roles or kill processes without safeguards. Implement a confirmation_required pattern or add a dry_run parameter.
No error handling guidance. Tools return raw errors without recovery hints. When a user is not found, the error should suggest 'Try list_users() to find available users' rather than a bare 404.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | <=2025-11-25 | v2 |
Get the configuration of a Stardog database. Optionally filter by specific keys.
Get documentation for Stardog database configuration options.
Get the estimated size of a Stardog database in triples.
Get the names of roles assigned to a specific user.
Get the server metrics from Stardog.
Get all the usernames of users assigned to a specific role.
Return the authenticated user's username.
Kill a specific process on the Stardog server.
List the names of all Stardog databases in the Stardog server.
List all the processes on the Stardog server.
List all of the roles names in the Stardog server. Optionally include role permissions. Optionally filter by specific role names.
List all stored queries in the Stardog server.
List all users in the Stardog server. Optionally include details about users like permissions, superuser status and whether they are enabled. Optionally filter by specific usernames.
Revoke a permission from a specific role.
Revoke a permission from a specific user.
Revoke a role from a specific user.
List tools lack pagination. list_users, list_roles, list_processes, and list_databases do not accept limit/offset or return total counts. Large result sets will blow context windows.
No per-tool permission declarations. Tools do not state what permissions they require (e.g., 'requires: admin, write:security'). This prevents least-privilege agent configuration and audit clarity.