Loading ToolBench
A real, working reference implementation of controlled AI-agent access to enterprise-shaped tools: read tools, one deny-by-default gated write path split into propose/execute steps, a real independent human-approval boundary (approve_pending.py, run out-of-process), dry-run mode, and a DB-enforced append-only audit log. Deterministic, zero-LLM-call business logic -- every field is either real synthetic-data evidence or explicitly 'unknown,' never fabricated.
Enterprise Agent Lab demonstrates solid fundamentals: all 5 tools have clear verb-noun naming (list_, classify_, propose_, execute_, get_), comprehensive descriptions (100-300 chars), and explicit input schemas with typed parameters. The write-path tools (propose_decision, execute_decision) show sophisticated security thinking with dry-run modes, approval gates, and audit logging. However, output schemas are not formally documented in the code, parameter descriptions lack constraint details (e.g., confidence enum values, decision string format), and error handling is incomplete (no recovery guidance for common failures). The tool composition is excellent, propose/execute separation enforces least-privilege, but the definitions could be more LLM-optimized.
Real, deterministic classification of one demo record (see `_classify_record`'s own docstring for the real, disclosed rule). Read-only -- no execution_guard authorization needed.
Step 2 of the ONE real, least-privilege write path. Reads a persisted approval_requests row by approval_id (never accepts approval status as a parameter), checks its real status from the database, and only if it is 'approved' (by a separate human via approve_pending.py) does it execute the decision by recording it in the decision_journal_entries table. Atomically claims the approval so concurrent calls cannot both execute the same approval_id. Returns the execution result, including whether the write succeeded and the new decision_journal_entry_id if it did.
Returns the most recent decision_journal_entries, ordered by recorded_at descending, up to the specified limit. Read-only audit trail query.
Lists real demo business records from the isolated local demo database (never any real production system). Optionally filtered to an exact `status` value.
Step 1 of the ONE real, least-privilege write path. `dry_run=True` (the default) previews the exact row that would be written, with zero side effects -- no approval request is created. `dry_run=False` creates a real, persisted PENDING `approval_requests` row and stops there: no write happens in this call, and this tool has no parameter that can mark that row approved. A real write only ever happens via a later, separate `execute_decision(approval_id)` call, after a separate human process (`approve_pending.py`) has approved it.
Output schemas not formally documented. Tools return dicts but LLMs cannot see field types, required fields, or structure. list_records returns list[dict] with unspecified fields; classify_record returns dict with 'category', 'score', 'basis' but no schema declaration.
Parameter descriptions lack constraint details. 'confidence' parameter has no enum values listed (should specify: 'low', 'medium', 'high', 'unknown'). 'decision' and 'reasoning' strings have no length limits or format guidance. LLMs cannot validate inputs without explicit constraints.
Error handling lacks recovery guidance. classify_record returns {'error': 'No demo record with id=...'} but does not suggest next steps (e.g., 'Call list_records() to see available IDs'). propose_decision/execute_decision do not document what errors are retryable vs. fatal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
STDIO transport only. Server is not remotely accessible and cannot be used by hosted MCP clients. Limits deployment to local/containerized environments.
Tool annotations missing. propose_decision and execute_decision are destructive/irreversible but lack destructiveHint/idempotentHint annotations. LLMs cannot infer retry safety without explicit hints.