Loading ToolBench
Runtime de engenharia baseado em evidências e interface VS Code fina para projetos Protheus ADVPL/TLPP.
24 tools with complete schemas and descriptions, but significant quality gaps. All tools have input schemas with proper types and required fields. Descriptions are present but often generic (avg ~80 chars, below 194-char baseline). Parameter descriptions are sparse or missing context. No output schemas documented. Error handling guidance absent. Tool naming follows verb_noun pattern (pea_* prefix) but lacks clarity on distinctions between similar tools (e.g., pea_read_context vs pea_session_context). No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite clear risk classifications. Security-sensitive tools (pea_write_memory, pea_build_run) lack confirmation/dry-run patterns.
Run a structured task through a host-configured governed AI provider.
Append one attributed, Git-friendly Project Memory record when context:write is granted.
Create one traceable bug report from source review, CodeGraph impact and residual risk.
Cancel one supervised build cancel.
Read one supervised build evidence.
Prepare one host-configured supervised build plan without executing it.
Run one prepared build request using a host-issued, one-time approval token.
No output schemas documented. Tools return data but LLMs cannot plan downstream calls or extract fields reliably. Violates pattern:tool and pattern:response-shaper.
Ambiguous tool naming: pea_read_context vs pea_session_context lack clear distinction. LLMs cannot determine which to call without trial-and-error. Violates pattern:tool naming clarity.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite explicit risk classifications (READ_ONLY, WRITE, IRREVERSIBLE). LLMs cannot infer safety properties. Violates current MCP spec (2026-07-28).
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2025-06-18+ | v2 |
Read one supervised build status.
Read one field from a configured, versioned Protheus dictionary snapshot.
Read one table from a configured, versioned Protheus dictionary snapshot.
Report local runtime, policy, integration, and Hermes adapter status.
Remove only structured Project Memory entries whose explicit expiry is elapsed.
Build a read-only ADVPL/TLPP symbol and call graph for the workspace.
Execute one configured read-only Oracle named query with bind variables.
Preview the exact attributed Memory record and hashes produced by promoting a Journal event.
Promote one Journal event to attributed Project Memory when context:write is granted.
Read bounded Project Memory and Journal data.
Append one attributed Journal event when context:write is granted.
Review one explicit staged, unstaged, working-tree or branch Git change set.
Run deterministic pre-review rules on one workspace source file.
Snapshot live project skills, rules, memory, journal, and isolated Hermes ACP/MCP descriptors.
Run one host-configured bounded child task with parent, depth and tool evidence.
Search a configured read-only, versioned TDN snapshot and return provenance.
Replace Project Memory when the active environment grants context:write.
Irreversible operations (pea_build_run, pea_bug_review) lack dry-run or confirmation step. Agents cannot preview consequences before executing. Violates pattern:confirmation-request.
Parameter descriptions sparse or missing context. E.g., pea_review_changes 'scope' enum lacks explanation of when to use each value (staged vs unstaged vs branch). Violates pattern:tool-description.