Loading ToolBench
28 tools with mostly complete schemas and descriptions, but significant gaps in parameter documentation and error guidance. Tool names follow verb_noun convention well (list_, create_, get_, write_, delete_, move_). Descriptions average ~120 chars, meeting baseline minimums but lacking LLM-optimized context (50-200 char sweet spot). Most tools have input schemas with types, but many parameters lack descriptions or constraints. Error handling is minimal, no recovery guidance, no actionable error messages, no dry-run patterns for destructive operations. Output schemas are not documented. Composition is reasonable (single responsibility per tool), but chaining IDs are not always returned.
Build a LAD (Ladder Logic) block from network definitions
Compile the PLC software and return compilation results
Create a new block from a template with parameters
Create a new global data block (DB)
Create a new group in Program Blocks
Create a new instance data block for a function block
Create a new PLC tag in a tag table
No output schemas documented. LLMs cannot infer what fields to expect from tool responses, forcing them to guess at downstream chaining and risking context loss.
Destructive operations (delete_item, write_block, write_scl_block, save_project) lack dry-run or confirmation patterns. Agents can irreversibly delete or overwrite without safeguards.
Error handling is minimal. No recovery guidance, no actionable error messages, no categorization of retryable vs fatal errors. Agents cannot self-correct on failure.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Create a new user-defined tag table
Create a new group in PLC data types
Delete a named item (block, data type, tag, or tag table) from the project
Apply a library layout manifest to Program Blocks and PLC data types
Export a block as XML
Get the raw XML template for a block template
List all blocks in the PLC software top-level block group
List all user-defined PLC data types (UDT) in the project
List all groups in Program Blocks and PLC data types
List all tag tables in the project
List all PLC tags across all tag tables
List all available block templates
Search for TIA Portal instructions by name or keyword
Move a block to a target group in Program Blocks
Move a PLC data type to a target group in PLC data types
Validate an SCL file for syntax and semantic errors before import
Preview a block that would be created from a template without actually creating it
Save the currently open TIA Portal project to disk
Check TIA Portal process status and list open projects
Import a block from XML file
Import a block from SCL (StructuredText) file
Parameter descriptions are sparse or generic. 'Optional project title/name match' (tia_status) lacks format guidance. 'Absolute path' (write_block) doesn't specify Windows vs POSIX or validation rules.
List tools (list_blocks, list_tags, list_templates) lack pagination parameters (limit, offset, cursor). Large projects could return thousands of items, exhausting context windows.