Loading ToolBench
A collection of Python utility scripts for validating, scanning, and managing lab artifacts in the C1756 course. Includes prohibited content scanning for non-WSQ courseware, lab file management, evidence recording, and architecture pack validation.
Five tools with clear, domain-specific names and detailed descriptions (avg 180 chars). All tools have input schemas with typed parameters and enums where appropriate (lab 1-9). However, output schemas are entirely undocumented, no response structure defined for any tool. Error handling is absent: no guidance on what failures mean or how to recover. Parameters lack validation rules (e.g., 'decision' requires 40+ chars but this is stated only in description, not enforced). No tool annotations (readOnlyHint/destructiveHint). Composition is sound (each tool has one responsibility), but the lack of output documentation and error recovery patterns significantly limits LLM usability.
Copy one lab's starter files without destroying existing learner work. Safely copies lab starter files with optional backup of existing files.
Validate, hash and checkpoint a completed C1756 lab. Records architecture decision evidence and creates a rejoin checkpoint with SHA256 artifact hashes.
Show learner edits even though labs/work is intentionally gitignored. Compares completed lab files with their incomplete starters using unified diff.
Scan non-WSQ courseware for prohibited WSQ/funding/assessment content. Reads generated PPTX/DOCX/MD artifacts and reports every hit with its exact location (slide number / paragraph index / line number).
Validate the connected C1756 Contoso architecture-pack lab outputs. Comprehensive validation of JSON, CSV, and Markdown artifacts across all 9 labs with schema, formula, and content verification.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields (e.g., what does scan_prohibited return? A list of hits? A count? Field names?). This forces agents to guess structure and wastes tokens on parsing.
No error handling guidance. Tools can fail (file not found, invalid lab number, permission denied) but provide no recovery hints. LLMs receive raw exceptions with no actionable next steps.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe reads from destructive writes. copy_lab_files and record_lab_evidence modify state but lack destructiveHint; scan_prohibited, review_lab_changes, and validate_architecture_pack are read-only but lack readOnlyHint.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Parameter 'decision' in record_lab_evidence requires 40+ chars and no TODO markers, but this constraint is only documented in description text, not enforced in schema. LLMs may pass invalid values; tool must validate and return actionable error.
copy_lab_files has a 'backup_existing' boolean with no default stated. Unclear whether omitting it skips backup (safe) or defaults to true (destructive). Defaults must not cause data loss.