Loading ToolBench
Bidirectional sync between .pen design files and frontend code via Claude CLI
pencil-sync defines 8 tools with consistent naming (verb_noun pattern: pencil_*) and reasonable descriptions (avg ~120 chars). All tools have input schemas with type definitions and parameter descriptions. However, output schemas are undocumented, responses are wrapped in a generic ok() helper returning untyped JSON. Error handling is basic (generic err() wrapper). No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications. Parameter descriptions are adequate but lack format constraints and validation guidance. Tools follow single-responsibility principle well (diff design, diff code, detect conflict, build prompt, apply fills, invalidate state, record sync, get config). Missing: pagination for list-like responses, idempotency guidance, and recovery hints in error messages.
Apply deterministic color (fill) changes directly to CSS variable files without needing LLM assistance.
Build the full sync prompt for a host agent to apply design changes to code (pen-to-code) or code changes to design (code-to-pen).
Detect whether both the .pen file and code files have changed since the last sync for a mapping.
Diff the current code directory file hashes against the last synced state for a mapping. Returns the list of changed code files.
Diff the current .pen file snapshot against the last synced state for a mapping. Returns the list of changed design properties.
Load and return the pencil-sync configuration for the given config file path.
Output schemas are undocumented. All tools return generic ok()/err() wrappers with untyped JSON content. LLMs cannot predict response structure or plan downstream tool calls.
No tool annotations despite clear risk classifications (READ_ONLY, WRITE, REVERSIBLE). Missing readOnlyHint, destructiveHint, idempotentHint in tool definitions. Agents cannot distinguish safe from destructive operations.
Error messages are generic ('Error: <message>') with no recovery guidance. When a mapping is not found or config is invalid, the LLM receives no hint about next steps (e.g., 'Try pencil_get_config() first to list available mappings').
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | <=2025-11-25 | v2 |
Clear the recorded code-file hashes for a mapping so the next sync treats all files as changed. Use this after manually rebuilding a design to force a full structural re-sync, or when state.json holds false 'synced' hashes that predate the actual canvas work.
Record that a sync has completed by updating the state store with current file hashes. Call this after the host agent has applied all edits.
Parameter descriptions lack format constraints. 'configPath' says 'Absolute path to pencil-sync.config.json' but does not specify: must exist, must be readable, must be valid JSON. 'mappingId' lacks guidance on valid format or how to discover valid IDs.
pencil_build_prompt has optional array parameters (diffs, changedFiles) with no guidance on mutual exclusivity or required combinations. Description does not clarify: when is diffs required vs changedFiles? What happens if both are omitted?