Loading ToolBench
LTP (Lean Task Protocol) — compile LLM goals into deterministic execution plans
Four tools with complete input schemas and basic descriptions. All tools are READ_ONLY and deterministic. Naming follows verb_noun pattern (parse_plan, validate_plan, visualize_plan, diff_plans). However, descriptions are minimal (10-80 chars), lack context on WHEN to use each tool, and parameter descriptions are sparse. No output schemas documented. Error handling returns JSON errors but lacks recovery guidance. Missing parameter constraints (e.g., minimum text length). No tool annotations (readOnlyHint present in metadata but not in schema). Schemas are properly typed but descriptions fall short of production baseline (194 chars avg).
Structural diff between two LTP plans (added/removed/changed/reordered steps).
Parse LTP text into a structured plan (JSON).
Validate an LTP plan and return structural issues.
Export an LTP plan as a Mermaid flowchart.
Tool descriptions are too brief (10-80 chars vs 194 char baseline). Missing WHEN to use each tool and prerequisites. E.g., 'Parse LTP text into a structured plan (JSON)' lacks context on what LTP is, when to call parse vs validate, and what the output enables.
Parameter descriptions are minimal. 'ltp_text' param lacks format guidance (e.g., 'Raw LTP plan text between PLAN_START and PLAN_END markers; minimum 50 chars'). No constraints on text length, encoding, or structure.
No output schemas documented. Callers cannot predict response structure. E.g., parse_plan returns JSON but schema for that JSON (fields, types, nesting) is not declared. LLMs must infer structure from examples.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
Error handling returns generic JSON errors ('invalid or empty LTP plan') without recovery guidance. No distinction between retryable vs user-fixable errors. LLM has no actionable next step.
Tool annotations (readOnlyHint) present in metadata but not exposed in input schema. LLMs cannot see that these tools are safe to call without confirmation.