Loading ToolBench
MCP server: snapshot tests for tool-call traces. Capture, normalize, and diff agent tool-use traces to catch silent regressions. Wraps @mukundakatta/agentsnap for Claude Desktop, Cursor, Cline, Windsurf, and Zed.
Three tools with clear, domain-specific names (normalize_trace, diff_traces, validate_snapshot) and solid descriptions (150 - 200 chars each). All tools have input schemas with required fields and property descriptions. However, schemas lack explicit type constraints on complex parameters (trace, baseline, current accept 'unknown' or bare 'object'), and output schemas are entirely undocumented. Parameters lack granular type info (e.g., 'trace' has no type constraint). Error handling is minimal, no recovery guidance or actionable error messages visible in the code. No tool annotations (readOnlyHint, etc.). Composition is sound: each tool has one responsibility, and tools chain logically (normalize → diff → validate). Security is adequate for read-only operations.
Diff two normalized agentsnap traces and surface meaningful differences. Returns same=true on a structural match, otherwise lists additions (paths only present in current), removals (paths only present in baseline), and changes (paths whose values differ). Use ignore_paths to drop noisy fields (e.g. ["fingerprint","tools[0].result_hash"]) before classification.
Coerce an arbitrary tool-call trace (an array of {name,args} entries or a partial Trace object) into the canonical agentsnap Trace shape. Returns the normalized trace plus a SHA-256 fingerprint hash computed over the canonical (key-sorted) JSON, so two runs can be byte-compared cheaply.
Sanity-check a snapshot object against the agentsnap Trace schema. Verifies required fields (version, tools, fingerprint), tool-entry shape (name + args), and surfaces actionable issues. Returns valid=true on success, otherwise a list of human-readable issues.
Output schemas completely undocumented. Tools return JSON text via content[].text, but LLMs cannot parse the structure without explicit schema documentation. normalize_trace returns {normalized, hash}, diff_traces returns {same, additions, removals, changes}, validate_snapshot returns {valid, issues}, none of these are formally declared.
Input parameter types are vague. 'trace' parameter accepts 'unknown' (no type constraint); 'baseline' and 'current' are bare 'object' with no shape documentation. LLMs cannot validate inputs or understand what structure is expected. Should specify: trace is array of {name, args, result?, error?} OR Trace object; baseline/current are Trace objects.
No error handling guidance. Code catches errors and returns errorResult('internal error: ...'), but LLMs receive no recovery hints. Should categorize errors (retryable vs user-fixable) and suggest next steps, e.g., 'Invalid trace format. Expected array of {name, args} or Trace object with tools[] field.'
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 66 | 2026-07-28+ | v2 |
No tool annotations. Tools are read-only (normalize, diff, validate) but lack readOnlyHint annotation. This prevents clients from optimizing caching or safety policies. Should add 'readOnlyHint': true to all three tools.
Parameter 'ignore_paths' in diff_traces lacks validation rules. Description says 'path prefixes' but does not specify format (dotted notation? array indices?). Should clarify: 'Array of dotted-path prefixes (e.g., "fingerprint", "tools[0].result_hash"). Matching is exact prefix on the path string.'