Loading ToolBench
An MCP server for the Jazz framework that enables AI assistants to interact with local-first apps built with Jazz
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output clarity. getAppData has 4 parameters with descriptions and Zod schemas (good), but descriptions are generic and lack actionable guidance. listApps has minimal parameters. Both tools lack error recovery guidance and output schemas are under-documented. No tool annotations (readOnlyHint, etc.). Naming is verb-forward (get_, list_) which is correct. However, parameter descriptions do not explain constraints, formats, or dependencies. Output schemas exist in code but are not surfaced to the LLM in a way that guides downstream tool chaining.
Get data from a Jazz app
List Jazz apps in a directory
Parameter descriptions lack actionable constraints. 'Path to the Jazz app' does not specify format, validation rules, or what happens if the path is invalid. 'Query to filter CoValues by' does not explain query syntax or matching behavior.
Output schemas are defined in code (Zod) but not exposed to the LLM. The tool descriptions do not explain what fields the LLM should expect or how to use returned data for downstream calls. This breaks tool chaining.
Error handling is generic. Errors like 'Schema not found' or 'Failed to get Jazz app data' do not guide the LLM on recovery steps. No suggestion of alternative schemas, paths, or retry strategies.
getAppData combines multiple concerns: lookup by ID, query filtering, and bulk retrieval. These should be separate tools so the agent can compose them independently.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 38 | 2025-06-18+ | v2 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). Both tools are read-only but this is not declared in the schema, forcing the LLM to infer safety from descriptions alone.