Loading ToolBench
context0 defines 3 tools with visible schemas and descriptions in src/mcp.rs. Tool naming follows the verb_noun pattern (get_context, save_context, list_context), which is appropriate. However, descriptions are minimal (28-94 characters), falling short of the production baseline of 194 chars. Parameter descriptions are present but terse. Output schemas are not explicitly documented in the source, the rubric requires evidence of documented return types for A-grade tools. Error handling is basic; there is no recovery guidance or actionable error messages visible in the code. The server accepts natural identifiers (repo_path, branch) which is good for the chat data model, but lacks input validation rules, enums for constrained values, and comprehensive parameter documentation. Overall execution is functional but lacks the detail and polish expected for confident production recommendations.
Retrieve the latest checkpoint for a given repo and branch
List recent checkpoints for a given repo and branch
Save a checkpoint with structured summary for current repo, branch, and commit
Output schemas not documented. The source code defines input parameters clearly but does not explicitly document what fields are returned by each tool. LLMs cannot plan chained calls without knowing the response structure (e.g., does get_context return commit_sha, done, next, blockers, tests, files, timestamp?).
Descriptions are too brief (28 - 94 chars vs. production baseline of 194 chars). They state WHAT the tool does but lack WHEN to use it, WHAT is returned, and dependencies between tools. For example, 'Retrieve the latest checkpoint for a given repo and branch' does not explain whether this returns structured metadata, raw diffs, or a summary suitable for feeding to save_context.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | 2025-06-18+ | v2 |
| 2026-03-09 | F | 23 | - | v1 |
Parameter descriptions are minimal. 'repo_path' is labeled 'The repository path to retrieve context for' (42 chars) but does not explain format expectations (absolute vs. relative), whether it must exist, or how it resolves if not found. 'branch' similarly lacks format guidance (e.g., can it contain slashes? spaces?).
No input validation rules visible in source. Parameters like limit in list_context declare a minimum (1) in the description but lack validation logic or enum constraints in the schema itself. This forces the LLM to reason about bounds rather than the tool rejecting invalid input with actionable error text.
Error handling is not described. Callers do not know what errors are possible (e.g., repo not found, permission denied, database error, invalid branch name) or what to do next. The code includes error handling (Result<T, Error>) but the tool definitions do not document error scenarios or recovery paths.
Optional parameters (done_text, next_text, blockers_text, tests_text, files, session_id in save_context) are not clearly marked as optional in descriptions. LLMs may assume all parameters are required and fail to construct minimal valid calls.
No pagination guidance in list_context despite accepting a limit parameter. Documentation does not clarify: does limit enforce a maximum result count? Is there a cursor or offset? Does the response include total_count or has_more? This forces the LLM to guess the pagination model.