Loading ToolBench
MCP server that records prompts/context/responses from Claude and Codex for small-model distillation.
mcp-distill has 9 tools with explicit schemas and descriptions visible in src/server.rs. Naming is verb-forward and clear (start_session, append_turn, export_dataset). Descriptions are present and moderately detailed (avg ~120 chars), meeting the 10-1024 char baseline. However, parameter descriptions are inconsistent: some params lack detail (e.g., 'message' in append_turn is just 'Provider-native message' without format guidance), and several optional params lack guidance on defaults or dependencies. Output schemas are not formally documented, callers must infer structure from code. Error handling is minimal; no recovery guidance or actionable error messages visible. No tool annotations (readOnlyHint/destructiveHint) despite clear WRITE vs READ_ONLY semantics. Composition is reasonable (each tool has one responsibility), but some tools could be better integrated (e.g., start_session + append_turn + end_session could offer a convenience wrapper).
Append a turn (request or response message) to a session. Adapts provider-native message shapes (Anthropic Messages or OpenAI Chat Completions) into canonical form.
Append token usage information to a session.
End a session (writes an `end` record). The session remains queryable but no more turns can be appended.
Export recorded sessions to a distillation dataset in one of three formats: openai_chat, sharegpt, or anthropic. Writes a JSONL file.
Record a complete interaction (request + response) in a single call. Creates a new session, records the system prompt, request messages, and response, then ends the session.
Status of the implicit recording session: {recording: bool, session_id?, root}.
Parameter 'message' in append_turn lacks format specification. Description says 'Provider-native message (Anthropic Messages or OpenAI Chat Completions shape)' but does not explain which fields are required, what structure is expected, or how to adapt between formats. LLMs cannot infer the correct shape without explicit schema or examples.
No output schemas documented for any tool. Callers must reverse-engineer return structure from code. For example, start_session returns 'session_id and trace file path' but the actual JSON structure is not specified. This forces LLMs to guess field names and types.
No tool annotations (readOnlyHint/destructiveHint/idempotentHint) despite clear semantic differences. start_session, append_turn, append_usage, end_session, record_interaction, and stop_recording are destructive WRITE operations; recording_status and export_dataset are READ_ONLY. Annotations would help agents reason about safety and retry logic.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 52 | <=2025-11-25 | v2 |
Resume recording: opens a new session and makes it the current one.
Start a new recording session and make it the currently-recording one. Returns the session_id and trace file path.
Stop the currently-recording session (writes an `end` record). Subsequent append_turn calls without an explicit session_id will fail until start_recording or start_session is called.
Error handling is absent from visible code. No guidance on what errors can occur, whether they are retryable, or what the LLM should do next. For example, if append_turn fails because the session_id is invalid, there is no recovery hint.
Provider enum in start_session and start_recording is documented as '"claude" | "codex" | "openai" | "anthropic" | "other"' in description text, but not enforced as a JSON Schema enum constraint. LLMs may pass invalid values like 'gpt-4' or 'claude-3' without validation.