Loading ToolBench
Drive the Prism multimodal playground API from an agent. Works with play.skyphusion.org and self-hosted prism.
Prism MCP demonstrates solid fundamentals: all 9 tools have descriptions (avg 156 chars), input schemas with typed properties, and tool annotations (readOnlyHint/destructiveHint). Naming follows verb_noun convention (health, list_models, update_prefs, chat, prism_request_*). However, parameter descriptions are sparse or missing in several tools (e.g., chat's 'attachments' array lacks element schema; prism_request_* params lack format/constraint guidance). Output schemas are undocumented, LLMs cannot predict response structure. Error handling is minimal; no recovery guidance or actionable error messages visible. Escape hatch tools (prism_request_read/write) are well-designed for extensibility but lack validation examples.
Run a prism model turn (POST /api/chat). Dispatches by model.type: chat, image, tts, video, stt, music. Required: model, user_input. Optional: system_prompt, conversation_id, project_id, use_docs, use_web_search, attachments, image_url, image_key. Persists history. Long video/music jobs return a job id; poll with poll_job. Live voice models use WebSocket /api/stt/stream (not available as an MCP tool; use prism_request is not enough).
Streaming chat (POST /api/chat/stream). Only for catalog entries with streaming: true. The MCP drains the SSE body into one text result (agents cannot hold open SSE). Prefer chat when stream is not required.
Get the session user's AI Gateway / control-plane prefs (GET /api/prefs). Tokens are masked.
Prism liveness probe (GET /health). No auth required on prism.
Prism deep health (GET /health/deep): D1, R2, Vectorize, gateway config. May return 503.
Output schemas undocumented. LLMs cannot predict response structure (fields, types, pagination). chat returns model output but structure is opaque; list_models returns catalog but field names unknown.
Complex parameter 'attachments' in chat tool lacks element schema. Description says 'Same shape as the SPA: mime, data (base64/data-URL), key, text, filename, etc.' but does not specify required fields, types, or constraints. LLMs cannot construct valid payloads.
Escape hatch tools (prism_request_read/write) accept free-form 'path' and 'query' without validation guidance. No examples of valid paths, no mention of path traversal protection or allowed route prefixes beyond '/'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | B | 70 | 2026-07-28+ | v2 |
List prism catalog models (GET /api/models). Returns models, auth mode, gateway/control-plane status. Boot probe is public on prism; still sent with session when configured.
Escape hatch: GET or HEAD any prism route (GET /api/* or HEAD /api/*). Stateless; no side effects. Useful for routes not yet curated as dedicated tools.
Escape hatch: POST, PATCH, PUT, or DELETE any prism route. Stateless; agent-driven polling for long jobs. Useful for routes not yet curated as dedicated tools.
Patch gateway or control-plane prefs (PATCH /api/prefs). Fields: gateway_id, cf_aig_token, clear_cf_aig_token, control_plane_key (pcp_…), clear_control_plane_key.
Error handling not visible in tool definitions. No recovery guidance (e.g., 'If model not found, call list_models first'). No error classification (retryable vs fatal). Agents cannot self-correct on failures.
Parameter descriptions in update_prefs lack format/constraint guidance. 'control_plane_key' description says 'pcp_<16 hex>_<43 base64url>' but does not specify this as a regex pattern or validation rule. LLMs may pass invalid formats.