Loading ToolBench
A FastAPI-based chat application with LangGraph state management, PostgreSQL persistence, and optional MCP (Model Context Protocol) tool integration. Supports streaming chat responses with reasoning capture.
RoseChat Backend presents 5 tools with significant definition gaps. While tool names follow the verb_noun convention (chat_stream, get_history, list_threads_endpoint, get_current_model, health_check), critical quality issues severely limit usability: (1) Parameter descriptions are present but descriptions themselves are extremely brief (10-30 chars, well below the 194-char baseline for A+ tools). (2) Input schemas lack proper JSON Schema validation, the chat_stream tool declares 'MessageRequest' as a type but does not expose the actual schema structure to the MCP client; parameters are nested in undocumented objects. (3) Output schemas are completely absent, no documentation of what chat_stream, get_history, or list_threads_endpoint return, forcing LLMs to guess at response structure. (4) No error handling guidance, tools have no documented recovery paths if a thread_id is invalid or the model is unavailable. (5) No tool annotations (readOnlyHint, destructiveHint, idempotentHint) despite all tools being read-only. These are production-grade tools in their domain (chat, thread history, health), but the MCP definitions do not expose enough structure for confident agent use.
Handle chat requests and stream responses.
Get the name of the currently configured chat model.
Retrieve chat history for a given thread ID.
Perform a health check of the service.
List all available chat thread IDs.
No output schemas documented for any tool. LLMs cannot plan downstream tool calls or extract response fields. chat_stream, get_history, and list_threads_endpoint all return unspecified structures.
Input schema for chat_stream nests parameters inside 'request_body' of type 'MessageRequest' but the actual MessageRequest schema is not exposed to the MCP client, only a description. This forces the agent to infer the structure.
Tool descriptions are extremely brief (10 - 30 characters): 'Handle chat requests and stream responses' (45 chars), 'Retrieve chat history for a given thread ID' (45 chars), 'List all available chat thread IDs' (35 chars). All fall well below the 194-char baseline and lack context about when/why to call each tool.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 35 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 0 | - | v1 |
No error handling guidance. If chat_stream receives an invalid thread_id or the model is unavailable, tools provide no recovery path. LLM has no way to know whether to retry, ask the user, or fail.
No tool annotations despite all tools being read-only. MCP spec supports readOnlyHint to signal to clients that these tools cannot modify state, enabling optimizations and safety checks.
get_history and chat_stream both work with thread IDs but no parameter description mentions whether thread IDs are UUIDs, numeric, or custom strings. Format constraints are absent.
list_threads_endpoint has zero input parameters but no description clarifies what pagination is supported, if any. Does it return all threads or a limited set? No limit, offset, or cursor parameters visible.