Loading ToolBench
A real-time chat and messaging platform with agent API, event streaming, push notifications, and progressive web app for human-agent communication
Finalechat implements 8 tools with basic structure but significant quality gaps. While tool names follow verb_noun conventions (send_message, list_threads, create_thread), parameter descriptions are minimal or absent, and output schemas are not documented. Most critically, many tools lack visible schema definitions in the provided code, and descriptions are generic without actionable guidance for LLM selection. The codebase shows CLI integration via Python (cli/finalechat) but actual tool registration logic is not visible in the provided source excerpts, forcing inference of tool definitions. This server would struggle in production agent scenarios requiring clarity on data flow and error recovery.
Ask a question to the user and wait for an answer
Create a new thread for messaging
Retrieve a specific thread
Get messages from a thread
List all threads for the authenticated user
Send a message from an agent to a thread
Set a live status message that appears in the interface
Output schemas not documented. Tools like list_messages and list_threads lack return type definitions, making it impossible for LLMs to understand what fields to extract or how to chain calls. For example, list_messages does not specify if it returns message objects with id, timestamp, and sender, critical for downstream operations.
Parameter descriptions are minimal or trivial. 'attachments' in send_message is described as 'Optional file attachments', this lacks detail on format (file paths? URLs? base64?), constraints (max count? file size limit?), and when to use. LLMs cannot infer these details and will guess wrong.
Tool registration is not visible in the provided source code. The tools are listed in cli/finalechat (Python integration) and internal/api/server.go (API handler), but no explicit MCP tool registration with schemas and capabilities is shown. This forces scoring assumptions and indicates the server may not expose tools via MCP at all, it appears to be a custom chat API, not an MCP server.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | <=2025-11-25 | v2 |
Upload a file attachment to a thread
No pagination or result limits documented. list_messages and list_threads provide no indication of how many results are returned, whether pagination is supported, or what happens if thousands of items exist. This violates the LLM context window and risks runaway token usage.
No error handling guidance. Tools have no documented error cases or recovery paths. For example, send_message does not specify what happens if thread_id is invalid, if the user lacks permission, or if the message is too large. LLMs receive no actionable error guidance.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). The server does not signal which tools are safe to retry, which are destructive, or which are read-only. This forces the LLM to infer from names alone, dangerous for irreversible operations like send_message or delete operations.
Ambiguous parameter types and constraints. 'options' in ask_question is described as 'Optional predefined answer options' with type array, but the element type, format, and structure are not specified. Is it an array of strings? Objects with id/label? LLMs cannot construct valid inputs.
No indication of authentication or permission requirements. Tools like send_message and create_thread do not indicate if they require specific user roles, authentication methods, or scope declarations. This violates least-privilege and audit trail expectations.