Loading ToolBench
MCP server that adds RAG-powered AI chat to any website. One command from Claude Code.
mcp-ragchat has 5 tools with mixed quality. Naming follows verb_noun convention (ragchat_setup, ragchat_test, ragchat_serve, ragchat_widget, ragchat_status), which is good. Tool descriptions are present and moderately detailed (50-180 chars), explaining WHAT each tool does and WHEN to use it. However, input schemas are VISIBLE but INCOMPLETE: parameters have type and description fields in the JSON spec provided, but critical constraints are missing (no enums for domain validation, no min/max for port numbers, no explicit error handling guidance). Parameter descriptions are present but generic, e.g., 'Domain to test' lacks context about what constitutes a valid domain. Output schemas are NOT documented, callers cannot anticipate the structure of responses. Error handling is present in the HTTP server (chat-server.ts) but not surfaced in tool definitions themselves, so LLMs cannot know what errors to expect or how to recover. The server validates inputs (domain existence, message length, history filtering) but does not return error guidance that could teach an LLM to self-correct. No tool declares permissions, audit requirements, or side effects beyond the Risk markers provided externally. Composition is reasonable, each tool has one clear responsibility, but no guidance on chaining (e.g., ragchat_setup before ragchat_test) appears in descriptions, forcing agents to discover prerequisites through trial.
Start a local HTTP chat server for a domain. The server runs on localhost and handles POST /chat requests. Use ragchat_widget to get the embed code that connects to this server.
Initialize a domain with a knowledge base from markdown content. Each ## section becomes a searchable document with vector embeddings. This is the first step — run this before testing or serving.
List all configured domains with document counts and config status. Shows what's been set up and what's ready to serve.
Send a test message to a domain's chat. Uses RAG search + LLM to generate a response, same as production. Good for verifying the knowledge base works.
Generate an embeddable chat widget. Returns a <script> tag that creates a floating chat bubble on any webpage. Connects to the chat server started with ragchat_serve.
Output schemas not documented. Tool descriptions do not specify what fields callers should expect in responses. LLMs cannot plan downstream tool calls or extract the right data.
Input parameters lack constraints. 'domain' parameter has no enum or regex pattern validating domain format. 'port' parameter has no min/max bounds. 'content' parameter mentions '50 chars minimum' in description but not enforced in schema. LLMs cannot validate inputs before calling.
No error recovery guidance in tool definitions. While chat-server.ts returns structured errors (e.g., 'Domain not configured. Run setup first.'), these error conditions and their recovery steps are not documented in tool definitions. LLMs will not know what to do when a tool fails.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 52 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 45 | - | v1 |
Prerequisites not documented in descriptions. 'ragchat_test' and 'ragchat_serve' require 'ragchat_setup' to have been called first, but this dependency is not stated. Agents must discover this through failure.
ragchat_widget parameter 'chatUrl' has a default ('http://localhost:3456') that is only valid during local development. In production, agents must override it, but the description does not warn about this or explain when to change it. Default could cause silent failures in deployed widgets.
No explicit permission or scope declarations. Tools like 'ragchat_setup' and 'ragchat_serve' are write operations that modify state, but no description states what permissions an agent needs to invoke them. No audit trail guidance.