Loading ToolBench
Model Context Protocol (MCP) tools for the Metis RAG application
Server has 17 tools with explicit schemas and descriptions, but quality is inconsistent. Naming follows verb_noun convention well (init_*, execute_*, get_*, list_*, add_*, delete_*, generate_*, query_*). Descriptions are present but often generic (avg ~80 chars, below 194-char baseline). Parameter descriptions exist but lack detail on constraints, formats, and valid ranges. Output schemas are not documented, LLMs cannot predict response structure. Error handling returns success/error flags but lacks recovery guidance. No tool annotations (readOnlyHint, destructiveHint). Critical security issue: init_llm_model and init_database_connection expose API keys and passwords as parameters, violating secret-injection pattern. No pagination support on list tools. Composition is reasonable (single responsibility per tool), but missing chaining IDs in responses.
Add documents to the vector store collection
Analyze the structure of a document
Split a document into chunks
Delete documents from the vector store collection
Execute a SQL query on the database
Extract code blocks from a document
Generate embeddings for text
Credentials exposed as tool parameters: init_database_connection accepts password, init_llm_model accepts apiKey. These leak into agent traces and logs. Use server-side secret injection via environment variables or vault.
No output schemas documented. LLMs cannot predict response structure (e.g., what fields does execute_query return? Is it {success, data, rowCount} or {rows, count}?). Document return types for all tools.
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). LLMs cannot distinguish safe reads from destructive writes. Mark delete_documents, execute_query (if UPDATE/DELETE), and init_* tools with appropriate hints.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 63 | 2026-07-28+ | v2 |
Generate text with an LLM model
Get or create a collection in the vector store
Get the schema of a database table
Initialize the database connection with the provided configuration
Initialize an LLM model
Initialize the vector store connection
List all tables in the database
Load a document from a file path
Query the vector store collection
Use a prompt template with an LLM model
Error handling lacks recovery guidance. Errors return {success: false, error: message} but do not tell LLM what to do next. E.g., 'Database connection not initialized' should suggest 'Call init_database_connection first.'
Parameter descriptions lack constraint details. E.g., chunkSize defaults to 1000 but no min/max specified. temperature defaults to 0.7 but valid range (0-2?) not stated. LLMs need explicit bounds to avoid invalid values.