Loading ToolBench
Weaviate MCP Server implemented in TypeScript
The Weaviate MCP server defines 4 tools with schemas visible in test/test-server.js and src/mcp.ts. However, critical gaps severely limit production readiness. Tool names lack clarity (weaviate-origin, weaviate-follow-ref are domain-specific jargon), descriptions exist but are terse (100-150 chars), and while input schemas are present with types and descriptions, output schemas are not documented anywhere. Error handling is minimal, the test shows a negative test case but no guidance on how errors map to recovery actions. The server lacks tool annotations (readOnlyHint, destructiveHint, idempotentHint), security documentation (permission scopes), and composition guidance. All four tools are READ_ONLY, which is good for safety, but no idempotency guarantees are stated. Parameter descriptions are present but often generic ('The search query string', 'The Weaviate collection/class name'). Most tools cluster around the same 4-5 parameters with minimal variation, suggesting potential for consolidation or better composition patterns.
Follow a cross-reference property from a base collection to related objects in a target class
Perform a hybrid search with generative AI text generation to produce answers based on search results
Hybrid search Etapa objects and return cross-references to Fluxo and Entidade with dynamic discovery of reference paths
Perform a hybrid search query against a Weaviate collection
Output schemas are not documented. The rubric requires 'Document the output schema' so LLMs know what fields to expect. All four tools have input schemas but zero documentation of return types, LLMs must infer the shape of search results, references, and generated text.
Tool names are domain-jargon and lack action verbs. 'weaviate-origin' and 'weaviate-follow-ref' assume deep familiarity with Weaviate's graph model. Baseline from 549 production tools shows 90% start with an action verb (get, list, create, search, update). These should be 'search_with_references' and 'follow_reference_path' respectively. Current names force LLMs to read the description to understand intent.
No error handling guidance. The negative test in test-server.js shows a call to a non-existent collection, but the server provides no documented error responses, recovery paths, or categorization (retryable vs user-fixable). The rubric requires 'Error responses must tell the LLM what to do next.' Currently there is no guidance.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 57 | <=2025-11-25 | v2 |
| 2026-03-09 | F | 31 | 2025-06-18+ | v1 |
No tool annotations (readOnlyHint, destructiveHint, idempotentHint). All tools are READ_ONLY but this is not declared in the tool definition via annotations. The spec (2026-07-28) includes tool annotations as a current pattern for clarity.
Parameter descriptions are generic and lack format/constraint details. E.g., 'The search query string (required, minimum length 1)', where is the maximum? 'Array of property names', what if the user passes a property that does not exist in the collection? Baseline from production tools shows descriptions should include expected format, range, and allowed values. The rubric specifies: 'Describe the expected format, range, and allowed values directly in the parameter description.'
No permission/scope declarations. The rubric requires 'Each tool should declare what permissions it requires (e.g. "read:email", "write:calendar").' All tools query Weaviate but there is no documentation of required authentication, scopes, or capabilities.
No pagination guidance in output. Tools accept 'limit' parameter but it is unclear if results are paginated, how to get the next page (offset? cursor?), or what the total count is. The rubric requires 'Tools returning lists should accept page/offset and limit parameters and return a total count or next_cursor.'
Redundant tool naming, three tools (weaviate-query, weaviate-origin, weaviate-follow-ref) perform variations of the same operation (search + optionally return references). The rubric warns 'Avoid multiple tools that do the same thing differently. If search_users and find_users both exist, the LLM wastes reasoning cycles deciding between them.' Consider consolidating with a 'follow_references' boolean parameter and a 'reference_path' parameter.