Loading ToolBench
Provides vector search capabilities using OpenAI embeddings with Qdrant database backend
The server defines 3 tools with basic structure, but suffers from significant gaps in schema completeness, parameter documentation, and error handling. All tools have descriptions (positive), but parameters lack detailed constraints and guidance. Output schemas are not formally documented, the tools return JSON strings without declared field schemas. Error handling is minimal: exceptions are caught but responses lack recovery guidance or categorization. The naming convention is reasonable (verb_noun), but parameter types and ranges are underspecified. This is a typical community-grade server with incomplete parameter descriptions and no output schema documentation.
Get information about a specific collection.
List all available collections in the Qdrant database.
Search a Qdrant collection using semantic search with OpenAI embeddings.
No output schemas documented for any tool. All tools return JSON strings ('str') but the structure of those strings (field names, types, presence/absence of fields) is undocumented. LLMs cannot predict response structure or plan downstream tool calls.
Parameter descriptions lack detail on format, constraints, and valid ranges. 'limit' has no min/max specified (default 5 is given, but unbounded max lets LLM pass huge values). 'model' has a default but no enum of valid models. 'collection_name' has no format guidance (e.g., alphanumeric, special chars allowed?).
Error handling is generic and non-actionable. All tools catch exceptions and return {'error': str(e)}, a raw exception message tells the LLM nothing about whether to retry, what went wrong, or what to do next. No categorization (retryable vs fatal). No recovery guidance.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 47 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 45 | - | v1 |
OpenAI API key and Qdrant API key are accessed via os.getenv() inside tool logic, not injected at server init. If an agent can inspect tool parameters or logs, credentials could be exposed. Best practice is server-side secret injection at initialization.
query_collection does not validate the 'model' parameter against known OpenAI embedding models. Invalid model names are passed directly to the OpenAI API, which fails with a raw API error. Should pre-validate or provide enum of supported models.
No pagination support. list_collections returns all collections at once. For a Qdrant instance with thousands of collections, response could explode context window. Should support limit/offset or cursor-based pagination.