Loading ToolBench
A simple MCP server for interacting with OpenAI assistants
The server defines 9 tools with consistent naming (verb_noun pattern) and generally adequate descriptions. All tools have documented input schemas with type information. However, output schemas are entirely undocumented, responses are strings only, no structured fields returned. Descriptions, while present, lack depth about when to use each tool, error recovery, and several parameters lack type-level constraints (e.g., model parameter accepts any string; no enum of valid OpenAI models). Tool annotations (readOnlyHint) are present, which is commendable. The ask_assistant_in_thread tool references streaming via Context but the actual streaming mechanism and output format are not visible in the provided code. Database-backed state management is present but not clearly exposed in tool schemas.
Sends a message to an assistant within a specific thread and streams the response. This provides progress updates and the final message in a single call. Use this to continue a conversation with an assistant in a specific thread. The thread ID can be retrieved from the list_threads tool. The assistant ID can be retrieved from the list_assistants tool. Threads are not inherently linked to a particular assistant, so you can use this tool to talk to any assistant in any thread.
Create a new OpenAI assistant to talk to about your desired topic. You can provide instructions that this assistant will follow and specify which of OpenAI's models it will use. NOTE: It is recommended to check existing assistants with list_assistants before creating a new one.
Creates a new, persistent conversation thread with a user-defined name and description for easy identification and reuse. These threads are stored in OpenAI's servers and are not deleted unless the user deletes them, which means you can re-use them for future conversations. Additionally, the thread name and description are stored in the local database, which means you can list them and update them later. Think how you can utilize threads in your particular use case.
Deletes a conversation thread from both OpenAI's servers and the local database. This action is irreversible.
No output schemas documented for any tool. All tools return unstructured strings instead of structured JSON objects. This forces LLMs to parse free-text output, risking errors, and breaks tool chaining because downstream tools cannot rely on field names (e.g., create_assistant returns 'Created assistant... with ID: xyz', LLM must regex-extract the ID rather than accessing assistant.id).
Model parameter in create_assistant and update_assistant accepts any string with no validation or enumeration. Should constrain to valid OpenAI model names (gpt-4, gpt-4o, gpt-3.5-turbo, etc.) or document the format/source of valid models.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | D | 58 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 46 | - | v1 |
List all available OpenAI assistants associated with the API key configured by the user. Returns a list of assistants with their IDs, names, and configurations. This can be used to select an assistant to use in the ask_assistant_in_thread tool instead of creating a new one.
Lists all locally saved conversation threads from the database. Returns a list of threads with their ID, name, description, and last used time. The thread ID can be used in the ask_assistant_in_thread tool to specify this thread to be continued.
Get detailed information about a specific assistant. The ID required can be retrieved from the list_assistants tool.
Modify an existing assistant's name, instructions, or model used. At least one optional parameter - what to change - must be provided, otherwise the tool will return an error. The ID required can be retrieved from the list_assistants tool.
Updates the name and/or description of a locally saved conversation thread. Both the local database and the OpenAI thread object will be updated. The thread ID can be retrieved from the list_threads tool.
delete_thread (irreversible operation) lacks confirmation pattern. No dry-run, confirmation parameter, or strong warning to prevent accidental deletion. Agents will invoke destructive tools without hesitation.
list_threads and list_assistants return no pagination metadata (total_count, next_cursor, has_more). If an agent has many threads or assistants, the tool will either truncate silently or return everything, risking context explosion.
ask_assistant_in_thread accepts a 'ctx: Context' parameter that is a framework artifact, not part of the MCP input schema. This may confuse client introspection tools and complicates schema documentation.
Tool descriptions lack recovery guidance for common failures. E.g., ask_assistant_in_thread gives no hint about what to do if the assistant_id or thread_id is invalid. Error messages should guide the LLM to list_assistants or list_threads.
update_thread requires at least one of name or description, but this constraint is enforced at runtime (via ToolError in code), not declared in the schema. Parameter descriptions should state the mutual-requirement relationship.