Loading ToolBench
Multi-LLM MCP server supporting concurrent calls to multiple LLM providers (DeepSeek, Kimi, Qwen, GPT) with session management, code execution via Codex, and batch processing
Server has 11 tools with basic schemas and descriptions, but significant quality gaps. Most tool descriptions are present but generic (10-50 chars), falling below the 50-200 char LLM-optimized baseline. Parameter descriptions exist but lack detail on constraints, formats, and valid ranges. No output schemas documented. Error handling is minimal, most tools return raw exceptions or generic dicts without recovery guidance. Tool naming is reasonable (verb_noun pattern mostly followed), but composition issues exist: ask/ask_many/review overlap in responsibility; chat_session/get_session_history/clear_session form a stateful session management cluster that could be better abstracted. Security concerns: API keys are environment-injected (good), but no permission gates, audit trails, or rate limiting visible. No tool annotations (readOnlyHint/destructiveHint) despite clear risk levels in metadata.
Call a single LLM model with a prompt and optional system message
Execute a prompt via Codex CLI with optional session resumption
Call multiple LLM models concurrently with the same prompt
Multi-turn conversation with an LLM model using session history
Clear conversation history for a session
Get status and output of a Codex job
Retrieve conversation history for a session
No output schemas documented. LLMs cannot infer what fields to expect from tool responses, forcing unstructured parsing and increasing hallucination risk.
Descriptions are too generic and short (10-50 chars). Baseline for LLM-optimized descriptions is 50-200 chars. Current descriptions lack WHEN to use, WHAT it returns, and dependencies.
No error handling guidance. Tools return raw exceptions or generic dicts without actionable recovery hints. LLMs cannot determine if errors are retryable or user-fixable.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | 2026-07-28+ | v2 |
Check server health status
Terminate a running Codex job
Request code review from multiple LLM models with focus area
Wait for a running Codex job to complete
Tool annotations missing. ask_codex and kill_codex_job are WRITE/DESTRUCTIVE but lack readOnlyHint/destructiveHint annotations. Agents cannot distinguish safe from risky tools.
Overlapping tool responsibilities. ask, ask_many, and review all call LLMs with prompts. ask_codex, wait_codex, get_codex_job, kill_codex_job form a job-management cluster. Session tools (chat_session, get_session_history, clear_session) should be unified.