Loading ToolBench
MCP server for local LLMs — connects to LM Studio or any OpenAI-compatible endpoint
Single tool 'generate' with decent structure but significant gaps. Tool name is verb-based (positive), description is present and informative (194 chars), and input schema is comprehensive with all parameters typed and described. However, output schema is not documented, error handling guidance is absent, and no indication of state-modifying behavior despite calling external LLM endpoints. The server follows basic MCP patterns but lacks production-grade robustness.
Send a chat message to the configured local LLM with support for system prompts, conversation history, streaming output, structured JSON schemas, and extended thinking. Handles automatic context overflow recovery and token accounting.
Read and analyze source code files with the local LLM for code review, refactoring, testing, or documentation. Supports multiple files, enforces size limits (default 10 MB per file), and includes optional directory confinement for security.
Execute a freeform prompt template against the local LLM with full control over context, formatting, and structured output. Supports variables, JSON schemas, and multi-turn reasoning.
List available and loaded models from the configured LM endpoint (LM Studio, Ollama, vLLM, or other OpenAI-compatible API). Returns model IDs, capabilities, and current context window.
Comprehensive listing of all available models including loaded status, architecture, quantization level, and context window. Returns rich metadata for model selection.
Output schema not documented. Tool description does not specify what fields are returned (e.g., generated_text, tokens_used, model_used, finish_reason). LLMs cannot plan downstream operations or extract results without knowing the response structure.
Missing error handling guidance. No indication of what errors can occur (timeout, invalid model, API unreachable, invalid parameters) or how the LLM should recover. Description does not state whether the tool modifies state (sends requests to external service) or guide retry behavior.
Tool description (194 chars) is well-structured but does not answer 'what does it return?' or 'when should I use this vs. calling the LLM directly?'. Missing context for tool selection and downstream data extraction.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-21 | F | 19 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 56 | - | v1 |
Parameters lack constraint documentation. 'temperature' accepts 0.0-2.0 but description does not state the range or recommend values for different use cases. 'max_tokens' is unbounded, no guidance on typical limits or API constraints. 'top_p', 'frequency_penalty', 'presence_penalty' descriptions are bare and do not explain when to use each.
No indication of how the 'model' parameter relates to configured defaults or available models. Description says 'Optional model name to override the default configured model' but does not explain where model names come from, what valid values are, or how to discover available models.
Parameter relationship between 'prompt' and 'messages' is underdocumented. Both accept text input, are they mutually exclusive? Do they conflict? Can both be provided simultaneously? If 'messages' is conversation history, what format do message objects use (role/content)? Undocumented dependencies cause misuse.