Loading ToolBench
MCP server for the oMLX local inference server. Exposes the oMLX local inference server (OpenAI-compatible API plus admin model-management routes) to MCP clients. Provides inference, inspection, and model lifecycle tools.
Server demonstrates solid tool definition quality with complete schemas, clear descriptions, and proper parameter validation. All 7 tools have explicit input schemas with type definitions and descriptions. Tool names follow verb_noun convention (list, status, chat, complete, embeddings, load, unload). Descriptions are substantive (100-250 chars), explaining what each tool does and when to use it. Error handling is comprehensive with actionable messages. Main gaps: output schemas are not documented (tools return raw JSON without field descriptions), and some parameter descriptions could be more specific about constraints and formats. No tool annotations (readOnlyHint/destructiveHint) despite clear risk classifications in metadata.
Run a chat completion against an oMLX model via the OpenAI-compatible endpoint. Calls POST /v1/chat/completions with the given messages and parameters. Returns the model's response as a single message.
Run a text completion against an oMLX model via the OpenAI-compatible endpoint. Calls POST /v1/completions with the given prompt and parameters. Returns the model's completion.
Generate embeddings via an oMLX embedding model using the OpenAI-compatible endpoint. Calls POST /v1/embeddings with the given input strings. Returns an array of embedding vectors.
List models available to oMLX via the OpenAI-compatible endpoint. Calls GET /v1/models. Returns the models the server exposes, by id or alias. For load state, memory size, and context window, use omlx_model_status instead.
Load a model into memory on the oMLX server via the admin endpoint. Calls POST /admin/api/models/{model_id}/load. Blocks until the model is loaded or an error occurs. Use omlx_model_status to check load progress.
Output schemas not documented. Tools return raw JSON responses without field descriptions, forcing LLMs to infer structure. This violates pattern:tool and increases hallucination risk.
Tool annotations missing. Tools have clear risk classifications (READ_ONLY vs WRITE) in metadata but do not expose readOnlyHint/destructiveHint/idempotentHint in schema. Agents cannot determine safety without reading descriptions.
Parameter descriptions lack format/constraint details. 'model_id' accepts strings up to 200 chars but description does not explain valid format (alphanumeric, hyphens, underscores?). Temperature range (0-2) is constrained in schema but not explained in description.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | C | 62 | 2026-07-28+ | v2 |
Report detailed model status via the oMLX admin endpoint. Calls GET /admin/api/models. Returns richer data than omlx_list_models, including load state (loaded, is_loading), estimated memory size, pinned state, engine and model type, and per-model settings such as the maximum context window.
Unload a model from memory on the oMLX server via the admin endpoint. Calls POST /admin/api/models/{model_id}/unload. Frees GPU/CPU memory occupied by the model.
Error handling returns raw HTTP responses. _handle_error() produces actionable messages for common cases (401, 404, 422, timeout) but tools return raw JSON from oMLX on success. If oMLX returns an error nested in JSON (e.g. {"error": "model not found"}), the agent sees raw JSON without guidance.
No pagination support. omlx_list_models and omlx_model_status return all models without limit or offset. If oMLX hosts 100+ models, response bloats context window. Pattern:paginated-result recommends limit/offset and total count.