Loading ToolBench
Read-only ops/health MCP server for a local vLLM server (WSL2/systemd-managed): liveness vs real-completion health tiers, GPU/VRAM status, systemd service status, and live serve-flag introspection. No lifecycle-control tools in this release.
6 tools with clear verb-noun naming (check_health, list_models, get_gpu_status, get_service_status, get_serve_config, test_completion). All have descriptions (194 - 392 chars, within baseline). Input schemas present with type declarations and descriptions. However, output schemas are NOT documented, descriptions mention return structure (e.g., '{ok, status, tier, model_id, latency_ms, completion_s, error}') but no formal schema definition is visible in code. Parameters lack enums where applicable (e.g., 'status' could be constrained to up|degraded|down). Error handling guidance is minimal, no recovery hints or actionable error messages shown. Rate-limiting is mentioned but not exposed as a tool parameter or error response structure.
Two-stage health probe for the local vLLM server, mirroring check-vllm.cmd. deep=False (default): GET /v1/models liveness check only. deep=True: additionally fires one minimal real /v1/chat/completions call to confirm the server actually generates text, not just reports a model loaded (rate-limited -- this consumes real GPU inference cycles on shared infra). Returns {ok, status: up|degraded|down, tier, model_id, latency_ms, completion_s, error}.
nvidia-smi wrapper: per-GPU VRAM used/total and utilization%, plus per-process VRAM usage (to catch VRAM-overshoot). Read-only.
Read-only introspection of the current vLLM launch flags. Prefers the LIVE process's real argv (/proc/<pid>/cmdline of the systemd unit's MainPID) as ground truth; falls back to the static exec script if the process isn't currently up. vLLM does not expose its own launch flags over the API, so this never queries the server itself. Any argv value following a flag matching a known-sensitive pattern (token/key/secret/password/credential) is redacted before being returned.
systemd unit status for the vLLM service (via `wsl -d <distro> -- systemctl show`, or native systemctl if this server runs inside WSL2/Linux). Returns load/active/sub state, restart count, main PID, and last-active timestamp. Read-only -- never starts/stops the unit.
Output schemas not formally documented. Descriptions mention return fields (e.g., '{ok, status, tier, model_id, latency_ms, completion_s, error}') but no JSON Schema definition is visible. LLMs cannot reliably parse unstructured text descriptions of output structure.
Enum constraints missing. 'status' field in check_health can be 'up|degraded|down' but is not declared as an enum in the input schema. Free-form strings invite hallucinated values.
Error handling lacks recovery guidance. Rate-limit refusals are mentioned in descriptions but no structured error response format is documented. LLMs need actionable guidance: 'Rate limit exceeded. Retry after 60 seconds.' vs a bare refusal.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 57 | 2026-07-28+ | v2 |
GET /v1/models passthrough -- lists model IDs currently loaded/served.
On-demand real /v1/chat/completions call with a caller-supplied prompt, for manual sanity-checking (same mechanism as check_health(deep=True) but user-controlled). Rate-limited -- consumes real GPU inference cycles on shared infra. max_tokens is clamped server-side to config.MAX_TEST_COMPLETION_TOKENS and the prompt is capped at config.MAX_TEST_COMPLETION_PROMPT_CHARS (oversized prompts are rejected) -- the rate limiter bounds call frequency, not the cost of a single call. Returns {ok, model_id, completion_s, text, error}.
Parameter constraints not formalized. 'max_tokens' defaults to 16 and is 'clamped server-side to config.MAX_TEST_COMPLETION_TOKENS' but no min/max bounds are declared in the schema. LLMs cannot infer valid ranges.
No pagination support documented. Tools like list_models may return large result sets but no limit, offset, or cursor parameters are visible. Large unbounded results risk context window exhaustion.