Loading ToolBench
MCP server to inspect and free NVIDIA GPU VRAM via Ollama + nvidia-smi.
vram-mcp demonstrates solid domain-specific tool design with 12 well-named, verb-prefixed tools (unload, ensure_free, reserve, claim, release, renew, list_claims, residency, audit_log, vram_status, vram_status_detailed, trend). All tools have descriptions (avg ~80 chars, within baseline 34-392). Input schemas are present and typed for all tools. However, descriptions are terse and lack LLM-optimized guidance on WHEN to use each tool vs. alternatives. Parameter descriptions exist but are minimal (e.g., 'Model name to unload' lacks format/validation hints). Output schemas are not documented in visible code, responses are inferred from implementation. Error handling is absent from tool definitions. No tool annotations (readOnlyHint/destructiveHint) despite clear risk stratification (READ_ONLY vs WRITE). Composition is strong: tools chain naturally (list_claims → release/renew). Naming follows verb_noun convention consistently.
Recent VRAM audit events.
Claim VRAM for a model to prevent eviction.
Free at least N MB by unloading models, respecting claims.
List all active claims and reservations.
Release a claim or reservation.
Extend the expiration of a claim or reservation.
Reserve VRAM for a model to prevent eviction.
Check if a model is loaded in VRAM.
Output schemas not documented. Tool descriptions state what is returned (e.g., 'Snapshot of GPU VRAM...') but do not specify the JSON structure, field types, or pagination. LLMs cannot plan downstream calls without knowing response shape.
Tool annotations missing. Tools are stratified by risk (READ_ONLY vs WRITE) but lack readOnlyHint and destructiveHint annotations in schema. LLMs cannot infer safety properties and may retry destructive operations (unload, ensure_free, reserve, claim, release, renew) inappropriately.
Parameter descriptions lack format/constraint hints. E.g., 'Model name to unload' does not specify valid format, length, or how to discover valid model names. 'Minutes of history' lacks min/max bounds. LLMs cannot validate inputs and will pass invalid values.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 59 | 2026-07-28+ | v2 |
Recent VRAM pressure trend with summary statistics.
Unload one Ollama model from VRAM.
Snapshot of GPU VRAM, loaded Ollama models, and pressure state.
Extended status including process table, claims, and operations.
No error handling guidance in tool definitions. Descriptions do not explain failure modes (e.g., 'model not found', 'insufficient VRAM', 'claim expired') or recovery steps. Agents will not know whether to retry, ask the user, or abort.
Descriptions are terse and lack LLM-optimized guidance. Average ~80 chars; baseline is 194 chars. Descriptions do not explain WHEN to use each tool vs. alternatives (e.g., when to call reserve vs. claim, or ensure_free vs. unload). LLMs will guess wrong.