MCP server for chunked prefill speculative inference with vLLM, managing batch inference with speculative decoding, KV cache optimization, and throughput metrics
This server presents a single tool with severe structural deficiencies across naming, description, parameter documentation, and output schema. The tool name 'execute_skill_action' is generic and violates verb_noun conventions (should be 'execute_speculative_inference' or similar). The description is superficial (59 chars, below the 100-char minimum for adequate context). Parameters have types and defaults but lack individual descriptions entirely. The output schema is completely undocumented, the return structure (inference_batch_id, batch_size, etc.) is visible in client.py but never formally specified in the tool definition. The server appears to be a minimal stub that prints a static JSON response; the actual tool implementation logic is absent, and the MCP protocol integration is non-functional.
Execute a speculative inference batch action through the vLLM server
Generic tool name 'execute_skill_action' violates verb_noun naming convention. Should clearly indicate the inference operation (e.g., 'execute_speculative_inference', 'submit_batch_inference'). 'execute_skill_action' is too abstract and does not convey what actually happens.
Tool description is critically incomplete (59 characters). Does not explain: when to use this tool vs. other inference methods, what 'speculative inference' means, what parameters control, or what fields the response contains. Current text 'Execute a speculative inference batch action through the vLLM server' assumes user knowledge and provides no context for LLM selection.
All input parameters lack descriptions. 'batch_requests_count', 'draft_model_spec', and 'target_model_spec' have types and defaults but no text explaining what each controls, why it matters, or what valid values are. LLM cannot reason about parameter selection without these.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 21 | 2026-07-28+ | v2 |
Output schema is completely undocumented. The client.py file shows the tool returns {inference_batch_id, batch_size, paged_kv_cache_fragmentation_pct, chunked_prefill_chunk_size_tokens, speculative_acceptance_rate_pct, time_to_first_token_p99_ms, aggregate_throughput_tok_per_sec}, but the tool definition in mcp_server.py contains no return type specification, field types, or documentation of what these metrics mean or how to use them downstream.
Parameter format/constraint documentation is absent. 'draft_model_spec' and 'target_model_spec' default to qwen2.5 variants, but nowhere does the tool specify: accepted model formats, valid model names, character length limits, or validation rules. LLM could pass arbitrary strings.
No error handling documentation. Tool provides no guidance on: what errors can occur (e.g., model not found, vLLM service down, batch too large), how to recover, or what to do on failure. LLM has no recovery path.
Tool implementation is a non-functional stub. mcp_server.py only prints a static JSON status message; it does not actually register the tool with the MCP server or implement request handling. The real logic appears to be in client.py (ChunkedPrefillSpeculativeInferenceServerClient), but there is no integration between the two.