Loading ToolBench
Production-grade MCP inference server for CI/CD log anomaly detection with LightGBM multiclass prediction, Prometheus metrics, and human-in-the-loop feedback
Server has 4 tools with complete input schemas and descriptions, but quality is uneven. analyze_log has excellent schema detail (18 parameters, all typed and described) and clear purpose. However, submit_human_correction and submit_ground_truth are near-duplicates with identical signatures and overlapping descriptions, violating the single-responsibility principle. get_feedback_stats lacks any input documentation. Descriptions are adequate (100-200 chars) but lack actionable guidance on when to use each tool vs. alternatives. No output schemas documented. Error handling descriptions are generic ('error', 'details') without recovery guidance. Tool naming is clear but composition is poor.
Classify a CI/CD log record into one of 10 failure types. Required fields: - build_duration_sec, test_duration_sec, deploy_duration_sec - cpu_usage_pct, memory_usage_mb, retry_count - error_message (string) Returns structured JSON with prediction, confidence, and top contributing features. Tracks Prometheus metrics.
Return summary statistics about collected feedback.
Submit a human ground-truth correction for a model prediction. Parameters ---------- log_id : Unique ID of the original CI/CD log record original_pred : What the model predicted (e.g., 'Network Error') corrected_label : The correct label (must be a valid failure type) author_id : GitHub username or ID of the reviewer Returns ------- On success: {"status": "accepted", "message": ..., "record_count": N} On error: {"error": "Validation failed", "details": [...]}
Submit a human ground-truth correction. Parameters ---------- log_id : Unique log record ID original_pred : Model's original prediction corrected_label : Correct label (validated against known types) author_id : Reviewer's ID (e.g., GitHub username) Returns ------- Success: {status, message, record_count, retrain_required} Error: {error, details}
Duplicate tools: submit_human_correction and submit_ground_truth have identical signatures and nearly identical descriptions. Both accept log_id, original_pred, corrected_label, author_id and return {status, message, record_count, error, details}. This violates single-responsibility and forces LLMs to guess which to call.
get_feedback_stats has only 7-word description ('Return summary statistics about collected feedback.') and no input schema documentation. Lacks context on what statistics are returned, when to call it, or what fields the response contains.
No output schemas documented for any tool. LLMs cannot plan downstream calls or extract required fields. analyze_log returns 'structured JSON with prediction, confidence, and top contributing features' but exact field names and types are not specified.
Inferred effective spec: <=2025-11-25.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | D | 54 | <=2025-11-25 | v2 |
Error responses use generic field names ('error', 'details') without actionable recovery guidance. E.g., 'Validation failed' tells LLM nothing about what to fix or retry. Should include constraint violations and suggestions.
analyze_log description includes example values ('Build Failure, Configuration Error, ...') in the corrected_label enum. LLMs may treat these as the only valid options rather than reading the enum constraint.