Loading ToolBench
MCP Server for Selective Answering Abstention Gatekeeper Skill. Balances empirical risk and coverage targets by deciding whether an agent should ANSWER autonomously, DELEGATE to tools, or ABSTAIN to human oversight.
Two tools with basic schemas and descriptions, but significant gaps in parameter documentation and output schema clarity. evaluate_action lacks descriptions for context_metadata parameter; set_risk_parameters has no required field specification and missing parameter descriptions. No output schemas documented. Error handling absent. Naming is reasonable (verb_noun pattern) but descriptions are generic and under 200 chars. No tool annotations (readOnlyHint/destructiveHint). Composition is sound (two separate concerns), but parameter constraints and validation guidance missing.
Evaluate whether agent should execute or abstain based on confidence and risk budget
Update confidence threshold and risk cost multipliers
Missing output schema documentation. Neither tool documents what fields are returned or their types. LLMs cannot plan downstream operations or extract required data.
Parameter descriptions incomplete. context_metadata in evaluate_action has no description; set_risk_parameters parameters lack descriptions explaining what each cost parameter controls or valid ranges.
No numeric constraints on parameters. confidence (0-1?), cost_of_error, cost_of_abstention, confidence_threshold lack min/max bounds. LLMs may pass invalid values like negative costs or confidence > 1.0.
No error handling guidance. Tool descriptions do not explain what happens on invalid input (e.g., confidence < 0, missing required params). No recovery hints for LLMs.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-23 | F | 45 | 2026-07-28+ | v2 |
Missing tool annotations. set_risk_parameters is a WRITE operation but lacks destructiveHint annotation. evaluate_action is READ_ONLY but lacks readOnlyHint. Agents cannot infer safety properties.