Loading ToolBench
MCP server for Keylime - integrates with Keylime verifier and registrar services to enable LLM-driven attestation management, agent enrollment, policy handling, and log investigation
Keylime MCP demonstrates moderate definition quality with consistent naming conventions and schema presence, but suffers from inconsistent parameter documentation, missing output schema details, and limited error guidance. All 18 tools follow verb_noun naming (get_, list_, enroll_, update_, stop_, reactivate_, unenroll_, registrar_, import_, investigate_), which is strong. Tool descriptions exist and generally explain the action (range 45-130 chars, within acceptable bounds). However, parameter descriptions are sparse or missing context about constraints, formats, and valid values. Output schemas are not explicitly documented in the source code, forcing LLMs to infer response structure. Error handling is minimal, no recovery guidance, no retryability hints, no examples of failure modes. Risk annotations (READ_ONLY, WRITE, DESTRUCTIVE) are present and useful, but not formalized as tool annotations per spec.
Enrolls an agent to the Keylime verifier with optional runtime and measured boot policies
Retrieves TPM policy, vTPM policy, metadata, and algorithm preferences for an agent
Fetches the operational status and details of a specific agent
Retrieves all agent UUIDs from the Keylime registrar
Returns a list of agents in failed operational states by checking all agents in parallel
Retrieves a specific runtime policy content from the Keylime verifier
Returns a flattened list of UUIDs for agents enrolled in the Keylime verifier
Checks the version and health status of both Keylime verifier and registrar services
Output schemas not documented. Tool definitions show input schemas but no explicit output type documentation, forcing LLMs to infer response structure from behavior. This violates the DIMENSION 1.D baseline that 100% of A+ tools have documented return types.
Parameter descriptions lack constraint details. Required UUID parameters have minimal descriptions (e.g., 'UUID of the agent (required)') with no format hints, examples of valid UUIDs, or error handling guidance. Missing guidance on format like 'UUID in format xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | F | 37 | - | v1 |
Imports a runtime policy from a JSON file into the Keylime verifier
Investigates Keylime verifier systemd logs with optional filtering by agent UUID or log type
Lists all available runtime policies from the Keylime verifier
Reactivates an agent that has been stopped in the Keylime verifier
Fetches agent details from the Keylime registrar for a specific agent UUID
Removes an agent from the Keylime registrar
Stops attestation for an agent in the Keylime verifier
Unenrolls an agent from the Keylime verifier
Updates an agent's policies by unenrolling and re-enrolling it with new runtime or measured boot policies
Updates an existing runtime policy by adding/removing excludes or digests
No error recovery guidance. Tool descriptions do not explain failure modes, retryability, or what LLMs should do next on errors. E.g., 'get_agent_status' could fail if agent UUID is invalid, but no guidance on how to recover.
Destructive operations lack confirmation pattern. 'registrar_remove_agent', 'unenroll_agent_from_verifier', and 'stop_agent' perform irreversible state changes without dry-run or confirmation options.
Tool annotations not formalized. Risk metadata (READ_ONLY, WRITE, DESTRUCTIVE) exists in annotations but is not mapped to MCP spec tool annotations (readOnlyHint, destructiveHint, idempotentHint). This prevents clients from enforcing safety policies programmatically.
Policy file paths exposed as parameters. 'import_runtime_policy' accepts 'file_path' as a required string parameter with no validation, format hints, or path traversal protection guidance. Per pattern:secret-injection and pattern:tool-gateway, file paths should be constrained and validated.
Array parameters lack element constraints. 'update_runtime_policy' accepts 'add_excludes', 'remove_excludes' as arrays of strings with no description of valid path formats, length limits, or examples. LLMs may pass malformed paths.
Enum constraint for 'filter' not emphasized in description. 'investigate_verifier_logs' has an enum constraint for 'filter' but the parameter description does not highlight valid values clearly (should read 'Must be one of: all, attestation_failures, errors').
Pagination not implemented. List tools ('get_all_agents', 'get_verifier_enrolled_agents', 'list_runtime_policies') do not document pagination parameters or result limits. If a Keylime deployment has thousands of agents or policies, responses could be unbounded and exhaust context windows.
Optional parameters not clearly marked. 'enroll_agent_to_verifier' and 'update_agent' have optional policy parameters but descriptions do not clarify defaults or what happens if both or neither is provided. LLMs may be confused about parameter relationships.