Loading ToolBench
MCPKI aims at providing an MCP based PKI for Large Language Models (LLMs). Provides tools for TLS certificate retrieval, EJBCA certificate authority operations, and cryptographic services.
MCPKI exposes 9 PKI-focused tools with proper noun-verb naming (GetServerCertificate, CreateCrl, etc.) and input schemas visible in source. However, descriptions are extremely terse (10-30 chars, well below the 194-char baseline for A+ tools), parameter descriptions are minimal or absent, and output schemas are not documented. Tool names use PascalCase rather than snake_case convention, parameter relationships are undocumented, and error handling is not visible in the provided source. The server implements HTTP transport with Spring Boot WebFlux (good), but definition quality falls in the 'fair' range due to significant gaps in documentation and schema completeness.
Issues a Certificate Revocation List (CRL) for the given issuer DN.
Enrolls a certificate given the PKCS#10 Certificate Signing Request (CSR) and returns it in the PEM format.
Returns the list of available Certification Authorities (CA).
Returns the PEM formatted CA certificate chain (last is root CA) including boundaries and Subject / Issuer annotation.
Returns the certificate profile with the given name.
Return the certificates about to expire within the given time in days.
Returns the number certificates in the database.
Tool descriptions are critically short (10-30 characters, well below the 194-char baseline). Descriptions like 'Returns the certificate by the TLS connection in PEM format' lack context about WHEN to use the tool, WHY it matters for PKI operations, and what the response structure contains. LLMs cannot effectively select tools with such minimal guidance.
Output schemas are not documented. The source code shows input parameters (e.g., 'issuerDn' for CreateCrl) but provides no documentation of what fields are returned, their types, or structure. Without output schema documentation, LLMs cannot plan downstream calls or extract required data for tool chaining.
| Scored | Grade | Overall | Spec posture | Rubric |
|---|---|---|---|---|
| 2026-09-22 | F | 48 | 2026-07-28+ | v2 |
| 2026-03-09 | D | 58 | - | v1 |
Returns the latest Certificate Revocation List (CRL) for the given issuer.
Returns the certificate by the TLS connection in PEM format.
Parameter descriptions are missing or trivial. Input schemas show types (string, boolean, integer) but lack actionable descriptions. For example, 'issuerDn' needs clarification: 'X.500 Distinguished Name of the issuing CA (e.g., CN=MyCA,O=MyOrg,C=US)'. LLMs cannot format complex inputs like DNs without explicit guidance.
Tool naming uses PascalCase (GetServerCertificate) rather than the MCP convention of snake_case (get_server_certificate). While naming is semantically clear (verb_noun structure is present), convention mismatch may confuse LLMs and client tools expecting standard naming.
Error handling is not visible in provided source. No evidence of actionable error messages, recovery guidance, or error classification (retryable vs user-fixable vs fatal). Tools that call EJBCA or perform TLS connections need explicit timeout/failure handling with guidance for LLM recovery actions.
Parameter relationships are undocumented. For GetCertificatesAboutToExpire, the interaction between 'offset' and 'max' pagination parameters is not explained. For EnrollCertificateWithCsr, dependencies between 'certificateProfileName', 'endEntityProfileName', and 'nameOfCa' are not documented. LLMs cannot reason about which combinations are valid.
CreateCrl and EnrollCertificateWithCsr are WRITE operations but contain no confirmation step or dry-run capability. These tools modify PKI state (issue certificates, revoke lists). Without a confirmation mechanism, agents could inadvertently issue invalid certificates or revoke legitimate CRLs. A confirm_before_execute pattern is needed for destructive operations.