Troubleshoot PKI Secrets Engine and ACME
Solve common problems related to ACME client integration with Vault PKI Secrets Engine's ACME server.
Error: ACME feature requires local cluster 'path' field configuration to be set
If ACME works on some nodes of a Vault Enterprise cluster but not on others, it likely means that the cluster address has not been set.
Symptoms
When a Vault client reads the ACME config (/config/acme
) on a
Performance Secondary nodes or when an ACME client attempts to connect to a
directory on this node, it will error with:
ACME feature requires local cluster 'path' field configuration to be set
Cause
In most cases, cluster path errors mean that the required cluster address is not set in your cluster configuration parameter.
Resolution
For each Performance Replication cluster, read the value of /config/cluster
and ensure the path
field is set. When it is missing, update the URL to
point to this mount's path on a TLS-enabled address for this PR cluster; this
domain may be a load balanced or a DNS round robin address. For example:
Once this is done, re-read the ACME configuration and make sure no warnings appear:
Error: Unable to register an account with the ACME server
Symptoms
When registering a new account without an External Account Binding (EAB), the Vault Server rejects the request with a response like:
Unable to register an account with ACME server
with further information provided in the debug logs (in the case of
certbot
):
Server requires external account binding.
or, if the client incorrectly contacted the server, an error like:
The request must include a value for the 'externalAccountBinding' field
In either case, a new account needs to be created with an EAB token created by Vault.
Cause
If a server has been updated to require eab_policy=always-required
in the
ACME configuration,
new account registration (and reuse of existing accounts will fail).
Resolution
Using a Vault token, fetch a new external account binding for the desired directory:
Then pass this new EAB token into the ACME client. For example, with
certbot
:
Ensure that the ACME directory passed to the ACME client matches that fetched from the Vault.
Error: Failed to verify eab
Symptoms
When initializing a new account against this Vault server, the ACME client might error with a message like:
The client lacks sufficient authorization :: failed to verify eab
This is caused by requesting an EAB from a directory not matching the one the client used.
Cause
If an EAB account token is incorrectly used with the wrong directory, the ACME server will reject the request with an error about insufficient permissions.
Resolution
Ensure the requested EAB token matches the directory. For a given directory
at /some/path/acme/directory
, fetch EAB tokens from
/some/path/acme/new-eab
. The remaining resolution steps are the same as
for debugging account registration
failures.
Error: ACME validation failed for {challenge_id}
Symptoms
When viewing the Vault server logs or attempting to fetch a certificate via an ACME client, an error like:
ACME validation failed for a465a798-4400-6c17-6735-e1b38c23de38-tls-alpn-01: ...
indicates that the server was unable to validate this challenge accepted by the client.
Cause
Vault can not verify the server's identity through the client's requested
challenge type (dns-01
,
http-01
, or tls-alpn-01
). Vault will not issue the certificate requested
by the client.
Resolution
Ensure that DNS is configured correctly from the Vault server's perspective, including setting any custom DNS resolver.
Ensure that any firewalls are set up to allow Vault to talk to the relevant
systems (the DNS server in the case of dns-01
, port 80 on the target
machine for http-01
, or port 443 on the target machine for tls-alpn-01
challenges).
Error: The client lacks sufficient authorization: account in status: revoked
Symptoms
When attempting to renew a certificate, the ACME client reports an error:
The client lacks sufficient authorization: account in status: revoked
Cause
If you run a manual tidy or have auto-tidy enabled with `tidy_acme=true, Vault will periodically remove stale ACME accounts.
Connections from clients using removed accounts will be rejected.
Resolution
Refer to the ACME client's documentation for removing cached local configuration and setup a new account, specifying any EABs as required.
Get help
Please provide the following information when contacting Hashicorp Support or filing a GitHub issue to help with our investigation and reproducibility:
- ACME client name and version
- ACME client logs and/or output
- Vault server DEBUG level logs
Tutorial
Refer to the Build Your Own Certificate Authority (CA) guide for a step-by-step tutorial.
Have a look at the PKI Secrets Engine with Managed Keys for more about how to use externally managed keys with PKI.
API
The PKI secrets engine has a full HTTP API. Please see the PKI secrets engine API for more details.