AWS secrets engine
The AWS secrets engine generates AWS access credentials dynamically based on IAM policies. This generally makes working with AWS IAM easier, since it does not involve clicking in the web UI. Additionally, the process is codified and mapped to internal auth methods (such as LDAP). The AWS IAM credentials are time-based and are automatically revoked when the Vault lease expires.
Vault supports three different types of credentials to retrieve from AWS:
iam_user
: Vault will create an IAM user for each lease, attach the managed and inline IAM policies as specified in the role to the user, and if a permissions boundary is specified on the role, the permissions boundary will also be attached. Vault will then generate an access key and secret key for the IAM user and return them to the caller. IAM users have no session tokens and so no session token will be returned. Vault will delete the IAM user upon reaching the TTL expiration.assumed_role
: Vault will call sts:AssumeRole and return the access key, secret key, and session token to the caller.federation_token
: Vault will call sts:GetFederationToken passing in the supplied AWS policy document and return the access key, secret key, and session token to the caller.
Setup
Most secrets engines must be configured in advance before they can perform their functions. These steps are usually completed by an operator or configuration management tool.
Enable the AWS secrets engine:
By default, the secrets engine will mount at the name of the engine. To enable the secrets engine at a different path, use the
-path
argument.Configure the credentials that Vault uses to communicate with AWS to generate the IAM credentials:
Internally, Vault will connect to AWS using these credentials. As such, these credentials must be a superset of any policies which might be granted on IAM credentials. Since Vault uses the official AWS SDK, it will use the specified credentials. You can also specify the credentials via the standard AWS environment credentials, shared file credentials, or IAM role/ECS task credentials. (Note that you can't authorize vault with IAM role credentials if you plan on using STS Federation Tokens, since the temporary security credentials associated with the role are not authorized to use GetFederationToken.)
Notice: Even though the path above is
aws/config/root
, do not use your AWS root account credentials. Instead generate a dedicated user or role.Configure a Vault role that maps to a set of permissions in AWS as well as an AWS credential type. When users generate credentials, they are generated against this role. An example:
This creates a role named "my-role". When users generate credentials against this role, Vault will create an IAM user and attach the specified policy document to the IAM user. Vault will then create an access key and secret key for the IAM user and return these credentials. You supply a user inline policy and/or provide references to an existing AWS policy's full ARN and/or a list of IAM groups:
For more information on IAM policies, please see the AWS IAM policy documentation.
Usage
After the secrets engine is configured and a user/machine has a Vault token with the proper permission, it can generate credentials.
Generate a new credential by reading from the
/creds
endpoint with the name of the role:Each invocation of the command will generate a new credential.
Unfortunately, IAM credentials are eventually consistent with respect to other Amazon services. If you are planning on using these credential in a pipeline, you may need to add a delay of 5-10 seconds (or more) after fetching credentials before they can be used successfully.
If you want to be able to use credentials without the wait, consider using the STS method of fetching keys. IAM credentials supported by an STS token are available for use as soon as they are generated.
Rotate the credentials that Vault uses to communicate with AWS:
Note: Due to AWS eventual consistency, after calling the
aws/config/rotate-root
endpoint, subsequent calls from Vault to AWS may fail for a few seconds until AWS becomes consistent again. See the AWS secrets engine API for further information onconfig/rotate-root
functionality.
Example IAM policy for Vault
The aws/config/root
credentials need permission to manage dynamic IAM users.
Here is an example AWS IAM policy that grants the most commonly required
permissions Vault needs:
Vault also supports AWS Permissions Boundaries when creating IAM users. If you wish to enforce that Vault always attaches a permissions boundary to an IAM user, you can use a policy like:
where the "iam:PermissionsBoundary" condition contains the list of permissions boundary policies that you wish to ensure that Vault uses. This policy will ensure that Vault uses one of the permissions boundaries specified (not all of them).
STS credentials
The above demonstrated usage with iam_user
credential types. As mentioned,
Vault also supports assumed_role
and federation_token
credential types.
STS federation tokens
Notice: Due to limitations in AWS, in order to use the federation_token
credential type, Vault must be configured with IAM user credentials. AWS
does not allow temporary credentials (such as those from an IAM instance
profile) to be used.
An STS federation token inherits a set of permissions that are the combination (intersection) of four sets of permissions:
- The permissions granted to the
aws/config/root
credentials - The user inline policy configured in the Vault role
- The managed policy ARNs configured in the Vault role
- An implicit deny policy on IAM or STS operations.
Roles with a credential_type
of federation_token
can specify one or more of
the policy_document
, policy_arns
, and iam_groups
parameters in the Vault
role.
The aws/config/root
credentials require IAM permissions for
sts:GetFederationToken
and the permissions to delegate to the STS
federation token. For example, this policy on the aws/config/root
credentials
would allow creation of an STS federated token with delegated ec2:*
permissions (or any subset of ec2:*
permissions):
An ec2_admin
role would then assign an inline policy with the same ec2:*
permissions.
The policy.json file would contain an inline policy with similar permissions,
less the sts:GetFederationToken
permission. (We could grant
sts:GetFederationToken
permissions, but STS attaches attach an implicit deny
that overrides the allow.)
To generate a new set of STS federation token credentials, we simply write to the role using the aws/sts endpoint:
STS AssumeRole
The assumed_role
credential type is typically used for cross-account
authentication or single sign-on (SSO) scenarios. In order to use an
assumed_role
credential type, you must configure outside of Vault:
- An IAM role
- IAM inline policies and/or managed policies attached to the IAM role
- IAM trust policy attached to the IAM role to grant privileges for Vault to assume the role
assumed_role
credentials offer a few benefits over federation_token
:
- Assumed roles can invoke IAM and STS operations, if granted by the role's IAM policies.
- Assumed roles support cross-account authentication
- Temporary credentials (such as those granted by running Vault on an EC2
instance in an IAM instance profile) can retrieve
assumed_role
credentials (but cannot retrievefederation_token
credentials).
The aws/config/root
credentials must have an IAM policy that allows sts:AssumeRole
against the target role:
You must attach a trust policy to the target IAM role to assume, allowing the aws/root/config credentials to assume the role.
When specifying a Vault role with a credential_type
of assumed_role
, you can
specify more than one IAM role ARN. If you do so, Vault clients can select which
role ARN they would like to assume when retrieving credentials from that role.
Further, you can specify both a policy_document
and policy_arns
parameters;
if specified, each acts as a filter on the IAM permissions granted to the
assumed role. If iam_groups
is specified, the inline and attached policies for
each IAM group will be added to the policy_document
and policy_arns
parameters, respectively, when calling sts:AssumeRole. For an action to be
allowed, it must be permitted by both the IAM policy on the AWS role that is
assumed, the policy_document
specified on the Vault role (if specified), and
the managed policies specified by the policy_arns
parameter. (The
policy_document
parameter is passed in as the Policy
parameter to the
sts:AssumeRole API call, while the policy_arns
parameter is passed in as the
PolicyArns
parameter to the same call.)
Note: When multiple role_arns
are specified, clients requesting credentials
can specify any of the role ARNs that are defined on the Vault role in order to
retrieve credentials. However, when policy_document
, policy_arns
, or
iam_groups
are specified, that will apply to ALL role credentials retrieved
from AWS.
Let's create a "deploy" policy using the arn of our role to assume:
To generate a new set of STS assumed role credentials, we again write to the role using the aws/sts endpoint:
Troubleshooting
Dynamic IAM user errors
If you get an error message similar to either of the following, the root credentials that you wrote to aws/config/root
have insufficient privilege:
If you get stuck at any time, simply run vault path-help aws
or with a subpath for
interactive help output.
STS federated token errors
Vault generates STS tokens using the IAM credentials passed to aws/config
.
Those credentials must have two properties:
- They must have permissions to call
sts:GetFederationToken
. - The capabilities of those credentials have to be at least as permissive as those requested by policies attached to the STS creds.
If either of those conditions are not met, a "403 not-authorized" error will be returned.
See http://docs.aws.amazon.com/STS/latest/APIReference/API_GetFederationToken.html for more details.
Vault 0.5.1 or later is recommended when using STS tokens to avoid validation errors for exceeding the AWS limit of 32 characters on STS token names.
AWS instance metadata timeouts
Affects Vault 1.4 and later
Anytime Vault uses the instance metadata service on an EC2 instance, such as for getting credentials from the instance profile, there may be a delay with the introduction of v2 of the instance metadata service (IMDSv2). The AWS SDK used by Vault first attempts to connect to IMDSv2, and if that times out, it falls back to v1. In Vault 1.4, this timeout can take up to 2 minutes. In Vault 1.5.5 and later, it can take up to 2 seconds with this fix: #10133.
The timeout occurs in situations where there is a proxy between Vault and IMDSv2, and the instance hop limit is set to less than the number of "hops" between Vault and IMDSv2. For example, if Vault is running in docker on an EC2 instance with the instance hop limit set to 1, the AWS SDK client will attempt to connect to IMDSv2, timeout, and fall back to IMDSv1 because of the extra network hop between docker and IMDS.
To avoid the timeout behavior, the hop limit may be adjusted on the underlying EC2 instances. With the docker example, setting the hop limit to 2 will allow the AWS SDK in Vault to connect to IMDSv2 without delay.
API
The AWS secrets engine has a full HTTP API. Please see the AWS secrets engine API for more details.