Learn to use the Vault HTTP API

All Vault's capabilities are accessible using the HTTP API. Calls from the CLI invoke the HTTP API and sometimes, Vault features are only accessible using the HTTP API.

Scenario

Oliver administers Vault at HashiCups. Part of Oliver's job is to create logins and passwords for developers at HashCups to login to Vault.

Danielle the developer needs to be able to login to Vault to create secrets used by the HashiCups application. Oliver will enable the userpass auth method, create a user, set a password, create and attach a policy to the user. Then Oliver creates a new secrets engine. Danielle then logs in, and creates their first secret.

For administrative and development tooling they both need to be able to use the HTTP API for their tasks.

Prerequisites

To complete this tutorial, you need the following:

• Vault binary installed and configured in your system PATH.
• curl to use the API command examples.
• jq to format JSON output.

You will use cURL to make requests to the HTTP API. Responses are JSON formatted and can be very long. jq helps make sure that the API responses are easier to read.

Set up the lab

1. Open a new terminal and start a Vault dev server with the literal string root as the root token value, and enable TLS.

   $ vault server -dev -dev-root-token-id root -dev-tls
   

   Copy

   The dev server listens on the loopback interface at 127.0.0.1 on TCP port 8200 with TLS enabled. At runtime, the dev server also automatically unseals, and prints the unseal key and initial root token values to the standard output.

   Root tokens

   The dev mode server starts with an initial root token value set.

   Root token use should be extremely guarded in production environments because they enable full access to the Vault server.

   You use the root token here for convenience, and to keep the tutorial steps focused on what you'll learn.

2. In a new terminal, export the VAULT_ADDR and VAULT_CACERT environment variables using the commands suggested in your Vault dev server output.

   Copy each command (without the $) from the server output, and paste it into the new terminal session.

   Example:

   $ export VAULT_ADDR='https://127.0.0.1:8200'
   

   Example:

   $ export VAULT_CACERT='/var/folders/qr/zgztx0sj6n1dxy86sl36ntnw0000gn/T/vault-tls3037226588/vault-ca.pem'
   

   Remember to use your dev server's values, not the examples shown here.

3. Export an environment variable for the vault CLI to authenticate with the Vault server.

   $ export VAULT_TOKEN=root
   

   Copy

   The Vault server is ready.

Check the server status

Oliver can use Vault's HTTP API for all interactions. They will start with checking the status of the Vault seal.

1. Set an environment variable CURL_CA_BUNDLE, CURL uses that locate a CA cert.

   $ export CURL_CA_BUNDLE=$VAULT_CACERT
   

   Copy

2. Use the HTTP API to check the server status.

   $ curl -s $VAULT_ADDR/v1/sys/seal-status | jq
   {
     "type": "shamir",
     "initialized": true,
     "sealed": false,
     "t": 1,
     "n": 1,
     "progress": 0,
     "nonce": "",
     "version": "1.15.5",
     "build_date": "2024-01-26T14:53:40Z",
     "migration": false,
     "cluster_name": "vault-cluster-cfe1cb9c",
     "cluster_id": "f507fd9f-89c4-e1f4-69ec-6919b0f9307b",
     "recovery_seal": false,
     "storage_type": "inmem"
   }
   

   Copy

   Use curl to send an HTTP GET request for the status of the Vault instance. This is an unauthenticated request, and does not require a client token. Vault responds to your request with a JSON based response.

Headers and paths

HTTP headers let the client and the server pass information with an HTTP request or response.

Almost every operation in HashiCorp Vault requires a client token. Set the X-Vault-token header with the token.

In this tutorial, in the shell you set the VAULT_TOKEN environment variable with the client token. That value is then sent in the X-Vault-token.

See the HTTP API docs for optional headers.

To see the curl equivalent of the CLI command to enable userpass auth method, use the -output-curl-string flag.

$ vault status -output-curl-string
curl -H "X-Vault-Token: $(vault print token)" -H "X-Vault-Request: true" http://localhost:8200/v1/sys/seal-status

Also required for HTTP calls is an address. The path after the address Vault is located on and the HTTP action determine what Vault does, and understanding them is important to effectively using Vault. In this tutorial, the address is http://localhost:8200 which is stored in VAULT_ADDR environment variable.

For example, in the health check above it hits $VAULT_ADDR/v1/sys/seal-status endpoint. So reading from the /v1/sys/seal-status path returns the status information.

Interestingly, using the CLI you can vault read against the same paths for the information.

$ vault read /sys/seal-status
Key              Value
---              -----
build_date       2024-01-26T14:53:40Z
cluster_id       4e4d0082-43e0-4636-223a-4b5e6d1e31f8
cluster_name     vault-cluster-1193e5a7
initialized      true
migration        false
n                1
nonce            n/a
progress         0
recovery_seal    false
sealed           false
storage_type     inmem
t                1
type             shamir
version          1.15.5

What parameter is used to pass a token to Vault?

Enable and configure an auth method

(Persona: operations)

Developers like Danielle need to authenticate with Vault in order to create secrets to be used by their applications and web services. Oliver will set up an authentication method that humans can use called userpass.

To enable an auth method with the CLI, you use the following command:

$ vault auth enable <auth_method_type>

1. This is how to enable the userpass auth method by invoking the Vault HTTP API.

   $ curl \
     -H "X-Vault-Token: $VAULT_TOKEN" \
     -X POST \
     -d '{"type": "userpass"}' \
     $VAULT_ADDR/v1/sys/auth/userpass
   

   Copy

   The HTTP API requires a POST against a new URL ending with the name of the auth method, with JSON data specifying the type.

   Notice that the request to enable the userpass endpoint needed an authentication token. In this case you are passing the root token generated when you started the Vault server. You can also generate tokens using any other authentication mechanisms, but initially you will use the root token for simplicity.

   Vault responds to this command with an HTTP 204 status code, but does not include any response data.

2. To determine if it was successful, check the terminal running Vault for a log entry resembling:

   [INFO]  core: core: enabled credential backend: path=userpass/ type=userpass version=""
   

   An alternative is to use the HTTP API list the enabled authentication engines.

   $ curl \
     -H "X-Vault-Token: $VAULT_TOKEN" \
     $VAULT_ADDR/v1/sys/auth  | jq ".data"
   

   Copy

   Example output:

   {
     "token/": {
       ...
     },
     "userpass/": {
       "accessor": "auth_userpass_0da1ad62",
       "config": {
         "default_lease_ttl": 0,
         "force_no_cache": false,
         "max_lease_ttl": 0,
         "token_type": "default-service"
       },
       "deprecation_status": "supported",
       "description": "",
       "external_entropy_access": false,
       "local": false,
       "options": {},
       "plugin_version": "",
       "running_plugin_version": "v1.15.5+builtin.vault",
       "running_sha256": "",
       "seal_wrap": false,
       "type": "userpass",
       "uuid": "5031c2b8-b766-6a2d-4c3c-582f8c7c3246"
     }
   }
   

3. Create an user in userpass.

   $ curl \
     -H "X-Vault-Token: $VAULT_TOKEN" \
     -X POST \
     -d '{"password":"Imprint Bacteria Marathon Aflutter","token_policies":"developer-vault-policy"}' \
     $VAULT_ADDR/v1/auth/userpass/users/danielle-vault-user
   

   Copy

   You should not expect any output from this command.

4. This user has a token_policy named developer-vault-policy and default. Vault has a default policy already created, and next you create ACL policies named developer-vault-policy. Use the /sys/policies/acl endpoint to create the policy via Vault API.

   $ curl \
       -H "X-Vault-Token: $VAULT_TOKEN" \
       -X PUT \
       -d '{"policy":"path \"dev-secrets/data/creds\" {\n  capabilities = [\"create\", \"update\"]\n}\n\npath \"dev-secrets/data/creds\" {\n  capabilities = [\"read\"]\n}\n"}' \
       $VAULT_ADDR/v1/sys/policies/acl/developer-vault-policy
   

   Copy

   You should not expect any output from this command.

5. List the Vault policies, and look for developer-vault-policy.

   $ curl -s -H "X-Vault-Token: $VAULT_TOKEN" $VAULT_ADDR/v1/sys/policy | jq ".data.policies"
   [
     "default",
     "developer-vault-policy",
     "root"
   ]
   

   Copy

Enable and configure a secrets engine

(Persona: operations)

Oliver will create a static secrets engine.

The policy developer-vault-policy expects dev-secrets/data path to exist. Starting the Vault server in development mode creates a key/value version 2 secrets engine at secrets/, and for the development team Oliver will create the dev-secrets secrets engine.

1. With this command, check which secrets engines are already mounted.

   $ curl -s \
     -H "X-Vault-Token: $VAULT_TOKEN" \
     $VAULT_ADDR/v1/sys/mounts | jq ".data"
   

   Copy

   Example output:

   {
     "cubbyhole/": {
       ...
     },
     "identity/": {
       ...
     },
     "secret/": {
       "accessor": "kv_49adaf9a",
       "config": {
         "default_lease_ttl": 0,
         "force_no_cache": false,
         "max_lease_ttl": 0
       },
       "deprecation_status": "supported",
       "description": "key/value secret storage",
       "external_entropy_access": false,
       "local": false,
       "options": {
         "version": "2"
       },
       "plugin_version": "",
       "running_plugin_version": "v0.16.1+builtin",
       "running_sha256": "",
       "seal_wrap": false,
       "type": "kv",
       "uuid": "3e67cdab-9fa2-0c1b-c689-f7eef8e5358c"
     },
     "sys/": {
       ...
     }
   }
   

2. Create a secrets engine at dev-secrets.

   $ curl \
       -H "X-Vault-Token: $VAULT_TOKEN" \
       -X POST \
       -d '{ "type":"kv-v2" }' \
       $VAULT_ADDR/v1/sys/mounts/dev-secrets
   

   Copy

   Vault responds to this command with an HTTP 204 status code, but does not include any response data.

Authenticate and create secrets

(Persona: development)

Oliver has created the login for Danielle, and set up a secrets engine. Danielle logs into Vault to create a static secret.

1. The following command logins in danielle-vault-user and retrieves the client-token.

   $ curl -s \
     -X POST \
     -d '{ "password": "Imprint Bacteria Marathon Aflutter" }' \
     $VAULT_ADDR/v1/auth/userpass/login/danielle-vault-user | jq ".auth.client_token"
   

   Copy

   You will get the client token under the auth.client_token key.

   "hvs.CAESIAs4W9No9XWCOPxY_G6fgxNLGO8CiO1pCayO0cuYlGSwGh4KHGh2cy13T0FHMInvAliDhycHhXNUEwSE9ZaE8"
   

   The Vault returns the client token (hvs.CAESIAs4W9No9XWCOPxY...) used to authenticate with Vault. This token has specific capabilities on all the resources encompassed by the developer-vault-policy.

2. Export the newly acquired client token as the DANIELLE_DEV_TOKEN environment variable value and used to authenticate later Vault requests.

   Copy the command (without the $), and paste it into the new terminal session.

   Example:

   $ export DANIELLE_DEV_TOKEN="hvs.CAESIAs4W9No9XWCOPxY_G6fgxNLGO8CiO1pCayO0cuYlGSwGh4KHGh2cy13T0FHMInvAliDhycHhXNUEwSE9ZaE8"
   

   Remember to use your login's values, not the examples shown here.

   When using the token in DANIELLE_DEV_TOKEN, HTTPS connections to Vault will interact with it with the capabilities defined by the policy developer-vault-policy.

3. Create a secret named creds with a key password and its value set to Driven Siberian Pantyhose Equinox.

   $ curl -s \
     -H "X-Vault-Token: $DANIELLE_DEV_TOKEN" \
     -X PUT \
     -d '{ "data": {"password": "Driven Siberian Pantyhose Equinox"} }' \
     $VAULT_ADDR/v1/dev-secrets/data/creds | jq ".data"
   

   Copy

   Example output:

   {
     "created_time": "2024-02-29T18:22:10.162571Z",
     "custom_metadata": null,
     "deletion_time": "",
     "destroyed": false,
     "version": 1
   }
   

4. Display the secret at dev-secrets/data/creds.

   $ curl -s \
     -H "X-Vault-Token: $DANIELLE_DEV_TOKEN" \
     $VAULT_ADDR/v1/dev-secrets/data/creds | jq ".data"
   

   Copy

   Example output:

   {
     "data": {
       "password": "Driven Siberian Pantyhose Equinox"
     },
     "metadata": {
       "created_time": "2024-03-05T20:04:04.073324Z",
       "custom_metadata": null,
       "deletion_time": "",
       "destroyed": false,
       "version": 1
     }
   }
   

Clean up

1. Unset the VAULT_TOKEN, VAULT_ADDR and the DANIELLE_DEV_TOKEN environment variable.

   $ unset VAULT_TOKEN && unset VAULT_ADDR && unset DANIELLE_DEV_TOKEN
   

   Copy

2. Use CTRL+C to stop the server process in the terminal window where you started the server, or use this command to kill the server process from any local terminal session:

   $ pkill vault
   

   Copy

Summary

All interactions with Vault use the Vault HTTP API. When you use the Vault CLI or UI, those interfaces are using the HTTP API. You can interact with the HTTP API through common utilities such as cURL or using programming languages such as Python (fetch, httpx) or NodeJS (https.get, https.request).