Dynamic, Synced Configuration
Warning
This content is part of the legacy version of Waypoint that is no longer actively maintained. For additional information on the new vision of Waypoint, check out this blog post and the HCP Waypoint documentation.
Waypoint can sync application configuration values with external systems such as Kubernetes ConfigMaps, HashiCorp Vault, Amazon Parameter Store, and more. These are known as dynamic configuration values. As the values change, Waypoint will automatically update your deployments and restart your application.
Dynamic values make it easy to utilize external configuration stores and the functionality they provide, such as automatic credential rotation in systems such as Vault or Amazon Parameter Store.
This functionality requires the Waypoint entrypoint.
Basic application configuration knowledge required. This
page assumes you know the basics of how to
set static application configuration. This page will
not thoroughly explain the config setting CLI or waypoint.hcl
stanzas.
Configuration Sources
The following is a list of currently available configuration sources and their associated documentation:
- Terraform Cloud - Read Terraform state outputs from Terraform Cloud.
- Kubernetes - Read values from ConfigMaps and Secrets.
- Vault - Read values from Vault secrets, including dynamic secrets such as database credentials.
- AWS SSM - Read configuration values from AWS SSM Parameter Store.
- Consul - Read values from Consul key value storage.
Custom configuration source plugins are coming soon. Currently it's overly difficult to get configuration source plugins into a place that they can be executed. In the future, we will enable this in a straightforward way.
Setting Dynamic Values via waypoint.hcl
Dynamic configuration values are set in the waypoint.hcl
file using
the dynamic
function within the same config
stanza as
static configurations. This specifies that a configuration value is
synced with an external system dynamically.
The example above syncs the PORT
environment variable with the "port"
key in the "my-config-map" Kubernetes ConfigMap. The "sync" actively watches
the defined source, updates the environment variable whenever the value
changes, and restarts your application.
The dynamic
function has the following signature:
The SOURCE
parameter is a string specifying the config source to sync
with and PARAMS
is a map of parameters for that config source. The set
of parameters is dependent on the source being used. The list of sources
supported is in the Configuration Sources section.
Unsetting Dynamic Values
To unset a dynamic configuration value, delete it from the waypoint.hcl
file and then use the CLI to set it to an empty string as shown below.
Deleting the key from the Waypoint file and syncing
it will not be enough.
Source Settings
Configuration sources sometimes support global settings for their behavior or
require global settings to be set for them to function at all.
For example, Vault has some global
settings around how to authenticate. Without setting these, Vault configurations
in applications will not work. These source settings are set at the server
level and used by all dynamic
calls.
To set configuration source settings, the waypoint config source-set
CLI command is used:
The documentation for the source settings available to a configuration source are alongside their parameters in the plugin documentation.
Scoped Source Settings
The waypoint config source-set
command's -scope
flag will configure a given
config source to be used within a specific scope. The supported values are below.
For a given app or project, the config source with the "most specific" scope
will be used.
- global: Source setting applied to all projects and their applications
- project: Applies source setting only to this project
- app: Applies source setting only to this application
For example, you may have three config sources configured in your Waypoint server,
one at the global
scope, one at the project
scope for project foo
, and one
at the app
scope for app bar
, in project foo
. These have been configured
with the commands below, for the Vault plugin. The Vault token set for each of
these config sources is tailored to each scope, with only the necessary set of
permissions for each.
In accordance with the scopes set, waypoint config source-get
will return the
config source corresponding with the specified scope
.
In the third of the series of commands above, were the app-scoped config source absent, the project-scoped source would be returned. Were the project-scoped config source absent as well, the global-scoped config source would be returned.
The Waypoint Entrypoint will always attempt to retrieve configuration for the scope of the deployed app.
In addition to the global
, project
, and app
scopes, when setting a config
source, it may also be scoped to a specific workspace. This can be achieved by
using the -workspace
(or -w
) flag. The config source set for a given workspace
will be used only for operations and app instances deployed within that workspace.
Specifying workspace-scoped config sources is especially useful when your
Waypoint server is used for releasing applications to multiple environments. A
Vault config source for the dev
workspace may have a different address or
less restrictive credentials to be used than deployments in the prod
workspace,
for example.
Deleting Configuration Source Settings
To unset the configuration after it has been set with waypoint config source-set
, specify the -delete
flag.
Debugging Dynamic Configuration
If a configuration value is not available (the environment variable is not
set or the value is not what you expect), then the best debugging tool
is waypoint logs
. The logs command will show entrypoint logs where
you will see any errors related to configuration syncing.
The waypoint logs
output below is from an application deployed with
Waypoint using Kubernetes ConfigMaps but deployed within a Nomad cluster. The
lines have been purposely shortened slightly to improve readability on the
website.
You can see the first line has an error reading configuration value. If
you keep scrolling right, you can see the source
is kubernetes
and the
err
is shown. In this case, it's telling us that it couldn't load any
in-cluster configuration. Upon thinking about this, we can realize that
we didn't deploy to a Kubernetes cluster so this error makes sense, and perhaps
we need to do something else to enable out-of-cluster authentication.
To iterate on configuration, change the waypoint.hcl
file and run
waypoint config sync
. This allows you to change configuration without
doing a full redeploy.
Listing Environment Variables
If you don't see any errors in the log, it may be possible the environment
variables are set correctly. To verify that environment variables are
set, open a shell with waypoint exec
and run env
. This command will
list the environment variables that are available to your application.
Note: the availability of /bin/sh
or env
is dependent on your application.
Most deployments support these tools but if your application is built
using a base Docker image or some other mechanism that doesn't include these,
then it is possible they are not there.