Register your services to Consul
In the previous tutorial, you deployed a Consul server with all security features enabled and explored how to use Consul as a KV store and DNS server.∂ç
Consul's value shines when you use them with Consul clients to serve as a distributed health monitoring platform for your services and as a centralized service catalog.
In this tutorial, you will deploy Consul client agents to your virtual machine (VM) workloads. Then, you will register the services to the Consul catalog and setup a distributed monitoring system using Consul health checks.
In this tutorial, you will use a test environment to:
- Configure Consul client agents for the different VMs
- Start Consul client instances on your workload VMs
- Configure your terminal to communicate with the Consul datacenter
- Verify Consul datacenter members
- Query Consul catalog using CLI, API and DNS interfaces
- Modify a service definition and update the service in Consul catalog
Note
This tutorial is part of the Get Started collection, for this reason all the steps used to configure Consul agents and services are shown and require to be executed manually. If you are setting up a production environment you should codify and automate the installation and deployment process. Refer to the VM production patterns tutorial collection for Consul production deployment best practices.
Tutorial scenario
This tutorial uses HashiCups, a demo coffee shop application made up of several microservices running on VMs.
At the beginning of the tutorial, you have an instance of HashiCups running on four VMs and one Consul server (you deployed this in the previous tutorial).
By the end of this tutorial, you will have deployed and started a Consul client agent on each VMs that hosts HashiCups. In addition, you will have registered the HashiCups services in the Consul service catalog and setup health checks for each service.
Prerequisites
This tutorial uses an interactive lab to guide you through how to deploy Consul client agents on the VMs hosting your services. The lab environment includes all required binaries and sample configurations.
Launch Terminal
This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.
Verify Consul binary in service servers
Verify that the VMs you want to deploy the Consul agents on have the Consul binary.
Configure environment
This tutorial and interactive lab environment uses scripts in the tutorial's GitHub repository to generate the Consul configuration files for your client agents.
The interactive lab environment includes these scripts and the pre-configured Consul server. In **Bastion Host**, list the files in the `ops` directory.
The scripts rely on default parameters to generate the configuration files. Set the following default values. Ensure you have permission to write in the specified paths.
Tip
We suggest using an unprivileged user to run Consul for security reasons.
The lab contains two environment files that will help you configure the terminal and generate the required configuration files.
Import the two files in your environment.
The script creates all the files in a destination folder. Export the path where you wish to create the configuration files for the scenario.
Make sure the folder exists.
Verify your Consul CLI can interact with your Consul server.
Generate Consul clients configuration
The interactive lab can generate the configuration using the info provided via the environment files previously loaded.
Since the Consul datacenter is configured with ACL enabled by default, you will need to define the ACL tokens you want to pass to the Consul clients when the configuration gets created.
First, export the token you generated for DNS so you can use it as default token for clients.
Then, for Consul service definition to be created properly, define a Consul token to be included in the service definition file.
In this example, you will use the bootstrap token.
Generate configuration for Database agent
First, define the Consul node name.
Then, generate the Consul configuration.
To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.
Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.
First, generate the service configuration for the agent.
Then, copy the service definition file into the general configuration directory.
Generate configuration for API agent
First, define the Consul node name.
Then, generate the Consul configuration.
To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.
Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.
First, generate the service configuration for the agent.
Then, copy the service definition file into the general configuration directory.
Generate configuration for Frontend agent
First, define the Consul node name.
Then, generate the Consul configuration.
To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.
Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.
First, generate the service configuration for the agent.
Then, copy the service definition file into the general configuration directory.
Generate configuration for NGINX agent
First, define the Consul node name.
Then, generate the Consul configuration.
To complete Consul agent configuration, you need to setup tokens for the client. For this tutorial, you are using the bootstrap token. We recommend you create more restrictive tokens for the client agents in production.
Once Consul agent configuration is generated, you can add the service configuration to the output folder. This will make sure your services are configured automatically at Consul startup.
First, generate the service configuration for the agent.
Then, copy the service definition file into the general configuration directory.
Check generated files
Once you have generated the configuration files, your directory should look like the following:
Each directory corresponds to a Consul client configuration for the respective node.
The scripts generated multiple configuration files so it is easier to read and tune them for your environment. The following are the generated files and a description of what they do:
- The
agent-acl-tokens.hcl
file contains tokens for the Consul agent. - The
agent-gossip-encryption.hcl
file configures gossip encryption. - The
consul-agent-ca.pem
file is the public certificate for Consul CA. - The
consul.hcl
file contains node specific configuration and it is needed, with this specific name, if you want to configure Consul as a systemd daemon. - The
svc-hashicups-*.hcl
files contains the service definition for the services that need to be registered on each node.
Visit the agent configuration and service configuration documentation to interpret the files or to modify them when applying them to your environment.
Copy configuration on client VMs
After the script generates the client configuration, you will copy these files into the respective Consul configuration directories in each client node.
Tip
In the lab environment, the HashiCups application nodes have a running SSH
server. As a result, you can use ssh
and scp
commands to perform the
following operations. If the nodes in your personal environment does not have
an SSH server, you may need to use a different approach to create the
configuration directories and copy the files.
First, define the Consul configuration directory.
Then, remove existing configuration from the VMs.
Finally, copy the configuration files into the different VMs.
Start Consul on client VMs
Now that you have copied the configuration files to each client VMs, you will start the Consul client agent on each VM.
Setup Database Consul client agent
Select the tab that corresponds with the service — in this case, Database.
Define the Consul configuration and data directories.
Ensure your user has write permission to the Consul data directory.
Finally, start the Consul server process.
The command starts the Consul server in the background to not lock the terminal.
You can access the Consul server log through the /tmp/consul-server.log
file.
Setup API Consul client agent
Select the tab that corresponds with the service — in this case, API.
Define the Consul configuration and data directories.
Ensure your user has write permission to the Consul data directory.
Finally, start the Consul server process.
The process is started in background to not lock the terminal. Consul server log
can be accessed in the /tmp/consul-client.log
file.
Setup Frontend Consul client agent
Select the tab that corresponds with the service — in this case, Frontend.
Define the Consul configuration and data directories.
Ensure your user has write permission to the Consul data directory.
Finally, start the Consul server process.
The process is started in background to not lock the terminal. Consul server log
can be accessed in the /tmp/consul-client.log
file.
Setup NGINX Consul client agent
Select the tab that corresponds with the service — in this case, NGINX.
Define the Consul configuration and data directories.
Ensure your user has write permission to the Consul data directory.
Finally, start the Consul server process.
The process is started in background to not lock the terminal. Consul server log
can be accessed in the /tmp/consul-client.log
file.
Verify Consul datacenter members
After you started all Consul agents, verify they successfully joined the Consul datacenter. If you are using the interactive lab environment, go to the Bastion Host tab.
Retrieve the agents in the Consul datacenter.
Query services in Consul catalog
When you started the Consul client agents, they registered the service running on their node into the Consul catalog. Each service definition also contained the service's health check. You can find the service definition files in each node's respective Consul configuration file you defined earlier.
Query the healthy services using the Consul CLI, API, or DNS.
Use the Consul CLI to query the service catalog.
Modify service definition tags
When using Consul CLI or the API endpoints, Consul will also show you the
metadata associated with the services. In this tutorial, you registered each
service with the v1
tag.
In this section, you will update the database service definition to learn how to update Consul service definitions. You must run these commands on the virtual machine that hosts the services.
Select the tab that corresponds with the service — in this case, Database.
Create the new configuration file with the following contents. Notice that this configuration adds a
v2
tag to the database service.
Once you have created the new service definition file, update the service in the Consul catalog.
Consul can automatically update some of its configuration by reloading the content of the configuration folder.
To use this feature, move the svc-hashicups-db.hcl
file into the Consul
configuration directory (/etc/consul.d
).
Setup the environment variables to connect with Consul.
Then, use the reload
command to update the service definition.
Query services by tags
After you have updated the database service definition, query it to verify the new tag.
Retrieve the tags associated with each service and verify the new v2
tag for
the database service.
Next steps
In this tutorial, you deployed Consul clients on each HashiCups' nodes VMs. In addition, you configured Consul to perform health checks on the registered services and updated a service definition.
You now have a distributed system to monitor and resolve your services all without changing your services' configuration or implementation. At this stage, you can use Consul to automatically configure and monitor your services. However, they have the same security they had before introducing Consul.
In the next tutorial, you will learn how to introduce zero trust security in your network by implementing Consul service mesh.
For more information about the topics covered in this tutorial, refer to the following resources: