Deploy seamless canary deployments with service splitters
Canary deployments (rolling deployments) let you release new software gradually, and identify and reduce the potential blast radius of a failed software release. This allows you to release new software with near-zero downtime. With canary deployments, you first route a small fraction of the service to the new version. When you confirm there are no errors, you slowly increase traffic to the new service until you fully promote the new environment.
In this tutorial, you will upgrade a demo application's service to a new version using the canary deployment strategy. In the process, you will learn how to use service resolvers and service splitters.
Scenario overview
HashiCups is a coffee shop demo application. It has a microservices architecture and uses Consul service mesh to securely connect the services. At the beginning of this tutorial, you will deploy the HashiCups microservices (nginx
, frontend
, public-api
, product-api
, product-db
, and payments
) to Consul service mesh. Unlike a typical HashiCups deployment, you will also deploy a second version of the frontend
service named frontend-v2
.
Next, you will use service resolvers to create service subset, one for each version (v1
and v2
). Then, you will use service splitters to progressive route more traffic to the new version of.
Prerequisites
The tutorial assumes that you are familiar with Consul and its core functionality. If you are new to Consul, refer to the Consul Getting Started tutorials collection.
For this tutorial, you will need:
- An AWS account configured for use with Terraform
- kubectl >= 1.27
- aws-cli >= 2.8
- terraform >= 1.4
- consul-k8s v1.2.0
- helm >= 3.0
Clone GitHub repository
Clone the GitHub repository containing the configuration files and resources.
Change into the directory with the newly cloned repository.
This repository contains Terraform configuration to spin up the initial infrastructure and all files to deploy Consul, the sample application, and the resources required for a canary deployment.
Here, you will find the following Terraform configuration:
eks-cluster.tf
defines Amazon EKS cluster deployment resourcesoutputs.tf
defines outputs you will use to authenticate and connect to your Kubernetes clusterproviders.tf
defines AWS and HCP provider definitions for Terraformvariables.tf
defines variables you can use to customize the tutorialvpc.tf
defines the AWS VPC resources
Additionally, you will find the following directories:
api-gw
contains the Kubernetes custom resource definitions (CRDs) required to deploy and configure the API gateway resourcesconsul
contains the Helm chart that configures your Consul instancehashicups
contains the Kubernetes definitions that deploys HashiCups, the sample applicationk8s-yamls
contains the service resolvers and service splitters Kubernetes CRDs that you will use for a canary deployment
Deploy infrastructure, Consul, and sample applications
Initialize your Terraform configuration to download the necessary providers and modules.
Then, create the infrastructure. Confirm the run by entering yes
. This will take about 15 minutes to deploy your infrastructure. Feel free to explore the next sections of this tutorial while waiting for the resources to deploy.
Configure your terminal to communicate with EKS
Now that you have deployed the Kubernetes cluster, configure kubectl
to interact with it.
Install Consul
You will now deploy Consul on your Kubernetes cluster with consul-k8s
. By default, Consul deploys into its own dedicated namespace (consul
). The Consul installation will use the Consul Helm chart file in the consul
directory.
Deploy Consul and confirm the installation with a y
.
Verify that you have installed Consul by inspecting the Kubernetes pods in the consul
namespace.
Deploy sample application
Now that your Consul service mesh is operational in your cluster, deploy HashiCups, the sample application you will use to explore canary deployments with Consul.
Deploy the HashiCups.
Check the pods to make sure they are all up and running. Notice there are two frontend
services.
Deploy API Gateway
In addition, deploy an API gateway. This lets you access HashiCups directly through a dedicated external IP address without any additional commands or configuration.
The following is the expected output:
Verify you have deployed API Gateway. You should find an output similar to the following.
Export the API gateway external IP address. You will reference this URL in the next sections to confirm you have configured the service splitters for canary deployments.
Configure your CLI to interact with Consul datacenter
In this section, you will set environment variables in your terminal so you can interact with your Consul datacenter.
Retrieve the ACL bootstrap token from the respective Kubernetes secret and set it as an environment variable.
Set the Consul destination address. By default, Consul runs on port 8500
for http
and 8501
for https
.
Review frontend services
The end-to-end deployment deploys two versions of the frontend service, one for v1
and the other for v2
.
Open hashicups/frontend-v1.yaml
. Notice that this deployment definition specifies a v1
tag and sets NEXT_PUBLIC_FOOTER_FLAG
to HashiCups-v1
. When you open the HashiCups UI, the footer will show this value. This will help you differentiate between the two frontend versions.
Open hashicups/frontend-v2.yaml
. Notice how all the values are the same except for the service tags (v2
) and the NEXT_PUBLIC_FOOTER_FLAG
value.
Confirm that there is only one Kubernetes frontend
service.
Retrieve the frontend service instances' Consul metadata. Each one will have a different version (v1
and v2
).
Run the following command to confirm two versions of the frontend service.
The response should be similar to the following:
Create service subsets with service resolvers
Service resolvers define how Consul can satisfy a discovery request for a given service name. For example, with service resolvers, you can:
- configure service subsets based on the instance's metadata
- redirect traffic to another service (redirect)
- control where to send traffic if frontend is unhealthy (failover)
- route traffic to the same instance based on a header (consistent load balancing)
Refer to the service resolver documentation for a full list of use cases and configuration parameters.
For canary deployments, you need to define a service resolver that creates service subsets based on the instance's version.
Inspect k8s-yaml/frontend-resolver.hcl
. This defines a service resolver named frontend
with two subsets (v1
and v2
) based on the instance's metadata (Service.Meta.version
). In addition, it specifies v1
as the default subset so Consul will send all traffic to the v1
frontend service.
Create the service resolver.
Confirm traffic goes to v1
only.
Open the frontend service's Routing page in the Consul dashboard. Notice the UI shows that the service splitter will route incoming traffic to the v1 frontend
service subset.
Send partial traffic to new service
Service splitters enable you to easily implement canary tests by splitting traffic across services and subsets. Refer to the service splitter documentation for a full list of use cases and configuration parameters.
Inspect k8s-yamls/frontend-splitter-50-50.yaml
. This defines a service splitter for the frontend service, routing 50% of the traffic to v1
subset and the rest to v2
.
Create the service resolver.
Confirm traffic goes to both v1
and v2
. There might not be an exact 50/50 split since you are only sending 10 requests.
Alternatively, retrieve the HashiCups URL and open it in your browser to confirm Consul splits traffic between the two versions.
Find the version at the footer. Refresh the page several times — you will find the footer will show Hashicups - v2
50% of the time.
Open the frontend service's Routing page in the Consul dashboard. Notice the UI shows that the service splitter will route incoming traffic to both the v1 and v2 frontend
service subset.
Send all traffic to new service
Now that the canary deployment was successful, you will route 100% of the traffic to v2 to fully promote the frontend service.
Inspect k8s-yamls/frontend-splitter-v2-only.yaml
. This defines a service splitter for the frontend service, routing all the to v2
.
Create the service resolver.
Confirm traffic goes to v2
only.
Configure future canary deployments
For future canary deployments, you will need to modify the service resolver to define the new service subset. If no services reference the old subset, we recommend you remove it from the service resolver to keep your environments clean. Remember to update the defaultSubset
if you do prune old service subsets to ensure your service points to the right version.
The following is an example of an updated service resolver, where x
represents the current version and y
represents the next version.
Once you do this, you need to update your service splitter to reference the new service subsets.
Clean up environment
Destroy the Terraform resources to clean up your environment. Enter yes
to confirm the destroy operation.
Due to race conditions with the various cloud resources created in this tutorial, you may need to run the destroy
operation twice to ensure all resources have been properly removed.
Next steps
In this tutorial, you upgraded a service using canary deployments before fully promoting the new service. This lets you release new versions gradually, allowing you to identify and address any potential blast radius from a failed software release. In the process, you learned how to use Consul's service splitter and service resolvers.
To learn more about service resolvers, refer to the service resolver documentation for a full list of use cases and configuration parameters.
To learn more about service splitters, refer to the service splitters documentation for a full list of use cases and configuration parameters.