Deploy Terraform Enterprise to Kubernetes
This topic describes how to deploy Terraform Enterprise to Kubernetes. You should have a deep understanding of Kubernetes before deploying Terraform Enterprise to a production Kubernetes environment.
Overview
You should deploy external service dependencies outside the Kubernetes cluster and scale reliably to accommodate Terraform Enterprise workloads. The following diagram shows the Terraform Enterprise architecture when deployed to Kubernetes-orchestrated containers:
Complete the following steps to install Terraform Enterprise:
- Complete the prerequisites.
- Install the Helm chart and apply your override values.
- Complete post installation tasks.
Complete post installation tasks, such creating the initial admin user account.
Prerequisites
Complete the following tasks before attempting to install Terraform Enterprise.
Prepare the deployment environment
Provide a DNS hostname for Terraform Enterprise and the associated TLS certificate. Additionally, you must configure your network so that your host can receive and send traffic. Refer to Prepare the host environment for details about preparing the host environment.
Deploy external storage systems
Deploy the database and other storage devices so that Terraform can connect to them when the application starts. Refer to Data storage settings overview for additional information.
Create the deployment configuration
Create a custom YAML configuration file, for example /tmp/overrides.yaml
, to override the default values in the Terraform Enterprise Helm chart. The file contains settings for the operational mode, license, TLS certificates, and network configuration. Add any additional configurations necessary for your environment. Refer to Configuration file overview for additional information.
Install Terraform Enterprise with Helm
Connect to the host instance.
Log in to the Terraform Enterprise container image registry.
Pull the Terraform Enterprise image from the registry.
Create a custom namespace.
Create an image pull secret in
<TFE_NAMESPACE>
to fetch theterraform-enterprise
container from the<DOCKER_REGISTRY_URL>
. This URL can beimages.releases.hashicorp.com
, or your internal container registry. If you are usingimages.releases.hashicorp.com
, useterraform
as the<DOCKER_REGISTRY_USERNAME>
parameter in the following command with--docker-password=$(cat /path/to/terraform.hclic)
Add the Hashicorp Helm registry:
Render the
terraform-enterprise
chart with your custom values file<OVERRIDES_FILE>
, for exampletmp/overrides.yaml
.Install
terraform-enterprise
, this step can take several minutes.Inspect
terraform-enterprise
pods to verify their successful start.If Terraform Enterprise pods fail to start, refer to Kubernetes Troubleshooting.
By default, Terraform Enterprise installs a load balancer service. Retrieve the external IP address of this service.
Set up a DNS record that points to your external IP address to enable routing to your
<TFE_HOSTNAME>
. A DNS address is required to communicate with Terraform Enterprise, and it is managed outside of Kubernetes and the Terraform Enterprise helm chart or application.Validate the readiness of the Terraform Enterprise application by querying the health check endpoint.
Post installation tasks
Complete the following tasks after the initial installation.
Review startup checks
When you start Terraform Enterprise, several startup checks also run to prevent errors related to invalid configurations or certificates, as well as other issues that could prevent the application from running successfully or safely. Refer to the startup checks reference for additional information.
Create initial admin user
Provision your first administrative user and start using Terraform Enterprise.
Configure the security context
Modify the .securityContext
Helm chart value to set the pod security configuration. Modify the .container.securityContext
Helm chart value to set the container security configuration. You must also omit the allowPrivilegeEscalation
container security context option or set it to true
.
Create a custom Helm chart fork
The Terraform Enterprise Helm Chart is designed to meet the needs of the majority of our users. You can fork our Helm chart and adapt it to your organization’s requirements.
If you contact HashiCorp support, include your custom helm chart alongside your support bundle to ensure support has all the information they need.
Custom ingress
You can define an optional ingress resource using the ingress controller. Refer the Terraform Enterprise Helm Chart documentation for additional information about the controller.
Specify values for the ingress section in the deployment configuration. Refer to the example values file in the Terraform Enterprise Helm chart repository for a demonstration of how to enable ingress configuration.
Complete the following steps to set up an custom ingress configuration with Nginx:
- Install the nginx controller in a different namespace.
- Deploy Terraform Enterprise with Ingress configured in your values file.
- Get the address from the ingress resource like so:
Example configurations
The following examples for each cloud-platform are based on cloud native hosted PostgreSQL, storage, or Redis cache services. You can copy an example configuration and modify the values to per your environment. Refer to Configuration Reference for a list of all configuration options.
The examples also depend on the following conditions:
Values under
.env.variables
are set as aConfigMap
and mounted as Terraform Enterprise environment variables.Values under
.env.secrets
are set as Kubernetes secrets and mounted as Terraform Enterprise environment variables.Extend the
env.configMapRefs[]
orenv.secretRefs[]
with your own resources to add additionalConfigMap
orSecret
resources within your environment configuration.Values marked
BASE_64_ENCODED*
indicate that the value given must be base 64 encoded. If you are using this certificate configuration to host Terraform Enterprise web traffic, this value must be valid with theenv.TFE_HOSTNAME
, or match the wildcard pattern.
AWS Elastic Kubernetes Service (EKS)
Google Kubernetes Engine (GKE)
Azure Kubernetes Service (AKS)
Using AKS with Workload Identity
If you are using AKS with Workload Identity, the configuration is slightly different. You must set omit the TFE_OBJECT_STORAGE_AZURE_ACCOUNT_KEY
secret, and configure
the AKS cluster as an OIDC provider.
The <AZURE_USER_ASSIGNED_IDENTITY_CLIENT_ID>
is the client_id from the Azure User Assigned Identity. One can find this value in the Azure portal.
This user assigned identity needs a federated identity credential with the following subject format
"system:serviceaccount:${KUBERNETES_NAMESPACE}:${SERVICE_ACCOUNT_NAME}"
.
The issuer of the federated identity should be the cluster's issuer URL. One can retrieve this URL by running the following command: