Manage credentials for HCP Terraform with Vault

As adoption of HCP Terraform grows, more organizations are incorporating it into their automated workflows and existing tooling. Interaction with the HCP Terraform API relies on auth tokens generated by the API. External systems use these tokens to automate actions in HCP Terraform, often as part of an organization’s CI/CD pipelines.

The Vault Terraform secrets engine enables you to generate, manage and revoke credentials for HCP Terraform and Terraform Enterprise while adhering to best practices of access and control.

In this tutorial, you will enable the secrets engine, configure it to generate credentials, and then manage those credentials.

Prerequisites

To perform the tasks described in this tutorial, you need to have:

• An HCP Vault cluster or a self-hosted Vault environment.
• HCP Terraform account and organization created.

Policy requirements

Each persona requires a different set of capabilities. These are expressed in policies. If you are not familiar with policies, complete the policies tutorial.

The admin tasks require these capabilities.

# Mount secrets engines
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Configure the Terraform secrets engine and create roles
path "terraform/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage the leases
path "sys/leases/+/terraform/creds/my-user/*" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

path "sys/leases/+/terraform/creds/my-user" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

# Write ACL policies
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage tokens for verification
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

The apps tasks require these capabilities.

# Get credentials from the terraform secrets engine
path "terraform/creds/my-user" {
  capabilities = [ "read" ]
}

Lab setup

The tutorial requires a HCP Terraform API key.

Retrieve HCP Terraform API key

1. Launch a web browser and navigate to https://app.terraform.io/session.

2. Click Continue with HCP account and sign in.

3. Expand your profile, and select Account settings.

4. From the side navigation, select Tokens.

5. Click Create an API token.

6. Enter "learn-vault" in the Description field.

7. Click Generate token.

8. Click the copy icon.

9. In a terminal, export the variable TF_TOKEN to the copied token value.

   $ export TF_TOKEN=<Copied Token>
   

   This token enables you to generate tokens for your HCP Terraform user account. Tokens generated for teams and organizations would be able to generate tokens for their respective scope.

Start Vault

Scenario Introduction

In this tutorial, you will configure the Terraform secrets engine, create a role to generate tokens and then manage the lifecycle of these tokens.

1. Enable the Terraform secrets engine
2. Configure Terraform secrets engine
3. Create a role
4. Request Terraform credentials
5. Manage leases

Enable the Terraform secrets engine

The Terraform secrets engine generates Terraform tokens dynamically based on configured roles.

$ vault secrets enable terraform
Success! Enabled the terraform secrets engine at: terraform/

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
       --request POST \
       --data '{"type":"terraform"}' \
       $VAULT_ADDR/v1/sys/mounts/terraform

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
   --request POST \
   --data '{"type":"terraform"}' \
   $VAULT_ADDR/v1/sys/mounts/terraform

The Terraform secrets engine is enabled.

Configure Terraform secrets engine

The Terraform secrets engine is configured by default to communicate with HCP Terraform. The HCP Terraform API key is set in the configuration to authenticate.

Configure the Terraform secrets engine to use the TF_TOKEN token.

$ vault write terraform/config token=$TF_TOKEN

$ tee payload.json <<EOF
{
    "token": "$TF_TOKEN"
}
EOF

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/terraform/config

$ tee payload.json <<EOF
{
    "token": "$TF_TOKEN"
}
EOF

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
   --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
   --request POST \
   --data @payload.json \
   $VAULT_ADDR/v1/terraform/config

Create a role

The secrets engine is configured with the credentials that you provided it. These credentials are used through roles that you define for each secrets engine. A role is a logical name within Vault that maps to HCP Terraform credentials. These roles are defined for an organization, a team, or a user.

Request the Terraform user ID for the account associated with the TF_TOKEN.

$ curl -s \
    --header "Authorization: Bearer $TF_TOKEN" \
    --header "Content-Type: application/vnd.api+json" \
    --request GET \
    https://app.terraform.io/api/v2/account/details | jq -r ".data.id"

NOTE: This example uses jq to process the JSON output for readability.

This unique ID is required to generate credentials for this user.

Create a variable to store the user ID.

$ USER_ID=$(!!)

Create a role named my-user with the USER_ID and a time-to-live of 2 minutes.

$ vault write terraform/role/my-user user_id=$USER_ID ttl=2m
Success! Data written to: terraform/role/my-user

This role is ready to generate credentials for this user that stay valid for two minutes.

Request Terraform credentials

The applications or users that require the Terraform credentials read them from the secrets engine's my-user role.

Read credentials from the my-user role.

$ vault read terraform/creds/my-user
Key                Value
---                -----
lease_id           terraform/creds/my-user/00hjA2DN0deht4z3Lp7oniEg
lease_duration     2m
lease_renewable    true
token              VgySVQ9kfyR6Yg.atlasv1.ogU7VVMAH8kH7A8chrCIOg64yxUM0l5iVldHncLOhWqUInh0aSure5uYYgRjyQBM3YA
token_id           at-adFBZFRExz9KiZqV

The Terraform credentials are displayed as the token value. The token_id represents its unique identifier that HCP Terraform uses to maintain that ID.

Create a variable to store the token created from the my-user role.

$ CREATED_TF_TOKEN=$(vault read -format=json terraform/creds/my-user | jq -r ".data.token")

Request all the authentication token IDs for the user account authenticating with the CREATED_TF_TOKEN token.

$ curl \
  --header "Authorization: Bearer $CREATED_TF_TOKEN" \
  --header "Content-Type: application/vnd.api+json" \
  --request GET \
  https://app.terraform.io/api/v2/users/$USER_ID/authentication-tokens | jq -r ".data[].id"

The results of the request authenticates with the new token and returns the list of all the token ids generated.

Manage leases

The credentials are managed by the lease ID and remain valid for the lease duration (TTL) or until revoked. Once revoked the credentials are no longer valid.

List the existing leases.

$ vault list sys/leases/lookup/terraform/creds/my-user

Keys
----
IQKUMCTg3M5QTRZ0abmLKjTX

All valid leases for Terraform credentials are displayed.

Create a variable that stores the first lease ID.

$ LEASE_ID=$(vault list -format=json sys/leases/lookup/terraform/creds/my-user | jq -r ".[0]")

Renew the lease for the database credential by passing its lease ID.

$ vault lease renew terraform/creds/my-user/$LEASE_ID

Key                Value
---                -----
lease_id           terraform/creds/my-user/aKziIqZM5GAm39swiJWEkp2g
lease_duration     2m
lease_renewable    true

The TTL of the renewed lease is set to 2m.

Revoke the lease without waiting for its expiration.

$ vault lease revoke terraform/creds/my-user/$LEASE_ID
All revocation operations queued successfully!

List the existing leases.

$ vault list sys/leases/lookup/terraform/creds/my-user
No value found at sys/leases/lookup/terraform/creds/my-user/

The lease is no longer valid and is not displayed.

Read new credentials from the my-user role.

$ vault read terraform/creds/my-user
Key                Value
---                -----
lease_id           terraform/creds/my-user/Y4CD97WFsEL13d4AMnEgOUlY
lease_duration     1m
lease_renewable    true
token              ndklHRyo22tSvQ.atlasv1.mmrM5mEOxZFg3yPtvdJiviWKgyjL2wQOIuVhvqVtfiszhSOwezypiHv7FydUMs7eVSE
token_id           at-c13pERGbFXRiGd56

All leases associated with a path may be removed.

Revoke all the leases with the prefix terraform/creds/my-user.

$ vault lease revoke -prefix terraform/creds/my-user

The prefix flag matches all valid leases with the path prefix of terraform/creds/my-user.

List the existing leases.

$ vault list sys/leases/lookup/terraform/creds/my-user
No value found at sys/leases/lookup/terraform/creds/my-user

All the leases with this path as a prefix have been revoked.

Next Step

The HCP Terraform tokens that you generated are able to perform all the abilities of the account that created them. Learn more about HCP Terraform with the HCP Terraform - Get Started tutorials.

The Terraform secrets engine is configured to work with your user account. Learn how to configure it for organizations and teams by by reading the Terraform secrets engine documentation.