Credential Management

This topic isn't crucial to understanding Concourse; if you're just getting started and have finished the Installing section, you may want to first move on to Using Concourse.

Going beyond Encryption, explicit credential management will provide credentials to your builds for a brief amount of time, without being persisted anywhere. It also allows for credentials to be rotated and managed external to the pipeline or team, and prevents them from being revealed by get-pipeline.

Currently, the only supported credential manager is Vault and Credhub.

Credential management works by replacing the credentials with ((parameters)) in your pipeline or task config. When the ATC is about to run the step or check that is configured with the parameters, it will resolve them by fetching the values from the credential manager. If the values are not present, the action will error.

What can be parameterized?

The following configurations can be parameterized with a credential manager:

Where these values are looked up and how the credential manager is configured depends on the backend. Consult the relevant section below for whichever backend you want to use.

Using Vault

Configuration

The ATC is statically configured with a Vault server URL (plus any TLS config), and either a client token or an auth backend.

For example, to point the ATC at an internal Vault server with TLS signed by a local CA, using an AppRole auth backend, you may configure:

concourse web ... \
  --vault-url https://10.2.0.3:8200 \
  --vault-ca-cert /etc/my-ca.cert \
  --vault-auth-backend approle \
  --vault-auth-param role_id=db02de05-fa39-4855-059b-67221c5c2f63 \
  --vault-auth-param secret_id=6a174c20-f6de-a53c-74d2-6018fcceff64

You may instead want to use the TLS auth backend, in which case you would specify a client certificate and private key, like so:

concourse web ... \
  --vault-url https://10.2.0.3:8200 \
  --vault-ca-cert /etc/my-ca.cert \
  --vault-client-cert /etc/my-client.cert \
  --vault-client-key /etc/my-client.key \
  --vault-auth-backend cert

In this case no params are necessary, as the TLS auth backend will check the certificate against all roles if no name is specified.

Alternatively, if you've got a periodic token created, you can pass it directly as --vault-client-token. Make sure the period is long enough to account for any ATC downtime, however, including the time between generating it and getting the ATC running.

concourse web ... \
  --vault-url https://10.2.0.3:8200 \
  --vault-ca-cert /etc/my-ca.cert \
  --vault-client-token c2c2fbd5-2893-b385-6fa5-30050439f698

For all of these configurations, the ATC will periodically renew its token, ensuring it doesn't expire.

Credential Lookup Rules

When resolving a parameter such as ((foo_param)), it will look in the following paths, in order:

  • /concourse/TEAM_NAME/PIPELINE_NAME/foo_param

  • /concourse/TEAM_NAME/foo_param

The leading /concourse can be changed by specifying --vault-path-prefix.

Vault credentials are actually key-value, so for ((foo)) Concourse will default to the field name value. You can specify the field to grab via . syntax, e.g. ((foo.bar)).

If the action is being run in the context of a pipeline (e.g. a check or a step in a build of a job), the ATC will first look in the pipeline path. If it's not found there, it will look in the team path. This allows credentials to be scoped widely if they're common across many pipelines.

If an action is being run in a one-off build, the ATC will only look in the team path.

Caveat: multiple fields of a rotating key

When a key has multiple fields and is re-generated on every fetch, for example when using the AWS secret backend, the ATC will currently fetch the credential once for each parameter, resulting in fields that don't match each other. This will be fixed in an upcoming release of Concourse.

Using Credhub

Configuration

The ATC is statically configured with a Credhub server URL with TLS and client config.

For example, to point the ATC at an internal Credhub server with TLS signed by a local CA, using client id and secret, you may configure:

concourse web ... \
  --credhub-url https://10.2.0.3:9000 \
  --credhub-ca-cert /etc/my-ca.cert \
  --credhub-client-id =db02de05-fa39-4855-059b-67221c5c2f63 \
  --credhub-client-secret 6a174c20-f6de-a53c-74d2-6018fcceff64

Credential Lookup Rules

When resolving a parameter such as ((foo_param)), it will look in the following paths, in order:

  • /concourse/TEAM_NAME/PIPELINE_NAME/foo_param

  • /concourse/TEAM_NAME/foo_param

The leading /concourse can be changed by specifying --credhub-path-prefix.

Credhub credentials actually have different types, which may contain multiple values. For example, the user type specifies both username and password. You can specified the field to grab via . syntax, e.g. ((foo_param.username)).

If the action is being run in the context of a pipeline (e.g. a check or a step in a build of a job), the ATC will first look in the pipeline path. If it's not found there, it will look in the team path. This allows credentials to be scoped widely if they're common across many pipelines.

If an action is being run in a one-off build, the ATC will only look in the team path.