/api/v1/workers API directly is a bit inconvenient. Your workers need credentials for the ATC, and must advertise an address that the ATC can reach. The API consumer would also have to continuously heartbeat so long as the server it's advertising is healthy.
A typical Concourse installation includes a component called the TSA. The TSA is used to securely register workers via SSH, and continuously health-check and heartbeat them to the ATC. These workers can come from any network that can reach the TSA, which is usually colocated with the ATC and possibly sitting behind the same load balancer (but on a different port, usually
This is slightly different from how most other CI systems work. Concourse inverts the dependency flow so that scaling up workers can be done easily without reconfiguring the ATC.
This flow also makes it easier to securely register remote workers living in private networks. Rather than making the worker VM publicly reachable (very dangerous!) or using a VPN (a bit sketchy depending on where the ATC lives, and often error prone), workers can listen on
127.0.0.1 and forward their connections through the TSA via a reverse SSH tunnel.
If you're using the standalone binaries, the SSH aspect of worker registration is automated for you; you'll just need to generate a key pair for the worker and configure the workers with the expected TSA host key. See Starting Workers for more details.
If you're registering standalone binary-provisioned workers with a BOSH-deployed Concourse, you'll want to go through the Supporting external workers section first.
If you have some other environment separate from your primary Concourse deployment, but still manageable by BOSH, configuration is done via properties, pointing at the main deployment.
To BOSH deploy your workers, follow largely the same instructions as in Clusters with BOSH, but you may strip things down a bit, e.g. don't bother deploying
postgresql for your workers.
If you're registering these workers workers with a BOSH-deployed Concourse, you'll want to go through the Supporting external workers section first.
You'll also need to configure the workers to register with your main cluster. The Concourse release includes a job called
groundcrew, which can be configured to register a (usually colocated) worker with a TSA.
To configure Groundcrew, you may want to set some of the following properties:
An array of tags to advertise for the worker. You'll probably want to specify this, so that only steps explicitly targeting the worker (via their own tags) run on it.
The platform to advertise for the worker. Defaults to
The address of the TSA server. This will probably be the address of your external Concourse cluster.
The port of the TSA server. Defaults to
The private key to use when authenticating with the TSA. This must be authorized by the TSA in advance
The public key to expect when connecting to the TSA.
Note that most properties have sane defaults. To see a full set of properties and what a manifest may look like when specifying them, browse around the docs at bosh.io.
With the above properties, we know what kind of worker we'll be advertising, but not how the ATC will reach it. There are two options: either directly advertising an address reachable by the ATC, or by forwarding a local address via a reverse SSH tunnel to the TSA, who will then advertise its tunnelled address to the ATC.
To directly advertise an address to the TSA, set the following properties on the
The address of the Garden server to advertise to the TSA. Note that this must be the external address. If omitted, Groundcrew will automatically determine this address, so you probably won't need to specify it.
The address of the Baggageclaim server to advertise to the TSA. Note that this must be the external address. If omitted, Groundcrew will automatically determine this address, so you probably won't need to specify it.
You would do this if your worker is not reachable by the outside world, but is reachable by the ATC. For example, a separate deployment within the same VPC. (Note: making a Garden server publicly reachable is a very bad idea.)
To forward a local Garden server through the TSA, set the following properties on the
The locally-reachable Garden address to forward through the TSA, e.g.
The locally-reachable Baggageclaim address to forward through the TSA, e.g.
You would do this if your worker lives in a private network (e.g. a local cluster), but your TSA is publicly reachable (which is much safer).