Concepts

Concourse's end goal is to provide an expressive system with as few distinct moving parts as possible.

Concourse limits itself to three core concepts: tasks, resources, and the jobs that compose them. Interesting features like timed triggers and synchronizing usage of external environments are modeled in terms of these, rather than as layers on top.

With these primitives you can model any pipeline, from simple (unit → integration → deploy → ship) to complex (testing on multiple infrastructures, fanning out and in, etc.).

There are no more nooks and crannies of Concourse introduced as your pipeline becomes more involved.

Tasks

A task is the execution of a script in an isolated environment with dependent resources available to it. For example, running myrepo/scripts/test in a Docker container, with the working directory containing all of the task's dependencies.

If the script exits 0, the task succeeds. Otherwise, it fails.

A task can either be executed by a Job or executed manually with the Fly CLI. Both execute the same configuration, giving the guarantee that locally-executed tasks with Fly are running the same way they would in your pipeline.

To learn more about configuring and running tasks, see Tasks.

Resources

A resource is any entity that can be checked for new versions, pulled down at a specific version, and/or pushed up to idempotently create new versions. A common example would be a git repository, but it can also represent more abstract things like time itself.

At its core, Concourse knows nothing about things like git. Instead, it consumes a generic interface implemented by resource types. This allows Concourse to be extended by configuring workers with resource type implementations.

This abstraction is immensely powerful, as it does not limit Concourse to whatever things its authors thought to integrate with. Instead, as a user of Concourse you can just reuse resource type implementations, or implement your own.

To use resources, configure them in your pipeline via the resources section, and use them in your Build Plans via the get and put steps.

See Resource Types to learn more about what types of resources Concourse supports.

Jobs

At a high level, a job describes some actions to perform when dependent resources change (or when manually triggered). For example, you may define a job that runs your unit tests whenever new code is pushed to a repository.

Jobs can be thought of as functions with inputs and outputs, that automatically run when new inputs are available. A job can depend on the outputs of upstream jobs, which is the root of pipeline functionality.

The definition of actions to perform is done via a Build Plan, which is a very powerful composition-based DSL that can express anything from running simple unit tests to running a matrix of tasks and aggregating the result.

Builds

An instance of execution of a job's plan is called a build. A build can either succeed or fail, or error if something unrelated to your code goes wrong (i.e. if one of your workers falls off the face of the earth).

When a build runs, the job's plan is realized. Each step described by the job's plan is executed, and so long as all Tasks succeed, the build succeeds. If a task fails, the build fails, and its resources do not propagate to the rest of the pipeline.

The containers running in a build can be accessed while they're running (and also shortly after they finish) via fly intercept, which can greatly help in debugging.