v2.5.0

November 16, 2016

Release Notes

  • feature

Teams can now be destroyed via destroy-team.

  • fix

Fixed a hairy deadlock that could lead to jobs getting stuck "waiting for a suitable set of input versions". We fixed it, like, really hard. Like the lock isn't even THERE anymore, man. (And it's not needed anymore, either. That's important too.)

  • feature

The Cf resource now has the latest CLI version again. Unbeknownst to us, the CLI team switched buckets, so we stopped getting new bits.

  • fix

We've fixed the CLI download links on the "no pipelines" page.

  • fix

The fallback flow in login for accepting the token manually is now fixed, thanks to a PR from @sharms!

  • feature

The Bosh Io Stemcell resource will now aggressively retry downloads, thanks to @zachgersh and @ljfranklin!

  • feature

The S3 resource now supports setting a Content-Type for the file being uploaded, thanks to a PR from @pdelagrave!

Downloadsv2.5.0

v2.4.0

October 28, 2016

Release Notes

  • feature

Worker keys can now be authorized for only a particular team. This prevents workers from being unintentionally (or maliciously) registered as a global worker, in the case where an operator is granting an external worker access to the cluster.

Consult web --help for CLI docs or bosh.io for BOSH docs.

  • feature

We've lowered the default memory/CPU usage of the concourse/lite Vagrant box to 2GB and 2 cores, down from 6GB and four cores, thanks to a PR from @jwiebalk!

  • feature

Baggageclaim will now be more durable to corrupt volumes. Previously a borked metadata file would effectively wedge the Baggageclaim API, making the worker unrecoverable. You would see an error like "failed to list volumes" in your builds. Baggageclaim will now pretend these volumes don't exist in the API, and reap them from the disk.

  • feature

on_failure, on_success, and ensure can now be attached to a job, thanks to a PR from @jmcarp!

  • feature

login will now automatically transfer the token to the CLI for the oAuth flow, rather than requiring you to copy-paste it.

  • fix

Fixed the behavior of the "home" button. It will now take you to your current pipeline, rather than always taking you to the first one.

  • fix

After logging in, the UI will now reflect that you're actually logged in. This used to require a refresh. Single page apps giveth and they taketh away.

  • fix

When viewing a build or a job, the groups the job are in will now be highlighted, rather than always the first group.

  • fix

Fixed a janky synchronization issues when updating the top bar while switching between pipelines; it used to sometimes show the previous pipeline and never update.

  • fix

The favicon will now reset back to the default "grey" flavor when switching from a build to any other page.

  • feature

Logging in will now redirect you back to where you were if it was initiated by some attempted action.

  • fix

The Bosh Io Stemcell resource now correctly returns versions in chronological order.

Downloadsv2.4.0

v2.3.1

October 13, 2016

Release Notes

  • fix

Fixed middle-clicking and other modifier keys when clicking on jobs/resources in the pipeline view.

Downloadsv2.3.1

v2.3.0

October 13, 2016

Release Notes

  • feature

The whole UI now runs as a single Elm app! Pages should load much quicker, and the pipeline sidebar now remains open as you navigate around.

There's still some UX work to be done to make things a big smoother, e.g. better handling for 404 cases and more consistent loading indicators, but this is the first big step on that path.

  • feature

The team name is now provided as $BUILD_TEAM_NAME along with the rest of the metadata available to resources, thanks to a PR from @SHyx0rmZ.

  • fix

Fixed the log out menu being unclickable on the build page.

  • fix

The sidebar no longer scrolls offscreen.

  • feature

The Github Release resource now supports publishing pre-releases, thanks to a PR from @ahelal!

  • feature

The Git resource can now have LFS disabled via a disable_git_ls param, thanks to a PR from @SHyx0rmZ!

  • feature

Unused resources in the pipeline config are now a validation error, thanks to a PR from @mmb!

  • feature

The BOSH release can now be configured with arbitrary Riemann tags, thanks to a PR from @combor!

  • feature

When configured with a CloudFront endpoint, the S3 resource will now download via CloudFront, which should be much faster. This is thanks to a PR from @cunnie and @ljfranklin!

  • feature

The S3 resource now supports v2 signature signing, thanks to a PR from @JamesClonk!

  • feature

The Bosh Deployment resource can now be configured to not redact properties from the deploy diff, thanks to a PR from @jszroberto!

Downloadsv2.3.0

v2.2.1

September 19, 2016

Release Notes

  • fix

Finished up the build rendering performance fix on Chrome, which only affected Chrome because Safari and Firefox didn't render Flexbox properly, which is also why their autoscrolling didn't work.

Web. Development.

  • fix

Fixed autoscrolling in Safari and Firefox.

  • fix

May have talked up the algorithm release note a bit much. Someone immediately found another case where the 100% CPU monster struck.

This is fixed now. Trust me.

  • feature

The Bosh Io Stemcell resource has been rewritten in Go with tests and such, thanks to a PR by @zachgersh!

It now does parallel downloads, to boot.

  • feature

The ATC can now be configured with a Riemann service prefix, thanks to a PR by @mastertinner!

  • fix

Fixed an issue where the exponential backoff when talking to a flaky worker would never give up.

  • fix

The Windows fly download link didn't work in the binary distribution. Because of the .exe suffix. Oi. Fixed.

  • fix

BOSH-deployed workers will now be named after a frankenguid taking parts from the BOSH instance ID and their hostname. This is to make it so you can correlate the worker to the BOSH instance, while also guaranteeing that when the worker is recreated it comes back under a new name.

Downloadsv2.2.1

v2.2.0

September 15, 2016

Release Notes

  • feature

The Algorithm has become much faster. The Algorithm is what computes the candidate set of inputs for a job, and is the second hardest problem in Concourse (behind the pipeline UI).

In some cases, e.g. when disabling an oft-used resource version, The Algorithm would go buck-wild and use 100% CPU trying to locate the new set of version candidates. This was undesirable.

Luckily, we prepared for this kind of degenerative case, and made it easy to capture the data sets that replicate the issue. We captured the data set, observed the slowness (had a build running for >12 hours before we gave up), thought long and hard, and did a bunch of work to bring that down to ~19 seconds.

Hopefully that's the last of the 100% CPU monster. Overall scheduling performance has also improved across the board.

  • feature

The pipeline view will now only redraw if the data has changed. We've also fixed a regression in v2.0.0 that led to redrawing multiple times on an interval, likely leading to the tab crashing if left in the background.

  • feature

Turns out set-team made it stupidly easy to configure a team (or reconfigure an existing team) with no auth credentials. I'm not saying something bad happened, but uh, it'll now warn you and force you to type a really long flag, and even shame you a little bit even when you use it.

  • feature

The fly binaries are now build natively on each platform, rather than cross-compiled. This removes a few surprises like native DNS and OS X Keychain functionality not working.

They're also now available for download alongside the rest of Concourse, rather than having to download from a Concourse installation.

  • feature

The --auth-duration flag introduced in v2.1.0 is now available as a BOSH property (auth_duration, surprise surprise). Thanks to @JamesClonk for the PR!

  • feature

checklist now generates a Checkfile with the team name present, thanks to a PR by @Amit-PivotalLabs.

Be sure to upgrade Checkman as well for this to work.

  • feature

The Generic oAuth provider now supports checking presence of a scope, thanks to a PR by @LinuxBozo!

  • feature

The Docker Image resource now supports build args, thanks to a PR from @o-orand!

  • feature

sync will bail early if the versions already match, thanks to a PR from @geofffranks!

  • fix

Turns out Chrome is really, really bad at rendering our build page now. We've made some improvements to this but I think more work is ahead of us.

Compared to Firefox and Safari, Chrome seems to redraw the entire dang page on every friggin update. Which means every second when we update that stupid little ticker up top, the whole page and all its output repaints.

For shame, Chrome.

If it's unbearable you can try Firefox or Safari, which seem to render more sensibly.

We've also fixed a bug that led to interpreting the event stream multiple times for builds that have a ton of output, thereby making things even slower and jankier.

  • fix

The Docker Image resource skip_download parameter now works again. This broke in v2.1.0. Sorry about that.

Downloadsv2.2.0

v2.1.0

September 8, 2016

Release Notes

  • feature

We've reduced the number of queries by about 60%, including removing constant write loads which may have led to increased CPU usage on RDS.

  • feature

The resource page is now much much more responsive. We've rewritten it in Elm, implementing live-updating along the way. It used to take a few (maybe quite a few) seconds to load, and now takes on the order of milliseconds. Pretty rad.

  • feature

Triggering a build will now update the UI in-place rather than redirecting.

  • fix

Autoscrolling is back and better than ever before. Keyboard controls for scrolling (e.g. Cmd+Down, Spacebar) should also now work as normal.

  • fix

We've updated to Go 1.7.1, which should fix a few DNS-related quirks.

  • fix

Cmd-click and other non-vanilla clicks should now work for build links in the header of the build view.

  • feature

The ATC can now be configured with a --auth-duration flag, making the duration for which tokens are valid configurable. This is thanks to a PR from @fmy!

  • feature

The Git resource now supports GPG verification for commits, thanks to a PR from @alext!

  • feature

The Docker Image resource now emits the output of docker inspect <image> as docker_inspect.json, thanks to a PR from @endzyme!

  • feature

Concourse now rejects traffic from web crawlers by providing a robots.txt. We may make this a bit more targeted in the future, but the intent is to reduce unwanted traffic as there are many many many links to click in Concourse.

  • feature

Thanks to @databus23 the Docker Image resource can now cache things once again! This regressed with Docker 1.10 as the semantics for caching and layer reuse changed to require some additional work/metadata.

  • feature

The Git resource now supports [skip ci] in addition to [ci skip], thanks to @fmy!

Downloadsv2.1.0

v2.0.2

August 30, 2016

Release Notes

Turns out when you wait a month between releases a few things can go wrong once you finally ship. This is, like, probably the last patch on v2.0.0. Maybe. We'll see. We may save our pride and release v2.1.x next irregardless.

Luckily we have this pipeline thing that lets us continuously fix our own mistakes, not just ship them!

  • fix

Previously a cluster of multiple ATCs could get into a deadlocked state when checking for resources. This would manifest itself as jobs being stuck in a "pending" state. This release, our first ever X.0.2 release, fixes that.

Downloadsv2.0.2

v2.0.1

August 29, 2016

Release Notes

  • fix

Previously if you were using Safari the pipeline would not render. Well, technically, it would render, but within a <div> element with 0px height. We have sighed, flailed at the CSS monster, triggered our web-development pipeline, and prevailed.

Also the top bar used to shrivel up and die if the size of the content page became too large. It, uh, doesn't anymore.

  • fix

We've restored the pre-teams API endpoint for the job status badges, so you all don't have to update your READMEs immediately. Sorry about that. (You should probably still update them, though.)

  • fix

We've improved the error message returned when the file used by a task step does not exist.

Downloadsv2.0.1

v2.0.0

August 26, 2016

Release Notes

  • feature

  • breaking

TEEEEEEEEEEEEEEEEEEEEEEEEAMS!

So, you'll notice that version number made quite a jump. This is why. The long-awaited "teams" feature brings (trusted) multi-tenancy to Concourse.

The following breaking changes have been made:

  • The --publicly-viewable configuration is gone, and is now set on a pipeline-by-pipeline basis, via expose-pipeline and hide-pipeline. Newly configured pipelines are hidden by default, and all existing pipelines will be hidden upon upgrade, so make sure to expose the ones you intend to be public!

  • Many API routes now require the team to specified in the URL, e.g. /api/v1/teams/foo/pipelines. Our API still not yet an official interface to Concourse; we continue to encourage using fly until we turn it into a properly versioned and documented API.

    The web UI routes now also have the team name in them. Old URLs should continue to work, and will now redirect to the new URL.

Everything else (pipelines and such) should continue to work the same as before, only now they'll belong to a team called main.

  • feature

  • deprecation

We have deprecated the /builds page, previously known as "the second hamburger menu button". Its button in the header was taking up valuable UI space, so we unceremoniously demoted it.

Many of you feel bad for this page. That is because you crazy. It has no feelings! builds is much better.

Before we kill it off completely, we'll make sure builds does everything you'd need from the page, which is really just a matter of having a column for the build URL so you can view it in the web UI. Aside from that, builds is better in every way: if your next step is intercept, it keeps you in the terminal. You can also change the number of results with -c, and filter it to a job with -j, both of which should be much faster for finding what you're looking for.

  • feature

The favicon will now change color when viewing a build to reflect its status. Pretty neat, right?

Thanks to @zachgersh and @rmasand for the inspiration! - <3 @kimeberz

  • feature

As an incremental step in our march towards a fully Elm-based single-page app, we've made some parts of the navigation much snappier. Switching between builds of a job will now update the UI in-place, and toggling pipeline groups now immediately re-renders the existing data set, rather than reloading the entire page.

We'll be focusing more in the upcoming weeks on bringing more of the web UI up to par, ultimately resulting in one big snappy single-page app (hopefully with none of the gotchas that made me hate them at first, i.e. inconsistent data that forces a page refresh and distrust of the entire app as a result).

  • fix

Previously if the database or network became sluggish, ATC's locking mechanism would stop functioning, resulting in multiple ATCs trying to manage the same build, among other things. We've switched to Postgres session locks, which should be much more airtight.

  • feature

The ATC now supports being configured with a generic oAuth provider. Huge thanks to @poida for doing the PR for this!

  • feature

The ATC now supports being configured with a UAA/CF auth provider.

  • feature

The Docker Image resource can now discover older versions. Previously it would only ever emit the current version. This can be used to roll back to a previously known-good image digest.

  • feature

We've bumped to Go 1.7 everywhere, and made this process continuous. Whenever Go 1.7.1 or 1.8 come out, we'll automatically pick it up. Turns out there's this pretty neat CI system that can do that kind of thing. You may have heard of it. (It's not Jenkins.)

  • feature

Fly learned the pause-resource and unpause-resource commands, thanks to pull requests from @gregarcara!

  • feature

The bar along the top of the page will now turn blue on already-rendered pages if the pipeline the page belongs to is paused.

  • feature

The login command now accepts a --ca-cert flag, which should be used instead of -k. The cert will be persisted for the target (even if its file goes away).

  • fix

We've refactored the internal scheduler component of the ATC, reducing query usage and generally making it easier to work on in the future.

This refactor also resulting in fixing behavior with version: every.

  • fix

Fixed volume deletion in BaggageClaim on a few platforms. May have been primarily situations where the root disk was btrfs.

  • feature

The S3 resource now supports encryption options, thanks to a PR from @jmcarp!

  • feature

The Github Release resource now creates a body file when fetching a resource, thanks to a pull request from @shinji62.

Now you can continuously read Concourse's release notes with to determine whether to auto-update!

  • fix

The Docker repository image now bakes in the default CA certs, thanks to a PR from @billimek!

  • fix

The Git resource's handling of merge commits now makes a lot more sense.

Previously, a merge commit would result in the history of the merged branch showing up in the version set. For tools like git log this makes sense, but from a CI standpoint, you only really care about the effect on the branch that it was merged into: it's all or nothing. This is now fixed, and only the merge commit itself will be yielded as a version.

Shout-out to @chipx86 for helping us reason through this on GitHub!

  • fix

The Time resource works now.

It was pretty broken before, because time is hard.

There were a couple issues:

  • If your start and stop were configured in some non-UTC timezone, say, -0700, it straight up wouldn't work if the times were late enough in the day. For real.

  • If you configured days and start and stop, the days would be treated as UTC, rather than respecting the timezone in start or stop. What's more, start and stop could be emitted, leaving there no place for a location for the days to even be specified.

    We've added a location field, which should be used instead of embedded offsets, and then days will respect it.

  • fix

The BOSH release will now leave 10GB of space free for the system, rather than allocating all of it for BaggageClaim. This is mainly to make the failure mode better. Without this overhead, BaggageClaim would fill up the host's disk, then fail to write to that, and then panic and go read-only, making it unrecoverable. Now the BaggageClaim volume will still fill up, but it'll at least be able to expire volumes and such, and the host machine will still function within its 10GB overhead.

This is all thanks to some sleuthing and a pull request from @alext.

  • feature

The Bosh Deployment resource now has the BOSH cli v1.3262.4, thanks to a PR from @alex-slynko!

Downloadsv2.0.0

v1.6.0

July 25, 2016

Release Notes

  • feature

We now provide an official Docker repository at concourse/concourse!

As part of this, the binary distribution has been updated to support environment variables for configuration, in addition to flags. Because the environment is perfectly safe.

Thanks to @gregarcara and @MeteoGroup for maintaining Concourse images until we started on this ourselves!

  • feature

The bosh.io Release resource will now verify SHA1 checksums, and place them in the fetched directory as sha1. The bosh.io Stemcell resource has also been updated so that they both have the same behavior.

  • feature

The Docker Image resource now supports ECR! There were a couple issues and pull requests opened for this; thanks to all who kept the ball rolling!

Downloadsv1.6.0

v1.5.1

July 20, 2016

Release Notes

  • fix

A bug introduced by v1.5.0 as part of the resource fetching synchronizing led to hanging get steps. It affected resources with large values in source or params. It is now fixed.

  • feature

A task can now specify the user to run the process as by configuring user in run.

Downloadsv1.5.1

v1.5.0

July 20, 2016

Release Notes

  • feature

When connectivity to Concourse is lost on the pipeline page, a fancy warning message will be shown.

This started as a PR from @fmy - thanks!

  • feature

Loading the logs of a build is now much faster (up to 12x improvements have been observed). Rendering performance is unchanged, but we found that for chatty builds the bulk of the time was spent simply downloading the logs.

  • feature

We will now only fetch a given resource (including image_resource) once per worker. Previously they would all fetch concurrently and each populate the cache, which would storm the worker with network traffic and CPU load. Now one will start fetching and the rest will wait.

  • feature

We will no longer create no-op containers for cache hits. This should reduce the number of overall containers used by the pipeline.

  • fix

The build view was only showing the last 100 builds. And none of you noticed! It'll show all of'em now.

  • fix

BOSH-deployed workers' names will be set to their BOSH instance ID, rather than their hostname. This should make identifying them a bit easier.

  • fix

The Docker Image resource will now correctly handle private registry URIs without their port included.

  • fix

We now limit the total number of database connections to 64 per ATC, and have removed a debugging utility that led to deadlocks when a connection limit was reached (and also may have led to those connection limits being reached in the first place).

Downloadsv1.5.0

v1.4.1

July 8, 2016

Release Notes

  • fix

A bug introduced by v1.4.0 caused custom resource types that override worker-provided resource types (e.g. git, s3, docker-image) to lead to containers being created repeatedly until your workers couldn't take anymore.

Fixed. Our bad.

  • fix

The TLS redirecting feature introduced as part of v1.3.0 made execute work only 50% of the time when running two ATCs. With three ATCs it would work 33.3%, repeating of course, of the time, and so on.

execute now works 100% of the time.

  • fix

The commit message format in the Pool resource has been once again tweaked so as to not incorrectly trigger GitHub's issue reference syntax, thanks to a PR from @geramirez.

Downloadsv1.4.1

v1.4.0

July 6, 2016

Release Notes

  • feature

We've revamped our container retention configuration.

Previously, containers used by failed builds would stick around for 1 hour, and containers for succeeded builds would stick around for 5 minutes. This was pretty dumb. It meant if you had frequently failing builds, containers (and disk usage) would pile up, and if you had a build that failed overnight, you wouldn't be able to investigate anything in the morning.

Instead, as long as the most recent build of a job is failed or errored, we'll keep it around indefinitely. It will be let go as soon as a new build finishes successfully, or fails, in which case that build will be retained instead.

  • fix

We've fixed a hairy issue that resulted in artifacts sometimes disappearing in the middle of a build. This issue primarily affected users with more than one worker.

  • fix

The new container retention semantics also fix the "volume mounted to container is missing" bug with hijacking.

  • fix

We've bumped the version of the Go AWS SDK used by the S3 resource. This should fix some issues related to long-running uploads and downloads.

  • fix

sync now shows a progress bar. You're welcome.

  • fix

Some of y'all with BIG DATA had volumes too large to fit their reported size in the database. That should work now.

  • feature

We've bumped to Garden-runC v0.4.0, which should fix the iptables "resource temporarily unavailable" error.

  • fix

We've gone back to a safer method of killing container process when aborting a build. We had initially switched to signalling the parent process and then killing it if it didn't exit after 10 seconds, however in a lot of cases this would just result in things not exiting when the process tree is sufficiently complex. This also resulted in the Pool resource not giving up in its attempt loop when aborted.

  • fix

Previously if a worker left the pool at an inopportune moment, Concourse would forget about its volumes, which led to things getting into a wedged state. This is now fixed. You should never have to pause your pipeline to "let it breathe" again.

Downloadsv1.4.0

v1.3.1

June 16, 2016

Release Notes

  • fix

Bumping Buildroot brought in git version 2.8.2, which breaks handling of nested submodules. We've moved ahead to master of Buildroot which bumps git to 2.8.3, which should fix the issue.

  • feature

The Github Release resource resource will now retry on failed uploads, up to 10 times.

  • fix

The build numbers made in automated commits to the Pool resource are now escaped with backticks so that GitHub doesn't auto-link them to bogus issues. Thanks @geramirez!

Downloadsv1.3.1

v1.3.0

June 13, 2016

Release Notes

  • feature

  • breaking

We have switched Garden backends to Garden runC. This new runC-based backend has proven in our testing to be far more portable, allowing our binaries to work on just about any stack that's using a recent enough Linux kernel (3.19+).

As part of this upgrade, your existing workers will need to be recreated.

With BOSH, you can do this with bosh deploy --recreate when deploying the new releases.

For the binaries, you'll need to stop the old worker, nuke the --work-dir, and then start the new one.

In addition, we now explicitly manage all aspects of container images. This should dramatically reduce disk usage on your workers, as there's no longer a redundant copy from importing the image into Garden's graph, as long as you're using image_resource. This also means we're now using btrfs for the whole stack, which makes running Docker in Concourse tasks much easier.

As part of this, the binary distribution no longer supports image. Supporting it has always been a portability nightmare, and we've been discouraging use of image for some time now.

  • feature

Jobs can now be configured with build_logs_to_retain, which is a number indicating how many builds for which to keep the build output. All build logs except for the most recent N builds will be reaped. You can flip this on for already-existing jobs with thousands of builds and we'll slowly reap them in batches.

  • feature

A task step in a plan can now be configured with an image field specifying an artifact source to use. This allows for build-and-test flows, where your pipeline produces an image and then propagates the exact image to a task that uses it as its rootfs.

  • feature

volumes now includes much more information about each volume, including its disk usage. This should help track down what's using so much disk, and whether you really just need more space to accomodate your workload.

  • feature

A Hg resource is now included as part of the core distribution.

  • feature

When a build is stuck "waiting for a suitable set of input versions", it will now show what input it cannot find versions for, and why.

  • feature

Previously workers could end up with very poor balancing of containers, in the worst case resulting in one worker handling the bulk of the resource checking load. We now balance checking across workers over time, by only reusing the check containers for up to an hour.

  • feature

The ATC itself can now be configured to listen with TLS, rather than relying on an upstream component like HAProxy or an ELB for SSL termination.

This also means the ATC can handle HTTP/2 traffic, thanks to Go's magic net/http package. We've seen noticeable speed boosts in the web UI from this alone.

When TLS is configured the ATC will redirect any non-HTTPS GET and HEAD requests to HTTPS.

  • feature

In addition to HTTP/2, we've done some optimizations that make the pipeline UI much faster and more responsive.

  • fix

intercept's help text now indicates that you can run an arbitrary command.

  • feature

The Git resource now includes branches and tags in its metadata for each commit.

  • fix

Previously the Time resource would accidentally report two versions within the boundary of a time range configured with start and stop. This has been fixed.

  • feature

The Docker Image resource can now be configured with SSL CA certs to trust when communicating with the registry. This allows you to use private registries securely, rather than listing the address as insecure.

  • fix

The Git resource will now detect the full history of tags when configured with tag_filter, rather than just the latest one.

  • feature

All core resources now include bash in their image, which should make hijacking more pleasant. We also stripped out extra stuff from some resources, so on the whole the resource images should be a bit smaller.

  • feature

We've bumped all core images to Buildroot v2016.05, and are now continously integrating with Buildroot.

  • feature

The Git resource can now be configured to NOT skip commits with [ci skip] in them, thanks to a PR from @zachgersh and @ryanmoran. This is useful when you're pointing at commits of an external repo with an unrelated CI.

  • feature

The Git resource, S3 resource, and Semver resource now support basic auth when talking to Git repos, thanks to PRs from @MatthiasWinzeler and @JamesClonk.

  • feature

The Docker Image resource can now be configured with a registry mirror, thanks to a PR from @gregarcara.

Downloadsv1.3.0

v1.2.0

April 27, 2016

Release Notes

  • feature

fly learned the check-resource command, which allows you to force detection of versions, notably those in the past. This is useful if you've configured a new resource but want to use a version that's not the latest one.

As part of this change we've slightly tweaked how check works (in a backwards-compatible way). Your check script should now include the requested version in the response if it's still valid. This is so that you can run check-resource with the version that you want, rather than the one before it.

  • feature

get steps can now be pinned to a specific version.

Example:

plan:
- get: my-repo
  version: {ref: cb0ed22c4cfc6b7524bcafc1664b2d27035521f9}

This will lock the my-repo step to the specified version. Note that the version must be valid, must be collected in the resource's version history (which means you may want to use check-resource), and must also satisfy any passed constraints listed on the step.

See version for more information.

  • feature

get steps can now be configured to run with every version of its resource, rather than skipping to the latest.

Example:

plan:
- get: pull-requests
  version: every

This will allow the build to run with every version of the resource, which is probably a bad idea for certain git repos (where folks may push 100 commits at once), but can make a lot of sense for other things (security auditing, handling all pull requests, processing commits across multiple branches, etc.).

See version for more information.

  • fix

We've fixed the rendering of multi-field versions in the UI to be substantially less confusing.

As part of this we've tweaked how we render steps in the UI. The checkboxes are now more subtle and less button-like, and aggregate steps look cooler.

  • fix

The ATC now validates that its URL flags are valid URLs. Previously you could configure an --external-url of example.com, which is missing the scheme, so some things would break.

  • feature

We've bumped to Go 1.6.1. You probably don't care.

  • fix

The Docker Image resource now requests the correct schema version of manifests from the registry, which should fix cases where it would pull the wrong digest.

This is thanks to a PR from @databus23.

  • fix

The S3 resource now issues a shouty warning if you're still using from and to.

It's configured to blink but our web UI doesn't (YET) support blinking text. Consider this a warning. You have one release to comply or be met with red, blinking text in your builds.

  • fix

The Semver resource can now be configured with an identify for the commits made with the git backend, thanks to a PR from @shinji62.

  • feature

trigger-job now has a -w flag for watching the build that was created.

  • fix

fly now respects $http_proxy and $https_proxy for communication to the Concourse server, thanks to a PR from @ArthurHlt.

  • feature

The Docker Image resource now has a tag_as_latest param for tagging the image with latest, in addition to any specified tag, thanks to a PR from @shinji62.

Downloadsv1.2.0

v1.1.0

April 14, 2016

Release Notes

  • feature

Workers can now configure proxies to use for containers that are spun up on them.

If you're using the binaries, all you have to do is set the standard $http_proxy, $https_proxy, and $no_proxy environment variables. There are also equivalent flags you can pass to concourse worker, which were added for discoverability's sake.

If you're using BOSH, just set the http_proxy_url, https_proxy_url, and no_proxy properties on the groundcrew job.

  • feature

A task's run can now specify the working directory by setting dir.

  • feature

fly learned the targets command, which, surprise surprise, lists the currently saved targets.

  • fix

The blackbox job in the BOSH release will now once again emit logs, by autodiscovering them from /var/vcap/sys/log/*/*.log.

  • fix

Fixed rendering of leading whitespace on lines of output in build logs.

  • fix

Fixed the scrolling behavior of the pipelines sidebar list to not cut off the last couple of entries.

  • fix

The Docker Image resource is now durable to resource images that do not contain a /etc/password file.

  • fix

Previously renaming a pipeline made bad things happen to the automatic resource checking and scheduling for said pipeline. Instead of doing this it now renames the pipeline and the pipeline continues to work.

  • fix

Previously a put occurring at the start of the plan would not have its source directory created (as there were no artifacts), which would cause some resources to break. We now ensure this directory exists.

  • fix

You can now scroll up more easily when viewing a finished build. You are all free now!

  • feature

You can now run fly help and it'll show its help text instead of "unknown command."

  • fix

Previously if you had an entry in resource_types and resources with the same name the ATC catch on fire. It now doesn't.

  • fix

Users who are present in more than 30 GitHub organizations and/or teams can now authorize with Concourse. You should be rewarded for your popularity.

  • fix

Piping input into intercept will now send an EOF when the input is exhausted (e.g. echo foo | fly intercept ... cat).

Downloadsv1.1.0

v1.0.0

March 29, 2016

Release Notes

We made it!

This release, although relatively small on its own, is built on years of feedback and iteration. So these notes will be a bit more broad and cover all the things you may have missed since you last checked in on our little CI system.

First off, a huge thanks to Pivotal for sponsoring our project and letting us work on it full-time. Over the past year we've had 17 team members rotating through, including 2 designers. Pretty sweet.

With 1.0.0 comes a more rigid release policy on our end. You may see deprecations here and there, so keep an eye out for those via the tags next to each release note, but nothing should change backwards-incompatibly until 2.0.0. We'll still be releasing at the same cadence as before, so we'll probably end up at v1.23.0 pretty soon.

Here's a text-form 80's montage of all the things you may have missed since v0.17.0, our first release:

  • feature

Build Plans replaced the old style job config.

  • feature

A standalone binary distribution of Concourse has been introduced. (Download links to the right.)

  • feature

Caching and more efficient artifact propagation: resources fetched by get steps are cached on the workers and efficiently propagated throughout steps in the build plan.

  • feature

A single Concourse can be configured with multiple pipelines dynamically.

  • feature

Custom resource types can now be added via resource_types in the pipeline, rather than reconfiguring your workers.

  • fix

Lots of performance improvements and optimizations, and resilience to flaky networks.

  • feature

The fly CLI has been entirely rewritten and is much more consistent in UX.

  • feature

Concourse knows its own version number and will warn you if your CLI is out of date.

  • feature

Tasks have explicit inputs and outputs, making artifact consumption and production a lot easier to follow.

  • feature

A new color scheme that's more colorblind-friendly.

  • feature

A whole bunch of improvements to core resources.

...and now for the actual 1.0 release notes, if you're upgrading from v0.76.0:

  • feature

  • breaking

The Concourse BOSH release is now built for BOSH 2.0. You will need a recent director to upgrade. Consult Clusters with BOSH for information on the new BOSH 2.0 deployment scheme.

  • fix

Resources backed by a resource type defined in resource_types will now periodically check for new versions of the resource type and use the latest one for checking. Previously the same container would be reused forever even if a new version of the resource type was released.

  • feature

We've added aria-label attributes to all buttons in the UI, which should improve accessibiltiy for folks using screen readers. Still a ways to go overall, but this is a start.

  • fix

Lots of dots in sequence in build output will now word-wrap once again.

  • feature

The BOSH release can now be configured to use GitHub enterprise endpoints for GitHub auth.

  • fix

Connections from ATC to Baggage Claim will now retry on connection errors.

  • fix

Fixed an issue where volumes would "expire" even though a build was still using them. Did a bunch of refactoring and now it should all be pretty airtight.

  • fix

We've fixed a goroutine leak on the ATC which would occur every time image_resource was used.

Downloadsv1.0.0

v0.76.0

March 23, 2016

Release Notes

  • fix

v0.75.0 introduced a client-side limit of 64 connections to the database, which no one would ever hit so we didn't bother putting it in the release notes. Then a bunch of people with large deployments hit it and their Concourse went cold. Sorry.

We're removing the limit and are going to do some investigation into the ATC's connection pool characteristics before considering adding it back.

  • feature

The pipelines sidebar is now scrollable. Some of y'all had a lot of them and got tired of buying larger monitors.

  • feature

Jobs can now have their manual triggering disabled, via disable_manual_trigger.

  • feature

The BOSH deployment resource now supports BOSH 2.0 manifests. Previously it would explode instead.

  • feature

The ATC can now be configured to authenticate against a GitHub Enterprise deployment, thanks to @aequitas!

  • fix

Cleaned up some internals to fix the root cause some noisy but harmless log lines (failed-to-lookup-ttl).

  • feature

The Semver resource now supports OpenStack Swift as a storage backend, thanks to @ChrisPRobinson!

  • feature

The Time resource can now be configured to only yield new timestamps on certain days of the week, thanks to @joek!

  • feature

fly learned the rename-pipeline command, thanks to @zachgersh!

  • feature

The Docker Image resource should now be more durable to flaky Docker registries, by retrying with exponential backoff on network errors or 5xx responses.

  • feature

The BOSH deployment resource now downloads the deployment manifest when used as a get.

  • fix

Previously the Pool resource would require you to specify retry_delay in nanoseconds, which was a bit silly. It now accepts Go duration format, e.g. 30s.

  • fix

The Tracker resource now correctly handles rejected stories by only delivering them if a new commit has been made after they were rejected.

Downloadsv0.76.0

v0.75.0

March 9, 2016

Release Notes

  • feature

  • deprecation

The Semver resource now creates a file called version containing the version number, making it consistent with other resources that provide a version.

We still create number for backwards compatibility, but you should switch.

  • feature

  • deprecation

Specifying both file and config on a task step is now deprecated. You should receive warnings when running set-pipeline and when running a task that specifies both.

Instead, you should be specifying params, input_mapping, and output_mapping.

  • feature

The Fly CLI and the web UI now know their own version! We probably should have done this years ago. So fly -v now works instead of printing a shrugging emoticon, and the web UI now has the version at the bottom right (it even live updates, for all your CI monitors out there).

In addition, fly will print a warning if the versions are slightly out of sync (patch release), and straight up prevent itself from running if they're significantly out of sync (i.e. minor or major).

  • fix

Tagged workers are now supported by image_resource.

  • feature

The duration that containers stick around for after finishing is now configurable via new atc.retention.* BOSH properties (and corresponding flags to the ATC).

  • feature

intercept now sorts its container list, which should aid in frustration with finding the container to intercept.

  • feature

containers now shows the TTL (as we've configured it) and validity (actual expiration, which counts down to 0) for each container. This will be useful to know which ones are sticking around because they failed, and which ones are sticking around because of a build that's running too frequently.

  • feature

fly learned the abort-build command, thanks to a pull request from @zachgersh.

  • feature

fly learned the trigger-job command, thanks to a pull request from @aminjam.

  • feature

The BOSH deployment resource now supports deploying to a director using UAA client ID/secret for auth.

  • feature

We've bumped the version of Buildroot that many of our resources are based on, which should bring in updated CA certificates and other miscellaneous things.

  • fix

Previously resources that had params involving lists of objects would cause Concourse to blow up instead of working. It should now work.

Downloadsv0.75.0

v0.74.0

February 26, 2016

Release Notes

  • feature

You can now configure resource types in your pipeline rather than redeploying your workers with additional resource types. This should make it much easier to use the community resources that people have built!

  • feature

Autoscroll on a build page is back and is now implemented in a way that doesn't kill the browser when you have many build events.

  • feature

If the Docker image you specify in the image_resource section of your task has a custom user then we will now respect that when running the task. This user will also be used when hijacking in to a build container.

  • feature

Hijacked connections will no longer cause connection timeouts at interim load balancers if there is no input or output.

  • feature

The pipeline graph rendering now has large portions of the computation cached. This should provide a significant speedup and decrease in CPU load when viewing a pipeline.

  • feature

We've made some tweaks to the ATC's build scheduling that should fix "deadlock" scenarios with serial groups.

Previously, if a pipeline of "A -> B -> C" had all 3 jobs in a serial group, and the builds were enqueued in order of C, B, then A (manually), nothing could ever run, as the scheduling was based on the order of the builds being enqueued, and C would never be satisifed. This is now fixed by collecting inputs and then scheduling only once they're available, so that C never gets scheduled, and so A is able to be scheduled, followed by B, and then C.

  • fix

The icon font that was broken in Safari by v0.73.0 are now unbroken by v0.74.0.

  • fix

Interrupting a fly execute that was fetching outputs will no longer panic if you cancel it in the middle.

  • feature

If you try and trigger a build while you are not logged in then we'll now redirect you back to the build page you were on rather than the main pipeline page.

  • feature

The Pool resource will now ignore in-place modifications when working out if a log aquisition is still valid.

  • feature

The Fly CLI will now print the target it will be interacting with at the start of every command.

  • breaking

The Fly CLI default value of the -t flag has been removed. If you're using the VirtualBox distribution then you'll need to start logging in and supplying a target. This is to get people in this habit before they progress to a bigger deployment.

  • feature

The Fly CLI has a more sensible timeout and a better error message if it cannot reach the targeted Concourse.

  • feature

We bumped to Go 1.6. You should see absolutely no change.

Downloadsv0.74.0

v0.73.0

February 18, 2016

Release Notes

  • feature

  • breaking

The destroy-pipeline command now runs much quicker. Unfortunately to implement this there's a massive database migration. Expect anywhere from a few minutes of downtime up to a few hours when you upgrade to this version, depending on how many builds you have and how chatty they are. Sorry about that.

There is no way of predicting how long this migration may take for your instance. It depends on the chattiness of your builds and the performance of your database. For a rough approximation: running the migration on the Concourse team's server took 20 minutes to migrate 13 million build events.

You can find out how many build events you have by running the following query against your Concourse database:

SELECT relname, n_live_tup
FROM pg_stat_user_tables
WHERE relname = 'build_events';

The root of the issue is the amount of data in the build_events table. If you don't have many build logs then then you can probably just upgrade and not worry about the rest of this. If the upgrade is going to take too long then you'll need to find some way to reduce the rows in that table.

Before you delete any data from the system you should make sure to take a backup of your database and make doubly sure you can restore it while blind-folded and upside-down in case anything goes wrong.

If you don't care about your old build logs then you can simply run TRUNCATE build_events; before upgrading and the migration will be quick and painless.

Many of you probably do care about your build events but maybe only those which were created in the past X months. If this sounds like a good idea then have I got the SQL query for you! Run this against your Concourse database (change the X to the number of months you'd like to keep):

DELETE FROM build_events
WHERE build_id IN (
  SELECT builds.id
  FROM jobs JOIN builds ON jobs.id = builds.job_id
  WHERE builds.end_time < NOW() - INTERVAL 'X month'
  ORDER BY builds.id
);

This query may take a while to execute but your Concourse can be online the entire time that it is running.

If you want to keep all of your build logs and have a fast migration then I'm sorry, I can't help you. :( Maybe upgrade over a weekend?

  • breaking

The Docker Image resource no longer produces the docker saved image by default. This is to reduce disk usage when using the resource as an image_resource. You must now pass save: true as part of params on the get step to produce the image file.

  • feature

Inputs to jobs that are not configured to trigger the job when new versions appear will now be rendered with a dashed line. This makes it easier to see which resources automate the pipeline flow, and which jobs are only ever manually triggered.

  • feature

A pending build will now indicate why it's pending, via a checklist that appears at the top of the build output. (Yay!)

  • breaking

The theme selector is gone. So are all but one of the themes. This new theme is the product of our research and your feedback. I'm confident that it's perfect in every way. But let us know if you have any major problems with it.

  • fix

Improved the caching of resources used for image_resource. Previously if the same version was fetched multiple times on the same worker, we'd keep all of them around so long as they were the latest version. We'll now only keep one.

  • feature

The destroy-pipeline command learnt the -n option which when used will not ask you to confirm the deletion of the pipeline. Useful for scripts. Dangerous for users.

  • fix

The Docker Image resource no longer worked for images configured with ENTRYPOINTs as of v0.72.0, and ended up running whatever the entrypoint was, with our internal binary tool as an argument. It now works again. Our bad.

  • fix

The GitHub release resource would have issues when fetching artefacts from S3 via GitHub when using an access token. It no longer has these issues.

  • feature

The Docker Image resource now supports a dockerfile parameter for specifying a path to the Dockerfile to use.

  • feature

The Git resource now supports producing annotated tags via the annotate parameter.

  • feature

The Git resource now supports checking for tag patterns like *-production via the tag_filter source configuration.

  • feature

The Git resource now includes git lfs.

Downloadsv0.73.0

v0.72.1

February 8, 2016

Release Notes

  • fix

Fixed a bug that would cause containers used by the new image_resource not to be released.

  • fix

Removed a uniqueness constraint on the database that checked a huge number of columns which caused PostgreSQL to error for certain pipelines.

Downloadsv0.72.1

v0.72.0

February 6, 2016

Release Notes

  • feature

  • deprecation

We've added a new way of specifying the image for a task. You can now use image_resource which will use Concourse resources to pull the image down.

For example:

image: docker:///ubuntu#14.04

would now become:

image_resource:
  type: docker-image
  source:
    repository: ubuntu
    tag: 14.04

If those keys look familiar that's because they're the same ones you would specify if you were defining a resource in your pipeline. This means that because the Docker Image resource supports private repositories you can now use private repositories for your task images. There are more details in the Tasks section of the documentation.

Further improvements to this feature are coming soon!

  • feature

  • breaking

Resources now have access to the external URL of the Concourse that they are associated with in the environment variable ATC_EXTERNAL_URL.

The property atc.external_url is now required for ATC to start up.

  • breaking

In order for a task to make something for a later step in the plan it now must use outputs. Previously the task's entire working directory was made available if outputs were not configured, which led to more network traffic than necessary if you didn't intend for the task to actually propagate anything to later steps.

  • feature

We now validate task configuration at the start of a task. Sneaky things like typos and overlapping inputs and outputs are now hard errors.

  • feature

The new default theme concourse is the culmination of our work on color schemes. It will soon take its place as the one and only color scheme.

If you've been changing your theme around, make sure to manually switch to the concourse theme to check it out, otherwise it'll stick with whatever scheme you last selected.

  • feature

Failed jobs are now displayed with an additional border and a symbol above the column that they are contained in. This change along with the improved color scheme should help the 315,000,000 potential and current Concourse users around the globe who are color blind.

  • feature

The atc.publicly_viewable flag now applies to the API endpoints as well as the web interface. There was no secret information exposed by the API but potentially sensitive things like job names for unannounced projects were there.

  • fix

We fixed the bug that would show your entire pipeline in the diff when uploading changes to your pipeline.

  • fix

New line characters in resource metadata output are now preserved again when viewing them in the web build view.

  • fix

If a job has both pending and running build we will now show the running build animation on the dashboard rather than the less interesting pending animation.

  • fix

If you view a job or build in a paused pipeline then the top bar on the page will be blue again.

  • feature

fly now presents a far more helpful error message if you're not logged in.

  • feature

The pipeline layout algorithm has changed to draw resources at an appropriate height so that more of the lines are straighter.

  • fix

The list of containers returned when you try and hijack is now correctly filtered to the build number you ask for.

  • fix

The ATC doesn't panic anymore if the database disappears in the middle of a build. It still gets pretty mad though so try and keep your database disappearing to a minimum.

  • feature

The Tracker Resource now understands variations on the word "completed" when scanning commit messages.

  • fix

ATC will now handle if you supply an atc.external_url with a trailing slash rather than not letting GitHub users log in. A significant improvement if I do say so myself.

Downloadsv0.72.0