| 1 | --- |
| 2 | title: Blue/green deployments |
| 3 | description: Deploy a new version, check it before serving traffic, and give the old version time to finish work. |
| 4 | --- |
| 5 | |
| 6 | Use blue/green when you want to start a checked replacement before stopping your running Service. Add the deployment policy to your application: |
| 7 | |
| 8 | ```typescript title="api.ts" |
| 9 | import * as Fly from "alchemy/Fly"; |
| 10 | import * as Effect from "effect/Effect"; |
| 11 | import * as HttpServerResponse from "effect/http/HttpServerResponse"; |
| 12 | |
| 13 | export default class Api extends Fly.Service<Api>()( |
| 14 | "Api", |
| 15 | { |
| 16 | main: import.meta.url, |
| 17 | deploy: { strategy: "bluegreen" }, |
| 18 | }, |
| 19 | Effect.succeed({ |
| 20 | fetch: Effect.succeed(HttpServerResponse.text("hello")), |
| 21 | }), |
| 22 | ) {} |
| 23 | ``` |
| 24 | |
| 25 | You still declare one Service. Alchemy creates the temporary replacement Machines during each rollout. |
| 26 | |
| 27 | ## Deploy named containers as a group |
| 28 | |
| 29 | `Fly.Machine` also accepts named containers for a blue/green deployment. Pin |
| 30 | **every** image to a full `repository@sha256:<digest>` reference and configure |
| 31 | Machine or service readiness checks that cover every critical component. |
| 32 | Container health checks and `dependsOn` govern startup; they do not replace |
| 33 | Alchemy's deployment readiness checks. |
| 34 | |
| 35 | A Machine runs in a [`Fly.App`](/fly/compute/apps) you declare: |
| 36 | |
| 37 | ```typescript |
| 38 | const app = yield* Fly.App("Previews"); |
| 39 | |
| 40 | const preview = yield* Fly.Machine("Preview", { |
| 41 | app, |
| 42 | deploy: { strategy: "bluegreen" }, |
| 43 | containers: [ |
| 44 | { name: "api", image: apiDigest, healthChecks: [{ http: { port: 3000, path: "/health" } }] }, |
| 45 | { name: "worker", image: workerDigest, dependsOn: [{ name: "api", condition: "healthy" }], |
| 46 | healthChecks: [{ http: { port: 3001, path: "/health" } }] }, |
| 47 | ], |
| 48 | checks: { |
| 49 | apiReady: { type: "http", port: 3000, path: "/health" }, |
| 50 | workerReady: { type: "http", port: 3001, path: "/health" }, |
| 51 | }, |
| 52 | }); |
| 53 | ``` |
| 54 | |
| 55 | The worker image in this example exposes `/health` on port 3001. Alchemy |
| 56 | replaces the full group, even if only one image changes. New workers |
| 57 | may start before HTTP promotion, so their jobs need overlap-safe ownership or |
| 58 | drain behavior. Attached volumes, including `MountVolume` bindings, remain |
| 59 | incompatible with blue/green; use rolling updates for a local database. |
| 60 | |
| 61 | Reordering containers or their startup dependencies does not create a new |
| 62 | generation. Changing a dependency condition, image, command or environment does. |
| 63 | The shutdown budget applies to the Machine. Each container must handle the |
| 64 | configured signal and finish its own in-flight work; setting `shutdown` does not |
| 65 | install a signal handler in your images. |
| 66 | |
| 67 | ## Put the Service in your stack |
| 68 | |
| 69 | Return the Service URL for your clients: |
| 70 | |
| 71 | ```typescript title="alchemy.run.ts" |
| 72 | import * as Alchemy from "alchemy"; |
| 73 | import * as Fly from "alchemy/Fly"; |
| 74 | import * as Effect from "effect/Effect"; |
| 75 | import Api from "./api.ts"; |
| 76 | |
| 77 | export default Alchemy.Stack( |
| 78 | "Site", |
| 79 | { providers: Fly.providers(), state: Alchemy.localState() }, |
| 80 | Effect.gen(function* () { |
| 81 | const api = yield* Api; |
| 82 | return { url: api.url }; |
| 83 | }), |
| 84 | ); |
| 85 | ``` |
| 86 | |
| 87 | Use `api.url` rather than a Machine ID or private IP in client configuration. The Service's App and its URL survive a rollout; the underlying Machines are replaced. |
| 88 | |
| 89 | ## Deploy a new response |
| 90 | |
| 91 | Change the handler in `api.ts`: |
| 92 | |
| 93 | ```diff lang="typescript" title="api.ts" |
| 94 | Effect.succeed({ |
| 95 | - fetch: Effect.succeed(HttpServerResponse.text("hello")), |
| 96 | + fetch: Effect.succeed(HttpServerResponse.text("hello v2")), |
| 97 | }), |
| 98 | ``` |
| 99 | |
| 100 | Deploy it with the same command you used for the first version: |
| 101 | |
| 102 | ```sh |
| 103 | alchemy deploy |
| 104 | ``` |
| 105 | |
| 106 | For the default single replica, expect this transition: |
| 107 | |
| 108 | ```text |
| 109 | Before deploy One Machine serves "hello". |
| 110 | Replacement ready Old and new Machines can both serve requests. |
| 111 | Deploy complete One Machine serves "hello v2" at the same URL. |
| 112 | ``` |
| 113 | |
| 114 | Keep API and database changes compatible with both versions during the overlap. Budget for the temporary extra Machine as well as your usual capacity. |
| 115 | |
| 116 | ## Check an HTTP response before switching traffic |
| 117 | |
| 118 | To require a successful HTTP response rather than just an open TCP port, add `services` to the `Api` props. This check calls `/`, which the handler above already serves: |
| 119 | |
| 120 | ```diff lang="typescript" title="api.ts" |
| 121 | main: import.meta.url, |
| 122 | deploy: { strategy: "bluegreen" }, |
| 123 | + services: [{ |
| 124 | + protocol: "tcp", |
| 125 | + internalPort: 3000, |
| 126 | + ports: [ |
| 127 | + { port: 80, handlers: ["http"], forceHttps: true }, |
| 128 | + { port: 443, handlers: ["tls", "http"] }, |
| 129 | + ], |
| 130 | + checks: [{ type: "http", path: "/", interval: "5s", timeout: "2s" }], |
| 131 | + }], |
| 132 | ``` |
| 133 | |
| 134 | If your application has a `/healthz` route that checks its dependencies, use that path instead. The replacement stays out of public traffic until its checks pass; every public service you configure needs its own check. |
| 135 | |
| 136 | ## Allow more time for startup checks |
| 137 | |
| 138 | If your application needs longer than the default 60-second readiness window, change the policy in `Api`: |
| 139 | |
| 140 | ```diff lang="typescript" title="api.ts" |
| 141 | - deploy: { strategy: "bluegreen" }, |
| 142 | + deploy: { strategy: "bluegreen", healthTimeout: "90 seconds" }, |
| 143 | ``` |
| 144 | |
| 145 | This changes how long Alchemy waits for passing checks after startup. It does not create a new generation by itself; the maximum is five minutes. |
| 146 | |
| 147 | ## Keep idle capacity with stop or suspend |
| 148 | |
| 149 | Provision three replicas by adding `count` to the Service props: |
| 150 | |
| 151 | ```diff lang="typescript" title="api.ts" |
| 152 | main: import.meta.url, |
| 153 | + count: 3, |
| 154 | ``` |
| 155 | |
| 156 | Then let Fly stop idle Machines and wake them for incoming requests. Add these settings to the public service declared above: |
| 157 | |
| 158 | ```diff lang="typescript" title="api.ts" |
| 159 | services: [{ |
| 160 | protocol: "tcp", |
| 161 | internalPort: 3000, |
| 162 | + autostop: "stop", |
| 163 | + autostart: true, |
| 164 | + minMachinesRunning: 1, |
| 165 | ``` |
| 166 | |
| 167 | You still own three Machines, but not all three have to run continuously. During deployment, Alchemy checks a running replacement and any additional replacements needed for Fly's configured running floor; unused capacity can stay idle. |
| 168 | |
| 169 | To allow suspension instead of stopping, change that one setting: |
| 170 | |
| 171 | ```diff lang="typescript" title="api.ts" |
| 172 | - autostop: "stop", |
| 173 | + autostop: "suspend", |
| 174 | ``` |
| 175 | |
| 176 | Use this only if your application can also cold-start: a resume can fall back to a new process, and a deployment does not carry suspended memory into the replacement. Suspension itself does not run SIGTERM cleanup. |
| 177 | |
| 178 | ## Give streaming responses time to finish |
| 179 | |
| 180 | If your API streams responses that can take 40 seconds, increase blue/green's default 30-second shutdown budget. Add `shutdown` to `Api`: |
| 181 | |
| 182 | ```diff lang="typescript" title="api.ts" |
| 183 | main: import.meta.url, |
| 184 | + shutdown: { timeout: "60 seconds" }, |
| 185 | ``` |
| 186 | |
| 187 | :::caution[Deploy the shutdown policy before relying on it] |
| 188 | The new policy applies to the version you are deploying, not the one already running. That first replacement may still interrupt requests. Later replacements use the old version's own shutdown policy, even if you give the new version a shorter timeout. |
| 189 | ::: |
| 190 | |
| 191 | On a subsequent replacement, the old process handles SIGTERM like this: |
| 192 | |
| 193 | ```text |
| 194 | 0s Stop accepting new requests. Existing responses continue. |
| 195 | 48s Close remaining HTTP connections (80% of the budget). |
| 196 | 54s Force exit if cleanup is still running (90%). |
| 197 | 60s Fly's configured termination deadline. |
| 198 | ``` |
| 199 | |
| 200 | A response with 40 seconds remaining has time to finish; a stream still open after 48 seconds is disconnected. Choose the budget using that cutoff, not the full 60 seconds. Sending another signal does not restart it. |
| 201 | |
| 202 | For jobs, [register your queue worker's cleanup](#close-your-queue-worker-on-shutdown). It runs while HTTP responses drain, with shared dependencies available until cleanup finishes or the process exits. |
| 203 | |
| 204 | ## Deploy a private queue worker |
| 205 | |
| 206 | Keep queue processing in `run` and expose a private readiness endpoint through `fetch`. This BullMQ example returns HTTP 503 until its initial Redis connection is ready: |
| 207 | |
| 208 | ```typescript title="jobs-service.ts" |
| 209 | import * as Fly from "alchemy/Fly"; |
| 210 | import * as Config from "effect/Config"; |
| 211 | import * as Effect from "effect/Effect"; |
| 212 | import * as Ref from "effect/Ref"; |
| 213 | import * as HttpServerResponse from "effect/http/HttpServerResponse"; |
| 214 | import { runJobs } from "./jobs.ts"; |
| 215 | |
| 216 | export default class Jobs extends Fly.Service<Jobs>()( |
| 217 | "Jobs", |
| 218 | { |
| 219 | main: import.meta.url, |
| 220 | services: [], |
| 221 | checks: { ready: { type: "http", port: 3000, path: "/" } }, |
| 222 | deploy: { strategy: "bluegreen" }, |
| 223 | shutdown: { timeout: "60 seconds" }, |
| 224 | }, |
| 225 | Effect.gen(function* () { |
| 226 | const host = yield* Config.String("REDIS_HOST"); |
| 227 | const ready = yield* Ref.make(false); |
| 228 | return { |
| 229 | fetch: Ref.get(ready).pipe(Effect.map((value) => |
| 230 | HttpServerResponse.text(value ? "ready" : "starting", { |
| 231 | status: value ? 200 : 503, |
| 232 | }), |
| 233 | )), |
| 234 | run: runJobs(host, ready), |
| 235 | }; |
| 236 | }), |
| 237 | ) {} |
| 238 | ``` |
| 239 | |
| 240 | `services: []` keeps the worker off the proxy; the worker runs in its own App. Fly can still call its named check on port 3000 before Alchemy retires the old worker. |
| 241 | |
| 242 | ## Close your queue worker on shutdown |
| 243 | |
| 244 | For an existing BullMQ queue, install the client in your application: |
| 245 | |
| 246 | ```sh |
| 247 | pnpm add bullmq |
| 248 | ``` |
| 249 | |
| 250 | Implement `runJobs` by pairing worker creation with `worker.close()` in an Effect scope. `processJob` below is your existing BullMQ processor: |
| 251 | |
| 252 | ```typescript title="jobs.ts" |
| 253 | import { Worker } from "bullmq"; |
| 254 | import * as Effect from "effect/Effect"; |
| 255 | import * as Ref from "effect/Ref"; |
| 256 | import { processJob } from "./process-job.ts"; |
| 257 | |
| 258 | export const runJobs = (host: string, ready: Ref.Ref<boolean>) => |
| 259 | Effect.gen(function* () { |
| 260 | const worker = yield* Effect.acquireRelease( |
| 261 | Effect.sync(() => new Worker("jobs", processJob, { |
| 262 | connection: { host }, |
| 263 | }).on("error", console.error)), |
| 264 | (worker) => Effect.promise(() => worker.close()), |
| 265 | ); |
| 266 | yield* Effect.promise(() => worker.waitUntilReady()); |
| 267 | yield* Ref.set(ready, true); |
| 268 | yield* Effect.never; |
| 269 | }).pipe(Effect.scoped); |
| 270 | ``` |
| 271 | |
| 272 | On shutdown, `worker.close()` stops taking new jobs and waits for active processors before closing the worker's connections. It runs inside `run`, so the deployment process never opens the queue connection. Configure Redis authentication and TLS in `connection` as required by your queue. |
| 273 | |
| 274 | With the 60-second budget above, unfinished jobs can still be cut off at 54 seconds. Make `processJob` [safe to retry](https://docs.bullmq.io/patterns/idempotent-jobs): old and new workers can consume the queue at the same time, and a terminated job can be delivered again. |
| 275 | |
| 276 | ## Add the worker to your stack |
| 277 | |
| 278 | Import `Jobs` in `alchemy.run.ts`: |
| 279 | |
| 280 | ```diff lang="typescript" title="alchemy.run.ts" |
| 281 | import Api from "./api.ts"; |
| 282 | +import Jobs from "./jobs-service.ts"; |
| 283 | ``` |
| 284 | |
| 285 | Yield it alongside the API: |
| 286 | |
| 287 | ```diff lang="typescript" title="alchemy.run.ts" |
| 288 | const api = yield* Api; |
| 289 | + yield* Jobs; |
| 290 | return { url: api.url }; |
| 291 | ``` |
| 292 | |
| 293 | Set `REDIS_HOST` in your deployment environment to a Redis host reachable from the Fly Machines, then run `alchemy deploy`. The API and worker now have independent deployment and cleanup lifetimes. |
| 294 | |
| 295 | ## Retry after a failed readiness check |
| 296 | |
| 297 | Fix the failing endpoint or startup error in your application, then redeploy: |
| 298 | |
| 299 | ```sh |
| 300 | alchemy deploy |
| 301 | ``` |
| 302 | |
| 303 | A definite check failure before the traffic switch leaves the old version serving. Alchemy removes confirmed unpromoted replacements; if the error reports uncertain creation or cleanup, the next deploy first reconciles what Fly actually created. |
| 304 | |
| 305 | ## Resume an interrupted deployment |
| 306 | |
| 307 | If your terminal disconnected or deployment failed while retiring old Machines, rerun the same stack with the same desired configuration: |
| 308 | |
| 309 | ```sh |
| 310 | alchemy deploy |
| 311 | ``` |
| 312 | |
| 313 | Alchemy can keep an already-promoted replacement and finish removing its predecessor. Keep the existing logical IDs and ownership metadata so it can find both versions; renaming the resource starts a different lifecycle. |
| 314 | |
| 315 | If recovery reports contradictory deployment metadata, inspect the reported |
| 316 | Machine before retrying. Removing or downgrading protocol metadata is not a |
| 317 | rollback: Alchemy preserves ambiguous capacity instead of treating it as a |
| 318 | completed deployment. Roll back application images using the same provider |
| 319 | version and logical resource IDs. |
| 320 | |
| 321 | If you want to remove the application instead, use normal stack cleanup: |
| 322 | |
| 323 | ```sh |
| 324 | alchemy destroy |
| 325 | ``` |
| 326 | |
| 327 | An ambiguous App-deletion error needs inspection before you reuse that App name. A delayed name-based delete could otherwise target a newly created App. |
| 328 | |
| 329 | ## Serialize production deployments in CI |
| 330 | |
| 331 | Add a concurrency group at the top level of your existing GitHub Actions deployment workflow: |
| 332 | |
| 333 | ```yaml title=".github/workflows/deploy.yml" |
| 334 | concurrency: |
| 335 | group: fly-site-prod |
| 336 | cancel-in-progress: false |
| 337 | ``` |
| 338 | |
| 339 | Use a distinct group for each stack and stage. This prevents a newer workflow from interrupting an active rollout; Fly's per-Machine leases do not serialize whole deployments or protect against a simultaneous manual deploy. |
| 340 | |
| 341 | ## Rotate secrets with your application |
| 342 | |
| 343 | When your API handler needs a credential, import Effect Config and your handler in `api.ts`: |
| 344 | |
| 345 | ```diff lang="typescript" title="api.ts" |
| 346 | import * as HttpServerResponse from "effect/http/HttpServerResponse"; |
| 347 | +import * as Config from "effect/Config"; |
| 348 | +import { handleRequest } from "./handler.ts"; |
| 349 | ``` |
| 350 | |
| 351 | Resolve the secret in the Service's initialization Effect and pass it to that handler: |
| 352 | |
| 353 | ```diff lang="typescript" title="api.ts" |
| 354 | - Effect.succeed({ |
| 355 | - fetch: Effect.succeed(HttpServerResponse.text("hello v2")), |
| 356 | + Effect.gen(function* () { |
| 357 | + const apiKey = yield* Config.Redacted("API_KEY"); |
| 358 | + return { fetch: handleRequest(apiKey) }; |
| 359 | }), |
| 360 | ``` |
| 361 | |
| 362 | After changing `API_KEY` in your deployment environment, run: |
| 363 | |
| 364 | ```sh |
| 365 | alchemy deploy |
| 366 | ``` |
| 367 | |
| 368 | Alchemy prepares the Service-bound secret before starting replacements. Keep both versions compatible with the new credential during overlap: both generations read the Service's App vault, and a failed rollout does not restore its old values. |
| 369 | |
| 370 | ## Require a secret version in a raw Machine |
| 371 | |
| 372 | If you rotate an App secret outside the deployment and Fly reports version 42, declare that minimum on your existing `Fly.Machine` resource in that App: |
| 373 | |
| 374 | ```diff lang="typescript" |
| 375 | const worker = Fly.Machine("Worker", { |
| 376 | app, |
| 377 | image: "registry.example.com/worker:v2", |
| 378 | + minSecretsVersion: 42, |
| 379 | ``` |
| 380 | |
| 381 | Then run `alchemy deploy` to reconcile the changed input. Version 42 is a minimum: another writer can advance the shared vault to 43, and an out-of-band write alone does not automatically redeploy this resource. |
| 382 | |
| 383 | ## Deploy a website with the same policy |
| 384 | |
| 385 | Add the policy to your website declaration inside its stack: |
| 386 | |
| 387 | ```diff lang="typescript" title="alchemy.run.ts" |
| 388 | const site = yield* Fly.Website.Vite("Web", { |
| 389 | rootDir: "./web", |
| 390 | + deploy: { strategy: "bluegreen" }, |
| 391 | + shutdown: { timeout: "60 seconds" }, |
| 392 | }); |
| 393 | ``` |
| 394 | |
| 395 | Run `alchemy deploy` after changing frontend assets or server code. The website builds a new image and replaces its hosted Service; other `Fly.Website` frameworks and `StaticSite` accept the same props. |
| 396 | |
| 397 | :::caution[Test your server's SIGTERM handling] |
| 398 | For a website server or raw container, these props configure Fly's stop request; they do not install Alchemy's managed Effect shutdown handler. Use your framework's graceful-shutdown support and test a deployment with an active request before relying on the budget. |
| 399 | ::: |
| 400 | |
| 401 | ## Use rolling updates for a volume-backed Service |
| 402 | |
| 403 | If the Service mounts a Fly volume, keep its deployment strategy rolling: |
| 404 | |
| 405 | ```diff lang="typescript" title="api.ts" |
| 406 | - deploy: { strategy: "bluegreen", healthTimeout: "90 seconds" }, |
| 407 | + deploy: { strategy: "rolling", healthTimeout: "90 seconds" }, |
| 408 | ``` |
| 409 | |
| 410 | A Fly volume cannot be shared by both generations, so blue/green rejects attached volumes, including `MountVolume` bindings. Keeping `shutdown` still gives scale-down and deletion a cleanup window, but a single Machine can be unavailable during its in-place update. |
| 411 | |
| 412 | ## Where next |
| 413 | |
| 414 | - [Services](/fly/compute/services) for application handlers and background programs. |
| 415 | - [Machines](/fly/compute/machines) for existing container images. |
| 416 | - [Networking](/fly/networking) for public addresses and domains. |
| 417 | - [Secrets](/fly/data/secrets) for App vault values and runtime configuration. |
| 418 | - [Machine reference](/providers/fly/machine) and [Service reference](/providers/fly/service) for supported settings and validation rules. |