| 1 | # Release Checklist |
| 2 | |
| 3 | > For maintainers. Using T3 Code? See [docs/user](../user/). |
| 4 | |
| 5 | This document covers the unified release workflow for stable and nightly desktop releases. |
| 6 | |
| 7 | ## What the workflow does |
| 8 | |
| 9 | - Workflow: `.github/workflows/release.yml` |
| 10 | - Triggers: |
| 11 | - manual `workflow_dispatch` with `channel=stable`, the normal way to ship stable. Stable |
| 12 | and nightly dispatches must select `main`; preview may select any branch. The channel defaults |
| 13 | to preview so an omitted selection cannot publish a stable release. |
| 14 | - push tag matching `v*.*.*` for a stable release of an explicit commit |
| 15 | - scheduled nightly check every 30 minutes |
| 16 | - manual `workflow_dispatch` with `channel=nightly` |
| 17 | - manual `workflow_dispatch` with `channel=preview`, the maintainers' test train. It exercises the whole release flow (build, sign, notarize, smoke, publish) for a commit that end users must never receive, which is how an unmerged branch or a risky change gets a real release run before it lands. It builds the triggering commit with nightly's versioning under the `preview` prerelease identifier (`0.0.41-preview.<date>.<run>`) and publishes a GitHub prerelease plus the npm packages under the `preview` dist-tag. Preview is not on the schedule, no default npm dist-tag points at it, its desktop builds carry no update feed, and no updater manifest (`latest*.yml`, `nightly*.yml`, blockmaps) is attached, so a stable or nightly install cannot be offered one. The only ways onto it are downloading the release by hand, `npx t3@preview`, `T3CODE_CHANNEL=preview` for the install scripts, or `t3 update --channel preview` from a terminal; each prints a warning, and the CLI asks for confirmation when the running build is not itself a preview. The release itself is named as a maintainer test build and its body is a warning rather than generated notes: a changelog of unmerged branch history is not a changelog, and nightly and stable notes are unaffected because each series resolves its previous tag within its own channel. The hosted web app, AUR, and Discord announcements are skipped. Keep it; it costs nothing when idle. |
| 18 | - A manual stable release builds the commit of the latest published nightly, not `main` HEAD. |
| 19 | Nightly is the release candidate: verify the nightly, then promote it. Merges to `main` keep |
| 20 | landing while you verify and never leak into the stable build. |
| 21 | - The version defaults to the one the nightly previewed (`0.0.39-nightly.*` ships as `0.0.39`). |
| 22 | Pass the `version` input to override it, for example for a minor bump. |
| 23 | - The stable tag is created on the nightly's commit when the GitHub Release is published. |
| 24 | - Pushing a `vX.Y.Z` tag by hand still works and builds exactly the tagged commit. Use it when |
| 25 | the commit to ship is not the latest nightly, such as a cherry-picked fix on a release branch. |
| 26 | - Runs lint, typecheck, and tests alongside artifact builds. Publishing waits for every check. |
| 27 | - Reads the shared production T3 Connect relay URL and Clerk client configuration before packaging clients. |
| 28 | - Builds the platform-independent JS (server bundle, web client, Electron main) once in the `build_bundle` job and hands it to every platform job as the `js-bundle` artifact; the platform jobs only package it, so no runner rebuilds it. |
| 29 | - Builds six desktop artifacts in parallel for both channels, each as its own job (`desktop_<platform>_<arch>`, one call of `release-desktop.yml`) on hardware of its own architecture, gated only on the bundle. The Windows jobs embed the same-arch Linux CLI archive as the WSL runtime and wait for that artifact partway through, not for the whole Linux job: |
| 30 | - macOS `arm64` DMG |
| 31 | - macOS `x64` DMG |
| 32 | - Linux `x64` and `arm64` AppImage and `.deb`, from one electron-builder run. The `.deb` updates in the app through electron-updater, which installs it with `dpkg`. |
| 33 | - Windows `x64` and `arm64` NSIS installer |
| 34 | - Publishes one GitHub Release with all produced files. |
| 35 | - Stable tags with a suffix after `X.Y.Z` (for example `1.2.3-alpha.1`) are published as GitHub prereleases. |
| 36 | - Only plain stable `X.Y.Z` releases are marked as the repository's latest release. |
| 37 | - Nightly runs are always GitHub prereleases and never marked latest. |
| 38 | - Automatically generated release notes are pinned to the previous tag in the same channel, so stable compares to the previous stable tag and nightly compares to the previous nightly tag. |
| 39 | - Includes Electron auto-update metadata (for example `latest*.yml`, `nightly*.yml`, and `*.blockmap`) in release assets. |
| 40 | - Builds a self-contained CLI archive per platform (`t3-<version>-<platform>-<arch>.tar.gz`, `.zip` on Windows) in the same job as that target's desktop artifact and attaches them to the GitHub Release with a `SHA256SUMS` file, on every channel, for five targets: macOS arm64, Linux x64 and arm64, Windows x64 and arm64. Every archive is built, signed, and smoke-tested on hardware of its own architecture. There is no macOS x64 archive: Node single-executables are unsupported on x64 macOS (the SEA docs list macOS as arm64 only) and the binary segfaults on start; the x64 desktop app is Electron and unaffected. |
| 41 | - The archive holds the server as a Node single-executable (`scripts/build-cli-archive.ts`), so unpacking it needs neither Node, npm, nor a compiler. It is the only form in which T3 Code manages a runtime: the desktop's SSH environments, the boot service, `t3 update`, and the install scripts all download and verify this archive against `SHA256SUMS`. The npm packages exist for people who run `npx t3` or `npm install -g t3` themselves and carry the same archive contents; nothing in the product installs from npm. The `curl | sh` installers are `scripts/install.sh` and `scripts/install.ps1`; the marketing site copies them into its `public/` at build time (`apps/marketing/scripts/stage-install-scripts.mjs`) and serves them at `t3.codes/install.sh` and `/install.ps1`. |
| 42 | - The executable is built with a Node that supports `--build-sea` (`VP_NODE_VERSION=26.8.2`, kept in step with `SEA_NODE_VERSION` in `apps/server/vite.config.ts`), while the repo stays on `engines.node`. |
| 43 | - macOS archives are signed with the Developer ID certificate and notarized when the Apple secrets are present (ad hoc otherwise, which still runs from `curl`/`tar` installs). Windows executables use the same Azure Trusted Signing setup as the installer. Every native addon in the macOS archive is signed too, since the hardened runtime refuses unsigned libraries. |
| 44 | - Each archive is extracted and executed on its build runner (`scripts/smoke-cli-archive.ts`) before it is uploaded. |
| 45 | - Publishes the CLI to npm with OIDC trusted publishing from the same workflow file, as the same bytes the GitHub Release carries: `scripts/build-npm-platform-packages.ts` unpacks the five CLI archives into `@t3code/t3-<platform>-<arch>` packages (each with `os`/`cpu` set so npm installs only the matching one) and generates the `t3` launcher, whose `bin/t3.js` lists them as `optionalDependencies` and execs the installed executable. `npx t3` therefore needs Node only to run the launcher, never to run the server. `node apps/server/scripts/cli.ts publish` publishes the platform packages first and the launcher last, after a `--dry-run` pass over all of them so an auth or scope error fails before anything is live. |
| 46 | - stable releases publish npm dist-tag `latest` |
| 47 | - nightly releases publish npm dist-tag `nightly` |
| 48 | - preview releases publish npm dist-tag `preview`, which nothing resolves unless asked for by name |
| 49 | - one-time setup: the `@t3code` npm scope (org) must exist, and `t3` and each `@t3code/t3-<platform>-<arch>` package needs a trusted publisher registered for this workflow file (see below). |
| 50 | - Builds the hosted web app on Vercel while the desktop jobs run, and makes it live only after a release is published: |
| 51 | - stable releases are aliased to the `latest` hosted app channel |
| 52 | - nightly releases are aliased to the `nightly` hosted app channel |
| 53 | - Signing is optional and auto-detected per platform from secrets. |
| 54 | |
| 55 | ## Pull request macOS previews |
| 56 | |
| 57 | Labeling a PR `preview:mac` publishes a signed, notarized Apple Silicon DMG with T3 Connect enabled |
| 58 | to the rolling `desktop-preview` prerelease, and works for fork PRs. The label is a one-shot request |
| 59 | for the commit it is applied to: the trusted workflow removes it once the build is in hand, and later |
| 60 | pushes do not build until a maintainer applies it again. Every signed preview is therefore a |
| 61 | per-commit maintainer decision, which matters because the result carries the Developer ID signature. |
| 62 | Vouching a contributor lets their labeled commits be signed; it is not a standing grant. The build is |
| 63 | split so the Developer ID certificate never shares a job with PR code: |
| 64 | |
| 65 | - `.github/workflows/desktop-macos-preview.yml` runs on `pull_request` with no secrets and builds |
| 66 | only the JS bundle from the PR (the same `js-bundle` artifact `release.yml` produces). |
| 67 | - `.github/workflows/desktop-macos-preview-publish.yml` runs on `workflow_run` from `main`. It |
| 68 | refuses unless the PR is open, still labeled, its head is the built commit, and the author is a |
| 69 | bot, a collaborator, or listed in `.github/VOUCHED.td` (read from the default branch, so a PR cannot vouch |
| 70 | for itself). It then packages and signs the bundle through `release-desktop.yml` checked out at |
| 71 | `main`, so packaging, native helpers, and the Electron/desktop dependencies come from `main`, not |
| 72 | the PR. Only the version and the public T3 Connect identifiers in `.env.example` are read from the |
| 73 | PR commit, as data, so the signed app's passkey entitlement matches the bundle. A PR that changes |
| 74 | packaging must use the `channel=preview` release train above instead. |
| 75 | |
| 76 | Before handing the bundle to the signing runner, the trusted workflow validates its ZIP entries |
| 77 | and accepts only regular files under `server/dist` and `desktop/dist-electron`, plus the directory |
| 78 | entries that lead to those roots. The artifact cannot |
| 79 | overwrite packaging code or installed dependencies. The bundle is copied into the app, never executed, |
| 80 | on the signing runner. The |
| 81 | `pull_request_target` cleanup job in the publish workflow removes the download when the PR closes, or |
| 82 | when the label is removed by hand before a build consumed it, and never checks out PR code. |
| 83 | |
| 84 | ## Required release credentials |
| 85 | |
| 86 | Stable releases require these GitHub Actions secrets in addition to the platform and deployment |
| 87 | credentials documented below: |
| 88 | |
| 89 | - `RELEASE_APP_ID` |
| 90 | - `RELEASE_APP_PRIVATE_KEY` |
| 91 | |
| 92 | The finalize job uses them to commit and push aligned package versions to `main` as the Release App. |
| 93 | GitHub Release publication uses the repository-scoped workflow token so it has a rate-limit quota |
| 94 | independent from the shared Release App installation. |
| 95 | |
| 96 | ## T3 Connect relay deployment |
| 97 | |
| 98 | The relay is a shared control plane versioned separately from client releases. Stable and nightly |
| 99 | client builds must point at the same relay so users see the same linked environments when switching |
| 100 | release channels. |
| 101 | |
| 102 | `.github/workflows/deploy-relay.yml` deploys Alchemy stage `prod` on every push to `main`. The |
| 103 | release workflow reads the relay URL and Clerk client configuration from the existing `production` |
| 104 | GitHub Actions environment before building desktop, CLI, or hosted web artifacts. |
| 105 | |
| 106 | Required repository variables shared by relay deployments: |
| 107 | |
| 108 | - `CLOUDFLARE_ACCOUNT_ID` |
| 109 | - `PLANETSCALE_ORGANIZATION` |
| 110 | - `AXIOM_ORG_ID` |
| 111 | |
| 112 | Required repository secrets shared by relay deployments: |
| 113 | |
| 114 | - `CLOUDFLARE_API_TOKEN` |
| 115 | - `PLANETSCALE_API_TOKEN_ID` |
| 116 | - `PLANETSCALE_API_TOKEN` |
| 117 | - `AXIOM_TOKEN` |
| 118 | |
| 119 | Required `production` environment variables: |
| 120 | |
| 121 | - `RELAY_API_ZONE_NAME` |
| 122 | - `RELAY_TUNNEL_ZONE_NAME` |
| 123 | - `CLERK_PUBLISHABLE_KEY` |
| 124 | - `CLERK_JWT_AUDIENCE` |
| 125 | - `CLERK_JWT_TEMPLATE` |
| 126 | - `CLERK_CLI_OAUTH_CLIENT_ID` |
| 127 | - `APNS_ENVIRONMENT` |
| 128 | - `APNS_TEAM_ID` |
| 129 | - `APNS_KEY_ID` |
| 130 | - `APNS_BUNDLE_ID` |
| 131 | |
| 132 | Optional `production` environment variables: |
| 133 | |
| 134 | - `RELAY_DOMAIN` when overriding the derived `relay.<RELAY_API_ZONE_NAME>` domain |
| 135 | - `RELAY_TUNNEL_CLEANUP_MODE` with `off`, `dry-run`, or `enabled`. Missing and blank values use |
| 136 | `off`. |
| 137 | - `RELAY_LEGACY_TUNNEL_CLEANUP_MODE` with the same values, for tunnels whose host never registered |
| 138 | recovery. Missing and blank values use `off`. |
| 139 | |
| 140 | Required `production` environment secrets: |
| 141 | |
| 142 | - `CLERK_SECRET_KEY` |
| 143 | - `APNS_PRIVATE_KEY` |
| 144 | |
| 145 | After changing a variable or secret, run the **Deploy T3 Connect relay** workflow manually from |
| 146 | `main` with **force** unchecked. Alchemy compares the values the Worker reads and redeploys it when |
| 147 | one changed. Check **force** only to redeploy resources with no detected change: a forced run also |
| 148 | replaces the Postgres runtime role and its password |
| 149 | ([alchemy-run/alchemy#1832](https://github.com/alchemy-run/alchemy/issues/1832)). |
| 150 | |
| 151 | The account-scoped repository credentials are consumed by Alchemy while provisioning relay stages; they |
| 152 | are not bound into the relay Worker. The production deployment uses an Axiom personal access token, |
| 153 | so `AXIOM_ORG_ID` must accompany `AXIOM_TOKEN`. The `prod` stage owns the retained PlanetScale |
| 154 | database. Local personal stages provision isolated branches from it and are never deployed by CI. |
| 155 | Production adopts the configured relay API and tunnel DNS zones as retained Cloudflare resources. |
| 156 | Personal stages reference the production-owned zones. |
| 157 | |
| 158 | Developers deploy personal stages locally rather than through pull-request automation: |
| 159 | |
| 160 | ```sh |
| 161 | vp run --filter t3code-relay deploy -- --stage "$USER" --env-file .env.local |
| 162 | ``` |
| 163 | |
| 164 | ### Managed tunnel cleanup rollout |
| 165 | |
| 166 | Keep `RELAY_TUNNEL_CLEANUP_MODE=off` for the first production deploy. That deploy applies the |
| 167 | nullable allocation migration and adds the recovery endpoints. Web and mobile clients need no |
| 168 | coordinated release. CLI and desktop server builds must reach users before cleanup is enabled, |
| 169 | because those builds register recovery and replace a deleted tunnel after wake. |
| 170 | |
| 171 | 1. Deploy the relay and migration with cleanup `off`. |
| 172 | 2. Release the server build and confirm current hosts register recovery. Older hosts stay marked |
| 173 | legacy and are only candidates under the legacy switch below. |
| 174 | 3. Set `dry-run`, run a relay deploy, and read the sweep counters (`scanned`, `wouldDelete`, |
| 175 | `skippedLegacy`, `skippedOrphan`, `failed`, `truncated`) across several sweeps. Each sweep records |
| 176 | them, and the active `mode`, as `relay.managed_endpoint_reaper.*` attributes on its |
| 177 | `relay.managed_endpoint_reaper.sweep` span in Axiom. |
| 178 | 4. Run the disposable-host canary below. |
| 179 | 5. Set `enabled` only after the canary recovers without a server restart. |
| 180 | |
| 181 | The job runs every five minutes with a five-minute grace period for tunnels that lost their |
| 182 | connector, so a candidate is usually removed five to ten minutes after it goes down. Tunnels that |
| 183 | never connected wait an hour. One sweep attempts at most 100 deletions, so a backlog takes longer. |
| 184 | Changing `RELAY_TUNNEL_CLEANUP_MODE`, including turning cleanup off during an incident, needs a relay |
| 185 | deploy without force. Confirm the new `mode` on the next sweep span. |
| 186 | |
| 187 | To roll back, set cleanup to `off` and run a relay deploy before downgrading any host. Keep the |
| 188 | recovery endpoints deployed while current server builds are in use. The nullable columns can stay. |
| 189 | |
| 190 | ### Legacy tunnel cleanup |
| 191 | |
| 192 | A legacy tunnel belongs to a host that never registered recovery, usually one that went offline |
| 193 | before the recovery build shipped. `RELAY_LEGACY_TUNNEL_CLEANUP_MODE` deletes these once Cloudflare |
| 194 | reports them down, or never connected, for more than 7 days. It is independent of |
| 195 | `RELAY_TUNNEL_CLEANUP_MODE`, and every other check still applies. |
| 196 | |
| 197 | A deleted legacy tunnel keeps its allocation, so its hostname is kept. When the host comes back: |
| 198 | |
| 199 | - On a build with recovery, the connector is rejected and the host requests a replacement tunnel at |
| 200 | the same hostname. |
| 201 | - On an older build with a CLI link, startup provisions a new tunnel. |
| 202 | - On an older build linked from web or mobile, the host stays offline until T3 Code on that computer |
| 203 | is updated. |
| 204 | |
| 205 | Ship the web and mobile builds that show the offline reason before enabling legacy cleanup, so a |
| 206 | user whose host is affected sees what to do. The relay adds the `tunnel_released_at` allocation |
| 207 | column in its first deploy with this change; the legacy switch stays `off` until you set it. |
| 208 | |
| 209 | 1. Run `vp run --filter t3code-relay tunnels:census` with a read-only Cloudflare token. It counts |
| 210 | tunnels in every relay stage. The reaper only sees its own stage's tunnels, so clean up the rest |
| 211 | by hand. |
| 212 | 2. Set the legacy mode to `dry-run`, deploy, and read `wouldDeleteLegacy`, `legacyOver30Days`, |
| 213 | `totalDown`, and `totalInactive` on the sweep spans for a day. `wouldDeleteLegacy` counts only the |
| 214 | tunnels a sweep inspected, at most 500 per status. `totalDown` and `totalInactive` are Cloudflare's |
| 215 | counts of this stage's tunnels down for over five minutes and never connected for over an hour. |
| 216 | They include ones the reaper skips, so they are an upper bound on the backlog. The share of `wouldDeleteLegacy` in each sweep's `scanned` estimates how |
| 217 | much of that total is eligible. |
| 218 | 3. Run the legacy steps of the disposable-host canary below. |
| 219 | 4. Before enabling, confirm the web and mobile builds that show the "update T3 Code on that computer" |
| 220 | message are live. Without them, a user whose older host lost its tunnel only sees it as offline. |
| 221 | 5. Set the legacy mode to `enabled`. One sweep deletes at most 100 tunnels, four at a time, and |
| 222 | stops starting new deletions after 90 seconds. A backlog of 20,000 takes about 17 hours if each |
| 223 | sweep finishes its 100. Watch `deletedLegacy`, `attempted`, `failed`, and `truncated`. An |
| 224 | `attempted` well under 100 with `truncated` set means the sweep stopped early: either the time |
| 225 | budget ran out or Cloudflare rate-limited a deletion. The counters don't say which; the relay |
| 226 | logs a warning with the Cloudflare error for each failed deletion. |
| 227 | |
| 228 | In Axiom, filter the relay traces dataset on `name == "relay.managed_endpoint_reaper.sweep"` and |
| 229 | chart the `attributes.custom.relay.managed_endpoint_reaper.*` fields over time. |
| 230 | |
| 231 | Set the legacy mode back to `off` and deploy if any of these happen: |
| 232 | |
| 233 | - `failed` stays above a few per sweep. Read the warning log for the Cloudflare error. |
| 234 | - Users report an environment that is offline with the update message after they have updated T3 |
| 235 | Code on that computer and restarted it. |
| 236 | - Relay request errors rise while sweeps run. Deletions share the Postgres connection pool with |
| 237 | request handlers. |
| 238 | |
| 239 | Turning the legacy mode off stops new legacy deletions; `RELAY_TUNNEL_CLEANUP_MODE` keeps deleting |
| 240 | tunnels of hosts with recovery while it is `enabled`. Deleted tunnels stay deleted; their hosts |
| 241 | recover as described above. |
| 242 | |
| 243 | ### Disposable-host canary |
| 244 | |
| 245 | This test has not been run against a real Cloudflare account. Run it against a disposable relay |
| 246 | stage, test Cloudflare account, disposable host, and disposable T3 home. Keep production cleanup at |
| 247 | `off` or `dry-run` until it passes. Do not stop a daily-use T3 server. |
| 248 | |
| 249 | 1. Deploy the disposable stage with cleanup `dry-run`. Link a first disposable environment through |
| 250 | web or mobile settings and confirm its tunnel is healthy and recovery is registered. |
| 251 | 2. Stop that host and restart the same T3 home on a different local port. Confirm the public |
| 252 | hostname reaches the new port and sends nothing to the old one. |
| 253 | 3. Link a second disposable environment with a server build that predates recovery registration. |
| 254 | Capture its managed `cloudflared` child PID, confirm it belongs to that host, and pause only that |
| 255 | child with `kill -STOP <legacy-pid>`. Wait until Cloudflare reports it down for over five minutes. |
| 256 | 4. Capture the first environment's `cloudflared` child PID from its server logs, confirm ownership, |
| 257 | and pause it with `kill -STOP <first-pid>`. Wait until Cloudflare reports it down for over five |
| 258 | minutes. |
| 259 | 5. Confirm dry-run counts the first tunnel in `wouldDelete` and the second in `skippedLegacy`. |
| 260 | 6. Set cleanup `enabled` on the disposable stage and deploy. Confirm in the test Cloudflare account |
| 261 | that the first tunnel is deleted and the legacy tunnel still exists. |
| 262 | 7. Resume the first child with `kill -CONT <first-pid>`. Confirm the running server detects the |
| 263 | repeated rejection, requests recovery, and becomes reachable at the same hostname without a |
| 264 | restart. |
| 265 | 8. Resume the legacy child with `kill -CONT <legacy-pid>` and confirm its tunnel reconnects. |
| 266 | 9. Repeat with a physical sleep and wake cycle on a disposable laptop before broad rollout. |
| 267 | |
| 268 | Legacy cleanup, on the same disposable stage: |
| 269 | |
| 270 | 10. Set `RELAY_LEGACY_TUNNEL_GRACE_MINUTES=10` and the legacy mode to `dry-run`, then deploy. The |
| 271 | override shortens the 7-day grace period and is ignored on `prod`. Pause the legacy child again |
| 272 | and wait until Cloudflare reports it down for over ten minutes. |
| 273 | 11. Confirm the sweep counts it in `wouldDeleteLegacy`, then set the legacy mode to `enabled` and |
| 274 | deploy. Confirm the legacy tunnel is deleted and its allocation row remains. |
| 275 | 12. With the legacy host still on its old build, resume the child. A CLI-linked host provisions a |
| 276 | new tunnel on its next restart; a web- or mobile-linked host stays offline. |
| 277 | 13. Update that host to the current build and start it. Confirm it requests recovery and is |
| 278 | reachable at the same hostname. |
| 279 | 14. Remove `RELAY_LEGACY_TUNNEL_GRACE_MINUTES` from the disposable stage. |
| 280 | |
| 281 | ## Marketing site deployment |
| 282 | |
| 283 | On nightly releases, the release workflow builds the same commit as a staged |
| 284 | production deployment of the marketing site's Vercel project while the desktop |
| 285 | jobs run, and promotes it with `vercel promote` after the release is published. |
| 286 | Stable releases do not deploy the marketing site because they can promote an |
| 287 | older nightly commit. |
| 288 | |
| 289 | The job looks up the `t3code-marketing` project using the existing `VERCEL_TOKEN` |
| 290 | and `VERCEL_ORG_ID` secrets. It also respects the optional `VERCEL_TEAM_SLUG` |
| 291 | variable. The Vercel project's root directory must be `apps/marketing`. |
| 292 | Git deployments remain disabled in `apps/marketing/vercel.ts`. |
| 293 | |
| 294 | ## Android APK mirror |
| 295 | |
| 296 | `.github/workflows/mobile-android-apk.yml` checks Google Play hourly and publishes the newest |
| 297 | build on the `v2` closed testing track to the rolling `android-nightly` prerelease, for nightly |
| 298 | users who can't join the Play testing group. Those v2-preview builds are the only Android builds that can |
| 299 | connect to a nightly server. The APK is the universal APK Play generates and signs from the |
| 300 | uploaded bundle, so no signing key lives in GitHub. The job skips itself until |
| 301 | `GOOGLE_PLAY_SERVICE_ACCOUNT_JSON` holds a Play service account key with the "Release to |
| 302 | production, exclude devices, and use Play App Signing" permission, which downloading generated |
| 303 | APKs requires. The job reads tracks without opening a Play edit, so it can share the service |
| 304 | account EAS Submit uses. |
| 305 | |
| 306 | ## Hosted web app release deployment |
| 307 | |
| 308 | The hosted app is intentionally not deployed by Vercel's Git integration. The |
| 309 | web project disables automatic Git deployments in `apps/web/vercel.ts` via |
| 310 | `git.deploymentEnabled: false`. `.github/workflows/release.yml` builds the web |
| 311 | app with Vercel CLI as a staged production deployment (`--skip-domain`) while |
| 312 | the desktop jobs run, and aliases the channel domains to it after the GitHub |
| 313 | Release succeeds. |
| 314 | |
| 315 | Required GitHub Actions secrets: |
| 316 | |
| 317 | - `VERCEL_TOKEN` |
| 318 | - `VERCEL_ORG_ID` |
| 319 | - `VERCEL_PROJECT_ID` |
| 320 | |
| 321 | Optional GitHub Actions variables: |
| 322 | |
| 323 | - `VERCEL_TEAM_SLUG`: overrides the Vercel CLI scope when the team slug is preferred over the `VERCEL_ORG_ID` secret. |
| 324 | - `T3CODE_WEB_ROUTER_URL`: defaults to `https://app.t3.codes`. |
| 325 | - `T3CODE_WEB_LATEST_DOMAIN`: defaults to `latest.app.t3.codes`. |
| 326 | - `T3CODE_WEB_NIGHTLY_DOMAIN`: defaults to `nightly.app.t3.codes`. |
| 327 | |
| 328 | Required Vercel domains: |
| 329 | |
| 330 | - `app.t3.codes`: the router domain users open, updated by stable releases. |
| 331 | - `latest.app.t3.codes`: channel alias updated by stable releases. |
| 332 | - `nightly.app.t3.codes`: channel alias updated by nightly releases. |
| 333 | |
| 334 | The router domain uses `apps/web/vercel.ts` routes. Users opt into a channel by |
| 335 | visiting `/__t3code/channel?channel=latest` or |
| 336 | `/__t3code/channel?channel=nightly`; the router stores the |
| 337 | `t3code_web_channel` cookie and rewrites future requests on `app.t3.codes` to |
| 338 | the matching channel alias. |
| 339 | |
| 340 | The release deploy job rewrites release package versions before upload so the |
| 341 | hosted app's About panel renders the release version. Stable deploys alias the |
| 342 | same deployment to both the `latest` channel and the router domain so the router |
| 343 | rules stay current. Nightly deploys only alias the `nightly` channel. The job |
| 344 | also passes `VITE_HOSTED_APP_CHANNEL=latest|nightly`, which renders the hosted |
| 345 | update track selector in the About panel. Changing the selector navigates |
| 346 | through `/__t3code/channel` on the router domain so the user's channel cookie is |
| 347 | updated before redirecting to the hosted app root. |
| 348 | |
| 349 | One-time Vercel dashboard setup: |
| 350 | |
| 351 | 1. Confirm the web project root directory remains `apps/web`. |
| 352 | 2. Add the three domains above to the web project. |
| 353 | 3. Disable automatic Git deployments in the dashboard if desired; the committed |
| 354 | `vercel.ts` setting is the source-of-truth, but disconnecting Git in the |
| 355 | dashboard is also safe. |
| 356 | 4. Run one stable release deployment, or manually alias the current stable |
| 357 | deployment, so `app.t3.codes` points at a deployment containing the router |
| 358 | rules in `apps/web/vercel.ts`. Future stable releases keep this alias current. |
| 359 | |
| 360 | ## Nightly builds |
| 361 | |
| 362 | - Workflow: `.github/workflows/release.yml` |
| 363 | - Triggers: |
| 364 | - scheduled check every 30 minutes |
| 365 | - manual `workflow_dispatch` with `channel=nightly` |
| 366 | - Automatic nightlies require new commits and at least six hours since the last nightly was published, including manual nightlies. |
| 367 | - Manual nightlies bypass the time and change checks. Nightly runs remain serialized. Scheduled runs wait for an active nightly to finish, then check the publication gap before building. |
| 368 | - Runs the same desktop quality gates and artifact matrix as the tagged release flow. |
| 369 | - Publishes a GitHub prerelease only: |
| 370 | - current tag format: `vX.Y.Z-nightly.YYYYMMDD.<run_number>` |
| 371 | - `nightly-v...` is accepted only as a legacy previous-nightly tag |
| 372 | - release name includes the short commit SHA |
| 373 | - `make_latest` is always `false` |
| 374 | - Uses the next stable patch version as the nightly base. For example, `0.0.17` produces nightlies on `0.0.18-nightly.*`. |
| 375 | - Publishes Electron auto-update metadata to the dedicated `nightly` updater channel, so desktop users can opt into that track independently from stable. |
| 376 | - Publishes the CLI npm packages (`t3` and `@t3code/t3-<platform>-<arch>`) to the `nightly` npm dist-tag using the same nightly version. |
| 377 | - Does not commit version bumps back to `main`. |
| 378 | |
| 379 | ## Server self-update release invariant |
| 380 | |
| 381 | Connected servers update to the client's exact version, not to an npm dist-tag. Every released |
| 382 | desktop or hosted client version must therefore have a matching `t3@<version>` package available on |
| 383 | npm before users can receive that client. |
| 384 | |
| 385 | The workflow enforces this ordering: |
| 386 | |
| 387 | 1. `publish_cli` publishes the exact release version to npm, on every channel. |
| 388 | 2. `release` depends on `publish_cli` before exposing desktop artifacts in GitHub Releases. |
| 389 | 3. `deploy_web` depends on `release` before moving the hosted channel to the new client. |
| 390 | `build_web` builds that client earlier with `vercel deploy --prod --skip-domain`, which |
| 391 | leaves the custom domains alone but moves the project's own `*.vercel.app` production |
| 392 | hostname. That hostname is behind Vercel SSO, so users only get the client through the |
| 393 | custom domains. |
| 394 | |
| 395 | Preserve these dependencies when changing the release graph. Publishing a client first would leave |
| 396 | the **Update server** action targeting a package version that does not exist yet. |
| 397 | |
| 398 | For a release smoke test, confirm `npm view t3@<version> version` returns the expected version, then |
| 399 | connect the new client to a server on the previous version and verify that the update action |
| 400 | reconnects to the matching server. When the release adds database migrations, verify that the |
| 401 | remote update applies them and reconnects. A failed trial must restore the database snapshot and |
| 402 | restart the previous server. If the installed launcher does not support the target protocol, |
| 403 | verify that the update stops before restart and run `npx t3@<version> service update` once on the |
| 404 | server machine. Also test the manual or desktop-managed guidance when those environments are |
| 405 | available. |
| 406 | |
| 407 | ## Desktop auto-update notes |
| 408 | |
| 409 | - Updater runtime: `apps/desktop/src/updates/DesktopUpdates.ts`. |
| 410 | - `electron-updater` adapter: `apps/desktop/src/electron/ElectronUpdater.ts`. |
| 411 | - `apps/desktop/src/main.ts` only wires the updater layers into the desktop runtime. |
| 412 | - Update UX: |
| 413 | - Background checks run on startup delay + interval. |
| 414 | - No automatic download or install. |
| 415 | - The desktop UI shows a rocket update button when an update is available; click once to download, click again after download to restart/install. |
| 416 | - Provider: GitHub Releases (`provider: github`) configured at build time. |
| 417 | - Repository slug source: |
| 418 | - `T3CODE_DESKTOP_UPDATE_REPOSITORY` (format `owner/repo`), if set. |
| 419 | - otherwise `GITHUB_REPOSITORY` from GitHub Actions. |
| 420 | - Required release assets for updater: |
| 421 | - platform installers (`.exe`, `.dmg`, `.AppImage`, `.deb`, plus macOS `.zip` for Squirrel.Mac update payloads) |
| 422 | - channel metadata: `latest*.yml` for stable releases, `nightly*.yml` for nightly releases |
| 423 | - `*.blockmap` files (used for differential downloads) |
| 424 | - macOS metadata note: |
| 425 | - `electron-updater` reads `latest-mac.yml` on stable and `nightly-mac.yml` on nightly, for both Intel and Apple Silicon. |
| 426 | - The workflow merges the per-arch mac manifests into one channel-specific mac manifest before publishing the GitHub Release. |
| 427 | |
| 428 | ### Windows payload topology and update validation |
| 429 | |
| 430 | Windows packages the bundled server and only its runtime-external/native |
| 431 | dependency closure in `resources/server.asar`. Native modules and helper |
| 432 | executables declared as unpacked by that archive must be present at the matching |
| 433 | paths below `resources/server.asar.unpacked`. The Windows-native backend reads |
| 434 | the archive in place through Electron. Packaged Windows builds also ship |
| 435 | `resources/wsl-runtime.tar.gz` plus its SHA-256 sidecar: the Linux CLI archive |
| 436 | (`t3-<version>-linux-<arch>.tar.gz`, the same arch as the Windows host) built |
| 437 | by the Linux desktop job and handed to the Windows desktop build as |
| 438 | `--wsl-runtime`, copied in verbatim so WSL runs the exact bytes a Linux user |
| 439 | downloads. WSL verifies and extracts that archive |
| 440 | into `~/.t3/wsl-runtime/sha256-<archive-digest>` inside the selected distro, |
| 441 | then reuses it for later launches of the same update. |
| 442 | |
| 443 | Windows keeps JavaScript and package metadata inside `app.asar` and unpacks only |
| 444 | native libraries and helper executables. Avoid enabling whole-package smart |
| 445 | unpacking: each loose file adds work to NSIS installation and counts against |
| 446 | the payload limit. |
| 447 | |
| 448 | The artifact builder rejects a Windows package when any of these invariants |
| 449 | break: |
| 450 | |
| 451 | - `resources/server.asar` is absent or does not contain the server entry. |
| 452 | - Any file marked unpacked in the ASAR header is absent from |
| 453 | `resources/server.asar.unpacked`. |
| 454 | - On same-architecture Windows builds, the packaged primary cannot load the fff |
| 455 | native library from inside `server.asar` through its `.unpacked` sibling. |
| 456 | - The isolated, extracted sidecar cannot load the server entry with plain Node. |
| 457 | - A Windows build given `--wsl-runtime` omits the WSL archive or SHA-256 |
| 458 | sidecar, or the sidecar digest does not match the emitted archive. |
| 459 | - The emitted WSL archive is not a Linux CLI release archive: it must unpack to |
| 460 | a single `t3-<version>-linux-<arch>` directory holding `t3`, `client/`, and |
| 461 | `node_modules/` with the Linux node-pty binary, and must not carry a loose |
| 462 | server bundle (`bin.mjs`). |
| 463 | - The external Windows resource monitor is absent. |
| 464 | - The unpacked Windows application contains more than 80 files. |
| 465 | |
| 466 | Cross-architecture Windows builds retain every structural and extracted-sidecar |
| 467 | check, but skip executing the target Electron binary. A same-architecture build |
| 468 | for each release target must exercise the primary native-load probe. |
| 469 | |
| 470 | NSIS differential packaging remains enabled. A sidecar layout transition can |
| 471 | produce a larger one-time download; subsequent small releases retain their |
| 472 | blockmaps, with a 60 MB maximum for a representative sidecar-to-sidecar update. |
| 473 | |
| 474 | ## 0) npm OIDC trusted publishing setup (CLI) |
| 475 | |
| 476 | The workflow runs `node scripts/build-npm-platform-packages.ts` on the downloaded CLI archives, then |
| 477 | `node apps/server/scripts/cli.ts publish --packages-dir npm-packages`, which runs `npm publish` on |
| 478 | each `@t3code/t3-<platform>-<arch>.tgz` and finally on `t3.tgz`, the launcher. The script publishes |
| 479 | tarballs it built itself rather than directories: `npm publish <dir>` strips `node_modules/` from the |
| 480 | tarball no matter what `files` says, and the executable loads its native addons from there. Seven |
| 481 | packages are published per release: `t3`, `@t3code/t3-darwin-arm64`, `@t3code/t3-darwin-x64`, |
| 482 | `@t3code/t3-linux-arm64`, `@t3code/t3-linux-x64`, `@t3code/t3-win32-arm64`, |
| 483 | `@t3code/t3-win32-x64`. |
| 484 | |
| 485 | Checklist: |
| 486 | |
| 487 | 1. Confirm the npm org owns package `t3` and the `@t3code` scope exists on npm (create the org if |
| 488 | it does not). |
| 489 | 2. For `t3` and each `@t3code/t3-<platform>-<arch>` package, configure a Trusted Publisher in the |
| 490 | npm package settings (a package that has never been published needs a first publish or a |
| 491 | placeholder before the setting exists; the `--dry-run` step in `publish_cli` reports which |
| 492 | names are still rejected): |
| 493 | - Provider: GitHub Actions |
| 494 | - Repository: this repo |
| 495 | - Workflow file: `.github/workflows/release.yml` |
| 496 | - Environment (if used): match your npm trusted publishing config |
| 497 | 3. Ensure npm account and org policies allow trusted publishing for every package. |
| 498 | 4. Create release tag `vX.Y.Z` and push; workflow will: |
| 499 | - build and smoke-test the five CLI archives |
| 500 | - build the npm packages from those archives |
| 501 | - publish them with npm dist-tag `latest` |
| 502 | 5. Nightly runs publish with npm dist-tag `nightly`; preview runs with `preview`. |
| 503 | |
| 504 | ## 1) Release validation and unsigned builds |
| 505 | |
| 506 | There is no dry-run tag path. Pushing any accepted non-nightly tag, including |
| 507 | `v0.0.0-test.1`, classifies the run as the stable channel. It publishes `t3` with npm dist-tag |
| 508 | `latest`, creates a real GitHub Release, aliases the hosted app to `latest.app.t3.codes` and |
| 509 | `app.t3.codes`, and can commit a version bump to `main` in the finalize job. Do not push a test tag |
| 510 | to validate the workflow. |
| 511 | |
| 512 | The workflow has no non-publishing `workflow_dispatch` mode. Use normal CI or local quality gates to |
| 513 | validate checks and builds without shipping. To exercise the complete release graph at lower stable |
| 514 | risk, manually dispatch `channel=nightly`; this still publishes a real nightly npm package, GitHub |
| 515 | prerelease, desktop updater release, hosted nightly alias, and marketing site, but it does not update stable app aliases or |
| 516 | commit a version bump to `main`. Only run it when a real nightly release is acceptable. |
| 517 | |
| 518 | Manual `channel=stable` is also a real stable-channel release. Omitting signing secrets only makes |
| 519 | platform artifacts unsigned; it does not prevent publication. |
| 520 | |
| 521 | ## 2) Apple signing + notarization setup (macOS) |
| 522 | |
| 523 | Required secrets used by the workflow: |
| 524 | |
| 525 | - `CSC_LINK` |
| 526 | - `CSC_KEY_PASSWORD` |
| 527 | - `APPLE_API_KEY` |
| 528 | - `APPLE_API_KEY_ID` |
| 529 | - `APPLE_API_ISSUER` |
| 530 | - `MACOS_PROVISIONING_PROFILE` (base64-encoded provisioning profile with Associated Domains) |
| 531 | |
| 532 | Required repository variables: |
| 533 | |
| 534 | - `APPLE_TEAM_ID` |
| 535 | |
| 536 | Optional repository variables: |
| 537 | |
| 538 | - `CLERK_PASSKEY_RP_DOMAINS`: comma-separated RP-domain override. By default, the build derives the |
| 539 | domain from the production Clerk publishable key. |
| 540 | |
| 541 | Checklist: |
| 542 | |
| 543 | 1. Apple Developer account access: |
| 544 | - Team has rights to create Developer ID certificates. |
| 545 | 2. Create an explicit App ID for `com.t3tools.t3code` and enable Associated Domains. |
| 546 | 3. Create a `Developer ID Application` certificate and a compatible provisioning profile for that |
| 547 | App ID with Associated Domains enabled. |
| 548 | 4. Export the certificate + private key as `.p12` from Keychain. |
| 549 | 5. Base64-encode the `.p12` and store as `CSC_LINK`. |
| 550 | 6. Base64-encode the provisioning profile and store it as `MACOS_PROVISIONING_PROFILE`. |
| 551 | 7. Store the `.p12` export password as `CSC_KEY_PASSWORD`, and set `APPLE_TEAM_ID` to the |
| 552 | 10-character Apple Developer Team ID. |
| 553 | 8. In App Store Connect, create an API key (Team key). |
| 554 | 9. Add API key values: |
| 555 | - `APPLE_API_KEY`: contents of the downloaded `.p8` |
| 556 | - `APPLE_API_KEY_ID`: Key ID |
| 557 | - `APPLE_API_ISSUER`: Issuer ID |
| 558 | 10. Complete the Clerk Native API and AASA setup in [T3 Connect setup](./connect-setup.md#desktop-passkeys). |
| 559 | 11. Re-run a tag release and confirm macOS artifacts are signed/notarized and contain the expected |
| 560 | `com.apple.developer.associated-domains` entitlement. |
| 561 | |
| 562 | Notes: |
| 563 | |
| 564 | - `APPLE_API_KEY` is stored as raw key text in secrets. |
| 565 | - The workflow writes it to a temporary `AuthKey_<id>.p8` file at runtime. |
| 566 | - The workflow decodes `MACOS_PROVISIONING_PROFILE`, validates it with `security cms`, and passes it |
| 567 | to the desktop packager. |
| 568 | - In-app browser passkeys depend on the same profile. The packager adds each entitlement only when |
| 569 | the profile grants it, because macOS will not launch an app that claims more than its profile |
| 570 | allows. The build log reports which ones it enabled. |
| 571 | - `keychain-access-groups` (`<TEAM_ID>.com.t3tools.t3code.webauthn`) enables Touch ID passkeys. |
| 572 | Profiles that grant the team's keychain groups (`<TEAM_ID>.*`) cover it. |
| 573 | - `com.apple.developer.web-browser.public-key-credential` lets the system passkey sheet (iCloud |
| 574 | Keychain, password managers, phones, security keys) serve any site. Apple grants it as a |
| 575 | managed capability: the Account Holder requests it through the |
| 576 | [macOS Browsers Passkeys form](https://developer.apple.com/contact/request/macos-browsers-passkeys/). |
| 577 | After approval, regenerate the profile and update `MACOS_PROVISIONING_PROFILE`. |
| 578 | |
| 579 | ## 3) Azure Trusted Signing setup (Windows) |
| 580 | |
| 581 | Required secrets used by the workflow: |
| 582 | |
| 583 | - `AZURE_TENANT_ID` |
| 584 | - `AZURE_CLIENT_ID` |
| 585 | - `AZURE_CLIENT_SECRET` |
| 586 | - `AZURE_TRUSTED_SIGNING_ENDPOINT` |
| 587 | - `AZURE_TRUSTED_SIGNING_ACCOUNT_NAME` |
| 588 | - `AZURE_TRUSTED_SIGNING_CERTIFICATE_PROFILE_NAME` |
| 589 | - `AZURE_TRUSTED_SIGNING_PUBLISHER_NAME` |
| 590 | |
| 591 | Checklist: |
| 592 | |
| 593 | 1. Create Azure Trusted Signing account and certificate profile. |
| 594 | 2. Record ATS values: |
| 595 | - Endpoint |
| 596 | - Account name |
| 597 | - Certificate profile name |
| 598 | - Publisher name |
| 599 | 3. Create/choose an Entra app registration (service principal). |
| 600 | 4. Grant service principal permissions required by Trusted Signing. |
| 601 | 5. Create a client secret for the service principal. |
| 602 | 6. Add Azure secrets listed above in GitHub Actions secrets. |
| 603 | 7. Re-run a tag release and confirm Windows installer is signed. |
| 604 | |
| 605 | ## 4) Ongoing release checklist |
| 606 | |
| 607 | 1. Pick the latest nightly and verify it: run the smoke test above against its artifacts and |
| 608 | check the nightly channel for regressions. |
| 609 | 2. Dispatch the Release workflow with `channel=stable`. Leave `version` empty unless the version |
| 610 | should differ from the one the nightly previewed. |
| 611 | 3. Confirm the `Resolve release commit` notice names the nightly tag and commit you verified. If a |
| 612 | newer nightly published in between, the run builds that one instead. |
| 613 | 4. Verify workflow steps: |
| 614 | - preflight passes |
| 615 | - release quality checks pass |
| 616 | - `build_bundle` and all platform builds pass |
| 617 | - `publish_cli` publishes the exact release version before the release job |
| 618 | - release job uploads expected files |
| 619 | 5. Smoke test downloaded artifacts. |
| 620 | |
| 621 | ## 5) Troubleshooting |
| 622 | |
| 623 | - macOS build unsigned when expected signed: |
| 624 | - Check all Apple secrets plus `APPLE_TEAM_ID` are populated and non-empty. |
| 625 | - Confirm the provisioning profile belongs to `APPLE_TEAM_ID.com.t3tools.t3code` and includes |
| 626 | Associated Domains. |
| 627 | - Windows build unsigned when expected signed: |
| 628 | - Check all Azure ATS and auth secrets are populated and non-empty. |
| 629 | - Build fails with signing error: |
| 630 | - Retry with secrets removed to confirm unsigned path still works. |
| 631 | - Re-check certificate/profile names and tenant/client credentials. |