1# Pub/Sub job queue on Cloud Run
2
3A background job queue. A public Cloud Run API enqueues jobs on a Pub/Sub topic; a Cloud Run worker pool pulls them, "processes" each payload (SHA-256, word and character counts), and writes the result to Firestore, where the API serves it.
4
5```sh
6curl -X POST "$URL/jobs" -d '{"payload":"hello world"}'
7# 202 {"id":"5f0c…","status":"pending"}
8curl "$URL/jobs/5f0c…"
9# 200 {"id":"5f0c…","status":"done","sha256":"b94d…","words":2,"chars":11,…}
10```
11
12## Architecture
13
14- [`Jobs`](./src/resources.ts) — Pub/Sub topic, the queue.
15- [`Results`](./src/resources.ts) — named Firestore Native database; one document per job at `jobs/{id}`.
16- [`Api`](./src/Api.ts) — `GCP.Function` (a Cloud Run service) with a public URL. `POST /jobs` publishes, `GET /jobs/:id` reads the result (`202 pending` until it exists).
17- [`Worker`](./src/Worker.ts) — `GCP.Run.WorkerPool` with one instance. No inbound URL; it pulls from the topic in a background loop.
18
19## Bindings and event sources
20
21| Host | Binding / event source | IAM granted to the host's service account |
22| ------ | ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
23| Api | `GCP.PubSub.WriteTopic(Jobs)` + `WriteTopicHttp` | `roles/pubsub.publisher` on the topic |
24| Api | `GCP.Firestore.ReadDatabase(Results)` + `ReadDatabaseHttp` | `roles/datastore.viewer` on the project, conditioned to the `Results` database |
25| Worker | `GCP.PubSub.consumeTopicMessages(Jobs)` + `GCP.Run.TopicPullEventSource` | creates a pull subscription on the topic; `roles/pubsub.subscriber` on that subscription (pull + ack) |
26| Worker | `GCP.Firestore.WriteDatabase(Results)` + `WriteDatabaseHttp` | `roles/datastore.user` on the project, conditioned to the `Results` database |
27
28Firestore databases have no resource-level IAM policy, so the Firestore grants are project-level with an IAM Condition on the database name. They can take a few minutes to propagate after the first deploy.
29
30## Delivery semantics
31
32Pub/Sub is at-least-once. The pull loop acks a batch only after the handler succeeds; a failed handler (or a crashed instance) leaves it unacked, and Pub/Sub redelivers it after the 60-second ack deadline. Redelivery is safe here because the worker writes each result with `set`, so a repeated job overwrites its document with the same values. Messages that can never parse are logged and acked instead of redelivered forever.
33
34This example has no dead-letter topic: a job that keeps failing is retried until the subscription's message retention (7 days) expires. For production, add a dead-letter policy and an alert on it.
35
36## Deploy
37
38Credentials come from your alchemy profile: run `alchemy profile` once and pick GCP (*Service account JSON* for a key file, or *Stored* for an access token or key kept in `~/.alchemy/credentials`, plus a default region), then deploy with `--profile <name>`.
39
40From the repository root:
41
42```sh
43pnpm install
44cd examples/gcp-pubsub-worker
45pnpm deploy --profile <name>
46```
47
48Deploying builds both container images locally, so Docker must be running.
49
50## Live test
51
52```sh
53ALCHEMY_PROFILE=<name> bun test test/integ.test.ts
54```
55
56The test deploys the stack, submits several jobs, polls `GET /jobs/:id` until every job is done, checks each result against the Firestore document out of band, then destroys the stack and checks the topic, subscription, worker pool, and database are gone.
57
58## Destroy
59
60```sh
61pnpm destroy --profile <name>
62```