clover's git

the-knife

1	---
2	title: One reconcile, no create vs. update
3	date: 2026-05-04
4	draft: true
5	excerpt: We replaced `create` and `update` with a single `reconcile` function. Both halves were already converging on the same defensive code.
6	---
7
8	Most IaC tools you've probably used are CRUD-based.
9	CloudFormation, Terraform, Pulumi, SST, and the CDK all
10	model a resource's lifecycle as four distinct operations:
11	**Create**, **Read**, **Update**, **Delete**. When you write
12	a provider, you write four functions, and the engine picks
13	which one to call based on whether prior state exists.
14
15	Kubernetes took a different approach. A controller doesn't
16	"create" or "update" — it *reconciles*. It looks at desired
17	state, looks at actual state, and does whatever it takes to
18	make them match. Same function, every time. But Kubernetes
19	pays for that simplicity with a **continuous control loop**:
20	controllers run forever, polling and reconciling on every
21	tick. There's no `plan` step, no `apply` step, no human in
22	the loop telling it "yes, do that now."
23
24	Alchemy wants both halves. We keep Terraform's *plan and
25	apply* model — you run `alchemy deploy`, the engine computes
26	a diff, shows you what's about to change, and applies it
27	once. But inside that apply step, each resource runs a
28	Kubernetes-style reconcile instead of a CRUD branch. One
29	shot, not a loop; but the function itself doesn't care
30	whether the resource is new, stale, drifted, or adopted —
31	it just makes the cloud match desired state and returns.
32
33	This post is about why we made that switch, and why the
34	providers got noticeably smaller for it.
35
36	## What `create` and `update` actually look like
37
38	If you've written an SST or CDK custom resource, or a
39	Terraform provider, you've seen the shape. Two functions —
40	`create` provisions from nothing, `update` mutates toward
41	new props.
42
43	Let's build one up for a DynamoDB `Table`, which we'll keep
44	using as the running example. Alchemy is Effect-native, so
45	the lifecycle functions are written as `Effect.fn` generators
46	that `yield*` AWS SDK calls instead of awaiting promises —
47	but the *shape* is the same as any CRUD provider. Start
48	with the skeleton:
49
50	```typescript
51	{
52	create: Effect.fn(function* ({ id, news }) {
53	// resource doesn't exist. provision it from `news`.
54	}),
55	update: Effect.fn(function* ({ id, news, olds, output }) {
56	// resource exists. mutate it from `olds` toward `news`.
57	}),
58	}
59	```
60
61	`create` runs once, when the engine has no prior state.
62	`update` runs every time after, with the previous props
63	(`olds`), the new props (`news`), and the attributes the
64	resource returned last time (`output`).
65
66	### Fill in `create`
67
68	DynamoDB's `CreateTable` call takes the whole table shape
69	in a single round trip — key schema, attributes, billing
70	mode, tags, all of it:
71
72	```diff lang="typescript"
73	create: Effect.fn(function* ({ id, news }) {
74	+ const tableName = yield* createTableName(id, news);
75	+ const desiredTags = yield* createTags(id, news.tags);
76	+
77	+ yield* dynamodb.createTable({
78	+ TableName: tableName,
79	+ KeySchema: toKeySchema(news),
80	+ AttributeDefinitions: toAttributeDefinitions(news.attributes),
81	+ BillingMode: news.billingMode ?? "PAY_PER_REQUEST",
82	+ StreamSpecification: news.stream,
83	+ Tags: createTagsList(desiredTags),
84	+ });
85	+
86	+ yield* waitForTableActive(tableName);
87	+ return { tableName, tableArn: ... };
88	}),
89	```
90
91	Notice `Tags` is a field on `CreateTable` itself — there's
92	no follow-up `TagResource` call. AWS designed it so a fresh
93	table comes out of the API fully tagged in one round trip.
94	This is the natural shape of `create`: take props, hand them
95	to the cloud, return attributes.
96
97	### Fill in `update` — tags
98
99	After the table exists, AWS won't let you change tags via
100	`UpdateTable`. There are separate `TagResource` and
101	`UntagResource` APIs, so `update` has to *diff* old props
102	against new props:
103
104	```diff lang="typescript"
105	update: Effect.fn(function* ({ id, news, olds, output }) {
106	+ const desiredTags = yield* createTags(id, news.tags);
107	+ const oldTags = yield* createTags(id, olds.tags);
108	+ const { removed, upsert } = diffTags(oldTags, desiredTags);
109	+
110	+ if (upsert.length > 0) {
111	+ yield* dynamodb.tagResource({
112	+ ResourceArn: output.tableArn,
113	+ Tags: upsert,
114	+ });
115	+ }
116	+ if (removed.length > 0) {
117	+ yield* dynamodb.untagResource({
118	+ ResourceArn: output.tableArn,
119	+ TagKeys: removed,
120	+ });
121	+ }
122	}),
123	```
124
125	This is the line we'll keep coming back to:
126	`diffTags(oldTags, desiredTags)`, where `oldTags` is derived
127	from `olds`. `update` is using `olds` as its baseline —
128	*the props we last wrote down* — rather than what's actually
129	attached to the table in AWS right now. Hold onto that.
130
131	### Fill in `update` — stream config
132
133	Stream configuration is mutable, but via `UpdateTable`
134	rather than `CreateTable`:
135
136	```diff lang="typescript"
137	update: Effect.fn(function* ({ id, news, olds, output }) {
138	// ...tag sync above...
139	+ if (news.stream !== olds.stream) {
140	+ yield* dynamodb.updateTable({
141	+ TableName: output.tableName,
142	+ StreamSpecification: news.stream,
143	+ });
144	+ yield* waitForTableActive(output.tableName);
145	+ }
146	+ return output;
147	}),
148	```
149
150	Same shape: compare `news` to `olds`, fire the delta API if
151	they differ. Every mutable aspect of the table — tags,
152	stream, TTL, PITR, GSIs — is its own `if (news.X !== olds.X)`
153	branch.
154
155	That's a textbook CRUD provider — `create` is a single-shot
156	bootstrap that consumes every prop at once, `update` is a
157	per-aspect diff between `news` and `olds`. Clean.
158	Reasonable. It falls apart in three places.
159
160	## Break #1 — adoption
161
162	A user has an existing DynamoDB table called `Orders` in
163	their account. They want to start managing it with Alchemy
164	without re-creating it:
165
166	```sh
167	alchemy deploy --adopt
168	```
169
170	The engine has no prior state for `Orders`. Whichever of
171	the two lifecycle functions we call, we end up writing the
172	same defensive code.
173
174	### If we call `create`
175
176	The first `yield*` hits `dynamodb.createTable(...)`. AWS
177	rejects it with `ResourceInUseException`. So we catch it.
178	Now we're inside the `create` body holding a table we may
179	or may not own, and we need to figure out which:
180
181	```typescript
182	yield* dynamodb.createTable({ ... }).pipe(
183	Effect.catchTag("ResourceInUseException", () =>
184	Effect.gen(function* () {
185	// It already exists — is it ours?
186	const liveTags = yield* dynamodb.listTagsOfResource({ ResourceArn });
187	if (yield* hasAlchemyTags(id, liveTags)) {
188	return; // ours — silent adopt, fall through to sync
189	}
190	// Foreign — gate on AdoptPolicy
191	const allowed = yield* AdoptPolicy;
192	if (!allowed) return yield* Effect.fail(new OwnedBySomeoneElse({ ... }));
193	}),
194	),
195	);
196
197	// ...and now what? We passed `BillingMode`, `StreamSpecification`,
198	// `Tags`, GSIs — none of them got applied, because we didn't actually
199	// create the table. We have to read live state and diff every aspect
200	// against desired, applying deltas where they differ.
201	```
202
203	By the time we're done, `create` contains:
204
205	1. A creation attempt.
206	2. A conflict-recovery branch with an ownership check.
207	3. An `AdoptPolicy` gate.
208	4. A per-aspect sync against live cloud state.
209
210	Steps 2-4 are exactly what `update` would have done.
211
212	### If we call `update`
213
214	Wait — we *can* route here, sort of. The engine has a `read`
215	lifecycle op precisely for this: when there's no prior state
216	but `--adopt` is set, `read` fetches live attributes and
217	seeds `output` before any lifecycle call. So `output` is
218	populated.
219
220	But `olds` is still `undefined` — there are no "previous
221	props" for a table we just met. And `update`'s tag-sync
222	line was:
223
224	```typescript
225	const oldTags = yield* createTags(id, olds.tags); // 💥 olds is undefined
226	```
227
228	The fix is the same shape as in `create`: don't trust
229	`olds`. Read tags from the cloud, diff observed against
230	desired:
231
232	```typescript
233	const observedTags = yield* dynamodb.listTagsOfResource({ ResourceArn });
234	const { removed, upsert } = diffTags(observedTags, desiredTags);
235	```
236
237	And every other `if (news.X !== olds.X)` branch in
238	`update` has to be rewritten the same way: read the live
239	value from the cloud, diff observed against desired, fire
240	the delta API.
241
242	Compare that to step 4 of the `create` body above. It's
243	the same code. `create` ended up needing it because the
244	table already existed when it tried to make one; `update`
245	needs it because `olds` is gone. Different reasons,
246	identical implementation.
247
248	### The point
249
250	Both branches converge on the same shape because **both
251	need to assume nothing and observe everything.** You can't
252	trust `olds` (it may be missing on adoption, or stale after
253	drift). You can't trust that `createTable` will succeed
254	(it'll conflict on adoption). The only thing that's
255	authoritative is what the cloud says right now.
256
257	Once `create` and `update` have both grown an "observe live
258	state, then converge per-aspect" prefix, the split is
259	purely cosmetic. This is the actual reconciler in
260	[`AWS/DynamoDB/Table.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/DynamoDB/Table.ts),
261	which collapses both into one flow:
262
263	```typescript
264	let state = yield* readTableState(tableName);
265
266	if (state === undefined) {
267	yield* dynamodb.createTable({ /* ...full props, tags inline... */ }).pipe(
268	// Race: a peer reconciler created the table between our observe and create.
269	Effect.catchTag("ResourceInUseException", () => adoptExistingTable(tableName)),
270	);
271	state = yield* readTableState(tableName);
272	}
273
274	// From here on, every aspect (tags, GSIs, PITR, TTL, stream) is synced
275	// against `state` — the OBSERVED cloud state — not against `olds`.
276	```
277
278	`create` is now just one step inside the flow ("if it
279	doesn't exist yet, make it"). The rest is sync. Adoption
280	just means `state` was already populated when the function
281	started — no separate code path.
282
283	## Break #2 — partial creates
284
285	A Cloudflare `Worker` is more than a script. It's a script,
286	plus a workers.dev subdomain toggle, plus any custom
287	domains, plus the binding/migration metadata for Durable
288	Objects. The `create` body has to:
289
290	1. `PUT` the script (`putWorker`)
291	2. Toggle the `workers.dev` subdomain on or off
292	3. Attach each custom domain via `putDomain`
293	4. Persist returned IDs (domain IDs, zone IDs) into state
294
295	Now imagine the process gets `SIGKILL`'d between step 1 and
296	step 3 — laptop lid closes, CI runner gets evicted, network
297	times out. The script is uploaded. The domain isn't. State
298	hasn't been persisted, because we crashed before returning.
299
300	Next deploy, the engine still has no prior state, so it
301	routes to `create` again. `putWorker` runs and succeeds
302	(it's an upsert). Then `putDomain` runs and tries to attach
303	the hostname — except now Cloudflare may already know
304	about it from the partial run, or it may not, or it may be
305	attached to a *different* Worker because someone retried
306	elsewhere. The `create` body wasn't written for any of those
307	cases; it was written assuming a clean account.
308
309	The textbook fix is "make `create` idempotent." But the
310	moment you do that — catch `WorkerAlreadyExists`, query the
311	domain table to see what's attached, conditionally PUT —
312	your `create` body is *already* a reconciler. It's reading
313	observed state and applying deltas. It just happens to be
314	named `create`.
315
316	[`Cloudflare/Workers/Worker.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/Cloudflare/Workers/Worker.ts)
317	makes the reconciler explicit. `putWorker` is a true upsert;
318	`reconcileDomains` queries the live domain list and computes
319	attach/detach deltas. There's no "first time" path. Crash
320	anywhere, re-run, converge.
321
322	## Break #3 — drift
323
324	Same Worker, fully deployed, state persisted, everyone
325	happy. Between deploys, a teammate opens the Cloudflare
326	dashboard and detaches `api.example.com` because they
327	were debugging something.
328
329	Next deploy, the engine has prior state, so it routes to
330	`update`. `update` compares `news.domain` to `olds.domain`,
331	sees no diff, and skips the domain logic. The cloud is
332	wrong, but `update` thinks everything is fine because
333	**`olds` is not the cloud — it's our cached belief about
334	the cloud.**
335
336	The fix, again, is to query live state instead of trusting
337	`olds`. From the real reconciler:
338
339	```typescript
340	// Always query the live state of domains attached to *this*
341	// Worker rather than trusting `_previous` from local state.
342	// State may have been wiped, populated by another machine, or
343	// simply be out of date. Without this we PUT domains that are
344	// already registered to this same Worker and Cloudflare
345	// returns a confusing "hostname already in use" error.
346	const liveAll = yield* listDomains({ accountId, service: scriptName });
347	```
348
349	Once you're querying live state and diffing against it,
350	`olds` has no role left to play. It's at best a hint to skip
351	a no-op API call, never the source of truth.
352
353	## The split was always a state machine
354
355	In all three breaks, the underlying problem is the same:
356	`create` and `update` form a two-state machine where
357	`update` assumes `create`'s postcondition holds. The cloud
358	doesn't honor that assumption. Adoption skips `create`
359	entirely. Partial creates leave the postcondition unmet.
360	Drift invalidates the postcondition after the fact.
361
362	By the time you've handled all three, `create` reads
363	observed state before doing anything, and `update` reads
364	observed state before doing anything. The two functions
365	have converged on the same shape — just under different
366	names, with different sets of edge cases papered over.
367
368	## The new shape
369
370	The provider surface is now three operations:
371
372	```typescript
373	{
374	diff, // replace? update? skip?
375	read, // load actual cloud state into the store
376	reconcile, // converge cloud → desired
377	delete: ...,
378	}
379	```
380
381	`read` is called first whenever there's no prior state. If
382	it finds a resource, the engine checks the
383	[`AdoptPolicy`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AdoptPolicy.ts)
384	— set globally with `--adopt`, or per-scope with
385	`.pipe(adopt(true))`. If adoption is allowed (or the
386	resource carries our ownership tags), the engine seeds state
387	from `read`'s output and proceeds. If not, it fails with
388	`OwnedBySomeoneElse` and tells the user how to opt in.
389
390	Then `reconcile` runs. Its contract is one sentence:
391
392	> Given desired state and current state, make the current
393	> state match.
394
395	It receives `output: Attributes | undefined` and
396	`olds: Props | undefined`. All three combinations are valid:
397
398	| `output` | `olds` | meaning |
399	| ----------- | ----------- | -------------------------------- |
400	| `undefined` | `undefined` | greenfield create |
401	| defined | defined | routine update |
402	| defined | `undefined` | adoption — first time we own it |
403
404	The body must work for all three. It's written as one flow:
405
406	```
407	1. Observe — derive the physical id; read live cloud state
408	2. Ensure — if missing, create it; tolerate AlreadyExists races
409	3. Sync — for each mutable aspect:
410	• read OBSERVED cloud state
411	• compute desired state
412	• diff observed vs. desired
413	• apply the delta (skip the API on no-op)
414	```
415
416	The forbidden pattern is `if (output === undefined) {
417	/* create body */ } else { /* update body */ }`. That's
418	rename-and-branch — it brings back every false assumption
419	we just deleted.
420
421	## Plan didn't change
422
423	This is important: the *plan* still talks about create and
424	update. Open [`Plan.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/Plan.ts)
425	and you'll find an `action` field on every node, with the
426	same values as before:
427
428	```typescript
429	type Action = "create" | "update" | "replace" | "delete" | "noop";
430	```
431
432	The planner still computes a diff and decides, per resource,
433	which of those actions applies. The terraform-style preview
434	you see in your terminal still says "Plan: 4 to create,
435	2 to update, 1 to replace, 0 to destroy."
436
437	What changed is what happens at *apply* time. Both `create`
438	and `update` actions now route into the same provider
439	function — `reconcile`. The distinction lives in the plan
440	(because users want to know "is this a fresh resource or a
441	modification?"), but it doesn't live in the provider. The
442	provider's job is identical either way: read live state,
443	converge it toward desired.
444
445	In other words, the create/update split was useful as a
446	*user-facing concept* — "tell me what's about to change" —
447	but harmful as a *provider-authoring concept*. We kept the
448	former and deleted the latter.
449
450	## Where to go next
451
452	This is part 1 of two. Part 2 covers how we actually
453	migrated all 50+ providers from `create` + `update` to
454	`reconcile`: we used a single AI prompt run in parallel
455	across the codebase, with the existing per-resource tests as
456	the ground truth, and the apply/plan integration tests as
457	the safety net for the engine changes themselves.
458
459	- [Part 2 — Migrating 50+ providers in an afternoon](/blog/2026-05-06-reconcile-ai-migration)
460	- [`AdoptPolicy`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AdoptPolicy.ts)
461	— the engine-side spec for `read` → `Unowned` → adopt/fail routing.
462	- Canonical reconcilers:
463	[`AWS/S3/Bucket.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/S3/Bucket.ts),
464	[`AWS/Kinesis/Stream.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/Kinesis/Stream.ts),
465	[`AWS/DynamoDB/Table.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/AWS/DynamoDB/Table.ts).
466	- [PR #179](https://github.com/alchemy-run/alchemy/issues/179)
467	— the migration itself, every provider in one diff.