clover's git

the-knife

1	---
2	title: Custom Provider
3	description: Add support for a new cloud or third-party API by declaring a Resource type and implementing its lifecycle as an Effect Layer.
4	---
5
6	Providers are Effect `Layer`s — adding support for a new cloud or
7	third-party API is "declare a type, implement a Layer". See
8	[Providers](/infrastructure-as-code/provider) for the operation contract.
9
10	This guide walks through building a Stripe `Product` provider
11	end-to-end: declaring props and attributes, defining the resource
12	type, implementing the lifecycle (`reconcile`, `delete`, `diff`,
13	`read`), bundling it into a `providers()` layer, and writing a
14	test.
15
16	See [Resource lifecycle](/infrastructure-as-code/resource-lifecycle) for the
17	semantics of when each lifecycle method fires.
18
19	## Declare props and attributes
20
21	Every resource has two sides:
22
23	- **Input properties** — the desired configuration you pass in
24	- **Output attributes** — the values the cloud returns after creation
25
26	Start with two plain TypeScript types. Both are pure data, so
27	they're trivial to share between the provider and call sites.
28
29	Create `src/stripe/Product.ts`:
30
31	```typescript
32	// src/stripe/Product.ts
33	export interface StripeProductProps {
34	name: string;
35	description?: string;
36	active?: boolean;
37	}
38
39	export interface StripeProductAttributes {
40	productId: string;
41	created: number;
42	}
43	```
44
45	## Declare the Resource type
46
47	A `Resource<Type, Props, Attributes>` is a phantom type that ties a
48	string `Type` to its props and attributes. The string `Type` (here
49	`"Stripe.Product"`) is what Alchemy uses to look up the provider at
50	plan time — it must be globally unique.
51
52	```diff lang="typescript"
53	// src/stripe/Product.ts
54	+import { Resource } from "alchemy";
55	+
56	export interface StripeProductProps {
57	name: string;
58	description?: string;
59	active?: boolean;
60	}
61
62	export interface StripeProductAttributes {
63	productId: string;
64	created: number;
65	}
66	+
67	+export type StripeProduct = Resource<
68	+ "Stripe.Product",
69	+ StripeProductProps,
70	+ StripeProductAttributes
71	+>;
72	```
73
74	## Declare the Resource constructor (the "tag")
75
76	`Resource<T>(type)` returns the value users actually call —
77	`StripeProduct("Pro", { ... })`. It also doubles as the **tag**
78	the provider Layer registers itself against, so by convention the
79	type and the value share the same name.
80
81	```diff lang="typescript"
82	// src/stripe/Product.ts
83	import { Resource } from "alchemy";
84
85	// ... props / attributes / type unchanged ...
86
87	export type StripeProduct = Resource<
88	"Stripe.Product",
89	StripeProductProps,
90	StripeProductAttributes
91	>;
92	+
93	+export const StripeProduct = Resource<StripeProduct>("Stripe.Product");
94	```
95
96	You can already use this constructor in a stack — but with no
97	provider registered, planning will fail with `Provider not found
98	for Stripe.Product`. Let's fix that.
99
100	## Scaffold the provider layer
101
102	A provider layer is a `Layer<Provider<R>>` produced by
103	`Provider.succeed(ResourceClass, service)`. The service is an
104	object with `reconcile`, `delete`, and `list` (required) plus
105	optional hooks like `diff`, `read`, `precreate`.
106
107	Start with stubs so the types compile, then fill them in:
108
109	```diff lang="typescript"
110	// src/stripe/Product.ts
111	-import { Resource } from "alchemy";
112	+import * as Provider from "alchemy/Provider";
113	+import { Resource } from "alchemy";
114	+import * as Effect from "effect/Effect";
115
116	// ... props / attributes / type / constructor unchanged ...
117
118	export const StripeProduct = Resource<StripeProduct>("Stripe.Product");
119	+
120	+export const StripeProductProvider = () =>
121	+ Provider.succeed(
122	+ StripeProduct,
123	+ StripeProduct.Provider.of({
124	+ reconcile: () => Effect.die("not implemented"),
125	+ delete: () => Effect.die("not implemented"),
126	+ list: () => Effect.succeed([]),
127	+ }),
128	+ );
129	```
130
131	A few patterns worth knowing:
132
133	- **`Provider.succeed`** wraps a `ProviderService` into a
134	`Layer<Provider<StripeProduct>>`.
135	- **`StripeProduct.Provider.of({...})`** is a typed constructor —
136	it forces every method's input/output to match the resource's
137	props and attributes.
138	- **`Provider.effect`** exists for services that need a dependency
139	resolved once at construction (a logger, `FileSystem`) — never a
140	live one; see the caution below.
141
142	## Credentials
143
144	Handlers that call an authenticated API resolve a lazy credentials
145	service — nothing touches the network until an operation actually runs.
146	The [Custom Auth Provider](/environments/custom-auth-provider) guide
147	builds the `StripeCredentials` service the snippets below import;
148	providers wrapping unauthenticated APIs skip this entirely.
149
150	## Implement `reconcile`
151
152	`reconcile` is the single lifecycle method that converges the cloud's
153	actual state to the desired state (`news`). It runs on every apply, so
154	one body must handle three situations, distinguished by `output` and
155	`olds`:
156
157	| `output` | `olds` | Situation |
158	| ----------- | ----------- | -------------------------------------------- |
159	| `undefined` | `undefined` | Greenfield — nothing exists yet |
160	| defined | defined | Routine update |
161	| defined | `undefined` | Adoption — the engine imported it via `read` |
162
163	Every reconciler follows the same shape:
164
165	```
166	1. Observe — derive the physical id; read live cloud state
167	2. Ensure — if missing, create it; tolerate AlreadyExists/etc.
168	3. Sync — for each mutable aspect: read observed, diff vs
169	desired, apply only the delta
170	4. Return — fresh Attributes
171	```
172
173	:::warning[Don't branch the body on `output === undefined`]
174	Writing `if (output === undefined) { /* create */ } else { /* update */ }`
175	just renames the old `create`/`update` split. Write one flow that
176	converges from any starting point — trust observed cloud state, not `olds`.
177	:::
178
179	Let's build the Stripe reconciler step by step.
180
181	### Resolve the client
182
183	```diff lang="typescript"
184	+import * as Redacted from "effect/Redacted";
185	+import Stripe from "stripe";
186	+import { StripeCredentials } from "./Credentials.ts";
187	+
188	return StripeProduct.Provider.of({
189	- reconcile: () => Effect.die("not implemented"),
190	+ reconcile: Effect.fn(function* ({ news, output }) {
191	+ const { apiKey } = yield* yield* StripeCredentials;
192	+ const stripe = new Stripe(Redacted.value(apiKey));
193	+ }),
194	delete: () => Effect.die("not implemented"),
195	});
196	```
197
198	The double-yield resolves credentials lazily, inside the handler — the
199	provider layer builds without ever touching the auth chain.
200
201	### Observe
202
203	```diff lang="typescript"
204	reconcile: Effect.fn(function* ({ news, output }) {
205	const { apiKey } = yield* yield* StripeCredentials;
206	const stripe = new Stripe(Redacted.value(apiKey));
207	+
208	+ let product = output?.productId
209	+ ? yield* Effect.tryPromise(() =>
210	+ stripe.products.retrieve(output.productId),
211	+ ).pipe(Effect.catchAll(() => Effect.succeed(undefined)))
212	+ : undefined;
213	}),
214	```
215
216	If a previous run cached a `productId`, look up its live state — a failed
217	`retrieve` just means "missing". `output` is a cache of identifiers, never
218	proof the resource still exists.
219
220	:::tip[Use `Effect.tryPromise`, not `Effect.promise`]
221	`Effect.tryPromise` surfaces SDK errors as Effect failures the lifecycle
222	machinery can report cleanly; `Effect.promise` turns rejections into
223	defects.
224	:::
225
226	### Ensure
227
228	```diff lang="typescript"
229	let product = output?.productId
230	? // ... observe ...
231	: undefined;
232	+
233	+ if (!product) {
234	+ product = yield* Effect.tryPromise(() =>
235	+ stripe.products.create({
236	+ name: news.name,
237	+ description: news.description,
238	+ active: news.active,
239	+ }),
240	+ );
241	+ }
242	}),
243	```
244
245	Create only when observation came back empty — greenfield creates and
246	"the cached id points at nothing" recover through the same line.
247
248	### Sync
249
250	```diff lang="typescript"
251	if (!product) {
252	// ... ensure ...
253	}
254	+
255	+ if (
256	+ product.name !== news.name ||
257	+ product.description !== (news.description ?? null) ||
258	+ product.active !== (news.active ?? true)
259	+ ) {
260	+ product = yield* Effect.tryPromise(() =>
261	+ stripe.products.update(product!.id, {
262	+ name: news.name,
263	+ description: news.description,
264	+ active: news.active,
265	+ }),
266	+ );
267	+ }
268	}),
269	```
270
271	Compare each mutable field of the *observed* product against `news` and
272	write only when something drifted — a no-drift deploy makes zero write
273	calls.
274
275	### Return fresh Attributes
276
277	```diff lang="typescript"
278	if (/* drifted */) {
279	// ... sync ...
280	}
281	+
282	+ return {
283	+ productId: product.id,
284	+ created: product.created,
285	+ };
286	}),
287	```
288
289	The returned Attributes are persisted to state and become the next run's
290	`output`.
291
292	### Why this shape works
293
294	The body is idempotent by construction: Alchemy can retry it after a
295	failed state write and it re-converges. Adoption needs no special case —
296	`retrieve` finds the imported resource and the sync step rewrites whatever
297	drifted from `news`.
298
299	:::note[Bindings]
300	`reconcile` also receives `bindings` — data attached by upstream policies
301	and event sources (IAM grants, queue subscriptions). Read them here if
302	your resource is a binding target; see
303	[Binding](/infrastructure-as-effects/binding).
304	:::
305
306	## Implement `delete`
307
308	`delete` runs when the resource is removed from code, replaced, or
309	when `alchemy destroy` runs.
310
311	:::tip[Treat 404 as success]
312	Like `reconcile`, `delete` **must be idempotent**. Alchemy may retry
313	it if persistence fails between the cloud call succeeding and state
314	being written. Catch "not found" errors and return `Effect.void`.
315	:::
316
317	```diff lang="typescript"
318	return StripeProduct.Provider.of({
319	reconcile: Effect.fn(function* ({ news, output }) { /* ... */ }),
320	- delete: () => Effect.die("not implemented"),
321	+ delete: Effect.fn(function* ({ output }) {
322	+ const { apiKey } = yield* yield* StripeCredentials;
323	+ const stripe = new Stripe(Redacted.value(apiKey));
324	+ yield* Effect.tryPromise(() =>
325	+ stripe.products.del(output.productId),
326	+ ).pipe(
327	+ Effect.catchAll((cause) =>
328	+ cause instanceof Error && cause.message.includes("No such product")
329	+ ? Effect.void
330	+ : Effect.fail(cause),
331	+ ),
332	+ );
333	+ }),
334	});
335	```
336
337	## Implement `diff` (optional)
338
339	Some property changes can't be applied in place. For Stripe
340	products the `name` is mutable but (hypothetically) the
341	`description` is not — changing it requires recreating the
342	product. Implement `diff` to tell Alchemy which kind of change to
343	plan.
344
345	`diff` runs at **plan time**, before `reconcile`, and returns one of:
346
347	- `{ action: "noop" }` — change is trivial, skip reconciling
348	- `{ action: "update", stables?: [...] }` — apply in place
349	- `{ action: "replace", deleteFirst?: boolean }` — destroy and
350	recreate
351	- `undefined` / `void` — fall back to default (plan an in-place
352	update)
353
354	:::note[`diff` runs during planning]
355	Props may still be unresolved `Output`s when `diff` is called —
356	they're only fully resolved at apply time. Always guard with
357	`isResolved(news)` and return `undefined` to defer to the default
358	path when you can't decide.
359	:::
360
361	```diff lang="typescript"
362	+import { isResolved } from "alchemy/Diff";
363	+
364	// ...
365
366	return StripeProduct.Provider.of({
367	+ diff: Effect.fn(function* ({ news, olds }) {
368	+ if (!isResolved(news)) return undefined;
369	+ if (news.description !== olds.description) {
370	+ return { action: "replace" } as const;
371	+ }
372	+ return undefined;
373	+ }),
374	reconcile: Effect.fn(function* ({ news, output }) { /* ... */ }),
375	delete: Effect.fn(function* ({ output }) { /* ... */ }),
376	});
377	```
378
379	For attributes that are immutable across **all** updates (e.g.
380	the Stripe `productId`, an ARN), declare them in
381	`stables` at the top level:
382
383	```diff lang="typescript"
384	return StripeProduct.Provider.of({
385	+ stables: ["productId"],
386	diff: Effect.fn(function* ({ news, olds }) { /* ... */ }),
387	// ...
388	});
389	```
390
391	## Implement `read` (optional, for recovery and adoption)
392
393	The engine calls `read` whenever a resource has no prior state, both
394	to recover from interrupted reconciles *and* to import pre-existing
395	cloud resources into a fresh state store. Returning `undefined` tells
396	Alchemy the resource doesn't exist and should be created.
397
398	```diff lang="typescript"
399	return StripeProduct.Provider.of({
400	stables: ["productId"],
401	diff: Effect.fn(function* ({ news, olds }) { /* ... */ }),
402	+ read: Effect.fn(function* ({ output }) {
403	+ if (!output?.productId) return undefined;
404	+ const { apiKey } = yield* yield* StripeCredentials;
405	+ const stripe = new Stripe(Redacted.value(apiKey));
406	+ const product = yield* Effect.tryPromise(() =>
407	+ stripe.products.retrieve(output.productId),
408	+ ).pipe(
409	+ Effect.catchAll(() => Effect.succeed(undefined)),
410	+ );
411	+ if (!product) return undefined;
412	+ return { productId: product.id, created: product.created };
413	+ }),
414	reconcile: Effect.fn(function* ({ news, output }) { /* ... */ }),
415	delete: Effect.fn(function* ({ output }) { /* ... */ }),
416	});
417	```
418
419	This example only finds resources by an ID we previously saved — without
420	a prior `output` it returns `undefined`, and that's a fine default.
421
422	### Ownership-aware reads
423
424	```typescript
425	import { Unowned } from "alchemy/AdoptPolicy";
426
427	read: Effect.fn(function* ({ id, olds }) {
428	const live = yield* lookupByName(olds.name);
429	if (!live) return undefined;
430	const attrs = { productId: live.id, created: live.created };
431	// Compare tags/owner against this stack/stage/id.
432	return ownsResource(id, live.tags) ? attrs : Unowned(attrs);
433	}),
434	```
435
436	If your provider can find an existing resource from props alone (tags,
437	deterministic naming), brand foreign ones with `Unowned` so the engine
438	gates the takeover behind `--adopt`. The full three-way return
439	semantics are in [Providers › read](/infrastructure-as-code/provider#read).
440
441	## Implement `list`
442
443	`list` enumerates every product in the account, returning the same
444	`Attributes` shape as `read` — it powers account-wide operations
445	like [`alchemy unsafe nuke`](/cli/nuke) and must paginate
446	exhaustively (see [Providers › list](/infrastructure-as-code/provider#list)):
447
448	```diff lang="typescript"
449	return StripeProduct.Provider.of({
450	stables: ["productId"],
451	- list: () => Effect.succeed([]),
452	+ list: Effect.fn(function* () {
453	+ const { apiKey } = yield* yield* StripeCredentials;
454	+ const stripe = new Stripe(Redacted.value(apiKey));
455	+ const products = yield* Effect.tryPromise(() =>
456	+ stripe.products.list({ limit: 100 }).autoPagingToArray({ limit: 10_000 }),
457	+ );
458	+ return products.map((p) => ({ productId: p.id, created: p.created }));
459	+ }),
460	// ...
461	});
462	```
463
464	Resources with no enumeration API (singletons, existence-only
465	resources) keep the `Effect.succeed([])` stub instead.
466
467	## Bundle into a `providers()` layer
468
469	Users expect the same one-line ergonomics as the built-ins
470	(`Cloudflare.providers()`, `AWS.providers()`). Bundle the resource
471	collection and every provider implementation into a single layer:
472
473	```typescript
474	// src/stripe/Providers.ts
475	import * as Provider from "alchemy/Provider";
476	import * as Layer from "effect/Layer";
477	import { StripeProduct, StripeProductProvider } from "./Product.ts";
478
479	export class Providers extends Provider.ProviderCollection<Providers>()(
480	"Stripe",
481	) {}
482
483	export const providers = () =>
484	Layer.effect(
485	Providers,
486	Provider.collection([StripeProduct]),
487	).pipe(Layer.provide(StripeProductProvider()));
488	```
489
490	`Layer.provide` wires each resource provider to the collection
491	privately. A provider with no credentials service is done here.
492
493	Ours isn't — the handlers' `StripeCredentials` requirement is still
494	unmet. The [Custom Auth Provider](/environments/custom-auth-provider)
495	guide finishes this bundle by `Layer.provideMerge`-ing in the
496	credential bridge and the auth-provider registration.
497
498	Re-export the public surface, dropping the redundant service prefix
499	(the namespaced-export convention: callers write `Stripe.Product`):
500
501	```typescript
502	// src/stripe/index.ts
503	export { StripeProduct as Product } from "./Product.ts";
504	export { Providers, providers } from "./Providers.ts";
505	```
506
507	Either way, users plug your providers in like any built-in — no
508	API key in sight:
509
510	```typescript
511	// alchemy.run.ts
512	import * as Alchemy from "alchemy";
513	import * as Effect from "effect/Effect";
514	import * as Stripe from "./src/stripe";
515
516	export default Alchemy.Stack(
517	"MyApp",
518	{ providers: Stripe.providers(), state: Alchemy.localState() },
519	Effect.gen(function* () {
520	const pro = yield* Stripe.Product("Pro", {
521	name: "Pro plan",
522	description: "Everything in Free, plus...",
523	});
524	return { productId: pro.productId };
525	}),
526	);
527	```
528
529	Users connect the provider with `alchemy profile edit --add Stripe` (CI
530	reads env vars instead); an unconfigured deploy fails with that exact
531	command to run. See [Auth Providers](/environments/auth-providers).
532
533	To mix with another cloud, merge the layers:
534
535	```typescript
536	import * as Layer from "effect/Layer";
537
538	providers: Layer.mergeAll(Cloudflare.providers(), Stripe.providers()),
539	```
540
541	## Test the lifecycle
542
543	Full `test.provider` reference and patterns: [Testing Providers](/testing/testing-providers).
544
545	Alchemy's test harness (`alchemy/Test/Vitest` or `alchemy/Test/Bun`)
546	configures providers + state once at the top of the file, then
547	exposes `test.provider(name, (stack) => ...)` for provider-level
548	tests. Each `test.provider` body receives a fresh in-memory scratch
549	stack with `.deploy(effect)` and `.destroy()` helpers.
550
551	:::note[Three things the harness gives you]
552	- **`Test.make({ providers })`** — wires your provider Layer into
553	every test in the file so the resource type resolves.
554	- **`stack.deploy(effect)`** — runs a sub-stack with an in-memory
555	state store. The same logical ID across two `stack.deploy` calls
556	triggers an `update` (or `replace`, depending on `diff`).
557	- **`stack.destroy()`** — replans against an empty desired state to
558	tear everything down. Call at the end of the test.
559	:::
560
561	Create `test/Product.test.ts`:
562
563	```typescript
564	// test/Product.test.ts
565	import * as Test from "alchemy/Test/Vitest";
566	import { expect } from "@effect/vitest";
567	import * as Effect from "effect/Effect";
568	import Stripe from "stripe";
569	import * as StripeProvider from "../src/stripe";
570
571	// Configure providers once per file. Credentials resolve through the
572	// Auth Provider — see /environments/custom-auth-provider.
573	const { test } = Test.make({ providers: StripeProvider.providers() });
574
575	const stripe = new Stripe(process.env.STRIPE_API_KEY!);
576
577	test.provider(
578	"create, update, delete a product",
579	(stack) => Effect.gen(function* () {
580	// Create
581	const created = yield* stack.deploy(
582	Effect.gen(function* () {
583	return yield* StripeProvider.Product("TestProduct", {
584	name: "v1",
585	description: "first version",
586	});
587	}),
588	);
589	expect(created.productId).toBeDefined();
590
591	const live1 = yield* Effect.tryPromise(() =>
592	stripe.products.retrieve(created.productId),
593	);
594	expect(live1.name).toBe("v1");
595
596	// Update (in place)
597	const updated = yield* stack.deploy(
598	Effect.gen(function* () {
599	return yield* StripeProvider.Product("TestProduct", {
600	name: "v2",
601	description: "first version",
602	});
603	}),
604	);
605	expect(updated.productId).toBe(created.productId);
606
607	const live2 = yield* Effect.tryPromise(() =>
608	stripe.products.retrieve(updated.productId),
609	);
610	expect(live2.name).toBe("v2");
611
612	// Destroy
613	yield* stack.destroy();
614	}),
615	);
616	```
617
618	To verify replacement semantics, change a `stables` field (or the
619	field your `diff` flags as `replace`) and assert that
620	`updated.productId !== created.productId`.
621
622	## Reference implementations
623
624	If you'd rather start from a real provider:
625
626	- [`Axiom/VirtualField.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/Axiom/VirtualField.ts) — minimal CRUD with `diff` and `read`
627	- [`Cloudflare/R2/Bucket.ts`](https://github.com/alchemy-run/alchemy/blob/main/packages/alchemy/src/Cloudflare/R2/Bucket.ts) — production provider with bindings and replace semantics
628
629	## Where next
630
631	- [Local Providers](/infrastructure-as-code/local-provider) — give your
632	resource a second, dev-mode implementation that emulates it locally
633	(`ProviderLayer.dual`, `LocalProvider.make`).
634	- [Custom Auth Provider](/environments/custom-auth-provider) — build
635	the `StripeCredentials` service and login flow this guide
636	consumes.
637	- [Auth Providers](/environments/auth-providers) — how credentials
638	resolve: lazy Effects, Profiles, auto-refresh.
639	- [Actions](/infrastructure-as-code/action) — deploy-time work that isn't a
640	resource.
641	- [Testing Providers](/testing/testing-providers) — the harness
642	patterns behind `test.provider`.
643	- [Providers](/infrastructure-as-code/provider) — the operation contract this
644	guide implements.
645	- [Bindings](/infrastructure-as-effects/binding) — connecting Resources
646	to Functions.