1---
2title: Effect AI
3description: Wire Effect's LanguageModel and Chat services into a Cloudflare Worker — read API keys with effect/Config, provide the model layer to your handler, plug in persistence.
4sidebar:
5 order: 5
6---
7
8Effect's [`@effect/ai`](https://effect.website/docs/ai/introduction)
9package gives you a single `LanguageModel.LanguageModel` service that
10every model provider (OpenAI, Anthropic, Workers AI, …) ships as a
11`Layer`. Once that layer is in scope, the rest of the surface
12(`generateText`, `streamText`, `Toolkit`, `Chat`, …) is
13provider-agnostic.
14
15This guide is about the *Alchemy* side of that picture: how to build
16the `LanguageModel` layer inside a Worker or Lambda
17([Runtime](/infrastructure-as-effects/runtime))
18(`Cloudflare.Worker`, `AWS.Lambda.Function`), where to read the API
19key from, and how to plug `Chat.Persistence` into a backing store
20that matches the platform. For the AI calls themselves —
21`LanguageModel.generateText`, `streamText`, `Toolkit`, structured
22outputs — refer to
23[Effect's AI documentation](https://effect.website/docs/ai/introduction).
24
25## The pattern
26
27Every Effect AI provider is two stacked layers:
28
291. A **client** Layer that holds the API key and HTTP transport
30 (`OpenAiClient.layer({ apiKey })`, `AnthropicClient.layer({ apiKey })`, …).
312. A **language model** Layer that picks the model and any defaults
32 (`OpenAiLanguageModel.layer({ model: "gpt-4o-mini" })`, …).
33
34You build that stack once in the Runtime's **Construction phase** and
35`Effect.provide(...)` it on the handler:
36
37```typescript
38Effect.gen(function* () {
39 // 1. Construct: build the LanguageModel layer (deploy bindings happen here).
40 const languageModel = ...;
41
42 return {
43 // 2. Exec: per-request handler with the model available.
44 fetch: Effect.gen(function* () {
45 const response = yield* LanguageModel.generateText({ prompt });
46 return HttpServerResponse.json({ text: response.text });
47 }).pipe(Effect.provide(languageModel)),
48 };
49});
50```
51
52`fetch` is just one example — the same `Effect.provide(languageModel)`
53pattern works in a Lambda handler, an HTTP API endpoint, an RPC
54procedure, a Workflow, or any other Effect.
55
56## Read the API key with `Config.Redacted`
57
58Every upstream provider's client Layer (`OpenAiClient.layer`,
59`AnthropicClient.layer`, …) takes an `apiKey: Redacted<string>`.
60[`Config.Redacted`](/environments/secrets) resolved in the Runtime's
61Construction phase gives you exactly that `Redacted<string>` — and Alchemy
62automatically binds the value as a `secret_text` binding
63(Cloudflare) or environment variable (Lambda) at deploy time:
64
65```typescript
66import * as Config from "effect/Config";
67
68// constructor — resolves the value AND records the binding
69const apiKey = yield* Config.Redacted("OPENAI_API_KEY");
70```
71
72`Config.Redacted("OPENAI_API_KEY")` reads `OPENAI_API_KEY` from your
73`ConfigProvider` (e.g. `.env`) at deploy time, records the binding,
74and resolves from that binding again at runtime — one line, both
75phases. See [Concepts › Secrets and Config](/environments/secrets) for
76the Construction/Runtime split, `.env`, and transformations.
77
78Because the value is in scope right in the constructor, the model Layer can be
79built eagerly — no `Layer.unwrap` needed:
80
81```typescript
82import * as Layer from "effect/Layer";
83
84const languageModel = OpenAiLanguageModel.layer({
85 model: "gpt-4o-mini",
86}).pipe(
87 Layer.provide(OpenAiClient.layer({ apiKey })),
88 Layer.provide(FetchHttpClient.layer),
89);
90```
91
92## In a Cloudflare Worker
93
94`FetchHttpClient.layer` plugs the runtime `fetch` API into the
95Effect HTTP client that every provider's client Layer depends on:
96
97```typescript
98// src/Worker.ts
99import * as Cloudflare from "alchemy/Cloudflare";
100import * as Config from "effect/Config";
101import * as Effect from "effect/Effect";
102import * as Layer from "effect/Layer";
103import { LanguageModel } from "effect/ai";
104import * as FetchHttpClient from "effect/http/FetchHttpClient";
105import * as HttpServerResponse from "effect/http/HttpServerResponse";
106import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai";
107
108export default Cloudflare.Worker(
109 "Worker",
110 { main: import.meta.url },
111 Effect.gen(function* () {
112 const apiKey = yield* Config.Redacted("OPENAI_API_KEY");
113
114 const languageModel = OpenAiLanguageModel.layer({
115 model: "gpt-4o-mini",
116 }).pipe(
117 Layer.provide(OpenAiClient.layer({ apiKey })),
118 Layer.provide(FetchHttpClient.layer),
119 );
120
121 return {
122 fetch: Effect.gen(function* () {
123 const response = yield* LanguageModel.generateText({
124 prompt: "Say hello.",
125 }).pipe(Effect.orDie);
126 return yield* HttpServerResponse.json({ text: response.text });
127 }).pipe(Effect.provide(languageModel)),
128 };
129 }),
130);
131```
132
133## Persisting chats
134
135For multi-turn conversations, Effect's `Chat` service holds the
136turn history and exposes the same `generateText` / `streamText`
137surface. `Chat.layerPersisted` wraps it with a
138`Chat.Persistence` interface that needs a `BackingPersistence`
139Layer underneath:
140
141```typescript
142import { Chat } from "effect/ai";
143import * as Persistence from "effect/persistence/Persistence";
144
145// inside your handler:
146const chat = yield* (yield* Chat.Persistence).getOrCreate("session-id");
147yield* chat.generateText({ prompt: "Hello again." });
148```
149
150The `BackingPersistence` Layer is what makes the chat history
151durable. Pick the one that matches your platform:
152
153| Platform | Backing |
154| --- | --- |
155| Cloudflare Worker + Durable Object | `Cloudflare.AI.DurableObjectChatPersistence` — bytes live in `state.storage`, one DO instance per session |
156| Any platform, in-memory (tests) | `Persistence.layerBackingMemory` — lost on restart |
157| Anything else (KV, R2, DynamoDB, …) | Implement `BackingPersistence` against your store |
158
159The `Chat.Persistence` API and handler code never change — only the
160backing layer does.
161
162## Cloudflare Workers AI binding
163
164For Workers AI models on Cloudflare, the shortest path is the native
165[Workers AI binding](/cloudflare/ai/workers-ai): no API key, no HTTP
166client layer, no resource to declare. `yield*
167Cloudflare.Workers.AI()` in the Construction phase attaches the binding, and
168`ai.model({...})` builds the `LanguageModel` Layer directly on top of
169it:
170
171```typescript
172Effect.gen(function* () {
173 const ai = yield* Cloudflare.Workers.AI();
174
175 const languageModel = ai.model({
176 model: "@cf/meta/llama-3.3-70b-instruct-fp8-fast",
177 parameters: { temperature: 0.7, maxTokens: 1024 },
178 });
179
180 return {
181 fetch: Effect.gen(function* () {
182 const response = yield* LanguageModel.generateText({
183 prompt: "Say hello.",
184 }).pipe(Effect.orDie);
185 return yield* HttpServerResponse.json({ text: response.text });
186 }).pipe(Effect.provide(languageModel)),
187 };
188}).pipe(Effect.provide(Cloudflare.Workers.AIBinding));
189```
190
191The [Workers AI guide](/cloudflare/ai/workers-ai) covers the rest of
192the binding surface — raw `ai.run(...)` inference, listing models,
193and declaring the binding on a plain async Worker's `env`.
194
195## Cloudflare AI Gateway
196
197To route Workers AI through an **AI Gateway** instead — caching, rate
198limiting, retries, and a unified request log — `Cloudflare.AI.Gateway`
199declares the gateway as a resource and
200`Cloudflare.AI.QueryGateway(Gateway)` returns a typed client whose
201`.model({...})` builds the same `LanguageModel` Layer shape, proxied
202through the gateway.
203
204Bind the gateway in the Construction phase and call `aiGateway.model({...})`
205with a Workers AI model id. It returns a `LanguageModel` Layer
206directly — no API key, no `Layer.unwrap`, since the gateway binding
207handles auth and the URL:
208
209```typescript
210const aiGateway = yield* Cloudflare.AI.QueryGateway(Gateway);
211
212const languageModel = aiGateway.model({
213 model: "@cf/meta/llama-3.1-8b-instruct",
214 parameters: { temperature: 0.7, maxTokens: 1024 },
215});
216```
217
218That `languageModel` Layer slots into the same `Effect.provide(...)`
219spot as any other provider — the rest of your handler doesn't
220change. Provide `Cloudflare.AI.QueryGatewayBinding` once at the
221bottom of the constructor's layer chain so the binding resolves at runtime:
222
223```typescript
224Effect.gen(function* () {
225 const aiGateway = yield* Cloudflare.AI.QueryGateway(Gateway);
226
227 const languageModel = aiGateway.model({
228 model: "@cf/meta/llama-3.1-8b-instruct",
229 parameters: { temperature: 0.7, maxTokens: 1024 },
230 });
231
232 return {
233 fetch: Effect.gen(function* () {
234 const response = yield* LanguageModel.generateText({
235 prompt: "Say hello.",
236 }).pipe(Effect.orDie);
237 return yield* HttpServerResponse.json({ text: response.text });
238 }).pipe(Effect.provide(languageModel)),
239 };
240}).pipe(Effect.provide(Cloudflare.AI.QueryGatewayBinding));
241```
242
243The [AI Gateway tutorial](/cloudflare/ai/ai-gateway) walks
244through it end to end — declaring the resource, streaming with
245`streamText`, and tuning caching, rate limits, and DLP.
246
247## Picking a provider
248
249Each upstream provider is its own `@effect/ai-*` package. The
250shape — `*Client.layer({ apiKey })` + `*LanguageModel.layer({
251model })` — is the same.
252
253- [`@effect/ai-openai`](https://www.npmjs.com/package/@effect/ai-openai)
254- [`@effect/ai-anthropic`](https://www.npmjs.com/package/@effect/ai-anthropic)
255- [`@effect/ai-google`](https://www.npmjs.com/package/@effect/ai-google)
256- [`@effect/ai-amazon-bedrock`](https://www.npmjs.com/package/@effect/ai-amazon-bedrock)
257- `Cloudflare.Workers.AI` (the native Workers AI binding, via Alchemy itself)
258- `Cloudflare.AI.Gateway` (Workers AI behind an AI Gateway, via Alchemy itself)
259- [`AWS.Bedrock.LanguageModel`](/aws/ai/bedrock) (Amazon Bedrock over the
260 Converse API with IAM-scoped bindings, via Alchemy itself)
261
262## Where to go next
263
264- [Effect AI documentation](https://effect.website/docs/ai/introduction)
265 — the API surface (`generateText`, `streamText`, `Toolkit`,
266 structured outputs, embeddings).
267- [Run Workers AI models](/cloudflare/ai/workers-ai) — the native
268 Workers AI binding, with or without the `LanguageModel` layer.
269- [Add an AI Gateway](/cloudflare/ai/ai-gateway) — wire
270 Workers AI behind a Cloudflare AI Gateway with caching and
271 streaming.
272- [Secrets & env](/cloudflare/security/secrets-env) — how `Config.Redacted`
273 feeds the upstream providers' API keys at deploy and runtime.