clover's git

the-knife

1	---
2	title: AI Gateway setup
3	description: Buy Neon AI Gateway credits and use a model from an Effect Function.
4	---
5
6	Use Neon AI Gateway from an Effect Function with a model your organization can
7	access. Complete [Setup](/neon/setup) first.
8
9	## Buy AI Gateway credits
10
11	[Buy AI Gateway credits in the Neon Console](https://console.neon.tech/app/billing)
12	before sending model requests. AI Gateway requires a paid Neon plan; follow
13	[Neon's purchase guide](https://neon.com/docs/ai-gateway/prepaid-credits) for the
14	steps and current requirements.
15
16	Choose a Chat Completions-compatible model from the
17	[model catalog](https://neon.com/docs/ai-gateway/models) that your organization
18	can access. The examples use the supported `aws-us-east-2` region.
19
20	## Declare the branch gateway
21
22	Inside your stack, using an existing `branch`:
23
24	```typescript
25	import * as Neon from "alchemy/Neon";
26
27	const gateway = yield* Neon.AIGateway("AI", { branch });
28	```
29
30	This discovers the endpoint without sending an inference request or changing
31	billing. See the [resource reference](/providers/neon/aigateway) for explicit
32	project/branch IDs and an optional credential reference.
33
34	## Bind an Effect model
35
36	Inside a Function's initialization Effect:
37
38	```typescript
39	const ai = yield* Neon.QueryAIGateway(gateway);
40	```
41
42	Provide `Neon.QueryAIGatewayHttp` on the application. It uses the injected
43	credential for a same-branch production Neon Function and manages a tracked
44	`ai_gateway:invoke` credential for a Worker, Lambda, local Function, or cross-branch
45	caller. An explicit credential on `Neon.AIGateway` overrides automatic injection.
46	Every path uses HTTP; Neon does not expose a Cloudflare-style native gateway
47	binding. The deployment management key is never the inference credential.
48
49	## Verify one small request
50
51	Inside an authenticated request handler, use a Chat Completions model ID your
52	organization can access. The following handler excerpt assumes `modelId` is that
53	configured string:
54
55	```typescript
56	import * as Effect from "effect/Effect";
57	import { LanguageModel } from "effect/ai";
58
59	const reply = yield* LanguageModel.generateText({
60	prompt: "Reply with hello.",
61	}).pipe(
62	Effect.provide(ai.model({
63	model: modelId,
64	parameters: { maxTokens: 16 },
65	})),
66	);
67	```
68
69	Call the deployed route once, confirm it returns model text, and check the
70	organization's usage. The request consumes credits; it is not a deployment-time
71	probe. Some models use `maxCompletionTokens` or need a larger reasoning budget;
72	use the catalog's supported settings rather than sending both token-limit fields.
73	Protect the route with authentication and rate limits before exposing it to users.
74
75	After basic inference works, exercise the actual features you need: streamed
76	output, tools or structured results, and cancelling a stream. Endpoint discovery
77	alone does not verify any of these behaviors. See
78	[AI Gateway bindings](/providers/neon/queryaigateway) and the
79	[Effect model layer](/providers/neon/makelanguagemodellayer).
80
81	## Diagnose access failures
82
83	| Symptom | Check | Action |
84	| --- | --- | --- |
85	| `403` with `ai gateway not enabled for account` | Correct organization, paid-plan access, and account enablement | Resolve access in the Console or with Neon Support before retrying; redeploying the Function does not enable the account |
86	| Model access denied | Approval for that model, not just gateway access | Select an accessible model or request foundation-model access |
87	| Authentication failure | Credential scope and target branch | Use an `ai_gateway:invoke` service credential or the same-branch Function's injected credential, not a management API key |
88	| `429` / `REQUEST_LIMIT_EXCEEDED` | Token rate, daily cap, and credit balance | Inspect rate-limit headers and Billing; distinguish transient rate limiting from a quota requiring operator action |
89	| Incompatible request or model | Chat Completions versus Responses/Anthropic API | Match the model to the adapter or use the appropriate bound SDK URL |
90
91	Preserve request IDs and sanitized errors when contacting
92	[Neon Support](https://neon.com/docs/introduction/support). Exclude bearer tokens,
93	private prompts, and response data from public reports. Neon's
94	[troubleshooting guide](https://neon.com/docs/ai-gateway/troubleshooting) covers
95	current service limits.
96
97	PrivateLink and IP Allow do not cover AI Gateway traffic. A private database
98	connection does not make this model endpoint private.
99
100	## Related guides
101
102	- [Setup](/neon/setup) — management credentials and feature prerequisites.
103	- [Private networking](/neon/guides/private-networking) — database-only network controls.
104	- [State and recovery](/neon/guides/state-recovery) — retain managed credential state.