1# gcp-vertex-ai
2
3A public chat endpoint on Cloud Run that answers with Gemini on Vertex AI —
4the GCP counterpart of [`aws-bedrock-ai`](../aws-bedrock-ai).
5
6| Route | Does |
7| ------------ | -------------------------------------------------------------------- |
8| `GET /` | health check |
9| `POST /chat` | `{ prompt, system?, temperature?, maxOutputTokens? }` → `{ text, finishReason, modelVersion, usage }` |
10
11## Architecture
12
13- `src/Chat.ts` — an Effect-native `GCP.Function` (a Cloud Run service) with
14 `main: import.meta.url` and `invokerIamDisabled: true`. Its init phase binds
15 `gemini-2.5-flash` with `GCP.AIPlatform.GenerateContent`; `fetch` calls it.
16- `alchemy.run.ts` — the stack; outputs the service URL, name, and runtime
17 service account.
18
19No API key exists anywhere. The container calls Vertex AI as its own runtime
20service account, using tokens from the Cloud Run metadata server.
21
22## Bindings and IAM
23
24| Binding | Grants |
25| ---------------------------------------------------------------- | ---------------------------------------------------------------- |
26| `GCP.AIPlatform.GenerateContent("gemini-2.5-flash")` (`GenerateContentHttp`) | `roles/aiplatform.user` to the service's runtime service account, on the project |
27| `invokerIamDisabled: true` on the service | no IAM check on invoke — the URL is public |
28
29Alchemy mints a dedicated runtime service account for the service; the grant
30goes to that account only and is revoked on destroy. Vertex AI publisher
31models have no resource-level IAM policy, so the role sits on the project.
32
33The model id is a plain string because publisher models are not resources
34you create. Pass `{ model, location }` to pin a region; the default location
35is `global`:
36
37```ts
38const gemini = yield* GCP.AIPlatform.GenerateContent({
39 model: "gemini-2.5-flash",
40 location: "us-central1",
41});
42const answer = yield* gemini.text("Write a haiku about rain.");
43```
44
45## Deploy
46
47Credentials come from your alchemy profile: run `alchemy profile` once and pick GCP (*Service account JSON* for a key file, or *Stored* for an access token or key kept in `~/.alchemy/credentials`, plus a default region), then deploy with `--profile <name>`.
48
49The Vertex AI API (`aiplatform.googleapis.com`) must be enabled on the
50project, and Docker must be running for the local image build.
51
52```sh
53pnpm deploy --profile <name>
54```
55
56```sh
57curl -X POST "$url/chat" -H 'content-type: application/json' \
58 -d '{"prompt":"Write a haiku about Cloud Run"}'
59```
60
61A fresh `roles/aiplatform.user` grant can take a minute or two to reach
62Vertex AI, during which `/chat` answers `500`.
63
64## Test
65
66```sh
67ALCHEMY_PROFILE=<name> bun test
68```
69
70Deploys the stack, checks the project IAM policy for the grant, sends
71`Reply with exactly: pong` to `/chat` and expects `pong` back, then destroys
72the stack and verifies the service is gone and the grant revoked. Skipped
73when Docker is not running.
74
75## Destroy
76
77```sh
78pnpm destroy --profile <name>
79```