clover's git

the-knife

1	---
2	title: Blob Store
3	description: Bulk, immutable, content-addressed bytes. R2 by default, S3 when data has to live in AWS, or any store that can do ranged reads and multipart writes.
4	---
5
6	`Git.BlobStore` holds everything big: compacted packs, clone bundles,
7	oversize objects, and the spilled bodies of large pushes. The contract
8	is what the pack plane needs and nothing more:
9
10	```typescript
11	interface BlobStoreShape {
12	get(key, range?): Effect<BlobBody | null>; // ranged read
13	put(key, body, { contentLength }): Effect<void>; // whole object
14	multipart(key): Effect<BlobMultipart>; // streaming write of unknown length
15	delete(keys): Effect<void>;
16	list(prefix): Stream<BlobMeta>; // purge only, never on a serving path
17	}
18	```
19
20	The package owns no bucket. You declare one and pass it in, and the one
21	Layer serves both places bytes are touched: the Worker streaming a
22	clone bundle and the Durable Object writing a pack.
23
24	## R2
25
26	```typescript
27	export const GitObjects = Cloudflare.R2.Bucket("GitObjects");
28
29	Layer.provide(Git.BlobStoreR2(GitObjects))
30	```
31
32	The default for a Cloudflare-hosted deployment. No egress fees and no
33	cross-internet hop on the serving path.
34
35	## S3
36
37	```typescript
38	export const GitObjects = AWS.S3.Bucket("GitObjects");
39
40	Layer.provide(Git.BlobStoreS3(GitObjects))
41	```
42
43	Compute stays on Cloudflare and bytes live in your S3 bucket.
44	`BlobStoreS3(GitObjects)` mints a least-privilege IAM identity for
45	the Worker (`GetObject`, `PutObject`, `DeleteObject`, `ListBucket`,
46	and the multipart actions on that bucket), and signs every request
47	with credentials assumed at runtime. The Layer is built over the S3
48	bindings, so it runs unchanged in the Worker and in the Durable
49	Object. The stack carries both provider sets:
50
51	```typescript
52	providers: Layer.mergeAll(Cloudflare.providers(), AWS.providers()),
53	```
54
55	Every read crosses the public internet, so expect higher serving
56	latency than R2. Reads are coalesced into few large ranges, but the hop
57	is real. [Bytes in S3, hashing on Lambda](/git/recipes/cloudflare-aws) is the
58	full stack.
59
60	## Your own
61
62	Implement the five operations and any object store works.
63	[Bring your own store](/git/recipes/your-own-store) walks through it:
64
65	```typescript
66	const BlobStoreMine: Layer.Layer<Git.BlobStore> = Layer.effect(
67	Git.BlobStore,
68	Effect.gen(function* () {
69	return {
70	get: (key, range) => /* ... */,
71	put: (key, body, options) => /* ... */,
72	multipart: (key) => /* ... */,
73	delete: (keys) => /* ... */,
74	list: (prefix) => /* ... */,
75	} satisfies Git.BlobStoreShape;
76	}),
77	);
78	```
79
80	Multipart parts are uniform in size except the last. Both R2 and S3
81	accept that, and it is what the push path writes.