clover's git

the-knife

1	# Compiler rebuild comparison
2
3	The `compiler-rebuild` suite measures 25 public SchemaParser operations with the
4	interpreter, selective JIT and generated AOT modules. Nineteen cases also run
5	against `z.compile(schema, { strict: true })`; eleven representative cases run
6	against Valibot and Zod with `{ jitless: true }`. The fixture families name the
7	execution mode. Cases
8	cover simple and nested Structs, Arrays, tuples, Records, transformed Record
9	keys, anyOf/oneOf Unions, transformations, middleware, recursive and suspended
10	schemas, Declarations, construction,
11	and successful and failing validation.
12
13	The Effect tuple has a trailing element after its rest element, Zod compilation
14	does not support recursive schemas or `z.xor`, and Zod has no equivalent
15	decoding middleware. Those four cases intentionally have no Zod fixture. Zod
16	compilation only accelerates forward parsing, so the encode case measures its
17	documented runtime fallback. Zod parsing stands in for Effect construction
18	because Zod has no separate constructor operation. Valibot parsing with defaults
19	is used for the same comparison.
20
21	```sh
22	pnpm runtimeperf-compare compiler-rebuild --base schema-compiler
23	pnpm runtimeperf-compare compiler-rebuild --base main --family interpreted
24	pnpm runtimeperf compiler-rebuild
25	```
26
27	The first command compares the current working tree with the archived compiler
28	branch. The second isolates changes to interpreted parsing against main. Schema
29	and parser construction are outside the steady-state measurements. AOT fixtures
30	generate their version-specific module in a separate process before loading it.
31
32	The standard paired harness records five alternating rounds, validates every
33	fixture before and after measurement, uses a common batch within each pair, and
34	reports bootstrap confidence intervals. Cross-mode rankings are descriptive;
35	the paired base/head comparison is the regression evidence.
36
37	## Retained heap and first use
38
39	For a current cross-library comparison of retained heap, V8 code memory, peak
40	RSS and preparation CPU, run:
41
42	```sh
43	node packages/effect/runtimeperf/suites/compiler-rebuild/run-resources.mts \
44	tmp/schema-compiler-resources.json 5 100,500
45	node packages/effect/runtimeperf/suites/compiler-rebuild/report-resources.mts \
46	tmp/schema-compiler-resources.json
47	```
48
49	Each sample runs in a fresh process. The report calculates median per-schema
50	slopes between 100 and 500 distinct schemas, which removes fixed module and
51	lazy initialization costs. Fixture inputs are released before retained-memory
52	measurement. Preparation CPU includes adapter creation and first use; AOT also
53	includes loading and installing the generated module. AOT source generation is
54	reported separately because it runs at build time. Hot synchronous CPU is
55	already represented by the normal runtime measurements.
56
57	The older Effect-only probe below remains available for comparisons with its
58	existing archived results.
59
60	The standalone cost probe accepts a checkout root, mode, operation, shape and
61	schema count. Run several fresh processes per combination and alternate the
62	checkout order. Supported modes are `interpreted`, `jit` and `aot`; operations
63	are `decode`, `invalid`, `is` and `make`; shapes are `struct`, `array`, `transform`
64	and `default`.
65
66	```sh
67	node --expose-gc packages/effect/runtimeperf/suites/compiler-rebuild/costs.mts heap "$PWD" jit decode struct 500
68	node --expose-gc packages/effect/runtimeperf/suites/compiler-rebuild/costs.mts cold "$PWD" jit decode struct 500
69	```
70
71	Heap measurements exclude schema construction and module imports, and retain
72	both schema objects and public parser functions. They include operation setup
73	and first use. Forced GC removes transient parse output and issues. The AOT
74	generator runs in a separate process, so its retained heap is not attributed to
75	the runtime application.
76
77	First-use measurements include constructing a fresh schema, installing its
78	compiler when requested, creating a public parser and calling it. They exclude
79	module imports and AOT source generation. Repeated shapes can benefit from V8's
80	source cache, so this is not process startup latency. Treat these timing probes
81	as exploratory, separately from the paired throughput results.