| 1 | # From Examples to Laws: Property-Based Testing with Effect's `Arbitrary` |
| 2 | |
| 3 | Most unit tests choose a few inputs by hand and check the expected result for each one. Property-based testing asks the |
| 4 | computer to try many inputs for us. |
| 5 | |
| 6 | Instead of listing every expected answer, we write a **property**: a rule that should be true for every allowed input. |
| 7 | If the rule fails, the test tries simpler versions of the failing input. This simplification step is called |
| 8 | **shrinking**, and the failing input reported to the user is called a **counterexample**. |
| 9 | |
| 10 | Effect divides the work into three parts: |
| 11 | |
| 12 | - `Schema` describes which inputs are allowed. |
| 13 | - `Arbitrary<A>` describes how to produce values of type `A` and how to simplify them after a failure. You can think of |
| 14 | it as an input generator with built-in shrinking. |
| 15 | - `it.prop`, `it.effect.prop`, or `Arbitrary.checkEffect` runs the rule against generated inputs. |
| 16 | |
| 17 | The API is available from `effect/Arbitrary` or as the `Arbitrary` export from `effect`. It is marked `@stability unstable`: |
| 18 | names, result types, generation policies, and replay format may still change before this module is promoted. |
| 19 | |
| 20 | ## Writing a First Property |
| 21 | |
| 22 | Consider the rule “adding zero does not change an integer.” With `@effect/vitest`, we can write it directly: |
| 23 | |
| 24 | ```ts |
| 25 | import { it } from "@effect/vitest" |
| 26 | import { Schema } from "effect" |
| 27 | import * as Arbitrary from "effect/Arbitrary" |
| 28 | |
| 29 | const integer = Arbitrary.schema(Schema.Int) |
| 30 | |
| 31 | it.prop("adding zero is identity", [integer], ([value]) => value + 0 === value) |
| 32 | ``` |
| 33 | |
| 34 | `Arbitrary.schema` turns a Schema into a generator for the values represented by that Schema. `it.prop` tries the rule |
| 35 | 100 times by default. It starts with small values, gradually tries more complex ones, and simplifies the first failing |
| 36 | input it finds. |
| 37 | |
| 38 | Effect keeps the allowed inputs visible. Even a one-argument property receives its inputs from an array or record: |
| 39 | |
| 40 | ```ts |
| 41 | it.prop( |
| 42 | "addition is commutative", |
| 43 | { left: integer, right: integer }, |
| 44 | ({ left, right }) => left + right === right + left |
| 45 | ) |
| 46 | ``` |
| 47 | |
| 48 | That explicitness becomes useful as a test grows. Input names remain visible, and each input may come from either a |
| 49 | Schema or a pre-built `Arbitrary`. |
| 50 | |
| 51 | ## Choosing the Inputs |
| 52 | |
| 53 | The rule and its allowed inputs belong together. A rule can be mathematically correct and still be unsuitable for the |
| 54 | values used by the program. |
| 55 | |
| 56 | For example, JavaScript cannot represent integers of every size exactly. If we want to test ordinary integer arithmetic |
| 57 | without overflow or loss of precision, we should restrict the inputs accordingly: |
| 58 | |
| 59 | ```ts |
| 60 | import { Schema } from "effect" |
| 61 | import * as Arbitrary from "effect/Arbitrary" |
| 62 | |
| 63 | const SmallInt = Schema.Int.check( |
| 64 | Schema.isBetween({ minimum: -100, maximum: 100 }) |
| 65 | ) |
| 66 | |
| 67 | const smallInt = Arbitrary.schema(SmallInt) |
| 68 | ``` |
| 69 | |
| 70 | For common checks, such as numeric bounds and collection lengths, Effect generates matching values directly instead of |
| 71 | generating unsuitable values and rejecting them afterward. |
| 72 | |
| 73 | Use Schema to describe basic values and data structures. Then use the `Arbitrary` operations below when you need to |
| 74 | combine or transform those generated values. |
| 75 | |
| 76 | ## Looking at Generated Values |
| 77 | |
| 78 | Sampling is useful while choosing your inputs. It is not a test by itself; it simply lets you see whether the generated |
| 79 | values have the shape and size you expected. |
| 80 | |
| 81 | ```ts |
| 82 | import { Effect } from "effect" |
| 83 | |
| 84 | const examples = await Effect.runPromise( |
| 85 | Arbitrary.sampleEffect(smallInt, { |
| 86 | count: 10, |
| 87 | size: 5, |
| 88 | seed: "small-integers" |
| 89 | }) |
| 90 | ) |
| 91 | |
| 92 | console.log(examples) |
| 93 | ``` |
| 94 | |
| 95 | The same generator, seed, size, and Effect version produce the same sequence. A fixed seed is therefore useful in |
| 96 | documentation and while investigating a problem. Ordinary tests usually do not need one: after a failure, Effect |
| 97 | returns a **replay token**, a string that can reproduce both the failing input and the simplification steps that |
| 98 | followed. |
| 99 | |
| 100 | ## Combining Generated Values |
| 101 | |
| 102 | In the rest of this guide, “generator” means an `Arbitrary` value. |
| 103 | |
| 104 | Use `Arbitrary.Constant` when a generator should always return an existing value. It is especially useful inside |
| 105 | `flatMap`, where one generated value chooses what should be generated next. A constant does not use randomness and |
| 106 | cannot be simplified further. If it contains an object, every run receives the same object, so the property must not |
| 107 | modify it. |
| 108 | |
| 109 | Use: |
| 110 | |
| 111 | - `map` to transform every generated value; |
| 112 | - `filter` to keep only values that pass a condition; |
| 113 | - `filterMap` to transform a value when the transformation may reject it. |
| 114 | |
| 115 | ```ts |
| 116 | import { Result, Schema } from "effect" |
| 117 | import * as Arbitrary from "effect/Arbitrary" |
| 118 | |
| 119 | const integers = Arbitrary.schema(Schema.Int) |
| 120 | |
| 121 | const integerOrZero = Arbitrary.schema(Schema.Boolean).pipe( |
| 122 | Arbitrary.flatMap((useFallback) => useFallback ? Arbitrary.Constant(0) : integers) |
| 123 | ) |
| 124 | |
| 125 | const nonNegativeLabels = integers.pipe( |
| 126 | Arbitrary.filter((value) => value >= 0), |
| 127 | Arbitrary.map((value) => `integer:${value}`) |
| 128 | ) |
| 129 | |
| 130 | const positiveLabels = Arbitrary.filterMap( |
| 131 | integers, |
| 132 | (value) => value > 0 ? Result.succeed(`positive:${value}`) : Result.fail(value) |
| 133 | ) |
| 134 | ``` |
| 135 | |
| 136 | These operations also apply while Effect simplifies a failing value. A value rejected by `filter` or `filterMap` is not |
| 137 | passed to the property. Effect limits how many generated values may be rejected, so an impossible condition stops with |
| 138 | `SampleError` or `Exhausted` instead of searching forever. |
| 139 | |
| 140 | Prefer a Schema check when it can describe the allowed inputs directly. Effect can often generate matching values |
| 141 | immediately, whereas `filter` must first generate a value and then test it. |
| 142 | |
| 143 | Use `all` to generate independent Arbitraries together. It accepts tuples, other iterables, and records while preserving |
| 144 | their shape: |
| 145 | |
| 146 | ```ts |
| 147 | const point = Arbitrary.all([ |
| 148 | Arbitrary.schema(Schema.Number), |
| 149 | Arbitrary.schema(Schema.Number) |
| 150 | ]) |
| 151 | |
| 152 | const person = Arbitrary.all({ |
| 153 | name: Arbitrary.schema(Schema.String), |
| 154 | age: Arbitrary.schema(Schema.Int) |
| 155 | }) |
| 156 | ``` |
| 157 | |
| 158 | Effect may generate the members in a different internal order, but the returned tuple positions and record keys always |
| 159 | match the input. After a failure, it simplifies one member at a time. Empty tuples and records produce empty values. |
| 160 | |
| 161 | Effect occasionally creates generated records without inherited `Object` methods. This can reveal code that assumes |
| 162 | methods such as `hasOwnProperty` always exist. Prefer `Object.hasOwn(value, key)` when checking generated objects. |
| 163 | |
| 164 | Use `array` for variable-length collections of an existing Arbitrary: |
| 165 | |
| 166 | ```ts |
| 167 | const command = integers.pipe(Arbitrary.map((value) => ({ _tag: "Add" as const, value }))) |
| 168 | const commands = Arbitrary.array(command, { maxLength: 50 }) |
| 169 | ``` |
| 170 | |
| 171 | `minLength` defaults to zero. Generation grows with `size`, honors explicit minima even at size zero, and respects |
| 172 | `maxLength`. Invalid bounds throw immediately. After a failure, `array` tries removing blocks, including prefixes and |
| 173 | interior blocks, while retaining the other elements in their original order. Retained objects keep their identity. |
| 174 | It also simplifies individual elements. The bounded search may stop before finding the smallest possible failure. |
| 175 | |
| 176 | This differs from generating a length and using `flatMap` with `all`: shrinking that length regenerates the dependent |
| 177 | contents. Use `array` when shrinking should remove commands from the failing sequence. `Arbitrary.schema` applied to |
| 178 | `Schema.Array` uses the same block-removal strategy; prefer it when the complete collection is already a Schema. |
| 179 | |
| 180 | Use `flatMap` when one generated value decides what can be generated next. Create the possible generators before the |
| 181 | callback when you can, because creating a Schema inside the callback repeats that work each time it runs: |
| 182 | |
| 183 | ```ts |
| 184 | import { Schema } from "effect" |
| 185 | import * as Arbitrary from "effect/Arbitrary" |
| 186 | |
| 187 | const Length = Arbitrary.schema( |
| 188 | Schema.Int.check(Schema.isBetween({ minimum: 1, maximum: 4 })) |
| 189 | ) |
| 190 | |
| 191 | const StringsByLength = globalThis.Array.from({ length: 4 }, (_, index) => { |
| 192 | const length = index + 1 |
| 193 | return Arbitrary.schema( |
| 194 | Schema.String.check(Schema.isMinLength(length), Schema.isMaxLength(length)) |
| 195 | ) |
| 196 | }) |
| 197 | |
| 198 | const SizedString = Length.pipe( |
| 199 | Arbitrary.flatMap((length) => StringsByLength[length - 1]) |
| 200 | ) |
| 201 | ``` |
| 202 | |
| 203 | After a failure, Effect first tries simpler values from the first generator and rebuilds the dependent value. It then |
| 204 | tries simpler values from the selected dependent generator. This usually produces a small pair of related values |
| 205 | without breaking the relationship between them. |
| 206 | |
| 207 | Callbacks passed to these operations must return normally, always produce the same result for the same input, finish |
| 208 | in a reasonable time, and avoid modifying generated values. Effect may call them again while simplifying or replaying |
| 209 | a failure. If a callback throws, `sampleEffect` or `checkEffect` reports a **defect**, Effect's term for an unexpected |
| 210 | failure. |
| 211 | |
| 212 | ### Targeting Rare Scenarios |
| 213 | |
| 214 | Some behavior can only be tested with a particular combination or sequence of inputs. Making one value appear more |
| 215 | often does not guarantee that combination. For example, generating `Remove` more often still does not guarantee that |
| 216 | it follows an `Insert` for the same key. |
| 217 | |
| 218 | Keep a general property over all command types, then add a focused property whose input already contains the |
| 219 | important transition: |
| 220 | |
| 221 | ```ts |
| 222 | import { Schema } from "effect" |
| 223 | import * as Arbitrary from "effect/Arbitrary" |
| 224 | |
| 225 | const Key = Schema.Int.check(Schema.isBetween({ minimum: 0, maximum: 20 })) |
| 226 | |
| 227 | const Insert = Schema.Struct({ |
| 228 | _tag: Schema.Literal("Insert"), |
| 229 | value: Key |
| 230 | }) |
| 231 | |
| 232 | const Remove = Schema.Struct({ |
| 233 | _tag: Schema.Literal("Remove"), |
| 234 | value: Key |
| 235 | }) |
| 236 | |
| 237 | const Contains = Schema.Struct({ |
| 238 | _tag: Schema.Literal("Contains"), |
| 239 | value: Key |
| 240 | }) |
| 241 | |
| 242 | const Command = Schema.Union([Insert, Remove, Contains]) |
| 243 | |
| 244 | const arbitraryHistory = Arbitrary.schema( |
| 245 | Schema.Array(Command).check(Schema.isMaxLength(50)) |
| 246 | ) |
| 247 | |
| 248 | const RemoveExistingScenario = Schema.Struct({ |
| 249 | before: Schema.Array(Command).check(Schema.isMaxLength(20)), |
| 250 | key: Key, |
| 251 | after: Schema.Array(Command).check(Schema.isMaxLength(20)) |
| 252 | }) |
| 253 | |
| 254 | const historyContainingRemoveExisting = Arbitrary.schema(RemoveExistingScenario).pipe( |
| 255 | Arbitrary.map(({ before, key, after }) => [ |
| 256 | ...before, |
| 257 | { _tag: "Insert" as const, value: key }, |
| 258 | { _tag: "Remove" as const, value: key }, |
| 259 | ...after |
| 260 | ]) |
| 261 | ) |
| 262 | ``` |
| 263 | |
| 264 | Use `arbitraryHistory` to explore interactions that were not anticipated. Use `historyContainingRemoveExisting` in a |
| 265 | separate property for behavior that specifically requires removing an existing key. |
| 266 | |
| 267 | This is more reliable than a `frequency` operation, which would only make one choice more likely: |
| 268 | |
| 269 | - every focused run contains the required sequence; |
| 270 | - requirements such as using the same key are visible in the generated input; |
| 271 | - `map` rebuilds the required `Insert` and `Remove` after each simplification, so the final counterexample still tests |
| 272 | the intended behavior; |
| 273 | - the general property remains free to explore all command types. |
| 274 | |
| 275 | If you need to reproduce production traffic proportions, measure throughput, or simulate a random process, use a |
| 276 | workload generator or simulation instead. Those tasks care about exact frequencies; a property test cares about |
| 277 | finding a small input that breaks a rule. |
| 278 | |
| 279 | ## Running a Property Directly |
| 280 | |
| 281 | `it.prop` is the shortest way to use a property in a test. Use `Arbitrary.checkEffect` when your program needs to inspect |
| 282 | the result instead of immediately failing a Vitest test. It accepts a function that returns either a boolean or an |
| 283 | `Effect`, and it simplifies the first failing input: |
| 284 | |
| 285 | ```ts |
| 286 | import { Effect, Schema } from "effect" |
| 287 | import * as Arbitrary from "effect/Arbitrary" |
| 288 | |
| 289 | const values = Arbitrary.schema(Schema.Array(Schema.Int)) |
| 290 | |
| 291 | const program = Arbitrary.checkEffect( |
| 292 | values, |
| 293 | (input) => input.slice().reverse().reverse().every((value, index) => value === input[index]), |
| 294 | { runs: 100, seed: "reverse" } |
| 295 | ) |
| 296 | |
| 297 | await Effect.runPromise(program) |
| 298 | // { _tag: "Passed", runs: 100, discards: 0 } |
| 299 | ``` |
| 300 | |
| 301 | A property may also return an `Effect`, so it can use Effect services or fail through the Effect error channel: |
| 302 | |
| 303 | ```ts |
| 304 | import { Effect, Schema } from "effect" |
| 305 | import * as Arbitrary from "effect/Arbitrary" |
| 306 | |
| 307 | const program = Arbitrary.checkEffect( |
| 308 | Arbitrary.schema(Schema.String), |
| 309 | (value) => Effect.succeed(value.length >= 0) |
| 310 | ) |
| 311 | ``` |
| 312 | |
| 313 | A failed check is returned as a value. Read its `_tag` to see what happened: |
| 314 | |
| 315 | | Result | Meaning | |
| 316 | | ---------------- | -------------------------------------------------------------------------------------- | |
| 317 | | `Passed` | Every requested run passed. | |
| 318 | | `Falsified` | The rule returned `false` or its Effect failed. Includes the simplified failing input. | |
| 319 | | `Exhausted` | Too many generated values were rejected. Includes the seed needed to repeat the run. | |
| 320 | | `ReplayMismatch` | A replay token no longer leads to the same kind of failure. | |
| 321 | |
| 322 | When the returned Effect fails, `Falsified.failure` is a `PropertyError` containing its error value. Returning `false` |
| 323 | produces `ReturnedFalse`. |
| 324 | |
| 325 | Effect keeps those two kinds of failure separate while simplifying an input. A proposed simpler input is not accepted |
| 326 | as the new counterexample if it changes from “returned false” to “Effect failed,” or the other way around. The actual |
| 327 | error value may change; only the kind of failure must stay the same. |
| 328 | |
| 329 | Defects and Effect interruption continue through the Effect returned by `checkEffect` instead of becoming `CheckResult` |
| 330 | values. This allows timeouts and the Effect that started the check to interrupt it normally. |
| 331 | |
| 332 | For the same input and environment, a property must always produce the same result. It must also treat generated values |
| 333 | as read-only. Effect may evaluate a value more than once while simplifying or replaying a failure, and it does not undo |
| 334 | changes made by the property. |
| 335 | |
| 336 | ## Turning Requirements into Properties |
| 337 | |
| 338 | Many useful properties are equations. Requirements often contain words such as “same,” “independent of order,” |
| 339 | “reversible,” or “normalizes.” Each word suggests a rule that compares two or more executions of the same code. Such a |
| 340 | rule is often called a **law**. |
| 341 | |
| 342 | ### Changing Order or Grouping (Commutativity and Associativity) |
| 343 | |
| 344 | ```ts |
| 345 | import { it } from "@effect/vitest" |
| 346 | import { Schema } from "effect" |
| 347 | import * as Arbitrary from "effect/Arbitrary" |
| 348 | |
| 349 | const smallInt = Arbitrary.schema( |
| 350 | Schema.Int.check(Schema.isBetween({ minimum: -100, maximum: 100 })) |
| 351 | ) |
| 352 | |
| 353 | it.prop( |
| 354 | "addition is commutative", |
| 355 | { a: smallInt, b: smallInt }, |
| 356 | ({ a, b }) => a + b === b + a |
| 357 | ) |
| 358 | |
| 359 | it.prop( |
| 360 | "addition is associative", |
| 361 | { a: smallInt, b: smallInt, c: smallInt }, |
| 362 | ({ a, b, c }) => (a + b) + c === a + (b + c) |
| 363 | ) |
| 364 | ``` |
| 365 | |
| 366 | The bounds are part of the meaning of these tests. They keep the generated values inside the region where JavaScript |
| 367 | integer arithmetic behaves like the algebra we intend to test. |
| 368 | |
| 369 | ### Running an Operation Twice Changes Nothing (Idempotence) |
| 370 | |
| 371 | Normalization is commonly idempotent: once a value is normalized, applying the operation again should do nothing. |
| 372 | |
| 373 | ```ts |
| 374 | const clampNonNegative = (value: number): number => Math.max(0, value) |
| 375 | |
| 376 | it.prop( |
| 377 | "clamping is idempotent", |
| 378 | [smallInt], |
| 379 | ([value]) => clampNonNegative(clampNonNegative(value)) === clampNonNegative(value) |
| 380 | ) |
| 381 | ``` |
| 382 | |
| 383 | ### Applying an Operation Twice Returns the Original (Involution) |
| 384 | |
| 385 | An involution returns to the original value when applied twice. Negation is the smallest example: |
| 386 | |
| 387 | ```ts |
| 388 | it.prop( |
| 389 | "negation is an involution", |
| 390 | [smallInt], |
| 391 | ([value]) => -(-value) === value |
| 392 | ) |
| 393 | ``` |
| 394 | |
| 395 | ### Comparing Two Implementations |
| 396 | |
| 397 | When replacing or optimizing an implementation, compare the two functions over the same generated inputs: |
| 398 | |
| 399 | ```ts |
| 400 | const doubleByAddition = (value: number): number => value + value |
| 401 | const doubleByMultiplication = (value: number): number => value * 2 |
| 402 | |
| 403 | it.prop( |
| 404 | "the two double implementations agree", |
| 405 | [smallInt], |
| 406 | ([value]) => doubleByAddition(value) === doubleByMultiplication(value) |
| 407 | ) |
| 408 | ``` |
| 409 | |
| 410 | ### Operations That Undo Each Other |
| 411 | |
| 412 | ```ts |
| 413 | const increment = (value: number): number => value + 1 |
| 414 | const decrement = (value: number): number => value - 1 |
| 415 | |
| 416 | it.prop( |
| 417 | "increment and decrement are inverses", |
| 418 | [smallInt], |
| 419 | ([value]) => decrement(increment(value)) === value |
| 420 | ) |
| 421 | ``` |
| 422 | |
| 423 | The `Arbitrary` module does not provide a separate helper for each kind of law. These comparisons are short to write |
| 424 | directly in TypeScript. A project that applies the same laws to many data types can still define its own reusable test |
| 425 | functions. |
| 426 | |
| 427 | ## Required Inputs and Common Traps |
| 428 | |
| 429 | Each property should be able to fail, be easy to explain, and come from a real requirement. Adding more conditions |
| 430 | without clarifying the allowed inputs often makes a test harder to understand rather than more useful. |
| 431 | |
| 432 | ### Operations That Reject Some Inputs |
| 433 | |
| 434 | If an operation only accepts some inputs, describe that restriction in the Schema when possible. For division, a union |
| 435 | can generate non-zero integers directly: |
| 436 | |
| 437 | ```ts |
| 438 | const NonZeroSmallInt = Schema.Union([ |
| 439 | Schema.Int.check(Schema.isBetween({ minimum: -100, maximum: -1 })), |
| 440 | Schema.Int.check(Schema.isBetween({ minimum: 1, maximum: 100 })) |
| 441 | ]) |
| 442 | |
| 443 | const nonZeroSmallInt = Arbitrary.schema(NonZeroSmallInt) |
| 444 | |
| 445 | it.prop("a non-zero integer divided by itself is one", [nonZeroSmallInt], ([value]) => value / value === 1) |
| 446 | ``` |
| 447 | |
| 448 | Use `Arbitrary.filter` when Schema cannot express the condition: |
| 449 | |
| 450 | ```ts |
| 451 | const odd = smallInt.pipe( |
| 452 | Arbitrary.filter((value) => value % 2 !== 0) |
| 453 | ) |
| 454 | ``` |
| 455 | |
| 456 | Rejected generated values count against `maxDiscards`. If very few values pass the filter, the check may return |
| 457 | `Exhausted` before running the property enough times. Prefer a Schema check or `flatMap` when either can generate valid |
| 458 | values directly. |
| 459 | |
| 460 | ### Floating-Point Laws |
| 461 | |
| 462 | Floating-point arithmetic needs a different rule. Use `Schema.Finite` when `NaN` and infinities are outside the allowed |
| 463 | inputs. When exact equality is not the real requirement, compare results with an acceptable error tolerance. |
| 464 | |
| 465 | ### Mutable Properties |
| 466 | |
| 467 | Effect does not clone generated inputs before evaluating them. A property that changes its input may change the |
| 468 | reported counterexample and make simplification or replay unreliable. Treat generated values as read-only, and create |
| 469 | mutable test data inside each property evaluation rather than sharing it between runs. |
| 470 | |
| 471 | ## Laws Can Still Miss Visible Bugs |
| 472 | |
| 473 | Passing a familiar list of laws does not prove that every public operation is correct. Two values may represent the |
| 474 | same logical result while an operation accidentally depends on how they are stored internally. |
| 475 | |
| 476 | Consider an immutable first-in-first-out queue represented by a front array and a reversed rear array. The balancing |
| 477 | and update operations below are correct, but `front` deliberately reads the last front element instead of the first: |
| 478 | |
| 479 | ```ts |
| 480 | import { assert, describe, it } from "@effect/vitest" |
| 481 | import { Schema } from "effect" |
| 482 | import * as Arbitrary from "effect/Arbitrary" |
| 483 | |
| 484 | interface Queue { |
| 485 | readonly front: ReadonlyArray<number> |
| 486 | readonly rear: ReadonlyArray<number> |
| 487 | } |
| 488 | |
| 489 | const balance = (front: ReadonlyArray<number>, rear: ReadonlyArray<number>): Queue => |
| 490 | front.length === 0 ? { front: [...rear].reverse(), rear: [] } : { front, rear } |
| 491 | |
| 492 | const empty = (): Queue => balance([], []) |
| 493 | const isEmpty = (queue: Queue): boolean => queue.front.length === 0 |
| 494 | |
| 495 | const enqueue = (value: number, queue: Queue): Queue => balance(queue.front, [value, ...queue.rear]) |
| 496 | |
| 497 | const dequeue = (queue: Queue): Queue => balance(queue.front.slice(1), queue.rear) |
| 498 | |
| 499 | // Deliberately wrong: a FIFO queue should return queue.front[0]. |
| 500 | const front = (queue: Queue): number => queue.front[queue.front.length - 1]! |
| 501 | |
| 502 | const toArray = (queue: Queue): ReadonlyArray<number> => [ |
| 503 | ...queue.front, |
| 504 | ...[...queue.rear].reverse() |
| 505 | ] |
| 506 | |
| 507 | const equals = (left: Queue, right: Queue): boolean => { |
| 508 | const a = toArray(left) |
| 509 | const b = toArray(right) |
| 510 | return a.length === b.length && a.every((value, index) => value === b[index]) |
| 511 | } |
| 512 | |
| 513 | const Item = Schema.Int.check( |
| 514 | Schema.isBetween({ minimum: -100, maximum: 100 }) |
| 515 | ) |
| 516 | const Items = Schema.Array(Item).check(Schema.isMaxLength(8)) |
| 517 | const NonEmptyItems = Items.check(Schema.isMinLength(1)) |
| 518 | |
| 519 | const item = Arbitrary.schema(Item) |
| 520 | const items = Arbitrary.schema(Items) |
| 521 | const nonEmptyItems = Arbitrary.schema(NonEmptyItems) |
| 522 | |
| 523 | const queue = Arbitrary.all({ |
| 524 | front: items, |
| 525 | rear: items |
| 526 | }).pipe( |
| 527 | Arbitrary.map(({ front, rear }) => balance(front, rear)) |
| 528 | ) |
| 529 | |
| 530 | const nonEmptyQueue = Arbitrary.all({ |
| 531 | front: nonEmptyItems, |
| 532 | rear: items |
| 533 | }).pipe( |
| 534 | Arbitrary.map(({ front, rear }) => balance(front, rear)) |
| 535 | ) |
| 536 | ``` |
| 537 | |
| 538 | The queue is intended to satisfy these equations: |
| 539 | |
| 540 | 1. `isEmpty(empty()) === true` |
| 541 | 2. `isEmpty(enqueue(x, q)) === false` |
| 542 | 3. `front(enqueue(x, empty())) === x` |
| 543 | 4. For non-empty `q`, `front(enqueue(x, q)) === front(q)` |
| 544 | 5. `dequeue(enqueue(x, empty()))` equals `empty()` |
| 545 | 6. For non-empty `q`, `dequeue(enqueue(x, q))` equals `enqueue(x, dequeue(q))` |
| 546 | |
| 547 | All six pass, even with the broken `front` operation: |
| 548 | |
| 549 | ```ts |
| 550 | describe("queue laws", () => { |
| 551 | it("Q1", () => assert.isTrue(isEmpty(empty()))) |
| 552 | |
| 553 | it.prop("Q2", [item, queue], ([x, q]) => !isEmpty(enqueue(x, q))) |
| 554 | |
| 555 | it.prop("Q3", [item], ([x]) => front(enqueue(x, empty())) === x) |
| 556 | |
| 557 | it.prop( |
| 558 | "Q4", |
| 559 | [item, nonEmptyQueue], |
| 560 | ([x, q]) => front(enqueue(x, q)) === front(q) |
| 561 | ) |
| 562 | |
| 563 | it.prop( |
| 564 | "Q5", |
| 565 | [item], |
| 566 | ([x]) => equals(dequeue(enqueue(x, empty())), empty()) |
| 567 | ) |
| 568 | |
| 569 | it.prop( |
| 570 | "Q6", |
| 571 | [item, nonEmptyQueue], |
| 572 | ([x, q]) => equals(dequeue(enqueue(x, q)), enqueue(x, dequeue(q))) |
| 573 | ) |
| 574 | }) |
| 575 | ``` |
| 576 | |
| 577 | The equations compare queues through `equals`, which checks their logical sequence. But replacing a queue with an equal |
| 578 | queue must not change the answer returned by `front`. |
| 579 | |
| 580 | We can test that hidden requirement by substituting both sides of Q6 into `front`: |
| 581 | |
| 582 | ```ts |
| 583 | it.prop( |
| 584 | "front agrees after the Q6 rewrite", |
| 585 | [item, nonEmptyQueue], |
| 586 | ([x, q]) => { |
| 587 | const left = dequeue(enqueue(x, q)) |
| 588 | const right = enqueue(x, dequeue(q)) |
| 589 | return front(left) === front(right) |
| 590 | }, |
| 591 | { |
| 592 | // This flag belongs only in the tutorial while `front` is intentionally broken. |
| 593 | fails: true, |
| 594 | arbitrary: { |
| 595 | runs: 1_000, |
| 596 | size: 10, |
| 597 | seed: "front-after-rewrite" |
| 598 | } |
| 599 | } |
| 600 | ) |
| 601 | ``` |
| 602 | |
| 603 | This property fails and simplifies to a small queue. The first six laws say that two queue expressions are equal. The |
| 604 | new property also checks that `front` gives the same answer for both expressions. |
| 605 | |
| 606 | The general lesson is simple: whenever a law says `left` and `right` are equal, try both values as inputs to public |
| 607 | operations. Their results should remain equal. This does not prove every possible combination, but it can reveal code |
| 608 | that accidentally reads an internal representation. |
| 609 | |
| 610 | ## Comparing with a Simple Model |
| 611 | |
| 612 | Some systems are easier to check against a small, straightforward implementation than against a list of equations. |
| 613 | This simpler implementation is called a **model**. It may be slower than the real code; its advantage is that it is |
| 614 | easy to understand and trust. |
| 615 | |
| 616 | The following property checks Effect's immutable `HashSet` against JavaScript's mutable `Set`. Instead of generating |
| 617 | sets directly, it generates command sequences and runs the same history against both implementations. |
| 618 | |
| 619 | ```ts |
| 620 | import { it } from "@effect/vitest" |
| 621 | import { HashSet, Schema } from "effect" |
| 622 | import * as Arbitrary from "effect/Arbitrary" |
| 623 | |
| 624 | const Key = Schema.Int.check( |
| 625 | Schema.isBetween({ minimum: 0, maximum: 20 }) |
| 626 | ) |
| 627 | |
| 628 | const Command = Schema.Union([ |
| 629 | Schema.Struct({ _tag: Schema.Literal("Insert"), value: Key }), |
| 630 | Schema.Struct({ _tag: Schema.Literal("Remove"), value: Key }), |
| 631 | Schema.Struct({ _tag: Schema.Literal("Contains"), value: Key }) |
| 632 | ]) |
| 633 | |
| 634 | const commands = Arbitrary.schema( |
| 635 | Schema.Array(Command).check(Schema.isMaxLength(50)) |
| 636 | ) |
| 637 | |
| 638 | it.prop("HashSet agrees with the Set model", [commands], ([input]) => { |
| 639 | const model = new Set<number>() |
| 640 | let actual = HashSet.empty<number>() |
| 641 | const modelTrace: Array<boolean> = [] |
| 642 | const actualTrace: Array<boolean> = [] |
| 643 | |
| 644 | for (const command of input) { |
| 645 | switch (command._tag) { |
| 646 | case "Insert": |
| 647 | model.add(command.value) |
| 648 | actual = HashSet.add(actual, command.value) |
| 649 | break |
| 650 | |
| 651 | case "Remove": |
| 652 | model.delete(command.value) |
| 653 | actual = HashSet.remove(actual, command.value) |
| 654 | break |
| 655 | |
| 656 | case "Contains": |
| 657 | modelTrace.push(model.has(command.value)) |
| 658 | actualTrace.push(HashSet.has(actual, command.value)) |
| 659 | break |
| 660 | } |
| 661 | } |
| 662 | |
| 663 | return modelTrace.length === actualTrace.length && |
| 664 | modelTrace.every((value, index) => value === actualTrace[index]) && |
| 665 | model.size === HashSet.size(actual) && |
| 666 | [...model].every((value) => HashSet.has(actual, value)) |
| 667 | }) |
| 668 | ``` |
| 669 | |
| 670 | The property compares two things: |
| 671 | |
| 672 | - the recorded answers verify each `Contains` operation; |
| 673 | - the final membership and size checks verify that both states ended with the same contents. |
| 674 | |
| 675 | When this test fails, Effect simplifies the command list and its values. The final counterexample is often a short |
| 676 | history that clearly shows which state change the implementation handled incorrectly. |
| 677 | |
| 678 | ## Replaying a Failure |
| 679 | |
| 680 | Every `Falsified` result contains a `replay` token. It is a string produced by Effect and is not meant to be edited. The |
| 681 | token records enough information to generate the same initial input and repeat the simplification steps that led to the |
| 682 | reported counterexample: |
| 683 | |
| 684 | ```ts |
| 685 | import { Effect, Schema } from "effect" |
| 686 | import * as Arbitrary from "effect/Arbitrary" |
| 687 | |
| 688 | const arbitrary = Arbitrary.schema(Schema.Int) |
| 689 | |
| 690 | const program = Effect.gen(function*() { |
| 691 | const first = yield* Arbitrary.checkEffect(arbitrary, (value) => value < 10, { |
| 692 | seed: "integer-bound" |
| 693 | }) |
| 694 | |
| 695 | if (first._tag === "Falsified") { |
| 696 | const replayed = yield* Arbitrary.checkEffect( |
| 697 | arbitrary, |
| 698 | (value) => value < 10, |
| 699 | { replay: first.replay } |
| 700 | ) |
| 701 | return replayed |
| 702 | } |
| 703 | |
| 704 | return first |
| 705 | }) |
| 706 | ``` |
| 707 | |
| 708 | Store the token in logs or failure output when you want to reproduce a failure locally. Because this module is |
| 709 | unstable, a token created by one Effect release may not work with another. For a permanent regression test, copy the |
| 710 | final failing input into an ordinary example-based test. |
| 711 | |
| 712 | When `replay` is present, its recorded settings replace `runs`, `size`, `maxDiscards`, `maxShrinks`, and `seed`. |
| 713 | |
| 714 | Effect returns `ReplayMismatch` when the Schema, property, or generator has changed enough that the token no longer |
| 715 | leads to the same kind of failure. The token does not store the exact final input or exact Effect error, so replay may |
| 716 | still succeed if the error value changes while the property continues to fail in the same way. |
| 717 | |
| 718 | ## Sampling Options |
| 719 | |
| 720 | `Arbitrary.sampleEffect` accepts the following options: |
| 721 | |
| 722 | | Option | Default | Meaning | |
| 723 | | ------------- | ---------------------------- | ---------------------------------------------------------- | |
| 724 | | `count` | `10` | Number of values to return. | |
| 725 | | `size` | `10` | Rough complexity of each generated value. | |
| 726 | | `maxDiscards` | `max(100, count * 10)` | Maximum rejected values before failing with `SampleError`. | |
| 727 | | `seed` | A value from Effect `Random` | String or number used to reproduce the generated sequence. | |
| 728 | |
| 729 | If too many values are rejected, the Effect fails with a `SampleError`. The error contains the number of accepted and |
| 730 | rejected values and the seed used for the run. Passing that seed to another `sampleEffect` call repeats the run even if |
| 731 | the first call did not specify a seed. |
| 732 | |
| 733 | ## Check Options |
| 734 | |
| 735 | `Arbitrary.checkEffect` accepts the following options: |
| 736 | |
| 737 | | Option | Default | Meaning | |
| 738 | | ------------- | ---------------------------- | ---------------------------------------------------------------------- | |
| 739 | | `runs` | `100` | Number of successful generations and property evaluations to complete. | |
| 740 | | `size` | `10` | Largest approximate input complexity. It grows during the check. | |
| 741 | | `maxDiscards` | `max(100, runs * 10)` | Maximum rejected values before returning `Exhausted`. | |
| 742 | | `maxShrinks` | `100` | Maximum simplifications tried after the first failure. | |
| 743 | | `seed` | A value from Effect `Random` | String or number used to reproduce generation. | |
| 744 | | `replay` | None | Token from a previous `Falsified` result. | |
| 745 | |
| 746 | Rejected values do not count toward `runs`, and the input size does not grow after a rejection. When `runs` is `1`, |
| 747 | Effect uses the configured `size` immediately. |
| 748 | |
| 749 | After a failure, `maxShrinks` counts every simpler input that Effect examines, including values rejected by a filter and |
| 750 | values that fail in a different way. The `shrinks` field in a `Falsified` result counts only the simplifications that |
| 751 | became the new best counterexample. |
| 752 | |
| 753 | `size` is a guide rather than a maximum length for the whole result. Separate strings, collections, and object fields |
| 754 | may each use it. Recursive structures share it so that the complete value remains finite. Explicit Schema minimum and |
| 755 | maximum checks still take priority. |
| 756 | |
| 757 | ## How Effect Builds a Generator from Schema |
| 758 | |
| 759 | This section is useful when a Schema contains custom checks, recursion, or declarations. If you only use ordinary |
| 760 | Schemas, you can skip to [Using `@effect/vitest`](#using-effectvitest). |
| 761 | |
| 762 | `Arbitrary.schema` generates the value represented by a Schema after decoding. For example, |
| 763 | `Schema.NumberFromString` represents a number encoded as a string, so its generator produces numbers rather than |
| 764 | strings. |
| 765 | |
| 766 | Effect prepares the generator when `Arbitrary.schema` is called. If it cannot support the Schema, the call throws |
| 767 | immediately rather than returning a generator that fails later. |
| 768 | |
| 769 | ### Schema Checks and Rejected Values |
| 770 | |
| 771 | Effect can use many common Schema checks while generating values, including: |
| 772 | |
| 773 | - minimum and maximum values, including values that use a custom `Order`; |
| 774 | - finite and integer numbers; |
| 775 | - minimum and maximum lengths for strings, collections, and object properties; |
| 776 | - supported regular-expression patterns; |
| 777 | - checks that collection values are unique. |
| 778 | |
| 779 | For these checks, Effect produces matching values directly. It applies other custom checks after generation. A value |
| 780 | that fails one of those checks is rejected. |
| 781 | |
| 782 | `maxDiscards` limits how many values may be rejected. An impossible or very selective check therefore produces |
| 783 | `SampleError` or `Exhausted` instead of searching forever. |
| 784 | |
| 785 | ### Custom Shrinking |
| 786 | |
| 787 | The default simplification is usually enough: Effect removes collection items, moves numbers and strings toward |
| 788 | simpler values, and simplifies object or tuple fields one at a time while preserving Schema checks. Use the `shrink` |
| 789 | option when your application has a useful simplification that Effect cannot infer from the data structure. |
| 790 | |
| 791 | For example, an addition expression can be replaced directly by either side. The default behavior cannot infer that |
| 792 | meaning from the object shape: |
| 793 | |
| 794 | ```ts |
| 795 | import { Schema } from "effect" |
| 796 | import * as Arbitrary from "effect/Arbitrary" |
| 797 | |
| 798 | interface Literal { |
| 799 | readonly _tag: "Literal" |
| 800 | readonly value: number |
| 801 | } |
| 802 | |
| 803 | interface Add { |
| 804 | readonly _tag: "Add" |
| 805 | readonly left: Expression |
| 806 | readonly right: Expression |
| 807 | } |
| 808 | |
| 809 | type Expression = Literal | Add |
| 810 | |
| 811 | const Expression: Schema.Codec<Expression> = Schema.suspend(() => |
| 812 | Schema.Union([ |
| 813 | Schema.Struct({ |
| 814 | _tag: Schema.Literal("Literal"), |
| 815 | value: Schema.Int |
| 816 | }), |
| 817 | Schema.Struct({ |
| 818 | _tag: Schema.Literal("Add"), |
| 819 | left: Expression, |
| 820 | right: Expression |
| 821 | }) |
| 822 | ]) |
| 823 | ) |
| 824 | |
| 825 | const shrinkExpression = (expression: Expression): ReadonlyArray<Expression> => { |
| 826 | switch (expression._tag) { |
| 827 | case "Literal": |
| 828 | return expression.value === 0 ? [] : [{ _tag: "Literal", value: 0 }] |
| 829 | case "Add": |
| 830 | return [ |
| 831 | expression.left, |
| 832 | expression.right, |
| 833 | ...shrinkExpression(expression.left).map((left) => ({ ...expression, left })), |
| 834 | ...shrinkExpression(expression.right).map((right) => ({ ...expression, right })) |
| 835 | ] |
| 836 | } |
| 837 | } |
| 838 | |
| 839 | const expressions = Arbitrary.schema(Expression, { |
| 840 | shrink: shrinkExpression |
| 841 | }) |
| 842 | ``` |
| 843 | |
| 844 | Initial values still come from `Expression`. After a property fails, `shrinkExpression` tells Effect which simpler |
| 845 | values to try next. Replacing `Add(left, right)` with `left` or `right` can reach a useful counterexample much faster |
| 846 | than changing one field at a time. |
| 847 | |
| 848 | Before the property sees a proposed value, Effect checks it with `Expression`. Invalid values are skipped and consume |
| 849 | one unit of `maxShrinks`; Effect does not ask the callback to simplify them further. The Schema therefore remains the |
| 850 | final authority even if the callback contains a cast or calls untyped code. |
| 851 | |
| 852 | Effect calls the callback only after a property fails. It may call it again during simplification or replay. The |
| 853 | callback must therefore finish normally, always return the same proposed values for the same input, and avoid modifying |
| 854 | data. It should list the most useful simplifications first. Returning `[]` means that the current value cannot be |
| 855 | simplified. |
| 856 | |
| 857 | A custom `shrink` callback replaces the default behavior. Keep the default when simplifying fields and collection |
| 858 | items is enough. Prefer `map` or `toCodecArbitrary` when you can generate a simpler representation and transform it. |
| 859 | Use `shrink` only for application-specific shortcuts that those approaches cannot express clearly. |
| 860 | |
| 861 | ### Recursive Schemas |
| 862 | |
| 863 | Recursive Schemas are supported as long as there is a way for generation to stop: |
| 864 | |
| 865 | ```ts |
| 866 | import { Schema } from "effect" |
| 867 | import * as Arbitrary from "effect/Arbitrary" |
| 868 | |
| 869 | interface Node { |
| 870 | readonly value: string |
| 871 | readonly children: ReadonlyArray<Node> |
| 872 | } |
| 873 | |
| 874 | const Node: Schema.Codec<Node> = Schema.Struct({ |
| 875 | value: Schema.String, |
| 876 | children: Schema.Array(Schema.suspend(() => Node)).check(Schema.isMaxLength(3)) |
| 877 | }) |
| 878 | |
| 879 | const nodes = Arbitrary.schema(Node) |
| 880 | ``` |
| 881 | |
| 882 | Here, an empty `children` array stops the recursion. Effect limits recursion across the whole generated value so that |
| 883 | separate branches cannot each grow without regard for the others. |
| 884 | |
| 885 | Effect ignores a recursive alternative that can never stop. If the complete Schema has no way to produce a finite |
| 886 | value, `Arbitrary.schema` throws immediately. You do not need to provide a special terminal generator or depth marker. |
| 887 | |
| 888 | ### Declaration Schemas |
| 889 | |
| 890 | `Schema.declare` can describe a type whose internal structure is hidden from the generic Schema machinery. To generate |
| 891 | such a value, Effect looks for a simpler Schema representation in this order: |
| 892 | |
| 893 | 1. an explicit `toCodecArbitrary`; |
| 894 | 2. a representation provided by Effect for one of its built-in types; |
| 895 | 3. `toCodecJson`; |
| 896 | 4. `toCodec`. |
| 897 | |
| 898 | Most declarations already provide a usable conversion and need no Arbitrary-specific setup. Add `toCodecArbitrary` |
| 899 | only when the usual representation cannot be generated or produces poor test inputs. |
| 900 | |
| 901 | If `toCodecJson` is present but returns `undefined`, the declaration explicitly says that its JSON representation is |
| 902 | hidden. Generation stops with an error instead of trying `toCodec`. |
| 903 | |
| 904 | `toCodecArbitrary` returns a Schema `Link`, not an `Arbitrary`. The first Schema in the link describes values that are |
| 905 | easy to generate, and the transformation converts them into the declared type: |
| 906 | |
| 907 | ```ts |
| 908 | import { Schema, SchemaTransformation } from "effect" |
| 909 | |
| 910 | class UserId { |
| 911 | readonly value: number |
| 912 | constructor(value: number) { |
| 913 | this.value = value |
| 914 | } |
| 915 | } |
| 916 | |
| 917 | const UserIdSchema = Schema.instanceOf(UserId, { |
| 918 | toCodecArbitrary: () => |
| 919 | Schema.link<UserId>()( |
| 920 | Schema.Int.check(Schema.isBetween({ minimum: 1, maximum: 1_000_000 })), |
| 921 | SchemaTransformation.transform({ |
| 922 | decode: (value) => new UserId(value), |
| 923 | encode: (id) => id.value |
| 924 | }) |
| 925 | ) |
| 926 | }) |
| 927 | ``` |
| 928 | |
| 929 | Effect checks converted values against the original declaration. The conversion may reject some values; those values |
| 930 | count toward `maxDiscards`, and Effect can continue with others. Normal decode errors reject a value, while unexpected |
| 931 | defects and interruption fail the sampling or checking Effect. |
| 932 | |
| 933 | The callback also receives: |
| 934 | |
| 935 | - the value types represented by declarations with type parameters; |
| 936 | - the common Schema checks that Effect recognized for the declaration. |
| 937 | |
| 938 | Built-in collection types use arrays as their simpler generation representation. Map keys and set values remain |
| 939 | unique. Effect collections such as `HashMap`, `HashSet`, and `Chunk`, and types such as `Graph`, `BigDecimal`, and date |
| 940 | and time values, provide their own links when that produces better inputs. A declaration whose usual conversion already |
| 941 | works needs no Arbitrary-specific annotation. |
| 942 | |
| 943 | ## Using `@effect/vitest` |
| 944 | |
| 945 | `@effect/vitest` accepts arrays or records containing Schemas, Arbitraries, or both: |
| 946 | |
| 947 | ```ts |
| 948 | import { assert, it } from "@effect/vitest" |
| 949 | import { Effect, Schema } from "effect" |
| 950 | import * as Arbitrary from "effect/Arbitrary" |
| 951 | |
| 952 | const Name = Arbitrary.schema(Schema.Literals(["Ada", "Grace"])) |
| 953 | |
| 954 | it.prop( |
| 955 | "integer addition is commutative", |
| 956 | [Schema.Int, Schema.Int], |
| 957 | ([a, b]) => a + b === b + a, |
| 958 | { arbitrary: { runs: 200, seed: "addition" } } |
| 959 | ) |
| 960 | |
| 961 | it.effect.prop( |
| 962 | "generated values can be checked in an Effect", |
| 963 | { name: Name, value: Schema.Int }, |
| 964 | ({ name, value }) => |
| 965 | Effect.sync(() => { |
| 966 | assert.include(["Ada", "Grace"], name) |
| 967 | assert.isTrue(Number.isInteger(value)) |
| 968 | }), |
| 969 | { arbitrary: { runs: 50 } } |
| 970 | ) |
| 971 | ``` |
| 972 | |
| 973 | Generators from other property-testing libraries are not supported. Use Effect Arbitraries when an input needs more |
| 974 | composition than a Schema alone can provide. |
| 975 | |
| 976 | `@effect/vitest` turns `Falsified`, `Exhausted`, and `ReplayMismatch` results into test failures. The failure message |
| 977 | includes the simplified failing input and replay token. Diagnostic values use `Formatter.format`, so strings remain quoted and |
| 978 | escaped, collections such as `Map` and `Set` retain their structure, and application values can provide a custom |
| 979 | `toString`. |
| 980 | |
| 981 | The Vitest integration reports a property failure when the property returns `false`, throws, or returns a failed Effect |
| 982 | for a reason other than interruption. It then simplifies and reports the failing input. Effect interruption remains an |
| 983 | interruption. Any normal return value other than `false`, including `void`, passes for that input. |
| 984 | |
| 985 | The Vitest `timeout` interrupts the Effect fiber that generates inputs, runs the property, and simplifies failures. |
| 986 | Effect finalizers still run. A timeout cannot stop a synchronous JavaScript callback until that callback returns. |
| 987 | |
| 988 | ## Current Limitations |
| 989 | |
| 990 | The module starts from Schema and currently does not provide: |
| 991 | |
| 992 | - a second catalog of Schema-like constructors for strings, numbers, and objects; describe those inputs with Schema; |
| 993 | - a public low-level generator constructor or direct access to simplification steps; |
| 994 | - weighted choice; use [targeted scenarios](#targeting-rare-scenarios) to guarantee that a test reaches an important |
| 995 | situation; |
| 996 | - support for generators from other property-testing libraries; |
| 997 | - running generated inputs in parallel; |
| 998 | - automatic test failure messages outside `@effect/vitest`; |
| 999 | - replay tokens guaranteed to work across releases of this unstable module. |
| 1000 | |
| 1001 | These limitations may change while the module remains unstable. |
| 1002 | |
| 1003 | ## Summary |
| 1004 | |
| 1005 | Property-based testing is most useful when the property says more than “this function did not throw.” Start by choosing |
| 1006 | the allowed inputs and a rule that comes from the requirement: |
| 1007 | |
| 1008 | - use Schema checks to generate valid values; |
| 1009 | - combine independent inputs with records, tuples, or `Arbitrary.all`; |
| 1010 | - use `it.prop` and `it.effect.prop` in tests, and `checkEffect` when result values or replay are part of the |
| 1011 | program; |
| 1012 | - write laws directly and remember the limits of JavaScript numbers; |
| 1013 | - when two values are considered equal, check that public operations treat them the same way; |
| 1014 | - use command sequences and a simple model for stateful behavior; |
| 1015 | - pay attention to rejected values, simplification, and replay because they affect what the test actually checks. |
| 1016 | |
| 1017 | The most important choice is not how many random inputs to generate. It is choosing inputs and a rule that make a small |
| 1018 | counterexample useful. |
| 1019 | |
| 1020 | ## Appendix: Design and Implementation Decisions |
| 1021 | |
| 1022 | This appendix is for people who maintain or extend the module. You can use `Arbitrary` without reading it. Internal type |
| 1023 | names appear only where they help connect a decision to the code, and each one is explained when it first appears. |
| 1024 | |
| 1025 | ### Why Effect Has Its Own Implementation |
| 1026 | |
| 1027 | Effect owns the property-testing implementation so it can provide: |
| 1028 | |
| 1029 | - **Automatic support for recursion.** Effect finds the paths that let recursive Schemas stop. If no such path exists, |
| 1030 | `Arbitrary.schema` fails immediately. All recursive branches in one value share the same limit. |
| 1031 | - **A limit on rejected values.** `maxDiscards` stops generation when a condition is impossible or accepts too few |
| 1032 | values. `maxShrinks` also limits rejected values examined while simplifying a failure. |
| 1033 | - **Normal Effect behavior.** Sampling and checking can be interrupted, use services, and fail through Effect. Direct |
| 1034 | checks, `TestSchema`, and `@effect/vitest` use the same implementation. |
| 1035 | - **Direct composition.** `all` combines existing Arbitraries without creating a temporary Schema. |
| 1036 | - **One replay token.** The token records the initial attempt and every accepted simplification. Replay regenerates the |
| 1037 | initial value, runs the property again, and follows those simplifications. |
| 1038 | - **Useful behavior for `flatMap`.** When the first generated value becomes simpler, Effect rebuilds the value that |
| 1039 | depends on it while keeping later random choices stable. Nested `flatMap` calls share the same recursion limit. |
| 1040 | - **Small common paths.** Common Schema shapes use direct loops, and sampling does not build simplification data. Bundle |
| 1041 | and runtime benchmarks watch these paths, but do not promise an exact performance level. |
| 1042 | |
| 1043 | ### Interface Design Decisions |
| 1044 | |
| 1045 | These decisions explain why the public API contains some operations and omits others. Each section also says what new |
| 1046 | use case would justify changing the decision. |
| 1047 | |
| 1048 | #### Use Schema as the Public Generation Language |
| 1049 | |
| 1050 | **Status:** Use Schema as the single public language for describing generated data. |
| 1051 | |
| 1052 | Schema already describes primitive values, data structures, checks, conversions, declarations, and recursion. |
| 1053 | Duplicating those features with `Arbitrary.String`, `Arbitrary.Array`, and similar constructors would give users two |
| 1054 | different ways to describe the same inputs. |
| 1055 | |
| 1056 | The module therefore exposes `schema` and a small set of operations for combining existing Arbitraries. A declaration |
| 1057 | whose structure is hidden can provide a simpler Schema through `toCodecArbitrary`; it does not need a second language |
| 1058 | for generators. |
| 1059 | |
| 1060 | `Arbitrary.array(item, options)` composes an existing element Arbitrary, including one built with `map`, `filter`, |
| 1061 | `flatMap`, or `all`. It supports deleting commands while retaining the failing sequence's remaining values, which a |
| 1062 | dependent length generator cannot preserve. Schema remains the place to describe primitive values and checked data |
| 1063 | structures. |
| 1064 | |
| 1065 | The implementation stored inside `Arbitrary<A>` remains private. Users can combine and run an Arbitrary without |
| 1066 | depending on how Effect currently generates or simplifies values. Reconsider this decision only if an important input |
| 1067 | cannot be described by Schema or built clearly from existing Arbitraries. |
| 1068 | |
| 1069 | #### Keep Low-Level Arbitrary Construction Internal |
| 1070 | |
| 1071 | **Status:** Do not expose `Arbitrary.make` for the current internal generator type. |
| 1072 | |
| 1073 | Applications can already build inputs through: |
| 1074 | |
| 1075 | - `Arbitrary.schema` for primitive values, data structures, checked values, and recursive values; |
| 1076 | - `Constant` for lifting an already constructed value into dependent generation; |
| 1077 | - `map`, `filter`, `filterMap`, `flatMap`, and `all` for composition; |
| 1078 | - `toCodecArbitrary` when a Schema declaration hides its structure and needs a simpler Schema for generation. |
| 1079 | |
| 1080 | The internal `Generator` does more than turn random numbers into values. It also tracks the minimum space needed for |
| 1081 | recursion, rejected values, later simplifications, interruption, and replay positions. Exposing it would require every |
| 1082 | custom generator to understand those rules. |
| 1083 | |
| 1084 | A simpler public constructor that ignores some rules would create Arbitraries that behave differently when combined or |
| 1085 | simplified. Reconsider this decision when at least two real generators cannot be expressed through Schema and the |
| 1086 | existing operations. Any proposal should keep the internal bookkeeping out of application code. |
| 1087 | |
| 1088 | #### Provide a Constant Constructor |
| 1089 | |
| 1090 | **Status:** Provide `Arbitrary.Constant` for an existing value. |
| 1091 | |
| 1092 | Without `Constant`, callers must generate an unrelated value and map it to the desired constant. `Schema.Literal` |
| 1093 | supports only literal values and is awkward to rebuild repeatedly inside `flatMap`. |
| 1094 | |
| 1095 | `Constant` uses no randomness and offers no simpler values of its own. The value that selected it through `flatMap` can |
| 1096 | still be simplified. Every run returns the same value, so objects are not cloned and properties must not modify them. |
| 1097 | |
| 1098 | #### Use Schema Union for Static Choice |
| 1099 | |
| 1100 | **Status:** Do not add a second operation for choosing uniformly among fixed alternatives. |
| 1101 | |
| 1102 | `Arbitrary.schema(Schema.Union([...]))` already chooses among alternatives. Because the alternatives remain Schemas, |
| 1103 | Effect can see their checks, recursion needs, and validation rules. `Arbitrary.oneOf` would duplicate that behavior |
| 1104 | without supporting a new use case. |
| 1105 | |
| 1106 | A weighted choice would add probabilities, but probabilities do not guarantee that a property reaches an important |
| 1107 | situation. The later decision about weighted choice explains the preferred alternative. |
| 1108 | |
| 1109 | #### Keep Size Management Internal |
| 1110 | |
| 1111 | **Status:** Do not pass the current `size` to an `Arbitrary.sized` callback. |
| 1112 | |
| 1113 | The runner's `size` option already lets strings, collections, and recursive Schemas become more complex as a check |
| 1114 | progresses. Most users therefore do not need to write their own size rules. |
| 1115 | |
| 1116 | Passing this value to callbacks would make the runner's growth policy part of the public composition API. It is mainly |
| 1117 | useful for low-level custom generators, which the module does not currently expose. Reconsider this decision when an |
| 1118 | important input needs a size rule that Schema checks and recursion cannot describe. |
| 1119 | |
| 1120 | #### Keep Custom Shrinking at the Schema Boundary |
| 1121 | |
| 1122 | **Status:** Add custom simplification only through `Arbitrary.schema(schema, { shrink })`. |
| 1123 | |
| 1124 | The Schema produces every initial value and checks every value proposed by the callback. Invalid proposals are skipped; |
| 1125 | valid proposals can be simplified again with the same callback. Effect calls the callback only when needed and records |
| 1126 | its stable order for replay. |
| 1127 | |
| 1128 | The callback replaces the default simplification order. Mixing two independent orders would make `maxShrinks` and |
| 1129 | replay positions hard to predict. |
| 1130 | |
| 1131 | A general `Arbitrary.reshrink` could not provide the same automatic validation. An Arbitrary built with `Constant`, |
| 1132 | `map`, `flatMap`, or `all` does not retain one original Schema. Reconsider a general operation only when a real input |
| 1133 | cannot be represented by Schema and needs its own simplification. A proposal must explain how values are checked, how |
| 1134 | the two simplification strategies interact, and how replay remains stable. |
| 1135 | |
| 1136 | #### Prefer Targeted Scenarios to Weighted Choice |
| 1137 | |
| 1138 | **Status:** Do not add `Arbitrary.frequency` merely to reach important test cases more often. |
| 1139 | |
| 1140 | Weights only change how often a branch is chosen. They cannot guarantee the combination of values that makes a behavior |
| 1141 | important. A general property over `Schema.Union`, plus a focused property whose input already contains the important |
| 1142 | situation, states the requirement directly and preserves it during simplification. |
| 1143 | |
| 1144 | Workloads that must match real probabilities belong in simulations or benchmarks. Reconsider weighted generation only |
| 1145 | if a correctness rule itself depends on a distribution, rather than merely needing better test coverage. |
| 1146 | |
| 1147 | #### Name Effectful Runners Explicitly |
| 1148 | |
| 1149 | **Status:** Keep the `Effect` suffix on `sampleEffect` and `checkEffect`. |
| 1150 | |
| 1151 | The names show that these functions return an Effect. They also leave clear names available if a real need for |
| 1152 | synchronous versions appears later; no such version is currently promised. |
| 1153 | |
| 1154 | #### Current Interface Scope |
| 1155 | |
| 1156 | The module focuses on inputs described by Schema and checks executed with Effect. It guarantees that generated values |
| 1157 | match their Schema, rejection is limited, recursion stops, and replay works within the same implementation. Exact |
| 1158 | generated sequences, probabilities, and intermediate simplifications may change. |
| 1159 | |
| 1160 | The unstable interface does not currently expose: |
| 1161 | |
| 1162 | - a second catalog of Schema-like constructors, or direct access to internal samples and simplification steps; |
| 1163 | - weighted distribution controls; |
| 1164 | - custom simplification that is not attached to a Schema; |
| 1165 | - support for generators from other libraries or a broad set of runner settings; |
| 1166 | - parallel property evaluation; |
| 1167 | - test-runner assertion integration outside `@effect/vitest`; |
| 1168 | - replay tokens guaranteed to work across releases. |
| 1169 | |
| 1170 | These omissions are not necessarily permanent. They let Schema remain responsible for describing valid values and let |
| 1171 | Effect change the internal generator while each proposed API addition is considered separately. |
| 1172 | |
| 1173 | ### Technical Decisions |
| 1174 | |
| 1175 | The remaining sections describe the current implementation. These details matter to maintainers because changing them |
| 1176 | can alter generated values, simplification, replay, performance, or bundle size without changing the public types. |
| 1177 | |
| 1178 | #### Representing Generation |
| 1179 | |
| 1180 | - The internal `Generator<A>` stores `minCost`, the minimum space needed to produce a value, and `generate`, the |
| 1181 | generation function. A Schema being prepared uses `Compiled<A>`, which also records its dependencies. Arbitrary |
| 1182 | operations outside Schema do not join that dependency graph. |
| 1183 | - One call returns `Generated` when it produced a value or `Discarded` when it rejected the attempt. It returns directly |
| 1184 | when possible and uses an Effect only when needed. Internal mapping uses Effect's eager operations so an immediate |
| 1185 | result remains synchronous. |
| 1186 | - A generated value may provide a `Pull`: an internal operation that returns one proposed simplification at a time. |
| 1187 | Effect creates this sequence only when needed. Sampling does not create it, and rejected proposals remain visible to |
| 1188 | the runner so they count toward `maxShrinks`. Operations such as `filter` can skip a rejected proposal and continue |
| 1189 | with simpler values that follow from it. |
| 1190 | |
| 1191 | #### Preparing a Schema |
| 1192 | |
| 1193 | - Preparation starts with `SchemaAST.toType`, which selects the value after decoding rather than its encoded form. |
| 1194 | - The implementation reads Schema's existing internal syntax tree, called its AST, instead of creating a second tree |
| 1195 | for Arbitrary. This preserves the original order of checks, declaration links, type parameters, recursive references, |
| 1196 | and error paths. |
| 1197 | - Preparation happens immediately and reuses work for repeated nodes in the same Schema. Unsupported declarations, |
| 1198 | contradictory bounds, and recursion with no stopping point fail during `Arbitrary.schema`. Combinations that are |
| 1199 | valid but impossible to satisfy eventually return `SampleError` or `Exhausted`. |
| 1200 | - Effect builds common checks into generation and still runs the original checks afterward. Ordinary Schema nodes do |
| 1201 | not use the complete Schema parser for every value. Declaration links are converted and then checked against the |
| 1202 | original declaration. |
| 1203 | - For supported regular expressions, Effect generates matching text directly and checks the pattern afterward. It |
| 1204 | chooses uniformly among supported patterns. Unsupported patterns act as ordinary filters instead of making the |
| 1205 | complete Schema unsupported. |
| 1206 | - Prepared patterns cache valid UTF-16 lengths and reusable character information. The same information helps simplify |
| 1207 | generated strings without changing which strings the pattern accepts. |
| 1208 | |
| 1209 | #### Declarations |
| 1210 | |
| 1211 | - Effect looks for a declaration representation in this order: `toCodecArbitrary`, a built-in representation, |
| 1212 | `toCodecJson`, then `toCodec`. If `toCodecJson()` returns `undefined`, the declaration says its JSON form is hidden, |
| 1213 | so preparation stops instead of silently trying another conversion. |
| 1214 | - `toCodecArbitrary` returns a Schema `Link`. Effect generates the source Schema, converts it, and checks the result |
| 1215 | against the original declaration. Rejected initial values count toward `maxDiscards`; rejected simplifications count |
| 1216 | toward `maxShrinks` while later valid proposals remain available. |
| 1217 | - The callback receives the represented type parameters and the common checks Effect recognized. `ReadonlyMap` and |
| 1218 | `ReadonlySet` use arrays internally. `HashMap`, `HashSet`, `Chunk`, and other specialized types keep local links when |
| 1219 | that avoids loading their implementation into every use of the generic Schema compiler. |
| 1220 | - Effect may use an `Order` while combining bounds, but passes only the final bounds to `toCodecArbitrary`. The link is |
| 1221 | responsible for producing suitable values, and the original declaration rejects incompatible results. |
| 1222 | - Links used only for generation cannot encode values; generation calls only their decode direction. |
| 1223 | |
| 1224 | #### Recursion and Size |
| 1225 | |
| 1226 | - Effect examines recursive references as a graph and computes the minimum space needed to reach a non-recursive value. |
| 1227 | A recursive alternative that can never stop is ignored; a complete Schema that can never stop is rejected |
| 1228 | immediately. |
| 1229 | - Each attempt receives the minimum required space plus the configured `size`. Crossing a recursive reference consumes |
| 1230 | some of that allowance. Effect reserves enough for required child values before generating optional siblings. |
| 1231 | Recursive siblings are tried in a changing order so one declaration position does not always receive more space, but |
| 1232 | the returned object keeps its declared order. |
| 1233 | - Every recursive branch in the generated value shares one allowance. Nested composition does not create fresh space. |
| 1234 | - Strings and collections also use `size` as a rough complexity target. Sampling uses a fixed value; checking raises it |
| 1235 | after successful runs. Schema minimums are always respected, maximums remain limits, and rejected attempts do not |
| 1236 | increase the size. |
| 1237 | |
| 1238 | #### Randomness and Probabilities |
| 1239 | |
| 1240 | - The runner starts from one seed and creates a separate random-number state for each attempt. Replay can therefore jump |
| 1241 | directly to a recorded attempt, and a property's use of Effect `Random` cannot change later generated inputs. |
| 1242 | - On some predictable attempts, generated records have no prototype. Choosing this case does not consume a random |
| 1243 | number, and every record produced during that attempt uses the same choice, including simplified values. This applies |
| 1244 | to `Schema.Struct`, `Schema.Record`, `Schema.Json`, and record-shaped `all`, but not to arrays, tuples, declarations, or |
| 1245 | collection classes. |
| 1246 | - Integer and BigInt generation avoids favoring some values accidentally. It tries boundary values more often on some |
| 1247 | runs. Unbounded BigInts select their bit length separately from their value, mixing ordinary magnitudes with much |
| 1248 | wider values even at a small `size`. Number generation includes signed zero, very small values, infinities, and `NaN` |
| 1249 | when the Schema permits them. Finite and integer checks exclude the values they promise to exclude. |
| 1250 | - BigDecimal generation selects precision and decimal exponent independently, while ordered constraints are projected |
| 1251 | to the selected scale. Its simplifications operate on the represented numeric value, so a fixed generated scale does |
| 1252 | not dictate the final counterexample. Local simplification continues between passing and failing values with a finite |
| 1253 | limit on added precision. Ordinary strings combine printable ASCII with a fixed set of JavaScript edge cases. |
| 1254 | Regular-expression length is measured in UTF-16 code units, matching JavaScript strings. |
| 1255 | - Exact probabilities, the value produced by a particular seed, and the order of simplifications may change. Source |
| 1256 | code comments credit algorithms adapted from other property-testing and random-number implementations. |
| 1257 | |
| 1258 | #### Generating and Simplifying Data Structures |
| 1259 | |
| 1260 | - Arrays try removing progressively smaller blocks at successive positions and simplifying individual items. Deletions |
| 1261 | retain the other elements and their shrink history, so an earlier element candidate can be tried again after deletion |
| 1262 | changes the property context. This history is lazy and local to a collection generation. Products and objects also |
| 1263 | retain shared child histories, so exploring a filtered descendant cannot exhaust a sibling's shrink candidates. |
| 1264 | Ordinary field shrinking continues from the field being simplified; structural deletions restart field traversal. |
| 1265 | Tuple prefixes and fixed tails cannot be deleted; optional positions can only be removed from the end when the tuple |
| 1266 | permits it. Whole-array minima apply to every deletion. Objects choose optional properties without favoring earlier |
| 1267 | declarations. If the first choice is too large for the available recursion space, Effect |
| 1268 | uses the smallest choice that still satisfies the Schema instead of rejecting a Schema that can produce a value. |
| 1269 | Simplification removes optional properties, simplifies values, and then simplifies generated keys while keeping keys |
| 1270 | unique. |
| 1271 | - Generation of unique collections has a retry limit. `Schema.isUnique()` compares complete values; |
| 1272 | `Schema.isUniqueKey()` compares the keys of Map entries. Both use Effect equality. Primitive values use specialized |
| 1273 | tracking, while objects use Effect `Hash` and `Equal`. |
| 1274 | - A Schema union chooses uniformly among alternatives that fit in the remaining recursion space. During simplification, |
| 1275 | it first tries the earliest alternative with the lowest required space when that alternative is cheaper, then |
| 1276 | simplifies the selected alternative. Separate random-number state keeps unrelated later values from changing this |
| 1277 | fallback. For `oneOf`, a value must match exactly one alternative; overlaps are rejected. |
| 1278 | - `all` adds the minimum space required by its members and makes them share one recursion allowance. It changes their |
| 1279 | internal generation order for fairness, then restores tuple positions and record keys. Simplification changes one |
| 1280 | member at a time. |
| 1281 | - `map` transforms every proposed value without consuming randomness or changing replay positions. `filter` and |
| 1282 | `filterMap` reject values that do not pass their condition. During simplification, Effect skips rejected proposals and |
| 1283 | can continue to later valid ones. `map` keeps duplicate transformed values because removing them would change replay |
| 1284 | positions. Each rejection consumes one unit of `maxShrinks` without running the property. |
| 1285 | - `filterMap` ignores the failure value in `Result`. Separate implementations for `map` and `filter` avoid allocating a |
| 1286 | `Result` in these common cases. |
| 1287 | - `Arbitrary.schema(schema, { shrink })` keeps normal generation but replaces default simplification. Effect calls the |
| 1288 | callback only when needed and checks each proposal against the Schema value after decoding. Invalid proposals consume |
| 1289 | `maxShrinks`, never reach the property, and are not simplified further. |
| 1290 | |
| 1291 | #### Dependent Generation with `flatMap` |
| 1292 | |
| 1293 | - Effect first generates the source value, calls the callback, and then runs the Arbitrary selected by that callback. |
| 1294 | The selected Arbitrary always receives enough space to produce its smallest possible value. |
| 1295 | - Simplification starts with the source. Each simpler source value selects a new dependent Arbitrary. Effect later tries |
| 1296 | simpler values from the current dependent Arbitrary. Once it accepts one of those dependent values, it no longer |
| 1297 | returns to source simplification on that path. |
| 1298 | - If the initial source or dependent Arbitrary rejects its value, the complete attempt is rejected. If a dependent value |
| 1299 | selected during simplification is rejected, it consumes one unit of `maxShrinks`, and Effect continues with later |
| 1300 | simplifications of the source. |
| 1301 | - Sampling needs no saved random state because it does not simplify values. Checking saves the random state immediately |
| 1302 | after source generation. The initial dependent value and every dependent value chosen from a simpler source receive a |
| 1303 | separate copy, so trying one simplification cannot change another or affect later generated inputs. |
| 1304 | - The dependent Arbitrary temporarily receives the minimum extra recursion space it needs. Any unused extra space is |
| 1305 | removed afterward, so nested `flatMap` calls still share one overall allowance. |
| 1306 | - Calling `Arbitrary.schema` inside the callback prepares that Schema on every callback call. Effect does not cache it |
| 1307 | automatically. |
| 1308 | |
| 1309 | #### Running, Replaying, and Interrupting Checks |
| 1310 | |
| 1311 | - `sampleEffect` fails with `SampleError`. `checkEffect` returns `Passed`, `Falsified`, `Exhausted`, or |
| 1312 | `ReplayMismatch`. `SampleError` and `Exhausted` include the seed so the run can be repeated. Only the boolean `true` |
| 1313 | passes. Returning `false` and returning a failed Effect are different kinds of property failure, and simplification |
| 1314 | preserves the original kind. The exact Effect error may change. Unexpected defects and interruption remain Effect |
| 1315 | failures rather than result values. |
| 1316 | - After a failure, Effect follows the first proposed simpler value that fails in the same way. `maxShrinks` counts every |
| 1317 | proposal examined, including rejected values; `shrinks` counts only proposals accepted as the new counterexample. |
| 1318 | Property runs do not include these extra evaluations. When the limit is reached, Effect returns the most simplified |
| 1319 | failing input found so far. It does not clone or freeze generated values. |
| 1320 | - A replay token records the seed, attempt number, size, original kind of failure, and the position of every accepted |
| 1321 | simplification. Replay rebuilds those values instead of storing them in the token. It returns `ReplayMismatch` when a |
| 1322 | position no longer exists or no longer fails in the same way. It does not compare the exact final input or Effect |
| 1323 | error. Malformed tokens may fail with a defect, and tokens need not work across releases while the module is unstable. |
| 1324 | Replay follows the recorded positions directly, so it ignores `maxShrinks`. |
| 1325 | - Long synchronous generation loops occasionally yield control according to `Scheduler.MaxOpsBeforeYield`. Generation |
| 1326 | that uses Effects, declaration conversion, property evaluation, and simplification can all be interrupted normally. |