Testing

Compiler testing is layered by contract. A source-to-runtime test proves user behavior, but it does not replace a parser recovery test or identify which MIR invariant failed. A pass unit test proves a local rewrite, but it does not show that the linked module executes.

Test pyramid

Changes should be tested at the lowest owning layer and at the highest integration boundary they affect.

Positive and negative fixtures

A language feature needs both:

• a positive fixture that checks the accepted shape and behavior;
• a negative fixture that checks the exact rejection, range, layer, and recovery behavior.

Backend work also needs a boundary fixture. If sema rejects an unsupported aggregate at a host boundary, test that diagnostic directly and separately test the closest supported aggregate through codegen.

Avoid fixtures that accidentally pass because the target code path never runs. A trigger test should assert a pass event, changed IR shape, selected import, or runtime result proving the intended path was exercised.

Unit and invariant tests

Small unit tests are appropriate for:

• lexer and parser productions;
• source-range arithmetic;
• type unification and substitution;
• specialization-key canonicalization;
• fact invalidation;
• individual MIR and WasmIR rewrites;
• linker codecs and remap tables;
• diagnostic construction and rendering.

For an internal transform, assert both the expected rewrite and the invariants it must preserve. A constant-fold test that never validates the resulting CFG is incomplete.

Snapshots

Snapshots are valuable for structured, reviewable output:

• HIR and MIR text;
• WasmIR or Wasm text;
• diagnostics with ranges and labels;
• debug metadata;
• trace event sequences.

A snapshot is an alarm, not an oracle. Review an update by asking why every changed line moved. Do not refresh snapshots merely because a test command offers an update flag.

Keep snapshots focused. One enormous whole-project dump makes unrelated changes noisy and hides the invariant a fixture intends to protect.

Differential tests

When two compiler paths should be equivalent, compare them directly:

Differential tests are the strongest guard for migrations because the existing path remains an executable specification until the new path proves parity.

Determinism tests

Run-to-run tests compile the same fixture repeatedly with fresh collections. Cross-process tests also reset process state and randomized seeds. Parallel tests vary worker schedule. See Reproducibility for the full contract.

Every deterministic test should compare the strongest practical output. A fingerprint helps failure messages, but complete byte equality prevents a hash collision from hiding drift.

End-to-end tests

CLI/runtime tests should use the same public path as users:

1. discover or create a project;
2. invoke checking, building, or running;
3. assert exit behavior and structured diagnostics;
4. inspect written unit artifacts where relevant;
5. execute the unit for runtime-sensitive semantics;
6. cleanly isolate temporary project and output paths.

Runtime tests are required for layouts, ownership, suspension, cancellation, arena growth, host calls, and any optimization whose bug can preserve valid Wasm while changing behavior.

Incremental tests

An incremental test needs at least three snapshots:

1. initial result;
2. no-edit repeat, proving cache hits;
3. edited result, proving correct invalidation.

Useful edits include:

• body-only change, which should preserve declaration resolution;
• signature change, which must invalidate dependents;
• trait implementation change, which can affect dispatch;
• unit-local change, which should preserve unaffected unit bytes or handles;
• file add/remove, which changes project enumeration.

Assert both reuse and correctness. A high cache-hit rate is meaningless if the stale result is wrong.

Diagnostic tests

For each new diagnostic, assert:

• code and severity;
• producer layer;
• file and primary byte range;
• secondary labels;
• help and suggestion edits where present;
• stable ordering among sibling diagnostics;
• internal-frame redirection behavior;
• the absence of redundant cascades.

String-only assertions are fragile and miss protocol fields used by editors.

Performance tests

Performance tests measure, rather than define, correctness. Keep:

• ignored or opt-in corpus measurements for large configurations;
• stable smoke thresholds only where the environment permits them;
• stage timings and allocation baselines as regression signals;
• correctness assertions in the same fixture so a faster skipped stage cannot look like an improvement.

Wall-clock timings are not reproducibility fingerprints.

Choosing commands

During development, run the narrow owning test first, then widen:

cargo test -p atoll-syntax
cargo test -p atoll-sema --test suite
cargo test -p atoll-mono --test suite
cargo test -p atoll-mir --test suite
cargo test -p atoll-codegen-wasm --test suite
cargo test -p atoll-db --test suite
cargo test -p atoll-cli --test suite

The repository’s CI or nextest configuration may provide faster suite-level execution; use the checked-in project command when it differs. A documentation example should not imply that running only one bundled suite replaces workspace checks.

Change gate

Before merging a compiler change:

1. identify the stage that owns the new invariant;
2. add a focused test at that stage;
3. add a differential or parity test if an execution path changed;
4. add runtime coverage if observable behavior, memory, or suspension changed;
5. add determinism coverage for any output-visible collection or parallel reduce;
6. validate malformed input and diagnostics;
7. run the narrow suite, dependent suites, and formatting/lint checks;
8. review snapshots and generated artifacts rather than updating them blindly.

The goal is a chain of evidence from the smallest rule to the public behavior.