Build & Test¶

For contributors and embedded/systems developers. The Makefile drives the build, the test pyramid, the metrics tool, and the docs site. Embedded/systems developers wanting to ship a feature-tailored binary will care about the Build section + the per-feature flags (Getting Started: Embedded). Contributors will care about everything.

Makefile reference¶

make help produces this listing dynamically. The targets, by category:

Build¶

Set CONFIG=full on any of these to select the everything-on config arm. Per-feature -Dfeature-*=on/off flags (ADR 002) compose freely with -Dconfig.

Test¶

The pyramid runs fast-first: unit → integration → e2e.

Metrics¶

Longitudinal recording of size + coverage + (opt-in) perf.

Coverage¶

Requires kcov (brew install kcov on macOS, apt-get install kcov on Linux):

Compatibility surface¶

Browser smoke¶

The tools/browser-smoke.py harness drives the web demos under headless Chromium. Use it for catching DOM-side regressions that the corpus + integration suite can't see.

See tools/browser-smoke.py for the full step primitives (click, type, select, eval, assert-text, screenshot, etc.).

Quality¶

The strict invariant: make check¶

$ make check
zig build -Dconfig=nano
python3 tools/progress.py --config nano ./zig-out/bin/nanopython
nanopython progress:  100.0%  [########################################]
  passing 541/541 corpus tests (byte-identical to the frozen reference)
  command: ./zig-out/bin/nanopython

This is the gate every commit must pass.

• Numerator: tests whose stdout+stderr+exit-status matches the frozen oracle byte-for-byte.
• Denominator: tests in corpus/nano/basics/ minus those in EXCLUDED.txt.
• Exit status: make check reports the count but always exits 0. make verify is the strict variant that exits non-zero on any miss.

If make verify reports any miss, you broke something. Identify the failing test and either fix it or revert. The 541/541 baseline is the long-standing strangler invariant; it's been at this level since the 2026-06-11 session closed builtin_help + gc1 + string_strip.

The wasm gate: make wasm-check¶

$ make wasm-check
nanopython wasm progress:  87.2% [###################################]
  passing 472/541 corpus tests
  GREEN — baseline holds: 68 known fails matched exactly

This gate is strict in both directions: - A new fail not on corpus/wasm-known-fails.txt → exits non-zero (you regressed something on wasm). - A test on the known-fails list that now passes → exits non-zero (you fixed something; remove the entry).

The list is documented in-place: each entry has a comment explaining why it's there (bignum needing MPZ, eval/exec hitting the longjmp wall, the two deliberate deque_* divergences).

The secondary differential gate: compile-exec-test¶

$ python3 tools/compile-exec-test.py
compile-exec-test: 540/542 pass, 2 fail, 0 compile-fail, 0 timeout, 0 runner-err

For each .py source, this compiles with our zigpy-compile (Zig frontend), then runs the resulting .mpy through the upstream MicroPython unix binary, and diffs stdout/stderr/exit-status against the frozen corpus/full/basics/*.out. Tests execution equivalence; catches cases where our compiler emits subtly-wrong bytecode that the byte-exact gate misses.

The "2 fail" are harness artifacts (mixed-sign bit-or constant folding), documented in the test runner's source.

Optimization mode comparison¶

For comparison, upstream MicroPython unix-standard (arm64, their -Os default): 652 KB.

The wasm artifact: 270 KB raw post-wasm-opt -Oz (see ADR 007 for the size arc and remaining targets).

CI strategy¶

No CI is wired into the public-facing pipeline yet. Local-only quality gates:

1. Before commit: make.
2. Before a tagged release: make on a fresh checkout (make clean && make) + a manual make smoke against the bundled web demos.

A future CI would run make on every PR. make verify and make wasm-check produce clean signals: diff the pass/fail list against main to localize any regression.

Adding your own tests¶

A Zig unit test¶

Add a test "..." { ... } block inline in any module under src/. Re-run make test-unit-compiler (for compiler-track files) or make test-unit-vm (for runtime-side files).

test "my_helper returns the right thing" {
    const std = @import("std");
    const testing = std.testing;
    try testing.expectEqual(@as(usize, 42), myHelper());
}

An integration smoke test¶

Edit tests/integration.py. Each entry is (name, py_source, expected_stdout) or (name, py_source, expected_stdout, expected_exit):

("my_test_case",
 "x = 5\nprint(x * 2)",
 "10\n"),

("my_exits_with_code",
 "import sys; sys.exit(7)",
 "",
 7),

Run with make test-integration (native) or make test-integration-wasm (wasm). The output of running py_source via the chosen binary is compared to expected_stdout.

Every bug fix lands with at least one regression pin here. If a wasm-only bug slipped past the corpus, that's the right place to plug the gap.

A corpus test¶

Corpus tests come from sandbox/micropython/tests/basics/*.py. Adding one is non-trivial: the goldens have to come from a known-good reference. See tools/corpus.py for the freeze logic; in practice, new corpus tests follow upstream MicroPython's test additions.

A golden program¶

Drop a .py file into the appropriate golden-programs/<bucket>/ directory with the per-program header comment:

# golden: source=mysource; needs=heapq,recursion
# golden: works-on=micropython,nanopython-native,nanopython-wasm
"""What this program demonstrates."""

Then make golden rebootstraps the buckets and re-verifies the annotations against three oracles (MicroPython, nanopython native, nanopython wasm). See ADR 008 for the bucket semantics.