Having the matrix of test targets for incremental compilation in the
individual test manifests has turned out to be inconvenient for a few
reasons: the tests are almost certain to accidentally get out of sync,
disabling targets entirely is annoying to do, and incr-check needs to
take care to print the target in all error messages (which currently
does not always happen).
If I recall correctly, I originally designed it this way because it
allows targets to be disabled at the granularity of individual tests,
but there's an easier approach to that: just let a test manifest that it
should be *skipped* on a certain target! As skipping is the rare case,
and also the case you want readers to notice, it makes sense for *it* to
be explicitly specified, like how unit tests use `error.SkipZigTest`.
So, `incr-check` no longer runs through a list of targets specified in
the manifest. Instead, it accepts (and, in fact, requires) a single
target on the command line, and runs the test for that specific target.
If the file contains a `#skip_target` directive for that target, then
`incr-check` exits immediately, so we can still disable targets at
individual test granularity, but you can also disable a target for all
tests by just commenting it out of the matrix in `test/tests.zig`.
As a nice bonus, this also allows the build system to run different
incremental test targets in parallel, because the targets are now
different steps.
This definitely seems like a better way to split the work between the
build system and incr-check---sorry for getting this wrong initially!
...but not in the way you'd expect. We were actually tracking them in
cases where we shouldn't have been! We cannot track a declaration if its
parent namespace has been lost, because that will cause analysis
failures immediately, but if we excluded a type from the mapping due to
a major change (such as a struct turning into a union, or a field being
added), we were still including any trackable instructions inside the
container's field expressions (e.g. struct field type expressions). This
meant we were tracking a type declaration while losing tracking on its
parent namespace, with predictably disastrous results.
Oh, also, tracking for opaque types was just totally wrong (I think this
was a typo from a while back). We could map it to things other than
opaque declarations, and we never mapped declarations inside opaques.
So, uh, I fixed that too.
There is no reason `inline fn`s should not be subject to error tracing:
they are still functions! So, push to the error trace when we return
from one, and add a test checking that inline functions do appear in
error traces.
This also changes how we emit error trace pushes: we no longer duplicate
the AIR `ret` instruction in the "error" and "non-error" code paths. I
suspect this will lead to slightly better unoptimized codegen, but I may
be wrong---I'll take some performance measurements before I merge this.
This PR enables all incremental tests under the `test/incremental` directory, except one: `change_exports`, which is currently ignored as it requires a non-trivial amount of work on the linker, since we do not currently support exporting data symbols.
To enable the other tests, the following fixes were needed:
1. `src/link/Wasm.zig`: instead of chasing function type through Nav, get it directly.
2. `src/target.zig`: `.panic_fn` appears to work fine with the wasm backend.
3. `src/codegen/wasm/CodeGen.zig`: there was a liveness related bug that caused some `ArenaAllocator` code to crash the backend.
More info on (3), the liveness and local reuse code in the backend for years in unfinished state. For example there is currently no branch merging and reuse happens only when inst die in same block level. I initially considered doing a large refactor to implement everything correctly, but aborted due to its sheer size and currently! no clear idea about how to do this efficiently.
Instead, I fixed the bug with minimal changes and removed useless code, keeping the old solution otherwise intact.
- New Features
-- Multiprocess Fuzzing
The fuzzer now is able to utilize multiple cores. This is controllable
with the `-j` build option. Limited fuzzing still uses one core.
-- Fuzzing Infinite Mode
When provided multiple tests, the fuzzer now switches between them and
prioritizes the most effective and interesting ones. Over time already
explored tests will become barely run compared to tests yielding new
inputs.
-- Crash Dumps
Crashing inputs are now saved to a file indicated by the crash message.
It is recommended to use these files to reproduce the crash using
`std.testing.FuzzInputOptions.corpus` and @embedFile.
- Design
Each fuzzing process is assigned an instance id which has the following
uses:
* In conjunction with the pc hash and running test index, they uniquely
identify input files in the case of a crash.
* It is combined with the test seed for a unique rng seed.
* Instance 0 is solely responsible for syncing the filesystem corpus.
When new inputs are found, they are sent to the build server. It then
distributes the new input to the other instances. Each instance has a
concurrent poller managed by the test runner which sends received
inputs to libfuzzer. (note that this is affected by #31718 and so can
(rarely) deadlock)
For fuzzing infinite mode, the test runner now receives a list of tests
from the build server. The fuzzer runs tests in batches of one second,
approximated in cycles by the previous batch's run speed. Tests finding
new inputs or with few runs are given a higher run chance. The baseline
run chance is based off the recency of the last find and the number of
pcs the test has hit.
The `rint` tests were commented out when added along with other functions to
the `libc-test` runner, presumably because they were failing. In the meantime,
we moved to using a Zig-implemented `rint` and I found that the tests are now
passing. To celebrate, I'm putting them back into commission.
Having these tests in a working state is nice because functions like `lrint`,
`llrint`, and `nearbyint` depend on `rint` implementation being correct.
Results:
```
$ ./build/stage3/bin/zig build -p stage4 -Denable-llvm -Dno-lib
$ stage4/bin/zig build test-libc -Dlibc-test-path=<LIBC-TEST-PATH> -Dtest-filter=rint -fqemu -fwasmtime --summary line
Build Summary: 2209/2209 steps succeeded
```
These all pass now! I have also removed the warning about the LLVM
backend not supporting incremental compilation; I expect it will work
sort of okay in practice by now.
Matthew Lugg