The active contributors and maintainers of Zig's linker code have
generally found the current linker test harness to be cumbersome. The
tests require a lot of maintenance, but do not provide a lot of
coverage, and when they fail it is painful to troubleshoot.
Furthermore, as part of working on #31691, I don't want to port over the
CheckObject step, because I don't like the code anyway.
The plan forward is to start enhancing `zig objdump` to assist in
linker development, as well as using it as the basis for snapshot
testing.
We absolutely need linker test coverage, but we need to try to improve
these things about the next attempt:
* less effort to create and maintain tests
* less CPU overhead - we should be able to add a lot of tests without
adding a lot of CI time.
* more helpful failures. A failed linker test should provide the next
steps a developer can take to understand why the test failed.
* a goal of porting over all of LLD's test suite, or at least the good
ones.
I'm not going to open an issue to track the lost linker test coverage,
because there was already so much lack of coverage for linker stuff.
However I will open issues to track this lost coverage:
* the deleted checks from test/standalone/glibc_compat/build.zig
* the deleted checks from test/standalone/compiler_rt_panic/build.zig
* the deleted checks from test/standalone/ios/build.zig
Before:
* test-zigc: run libzigc unit tests (part of test-modules)
* test-libc: run libc-test cases
Now:
* test-libc: run libc API unit tests (part of test-modules)
* test-libc-nsz: run libc-test cases
libc API unit tests (previously referred to as libzigc unit tests) now run for
all supported targets, even those we don't provide libzigc for. The idea is that
this will help us catch bad assumptions in the unit tests, as well as bugs in
other libcs.
I considered this setup:
* test-c: run libc API unit tests (part of test-modules)
* test-libc-nsz: run libc-test cases
* test-libc: both of the above
However, I do not like it because it gives a false sense of security; the full
module and C ABI test suites are still liable to catch libzigc bugs that test-c
and test-libc-nsz might not. So contributors should just run the test steps
outlined in https://codeberg.org/ziglang/zig/issues/30978.
Co-authored-by: rpkak <rpkak@noreply.codeberg.org>
For a test target with link_libc = null, we need to skip it if we would
implicitly link libc because the target requires it or because we don't yet have
a syscall layer for it.
This is distinct from the question of whether the target formally considers libc
to be the only stable syscall interface; for example, FreeBSD and NetBSD have
stable syscalls, but we don't yet have syscall layers for them in std.os.
...but not in the way you'd expect. We were actually tracking them in
cases where we shouldn't have been! We cannot track a declaration if its
parent namespace has been lost, because that will cause analysis
failures immediately, but if we excluded a type from the mapping due to
a major change (such as a struct turning into a union, or a field being
added), we were still including any trackable instructions inside the
container's field expressions (e.g. struct field type expressions). This
meant we were tracking a type declaration while losing tracking on its
parent namespace, with predictably disastrous results.
Oh, also, tracking for opaque types was just totally wrong (I think this
was a typo from a while back). We could map it to things other than
opaque declarations, and we never mapped declarations inside opaques.
So, uh, I fixed that too.
Having the matrix of test targets for incremental compilation in the
individual test manifests has turned out to be inconvenient for a few
reasons: the tests are almost certain to accidentally get out of sync,
disabling targets entirely is annoying to do, and incr-check needs to
take care to print the target in all error messages (which currently
does not always happen).
If I recall correctly, I originally designed it this way because it
allows targets to be disabled at the granularity of individual tests,
but there's an easier approach to that: just let a test manifest that it
should be *skipped* on a certain target! As skipping is the rare case,
and also the case you want readers to notice, it makes sense for *it* to
be explicitly specified, like how unit tests use `error.SkipZigTest`.
So, `incr-check` no longer runs through a list of targets specified in
the manifest. Instead, it accepts (and, in fact, requires) a single
target on the command line, and runs the test for that specific target.
If the file contains a `#skip_target` directive for that target, then
`incr-check` exits immediately, so we can still disable targets at
individual test granularity, but you can also disable a target for all
tests by just commenting it out of the matrix in `test/tests.zig`.
As a nice bonus, this also allows the build system to run different
incremental test targets in parallel, because the targets are now
different steps.
This definitely seems like a better way to split the work between the
build system and incr-check---sorry for getting this wrong initially!
Alex Rønne Petersen