`zig build` CLI kicks off async task to compile optimized make runner
executable, does fetch, compiles configure process in debug mode, then
checks cache for the CLI options that affect configuration only. On hit,
skips building/running the configure script. On miss, runs it, saves
result in cache.
The cached artifact is a "configuration" file - a serialized build step
graph, which also includes unlazy package dependencies and additional
file system dependencies.
Next, awaits task for compiling optimized make runner executable, passes
configuration file to it. Make runner is responsible for the CLI after
that point.
For the use case of detecting when `git describe` needs to be rerun, we
can allow the configure process to manually add a file system mtime
dependencies, in this case it would be on `.git/index` and `.git/HEAD`.
This will enable two optimizations:
1. The bulk of the build system will not be rebuilt when user changes
their configure script.
2. The user logic can be completely bypassed when the CLI options
provided do not affect the configure phase - even if they affect the
make phase.
Remaining tasks in the branch:
* some stuff in `zig build` CLI is `@panic("TODO")`.
* configure runner needs to implement serialization of build graph using
std.zig.Configuration
* build runner needs to be transformed into make runner, consuming
configuration file as input and deserializing the step graph.
* introduce depending only on a file's metadata and *not* its contents
into the cache system, and add a std.Build API for using it.
This PR merges the functionality of the `getLastOrNull` method into `getLast`, which improves consistency as its
based on methods like `front`, `back`, and `peek` in the `Deque` and `PriorityQueue` containers.
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/32008
Reviewed-by: Andrew Kelley <andrew@ziglang.org>
Followup to #30769
I grepped for `try .*toOwnedSlice` and checked all of them by hand.
Fixes a bunch of memory leaks removes usages or `errdefer` and `vars` in some places. I also switched array_list.Managed to ArrayList where it was convenient.
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/32001
Reviewed-by: Andrew Kelley <andrew@ziglang.org>
- New Features
-- Multiprocess Fuzzing
The fuzzer now is able to utilize multiple cores. This is controllable
with the `-j` build option. Limited fuzzing still uses one core.
-- Fuzzing Infinite Mode
When provided multiple tests, the fuzzer now switches between them and
prioritizes the most effective and interesting ones. Over time already
explored tests will become barely run compared to tests yielding new
inputs.
-- Crash Dumps
Crashing inputs are now saved to a file indicated by the crash message.
It is recommended to use these files to reproduce the crash using
`std.testing.FuzzInputOptions.corpus` and @embedFile.
- Design
Each fuzzing process is assigned an instance id which has the following
uses:
* In conjunction with the pc hash and running test index, they uniquely
identify input files in the case of a crash.
* It is combined with the test seed for a unique rng seed.
* Instance 0 is solely responsible for syncing the filesystem corpus.
When new inputs are found, they are sent to the build server. It then
distributes the new input to the other instances. Each instance has a
concurrent poller managed by the test runner which sends received
inputs to libfuzzer. (note that this is affected by #31718 and so can
(rarely) deadlock)
For fuzzing infinite mode, the test runner now receives a list of tests
from the build server. The fuzzer runs tests in batches of one second,
approximated in cycles by the previous batch's run speed. Tests finding
new inputs or with few runs are given a higher run chance. The baseline
run chance is based off the recency of the last find and the number of
pcs the test has hit.
Users can proceed by telling the build system how much memory to assume
the system has. I have improved the failure message to communicate this.
Partially reverts 87f8f47ba5
Even if we wanted to unrevert this reverted commit, it's not sufficient
to merely downgrade the failure to a warning, because the main
scheduling logic will fail to schedule steps that have a max_rss
exceeding the detected system value, causing some steps to never get
executed, eventually tripping an assert. Such a change would need to
also override the max_rss value to be equal to the greatest value across
all steps.
closes#31510
The indexes can change between recompilation due to conditional
compilation and compiler quirks. While unit test names are still not a
perfect solution, they are better than indexes.
Now that zig1.wasm is updated, apply the matching standard library
changes.
The main one is in `std.builtin.Type`, where `alignment` fields now have
type `?usize` rather than `comptime_int`.
Additionally, we need to add explicit backing integers to some packed
unions, because (due to https://github.com/ziglang/zig/issues/24714)
they need explicit backing integers to be used in `extern` contexts.
This change could not happen before now, because prior to this branch,
packed unions did not allow explicit backing integer types (that is,
this branch implemented https://github.com/ziglang/zig/issues/25350).
On Windows, it is sometimes problematic to depend on ws2_32.dll. Before,
users of std.Io.Threaded would have to call ioBasic() rather than io()
in order to avoid unnecessary dependencies on ws2_32.dll. Now, the
application can disable networking with std.Options.
This change is necessary due to moving networking functionality to
be based on Io.Operation, which is a tagged union.
The tar format expects `/`, although some untar implementations do seem to handle Windows-style `\` path separators (7-Zip at least).
The tar.Writer API can't really enforce this, though, as doing so would effectively make `\` an illegal character when it's really not. So, it's up to the user to provide paths with the correct path separators.
`Build/WebServer.zig` will still output tars with `\` as a path separator on Windows, but that's currently only used during fuzzing which is not yet implemented on Windows.
Andrew Kelley