- New Features
-- Multiprocess Fuzzing
The fuzzer now is able to utilize multiple cores. This is controllable
with the `-j` build option. Limited fuzzing still uses one core.
-- Fuzzing Infinite Mode
When provided multiple tests, the fuzzer now switches between them and
prioritizes the most effective and interesting ones. Over time already
explored tests will become barely run compared to tests yielding new
inputs.
-- Crash Dumps
Crashing inputs are now saved to a file indicated by the crash message.
It is recommended to use these files to reproduce the crash using
`std.testing.FuzzInputOptions.corpus` and @embedFile.
- Design
Each fuzzing process is assigned an instance id which has the following
uses:
* In conjunction with the pc hash and running test index, they uniquely
identify input files in the case of a crash.
* It is combined with the test seed for a unique rng seed.
* Instance 0 is solely responsible for syncing the filesystem corpus.
When new inputs are found, they are sent to the build server. It then
distributes the new input to the other instances. Each instance has a
concurrent poller managed by the test runner which sends received
inputs to libfuzzer. (note that this is affected by #31718 and so can
(rarely) deadlock)
For fuzzing infinite mode, the test runner now receives a list of tests
from the build server. The fuzzer runs tests in batches of one second,
approximated in cycles by the previous batch's run speed. Tests finding
new inputs or with few runs are given a higher run chance. The baseline
run chance is based off the recency of the last find and the number of
pcs the test has hit.
The cmpxchg is there to recover alignment padding that isn't needed (which
can only be determined after the fetch-and-add that reserves it as allocated
memory). As cmpxchg tends to be a very expensive operation, it is actually
faster to introduce an additional branch here that checks if the cmpxchg
would be a noop (because all of the reserved alignment padding was in fact
necessary) and skips it if that's the case.
This does not measurably regress performance if the arena is only accessed
by a single thread and yields slight performance benefits for multi-threaded
usage. If the arena is commonly used for unaligned allocations, the perf
benefits are quite significant.
Co-authored-by: Jacob Young <amazingjacob@gmail.com>
The implementations of `std.ArrayList.replaceRange` and `std.ArrayList.replaceRangeAssumeCapacity` were needlessly complicated, called `@memcpy` too many times, and left the structure in a semi-modified state in OOM conditions. This commit fixes that.
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/31710
Reviewed-by: Andrew Kelley <andrew@ziglang.org>
Co-authored-by: Hila Friedman <red.black.liquorice@gmail.com>
Co-committed-by: Hila Friedman <red.black.liquorice@gmail.com>
I hit an unexpected errno 30 on macos attempting to call createFile on a
directory in PATH. errno 30 is EROFS, this change propagates that error
as error.ReadOnlyFileSystem.
This isn't causing any functional problem today, but technically
`mapped[tls_offset..]` runs past the tls part of `mapped` and into
the `Instance` storage, and currently `prepareArea()` memsets its
entire argument to zero. It is only the current layout and
initialization order of `mapped` that prevents this from being a
problem. Being more precise here avoids future footguns if any of
that changes.
The main goal here is to make incremental compilation work a bit better.
I also slightly expanded some `std.zig.llvm.Builder` APIs so that we
don't need to pointlessly create new `Global`s whenever e.g. a function
turns into a variable or vice versa.
Also, lean into aliases for exports! If we just use aliases for every
export, everything becomes simpler. Besides, we can't just go around
renaming the globals of `Nav`s: the export could disappear on a future
update, in which case we'd have to somehow revert that change, which is
easier said than done.
RFC 9110 says
> It is RECOMMENDED that all senders and recipients support, at a minimum,
> URIs with lengths of 8000 octets in protocol elements.
closes#30207
Previously, each message requires an unseekable error to be returned
from a syscall before proceeding. Ideally, the code would just pass
around `*std.Io.Writer` instead of `std.Io.File` in the first place, but
even then, you could argue for saving a syscall with `writerStreaming`.
Jacob Young