Implements a thread-safe allocator with the following guarantees:
* `deinit` reports all leaks and frees all backing memory.
* All allocation mismatches result in either a panic or segmentation
fault.
* Allocations from other `SafeAllocator` instances cause a panic (if
`Options.canary` differ).
* Double frees and operation (resize, remap, and free) races panic or
segmentation fault.
Given the backing allocator does not reuse memory, it does not reuse
memory either and
* Most writes after free will segmentation fault or are eventually
detected and panic.
`std.heap.DebugAllocator` has been deprecated (I have also deprecated
`std.heap.Check` since this was its last usage and returning a `usize`
leak count is a much cleaner approach).
- General Design
Every allocation is trailed by an `AllocFooter` which contains metadata
for the allocation and stack traces. It is protected by a checksum to
catch corruption from allocation overwrites and report canary
mismatches. An allocation's memory has a minimum alignment of
`AllocFooter` so that the footer is at a fixed offset determined from
the allocation size. An allocation's memory is stored either:
* Inside linearly-filled buckets for small allocations.
* Inside an allocation directly from the backing allocator.
To track allocations, each thread maintains a table of backing
allocations. The table may be modified by other threads in the case of
a producer-consumer operation, so the table is a linked list only
expanded by creating new segments. Each thread maintains a linked list
of free entries, which may contain entries from other threads' tables.
In the case of producer-consumer operations, acquire/release ordering
is assumed to be provided externally. This is also assumed by all other
thread-safe allocators that reuse memory as otherwise there would be
data races on reuse of allocated memory.
- Fuzz Tests
Two fuzz tests have also been added for the allocator. They check that
there is no memory reuse, that returned memory is writable, and that
it is not overwritten. The multi-threaded fuzz test spawns a number of
worker threads which are used for all the test runs. I have run these
tests extensively under TSAN.
- Performance Measurements
Building the standard library tests with a RelaseSafe compiler build
and `-Ddebug-allocator`:
```
Benchmark 1 (3 runs): ./master-out/bin/zig test --zig-lib-dir lib lib/std/std.zig -femit-bin=test --test-no-exec
measurement mean ± σ min … max outliers delta
wall_time 29.4s ± 157ms 29.2s … 29.5s 0 ( 0%) 0%
peak_rss 2.24GB ± 3.49MB 2.23GB … 2.24GB 0 ( 0%) 0%
cpu_cycles 143G ± 999M 142G … 144G 0 ( 0%) 0%
instructions 268G ± 5.22M 268G … 268G 0 ( 0%) 0%
cache_references 13.1G ± 88.8M 13.0G … 13.2G 0 ( 0%) 0%
cache_misses 2.38G ± 30.7M 2.35G … 2.41G 0 ( 0%) 0%
branch_misses 634M ± 6.22M 629M … 641M 0 ( 0%) 0%
Benchmark 2 (3 runs): ./branch-out/bin/zig test --zig-lib-dir lib lib/std/std.zig -femit-bin=test --test-no-exec
measurement mean ± σ min … max outliers delta
wall_time 22.1s ± 88.6ms 22.0s … 22.2s 0 ( 0%) ⚡- 24.7% ± 1.0%
peak_rss 1.11GB ± 799KB 1.11GB … 1.11GB 0 ( 0%) ⚡- 50.3% ± 0.3%
cpu_cycles 136G ± 480M 136G … 137G 0 ( 0%) ⚡- 4.4% ± 1.2%
instructions 273G ± 2.07M 273G … 273G 0 ( 0%) 💩+ 1.6% ± 0.0%
cache_references 12.3G ± 71.3M 12.2G … 12.4G 0 ( 0%) ⚡- 6.0% ± 1.4%
cache_misses 2.02G ± 11.5M 2.01G … 2.03G 0 ( 0%) ⚡- 14.9% ± 2.2%
branch_misses 569M ± 2.65M 567M … 572M 0 ( 0%) ⚡- 10.2% ± 1.7%
```
Currently the only function that handles sentinel terminated slices
properly is free. All uses of mem.sliceAsBytes() in the allocator
interface lack proper handling of a possible sentinel.
This commit changes the Allocator interface to use @ptrCast() plus
the new mem.absorbSentinel() instead.
Reported-by: David Vanderson <david.vanderson@gmail.com>
References: https://github.com/ziglang/zig/pull/19984
References: https://github.com/ziglang/zig/pull/23020
Now that zig1.wasm is updated, apply the matching standard library
changes.
The main one is in `std.builtin.Type`, where `alignment` fields now have
type `?usize` rather than `comptime_int`.
Additionally, we need to add explicit backing integers to some packed
unions, because (due to https://github.com/ziglang/zig/issues/24714)
they need explicit backing integers to be used in `extern` contexts.
This change could not happen before now, because prior to this branch,
packed unions did not allow explicit backing integer types (that is,
this branch implemented https://github.com/ziglang/zig/issues/25350).
Reversal on the decision: the Allocator interface is the correct place
for the memset to undefined because it allows Allocator implementations
to bypass the interface and use a backing allocator directly, skipping
the performance penalty of memsetting the entire allocation, which may
be very large, as well as having valuable zeroes on them.
closes#4298
This one changes the size of an allocation, allowing it to be relocated.
However, the implementation will still return `null` if it would be
equivalent to
new = alloc
memcpy(new, old)
free(old)
Mainly this prepares for taking advantage of `mremap` which I thought
would be a bigger deal but apparently is only available on Linux. Still,
we should use it on Linux.
This keeps the implementation matching master branch, however,
introduces a compile error that applications can work around by
explicitly setting page_size_max and page_size_min to match their
computer's settings, in the case that those values are not already
equal.
I plan to rework this allocator in a follow-up enhancement with the goal
of reducing total active memory mappings.
* fix merge conflicts
* rename the declarations
* reword documentation
* extract FixedBufferAllocator to separate file
* take advantage of locals
* remove the assertion about max alignment in Allocator API, leaving it
Allocator implementation defined
* fix non-inline function call in start logic
The GeneralPurposeAllocator implementation is totally broken because it
uses global state but I didn't address that in this commit.
heap.zig: define new default page sizes
heap.zig: add min/max_page_size and their options
lib/std/c: add miscellaneous declarations
heap.zig: add pageSize() and its options
switch to new page sizes, especially in GPA/stdlib
mem.zig: remove page_size
This check doesn't make sense with the modern Allocator API; it's left over
from when realloc could change alignment. It's statically known (but not
comptime-known) to be true always. This check was one of the things
blocking Allocator from being used at comptime (related: #1291).
This was done by regex substitution with `sed`. I then manually went
over the entire diff and fixed any incorrect changes.
This diff also changes a lot of `callconv(.C)` to `callconv(.c)`, since
my regex happened to also trigger here. I opted to leave these changes
in, since they *are* a correct migration, even if they're not the one I
was trying to do!
It wasn't immediately clear from the implementation whether passing
zero-length memory to free() was undefined behavior or intentionally
supported. Since ArrayList and other core data structures rely on
this behavior working correctly, this should be explicitly documented
as part of the public API contract.
The compiler actually doesn't need any functional changes for this: Sema
does reification based on the tag indices of `std.builtin.Type` already!
So, no zig1.wasm update is necessary.
This change is necessary to disallow name clashes between fields and
decls on a type, which is a prerequisite of #9938.
* Move functionality from generic functions that doesn't depend on the type into a function that only depends on comptime alignment.
This reduces comptime code duplication because e.g. `alloc(u32, )` and `alloc(i32, )` now use the same function `allocWithFoo(4, 4, )` under the hood.
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:
* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
Anecdote 1: The generic version is way more popular than the non-generic
one in Zig codebase:
git grep -w alignForward | wc -l
56
git grep -w alignForwardGeneric | wc -l
149
git grep -w alignBackward | wc -l
6
git grep -w alignBackwardGeneric | wc -l
15
Anecdote 2: In my project (turbonss) that does much arithmetic and
alignment I exclusively use the Generic functions.
Anecdote 3: we used only the Generic versions in the Macho Man's linker
workshop.
* Sema: upgrade operands to array pointers if possible when emitting
AIR.
* Implement safety checks for length mismatch and aliasing.
* AIR: make ptrtoint support slice operands. Implement in LLVM backend.
* C backend: implement new `@memset` semantics. `@memcpy` is not done
yet.
Now they use slices or array pointers with any element type instead of
requiring byte pointers.
This is a breaking enhancement to the language.
The safety check for overlapping pointers will be implemented in a
future commit.
closes#14040
This reverts commit abc9530a88.
This patch implies that the idiomatic Zig way of handling anytype
parameter is to write a bunch of boilerplate instead of directly
accessing type information and relying on the compiler to be useful.
I don't want it to be this way.
It is the compiler's job to make useful error messages when the wrong
field of a type info result is accessed, and it is the zig programmer's
job to understand what it means when a compile error points at the field
access of `@typeInfo` (along with the relevant callsites).
One thing that might be useful would be having the compiler be aware of
module boundaries and highlighting the boundaries of them. The first
reference note after crossing a module boundary is likely the most
interesting one.
There are still a few occurrences of "stage1" in the standard library
and self-hosted compiler source, however, these instances need a bit
more careful inspection to ensure no breakage.
Fixes#11353
The renderer treats comments and doc comments differently since doc
comments are parsed into the Ast. This commit adds a check after getting
the text for the doc comment and trims whitespace at the end before
rendering.
The `a = 0,` in the test is here to avoid a ParseError while parsing the
test.
Kendall Condon