options which are passed to configurer and therefore observable by the
build script are added to the cache hash. A sorted list is hashed since
they are unordered.
I had this idea to make b.dupe() also intern the strings since they will
be ultimately serialized to Configuration. Unfortunately the idea does
not work, because although a process-lived arena is used for the
string_bytes ArrayList of the Configuration.Wip, when the ArrayList is
resized, Allocator.free() memsets the freed memory to undefined, even
though it still technically lives due to being in a process-scoped
arena. So this commit will need to be partially reverted. However, I
kept it for posterity, and there are some more changes which I will now
note below.
- dupePaths: don't rewrite backslashes to forward slashes. backslashes
are valid in filenames on non-windows systems.
- always compile configurer in single-threaded mode
- use arena allocator for everything, no gpa for anything
- construct the Configuration.Wip instance earlier, so some stuff can be
prepopulated as desired.
- don't forget to flush
Number of generated files is recorded in serialized Configuration. Maker
preallocates array of generated files so that loads and stores can be
synchronization-free (protected by the dependency tree ordering).
More progress on Compile Step Zig CLI lowering.
next thing to do is figure out how LazyPath is supposed to work now.
something like this:
* each Step that provides LazyPath objects has a setLazyPath and
getLazyPath function which takes a tagged union identifying which one
to access
* steps that fulfill LazyPath objects can freely call setLazyPath
without obtaining a lock because the dependency graph prevents
simultaneous access.
* similarly, steps that access LazyPath results can freely call
getLazyPath without obtaining a lock, because after modification,
there may be simultaneous reads from dependencies but they will all be
read-only
* a fulfilled LazyPath object is a read-only std.Build.Cache.Path.
`zig build` CLI kicks off async task to compile optimized make runner
executable, does fetch, compiles configure process in debug mode, then
checks cache for the CLI options that affect configuration only. On hit,
skips building/running the configure script. On miss, runs it, saves
result in cache.
The cached artifact is a "configuration" file - a serialized build step
graph, which also includes unlazy package dependencies and additional
file system dependencies.
Next, awaits task for compiling optimized make runner executable, passes
configuration file to it. Make runner is responsible for the CLI after
that point.
For the use case of detecting when `git describe` needs to be rerun, we
can allow the configure process to manually add a file system mtime
dependencies, in this case it would be on `.git/index` and `.git/HEAD`.
This will enable two optimizations:
1. The bulk of the build system will not be rebuilt when user changes
their configure script.
2. The user logic can be completely bypassed when the CLI options
provided do not affect the configure phase - even if they affect the
make phase.
Remaining tasks in the branch:
* some stuff in `zig build` CLI is `@panic("TODO")`.
* configure runner needs to implement serialization of build graph using
std.zig.Configuration
* build runner needs to be transformed into make runner, consuming
configuration file as input and deserializing the step graph.
* introduce depending only on a file's metadata and *not* its contents
into the cache system, and add a std.Build API for using it.
Previously, when using `zig build -fincremental --watch some-run-step`,
if the binary initially builds fine but a future update introduces a
compile error, the old file system inputs of the `Step.Run`---containing
the executable file itself---would remain. This is a bug, because that
file *has* changed since the input was registered (due to incremental
compilation), and yet the `Run` step is not out-of-date because it was
skipped due to a transitive failure.
A simple reproduction for this issue was:
$ zig init
$ zig build test --watch -fincremental
Then, in another terminal, introduce a compile error:
$ sed -i 's/const/onst/' src/main.zig
Before this commit, this would cause the `zig build` command to repeat
updates forever, separated only by the 50ms debounce interval.
To fix this, when we reset a step with intent to re-run it, we should
clear its inputs immediately, so that if the step is skipped, it is
correctly marked as having no inputs.
This relates to a more general problem with the file system watching
logic which is that the set of inputs includes files in the cache which
are actually artifacts of other steps. Eventually, the build system
should learn to identify such files and exclude them from the set of
file system inputs. In other words, the fact that a `Run` step depends
on the executable generated by a `Compile` step should be modeled by a
dependency in the build step graph, *not* by a dependency on the path
where the `Compile` step happens to have emitted its executable file.
Andrew Kelley