Commit Graph

718 Commits

Author SHA1 Message Date
Andrew Kelley 2adfd4d107 std.Io: fix and improve Group API
Rename `wait` to `await` to be consistent with Future API. The
convention here is that this set of functionality goes together:
* async/concurrent
* await/cancel

Also rename Select `wait` to `await` for the same reason.

`Group.await` now can return `error.Canceled`. Furthermore,
`Group.await` does not auto-propagate cancelation. Instead, users should
follow the pattern of `defer group.cancel(io);` after initialization,
and doing `try group.await(io);` at the end of the success path.
Advanced logic can choose to do something other than this pattern in the
event of cancelation.

Additionally, fixes a bug in `std.Io.Threaded` future await, in which it
swallowed an `error.Canceled`. Now if a task is canceled while awaiting
a future, after propagating the cancel request, it also recancels,
meaning that the awaiting task will properly detect its own cancelation
at the next cancelation point.

Furthermore, fixes a bug in the compiler where `error.Canceled` was
being swallowed in `dispatchPrelinkWork`.

Finally, fixes std.crypto code that inappropriately used
`catch unreachable` in response to cancelation without even so much as a
comment explaining why it was believed to be unreachable. Now, those
functions have `error.Canceled` in the error set and propagate
cancelation properly.

With this way of doing things, `Group.await` has a nice property: even if
all tasks in the group are CPU bound and without cancelation points, the
`Group.await` can still be canceled. In such case, the task that was
waiting for `await` wakes up with a chance to do some more resource
cleanup tasks, such as canceling more things, before entering the
deferred `Group.cancel` call at which point it has to suspend until the
canceled but uninterruptible CPU bound tasks complete.

closes #30601
2025-12-29 22:47:34 -08:00
Ryan Liptak 51a6f3a525 Update a few more callsites for std.Io changes 2025-12-23 22:15:11 -08:00
Andrew Kelley a8088306f6 std: rename other Dir "make" functions to "create" 2025-12-23 22:15:11 -08:00
Andrew Kelley 7ce5ee2e92 std: update remaining unit tests for std.Io API changes 2025-12-23 22:15:10 -08:00
Andrew Kelley 21d0264c61 std.dynamic_library: use a global static single threaded Io
See #30150
2025-12-23 22:15:10 -08:00
Andrew Kelley 608145c2f0 fix more fallout from locking stderr 2025-12-23 22:15:10 -08:00
Andrew Kelley 22afd168d9 fix build runner compilation errors on macOS 2025-12-23 22:15:09 -08:00
Andrew Kelley 16bd2e137e compiler: fix most compilation errors from std.fs changes 2025-12-23 22:15:09 -08:00
Andrew Kelley 1925e0319f update lockStderrWriter sites
use the application's Io implementation where possible. This correctly
makes writing to stderr cancelable, fallible, and participate in the
application's event loop. It also removes one more hard-coded
dependency on a secondary Io implementation.
2025-12-23 22:15:09 -08:00
Andrew Kelley 16f8af1b9a compiler: update various code to new fs API 2025-12-23 22:15:09 -08:00
Andrew Kelley 68621afd2e std.tar: update fs API calls to take io argument 2025-12-23 22:15:09 -08:00
Andrew Kelley c4cefd6835 std.Build.Step.Run: restore Color.inherit logic 2025-12-23 22:15:09 -08:00
Andrew Kelley b042e93522 std: update tty config references in the build system 2025-12-23 22:15:09 -08:00
Andrew Kelley bee8005fe6 std.heap.DebugAllocator: never detect TTY config
instead, allow the user to set it as a field.

this fixes a bug where leak printing and error printing would run tty
config detection for stderr, and then emit a log, which is not necessary
going to print to stderr.

however, the nice defaults are gone; the user must explicitly assign the
tty_config field during initialization or else the logging will not have
color.

related: https://github.com/ziglang/zig/issues/24510
2025-12-23 22:15:08 -08:00
Andrew Kelley 4a53e5b0b4 fix a handful of compilation errors related to std.fs migration 2025-12-23 22:15:08 -08:00
Andrew Kelley ebdbbd20ac update makeDir() sites to specify permissions 2025-12-23 22:15:08 -08:00
Andrew Kelley 3725f72293 update std.process.Child.run occurences to use io 2025-12-23 22:15:08 -08:00
Andrew Kelley 1dcfc8787e update all readFileAlloc() to accept Io instance 2025-12-23 22:15:08 -08:00
Andrew Kelley 4be8be1d2b update all rename() to rename(io) 2025-12-23 22:15:08 -08:00
Andrew Kelley 4218344dd3 std.Build.Cache: remove readSmallFile and writeSmallFile
These were to support optimizations involving detecting when to avoid
calling into LLD, which are no longer implemented.
2025-12-23 22:15:08 -08:00
Andrew Kelley 950d18ef69 update all access() to access(io) 2025-12-23 22:15:08 -08:00
Andrew Kelley 314c906dba std.debug: simplify printLineFromFile 2025-12-23 22:15:08 -08:00
Andrew Kelley 9ccd68de0b std: move abort and exit from posix into process
and delete the unit tests that called fork()

no forking allowed in the std lib, including unit tests, except to implement child process spawning.
2025-12-23 22:15:08 -08:00
Andrew Kelley 7f5bb118d4 std.Io: make all the close functions batched 2025-12-23 22:15:08 -08:00
Andrew Kelley 6f46570958 link.MachO: update parallel hasher to std.Io 2025-12-23 22:15:08 -08:00
Andrew Kelley 9f4d40b1f9 update all stat() to stat(io) 2025-12-23 22:15:08 -08:00
Andrew Kelley 264d714321 update all openDir() sites to accept io instance 2025-12-23 22:15:08 -08:00
Andrew Kelley f53248a409 update all std.fs.cwd() to std.Io.Dir.cwd() 2025-12-23 22:15:08 -08:00
Andrew Kelley 8328de24f1 update all occurrences of openFile to receive an io instance 2025-12-23 22:15:08 -08:00
Andrew Kelley dd1d15b72a update all occurrences of std.fs.Dir to std.Io.Dir 2025-12-23 22:15:08 -08:00
Andrew Kelley 3204fb7569 update all occurrences of std.fs.File to std.Io.File 2025-12-23 22:15:07 -08:00
Andrew Kelley aafddc2ea1 update all occurrences of close() to close(io) 2025-12-23 22:15:07 -08:00
Andrew Kelley d1d2c37af2 std: all Dir functions moved to std.Io 2025-12-23 22:15:07 -08:00
Alex Rønne Petersen aa0249d74e Merge pull request 'std.ascii: rename indexOf functions to find' (#30101) from adria/zig:indexof-find into master
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/30101
Reviewed-by: Andrew Kelley <andrewrk@noreply.codeberg.org>
Reviewed-by: mlugg <mlugg@noreply.codeberg.org>
2025-12-22 12:50:46 +01:00
Matthew Lugg b4ee54b580 std.Io: reimplement Mutex and Condition atop futex
This work was partially cherry-picked from Andrew's WIP std.fs branch.
However, I also analyzed and simplified the Mutex and Condition
implementations, and brought them in line with modern Zig style.

Co-authored-by: Andrew Kelley <andrew@ziglang.org>
2025-12-21 13:07:03 +00:00
Jacob Young c13857e504 windows: type safety improvements and more ntdll functions 2025-12-12 01:58:21 -05:00
mlugg dbb4c8d151 Merge pull request 'Remove things deprecated during the 0.15 release cycle' (#30018) from linus/zig:remove-deprecated-stuff into master
Reviewed-on: https://codeberg.org/ziglang/zig/pulls/30018
2025-12-06 08:51:15 +01:00
Adrià Arrufat 02c5f05e2f std: replace usages of std.mem.indexOf with std.mem.find 2025-12-05 14:31:27 +01:00
Alex Rønne Petersen d22231c039 Compilation: track indirect file system inputs from clang's depfile
Co-authored-by: Matthew Lugg <mlugg@mlugg.co.uk>
2025-12-05 11:22:10 +01:00
Linus Groh 39fa831947 std: Remove a handful of things deprecated during the 0.15 release cycle
- std.Build.Step.Compile.root_module mutators -> std.Build.Module
- std.Build.Step.Compile.want_lto -> std.Build.Step.Compile.lto
- std.Build.Step.ConfigHeader.getOutput -> std.Build.Step.ConfigHeader.getOutputFile
- std.Build.Step.Run.max_stdio_size -> std.Build.Step.Run.stdio_limit
- std.enums.nameCast -> @field(E, tag_name) / @field(E, @tagName(tag))
- std.Io.tty.detectConfig -> std.Io.tty.Config.detect
- std.mem.trimLeft -> std.mem.trimStart
- std.mem.trimRight -> std.mem.trimEnd
- std.meta.intToEnum -> std.enums.fromInt
- std.meta.TagPayload -> @FieldType(U, @tagName(tag))
- std.meta.TagPayloadByName -> @FieldType(U, tag_name)
2025-11-27 20:17:04 +00:00
IOKG04 e23af9d31d Build.Step.Options: add comptime_float support
It seems to me this was simply forgotten.
Or there is some reason I don't know why this code doesn't work for `comptime_float`.

For a more comprehensive fix, https://github.com/ziglang/zig/pull/24057 is the place to look.
2025-11-26 09:57:49 +01:00
Andrew Kelley 5816646aa0 Merge pull request #26037 from ziglang/build-runner-std.Io
build runner: update from std.Thread.Pool to std.Io
2025-11-25 02:49:37 -08:00
Ryan Liptak 53e615b920 Merge pull request #25993 from squeek502/windows-paths
Teach `std.fs.path` about the wonderful world of Windows paths
2025-11-24 15:27:24 -08:00
Andrew Kelley 3f34f5e433 build runner: update Mutex and Condition usage to std.Io 2025-11-24 14:34:18 -08:00
Andrew Kelley a242292644 build runner: update from std.Thread.Pool to std.Io 2025-11-24 14:34:18 -08:00
Ryan Liptak 59b8bed222 Teach fs.path about the wonderful world of Windows paths
Previously, fs.path handled a few of the Windows path types, but not all of them, and only a few of them correctly/consistently. This commit aims to make `std.fs.path` correct and consistent in handling all possible Win32 path types.

This commit also slightly nudges the codebase towards a separation of Win32 paths and NT paths, as NT paths are not actually distinguishable from Win32 paths from looking at their contents alone (i.e. `\Device\Foo` could be an NT path or a Win32 rooted path, no way to tell without external context). This commit formalizes `std.fs.path` being fully concerned with Win32 paths, and having no special detection/handling of NT paths.

Resources on Windows path types, and Win32 vs NT paths:

- https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html
- https://chrisdenton.github.io/omnipath/Overview.html
- https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file

API additions/changes/deprecations

- `std.os.windows.getWin32PathType` was added (it is analogous to `RtlDetermineDosPathNameType_U`), while `std.os.windows.getNamespacePrefix` and `std.os.windows.getUnprefixedPathType` were deleted. `getWin32PathType` forms the basis on which the updated `std.fs.path` functions operate.
- `std.fs.path.parsePath`, `std.fs.path.parsePathPosix`, and `std.fs.path.parsePathWindows` were added, while `std.fs.path.windowsParsePath` was deprecated. The new `parsePath` functions provide the "root" and the "kind" of a path, which is platform-specific. The now-deprecated `windowsParsePath` did not handle all possible path types, while the new `parsePathWindows` does.
- `std.fs.path.diskDesignator` has been deprecated in favor of `std.fs.path.parsePath`, and same deal with `diskDesignatorWindows` -> `parsePathWindows`
- `relativeWindows` is now a compile error when *not* targeting Windows, while `relativePosix` is now a compile error when targeting Windows. This is because those functions read/use the CWD path which will behave improperly when used from a system with different path semantics (e.g. calling `relativePosix` from a Windows system with a CWD like `C:\foo\bar` will give you a bogus result since that'd be treated as a single relative component when using POSIX semantics). This also allows `relativeWindows` to use Windows-specific APIs for getting the CWD and environment variables to cut down on allocations.
- `componentIterator`/`ComponentIterator.init` have been made infallible. These functions used to be able to error on UNC paths with an empty server component, and on paths that were assumed to be NT paths, but now:
  + We follow the lead of `RtlDetermineDosPathNameType_U`/`RtlGetFullPathName_U` in how it treats a UNC path with an empty server name (e.g. `\\\share`) and allow it, even if it'll be invalid at the time of usage
  + Now that `std.fs.path` assumes paths are Win32 paths and not NT paths, we don't have to worry about NT paths

Behavior changes

- `std.fs.path` generally: any combinations of mixed path separators for UNC paths are universally supported, e.g. `\/server/share`, `/\server\share`, `/\server/\\//share` are all seen as equivalent UNC paths
- `resolveWindows` handles all path types more appropriately/consistently.
  + `//` and `//foo` used to be treated as a relative path, but are now seen as UNC paths
  + If a rooted/drive-relative path cannot be resolved against anything more definite, the result will remain a rooted/drive-relative path.
  + I've created [a script to generate the results of a huge number of permutations of different path types](https://gist.github.com/squeek502/9eba7f19cad0d0d970ccafbc30f463bf) (the result of running the script is also included for anyone that'd like to vet the behavior).
- `dirnameWindows` now treats the drive-relative root as the dirname of a drive-relative path with a component, e.g. `dirname("C:foo")` is now `C:`, whereas before it would return null. `dirnameWindows` also handles local device paths appropriately now.
- `basenameWindows` now handles all path types more appropriately. The most notable change here is `//a` being treated as a partial UNC path now and therefore `basename` will return `""` for it, whereas before it would return `"a"`
- `relativeWindows` will now do its best to resolve against the most appropriate CWD for each path, e.g. relative for `D:foo` will look at the CWD to check if the drive letter matches, and if not, look at the special environment variable `=D:` to get the shell-defined CWD for that drive, and if that doesn't exist, then it'll resolve against `D:\`.

Implementation details

- `resolveWindows` previously looped through the paths twice to build up the relevant info before doing the actual resolution. Now, `resolveWindows` iterates backwards once and keeps track of which paths are actually relevant using a bit set, which also allows it to break from the loop when it's no longer possible for earlier paths to matter.
- A standalone test was added to test parts of `relativeWindows` since the CWD resolution logic depends on CWD information from the PEB and environment variables

Edge cases worth noting

- A strange piece of trivia that I found out while working on this is that it's technically possible to have a drive letter that it outside the intended A-Z range, or even outside the ASCII range entirely. Since we deal with both WTF-8 and WTF-16 paths, `path[0]`/`path[1]`/`path[2]` will not always refer to the same bits of information, so to get consistent behavior, some decision about how to deal with this edge case had to be made. I've made the choice to conform with how `RtlDetermineDosPathNameType_U` works, i.e. treat the first WTF-16 code unit as the drive letter. This means that when working with WTF-8, checking for drive-relative/drive-absolute paths is a bit more complicated. For more details, see the lengthy comment in `std.os.windows.getWin32PathType`
- `relativeWindows` will now almost always be able to return either a fully-qualified absolute path or a relative path, but there's one scenario where it may return a rooted path: when the CWD gotten from the PEB is not a drive-absolute or UNC path (if that's actually feasible/possible?). An alternative approach to this scenario might be to resolve against the `HOMEDRIVE` env var if available, and/or default to `C:\` as a last resort in order to guarantee the result of `relative` is never a rooted path.
- Partial UNC paths (e.g. `\\server` instead of `\\server\share`) are a bit awkward to handle, generally. Not entirely sure how best to handle them, so there may need to be another pass in the future to iron out any issues that arise. As of now the behavior is:
  + For `relative`, any part of a UNC disk designator is treated as the "root" and therefore isn't applicable for relative paths, e.g. calling `relative` with `\\server` and `\\server\share` will result in `\\server\share` rather than just `share` and if `relative` is called with `\\server\foo` and `\\server\bar` the result will be `\\server\bar` rather than `..\bar`
  + For `resolve`, any part of a UNC disk designator is also treated as the "root", but relative and rooted paths are still elligable for filling in missing portions of the disk designator, e.g. `resolve` with `\\server` and `foo` or `\foo` will result in `\\server\foo`

Fixes #25703
Closes #25702
2025-11-21 00:03:44 -08:00
Benjamin Jurk 4b5351bc0d update deprecated ArrayListUnmanaged usage (#25958) 2025-11-20 14:46:23 -08:00
Matthew Lugg bc524a2b1a std.Build: fix crashes running fuzz tests 2025-11-20 10:42:21 +00:00
Matthew Lugg 010dcd6a9b fuzzer: account for runtime address slide
This is relevant to PIEs, which are notably enabled by default on macOS.
The build system needs to only see virtual addresses, that is, those
which do not have the slide applied; but the fuzzer itself naturally
sees relocated addresses (i.e. with the slide applied). We just need to
subtract the slide when we communicate addresses to the build system.
2025-11-20 10:42:20 +00:00
Matthew Lugg 0a330d4f94 std.debug.Info: basic Mach-O support 2025-11-20 10:42:20 +00:00