Add implementations of last in terms of next_back on a bunch of DoubleEndedIterators
Provided a `DoubleEndedIterator` has finite length, `Iterator::last` is equivalent to `DoubleEndedIterator::next_back`. But searching forwards through the iterator when it's unnecessary is obviously not good for performance. I ran into this on one of the collection iterators.
I tried adding appropriate overloads for a bunch of the iterator adapters like filter, map, etc, but I ran into a lot of type inference failures after doing so.
The other interesting case is what to do with `Repeat`. Do we consider it part of the contract that `Iterator::last` will loop forever on it? The docs do say that the iterator will be evaluated until it returns None. This is also relevant for the adapters, it's trivially easy to observe whether a `Map` adapter invoked its closure a zillion times or just once for the last element.
This distinction is fairly abstract, but in practice, the main advantage
here is that LLVM's triple code considers WASI to be an OS, so this
makes rustc agree with that.
Fix potential integer overflow in SGX memory range calculation.
Thanks to Eduard Marin and David Oswald at the University of Burmingham, and Jo Van Bulck at KU Leuven for discovering this issue.
Make `std::fs::copy` attempt to create copy-on-write clones of files on MacOS
The behaviour of MacOS now matches Linux which uses `copy_file_range` to perform CoW file copies where available and supported by the underlying filesystem.
use SecRandomCopyBytes on macOS in Miri
This is a hack to fix https://github.com/rust-lang/miri/issues/686: on macOS, rustc will open `/dev/urandom` to initialize a `HashMap`. That's quite hard to emulate properly in Miri without a full-blown implementation of file descriptors. However, Miri needs an implementation of `SecRandomCopyBytes` anyway to support [getrandom](https://crates.io/crates/getrandom), so using it here should work just as well.
This will only have an effect when libstd is compiled specifically for Miri, but that will generally be the case when people use `cargo miri`.
This is clearly a hack, so I am opening this to start a discussion about whether we are okay with such a hack or not.
Cc @oli-obk
This renames `std::io::IoVec` to `std::io::IoSlice` and
`std::io::IoVecMut` to `std::io::IoSliceMut`, and stabilizes
`std::io::IoSlice`, `std::io::IoSliceMut`,
`std::io::Read::read_vectored`, and `std::io::Write::write_vectored`.
Closes#58452
std: Add `{read,write}_vectored` for more types
This commit implements the `{read,write}_vectored` methods on more types
in the standard library, namely:
* `std::fs::File`
* `std::process::ChildStd{in,out,err}`
* `std::io::Std{in,out,err}`
* `std::io::Std{in,out,err}Lock`
* `std::io::Std{in,out,err}Raw`
Where supported the OS implementations hook up to native support,
otherwise it falls back to the already-defaulted implementation.
This commit implements the `{read,write}_vectored` methods on more types
in the standard library, namely:
* `std::fs::File`
* `std::process::ChildStd{in,out,err}`
* `std::io::Std{in,out,err}`
* `std::io::Std{in,out,err}Lock`
* `std::io::Std{in,out,err}Raw`
Where supported the OS implementations hook up to native support,
otherwise it falls back to the already-defaulted implementation.
wasi: Use shared API for preopened fds
This commit updates the wasi target with supported added in
CraneStation/wasi-sysroot#10. That function allows both C and Rust to
cooperate in how preopened files are managed, enabling us to learn about
propened files through the same interface. The `open_parent` function in
the wasi `fs` module was updated to avoid its own initialization of a
global preopened map and instead delegate to libc to perform this
functionality.
This should both be more robust into the future in terms of handling
path logic as well as ensuring the propened map is correctly set up at
process boot time. This does currently require some unfortunate
allocations on our side, but if that becomes an issue we can always
paper over those in time!
This commit updates the wasi target with supported added in
CraneStation/wasi-sysroot#10. That function allows both C and Rust to
cooperate in how preopened files are managed, enabling us to learn about
propened files through the same interface. The `open_parent` function in
the wasi `fs` module was updated to avoid its own initialization of a
global preopened map and instead delegate to libc to perform this
functionality.
This should both be more robust into the future in terms of handling
path logic as well as ensuring the propened map is correctly set up at
process boot time. This does currently require some unfortunate
allocations on our side, but if that becomes an issue we can always
paper over those in time!
std: Avoid usage of `Once` in `Instant`
This commit removes usage of `Once` from the internal implementation of
time utilities on OSX and Windows. It turns out that we accidentally hit
a deadlock today (#59020) via events that look like:
* A thread invokes `park_timeout`
* Internally, only on OSX, `park_timeout` calls `Instant::elapsed`
* Inside of `Instant::elapsed` on OSX we enter a `Once` to initialize
global timer data
* Inside of `Once`, it attempts to `park`
This means on the same stack frame, when there's contention, we're
calling `park` from inside `park_timeout`, causing a deadlock!
The solution implemented in this commit was to remove usage of `Once`
and instead just do a small dance with atomics. There's no real need we
need to guarantee that the global information is only learned once, only
that it's only *stored* once. This implementation may have multiple
threads invoke `mach_timebase_info`, but only one will store the global
information which will amortize the cost for all other threads.
A similar fix has been applied to windows to be uniform across our
implementations, but looking at the code on Windows no deadlock was
possible. This is purely just a consistency update for Windows and in
theory a slightly leaner implementation.
Closes#59020
This commit removes usage of `Once` from the internal implementation of
time utilities on OSX and Windows. It turns out that we accidentally hit
a deadlock today (#59020) via events that look like:
* A thread invokes `park_timeout`
* Internally, only on OSX, `park_timeout` calls `Instant::elapsed`
* Inside of `Instant::elapsed` on OSX we enter a `Once` to initialize
global timer data
* Inside of `Once`, it attempts to `park`
This means on the same stack frame, when there's contention, we're
calling `park` from inside `park_timeout`, causing a deadlock!
The solution implemented in this commit was to remove usage of `Once`
and instead just do a small dance with atomics. There's no real need we
need to guarantee that the global information is only learned once, only
that it's only *stored* once. This implementation may have multiple
threads invoke `mach_timebase_info`, but only one will store the global
information which will amortize the cost for all other threads.
A similar fix has been applied to windows to be uniform across our
implementations, but looking at the code on Windows no deadlock was
possible. This is purely just a consistency update for Windows and in
theory a slightly leaner implementation.
Closes#59020
wasi: Implement more of the standard library
This commit fills out more of the `wasm32-unknown-wasi` target's standard library, notably the `std::fs` module and all of its internals. A few tweaks were made along the way to non-`fs` modules, but the last commit contains the bulk of the work which is to wire up all APIs to their equivalent on WASI targets instead of unconditionally returning "unsupported". After this some basic filesystem operations and such should all be working in WASI!
This commit fills out the `std::fs` module and implementation for WASI.
Not all APIs are implemented, such as permissions-related ones and
`canonicalize`, but all others APIs have been implemented and very
lightly tested so far. We'll eventually want to run a more exhaustive
test suite!
For now the highlights of this commit are:
* The `std::fs::File` type is now backed by `WasiFd`, a raw WASI file
descriptor.
* All APIs in `std::fs` (except permissions/canonicalize) have
implementations for the WASI target.
* A suite of unstable extension traits were added to
`std::os::wasi::fs`. These traits expose the raw filesystem
functionality of WASI, namely `*at` syscalls (opening a file relative
to an already opened one, for example). Additionally metadata only
available on wasi is exposed through these traits.
Perhaps one of the most notable parts is the implementation of
path-taking APIs. WASI actually has no fundamental API that just takes a
path, but rather everything is relative to a previously opened file
descriptor. To allow existing APIs to work (that only take a path) WASI
has a few syscalls to learn about "pre opened" file descriptors by the
runtime. We use these to build a map of existing directory names to file
descriptors, and then when using a path we try to anchor it at an
already-opened file.
This support is very rudimentary though and is intended to be shared
with C since it's likely to be so tricky. For now though the C library
doesn't expose quite an API for us to use, so we implement it for now
and will swap it out as soon as one is available.
I've since learned that the mapping between libc fds and wasi fds are
expected to be one-to-one, so we can use the raw syscalls for writing to
stdout/stderr and reading from stdin! This should help ensure that we
don't depend on a C library too unnecessarily.
This commit switches the wasi target to loading CLI arguments via the
syscalls provided by wasi rather than through the argc/argv passed to
the main function. While serving the same purpose it's hoped that using
syscalls will make us a bit more portable (less reliance from libstd on
an external C library) as well as avoiding the need for a lock!
Steven Fackler