Fix Instant/Duration math precision & associativity on Windows
**tl;dr** Addition and subtraction on Duration/Instant are not associative on windows because we use the perfcounter frequency in every calculation instead of just when we measure time.
This is my first contrib (PR or Issue) to Rust, so please lmk if I've done this wrong. I followed CONTRIBUTING to the extent I could given my system doesn't seem to be able to build the compiler with changes in the source tree. I also asked about this issue in #rust-beginners a week or so ago, before digging through libstd -- I'm unsure if there's a good way to follow up on that, but I'd be happy to update the docs on the timing structs if this fixes the problem.
## Issue
The `Duration` type keeps seconds in the upper-64 and nanoseconds in the lower-32 bits. In theory doing math on these ought to be basically the same as doing math on any other 64 or 32 bit integral number.
On windows (and I think macos too), however, our math gets messy because the Instant type stores the current point in time in units of HPET Performance Counter counts, not nanoseconds, and does unit conversions on every math operation, rather than just when we measure the time from the system clock.
I tried this code:
```
use std::time::{Duration, Instant};
fn main() {
let now = Instant::now();
let offset = Duration::from_millis(5);
assert_eq!((now + offset) - now, (now - now) + offset);
}
```
On UNIX machines (linux and macos) it behaves as you'd expect -- [no crash](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=cf2206c0b7e07d8ecc7767a512364094).
On Windows hosts, however, it blows up because of a precision problem in the Instant +/- Duration math:
```
C:\Users\aberg\work\timetest (master -> origin)
λ cargo run
Finished dev [unoptimized + debuginfo] target(s) in 0.02s
Running `target\debug\timetest.exe`
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `4.999914ms`,
right: `5ms`', src\main.rs:6:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: process didn't exit successfully: `target\debug\timetest.exe` (exit code: 101)
C:\Users\aberg\work\timetest (master -> origin)
λ cat src\main.rs
use std::time::{Duration, Instant};
fn main() {
let now = Instant::now();
let offset = Duration::from_millis(5);
assert_eq!((now + offset) - now, (now - now) + offset);
}
```
On windows I think this is a consequence of doing the HPET-PerfCounter-Unit conversion on each math operation. I suspect the reason it works on macs is that (from what I could find online) most apple machines report timing in nanoseconds anyway. For anyone interested, the equivalent functions on macos, with a little work to fish out the numerator/denominator from a timebase struct:
* `QueryPerformanceCounter()` -> `mach_absolute_time()`
* `QueryPerformanceFrequency()` -> `mach_timebase_info()`
If this PR ends up working as I expect it to when CI runs the tests, I can make the same changes to the macos implementation.
## Potential Fix
We ought to be able to sort this out by storing nanoseconds, rather than PerfCounter units, that way intermediate math is done in the most precise units we support and we're only doing unit conversions when we actually measure the system clock (and it might even translate to a small perf gain for people doing tons of Instant/Duration math).
I believe this will address the underlying cause of #56034, and make the guessed epsilon constant from #56059 unnecessary. If it's of interest, I can write up how these timing types work on the tier 1 platforms to address #32626 as well, since I'm already in here figuring it out.
## This Patch
To that end, I've got this patch, which I think should fix it on windows, but I'm having trouble testing it -- any time I change anything in libstd I start getting this error, which no amount of clean building seems to resolve:
```
C:\Users\aberg\work\rust (master -> origin)
λ python x.py test --stage 0 --no-doc src/libstd
Updating only changed submodules
Submodules updated in 0.27 seconds
Finished dev [unoptimized] target(s) in 2.41s
Building stage0 std artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
Finished release [optimized] target(s) in 6.78s
Copying stage0 std from stage0 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Building stage0 test artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
Compiling test v0.0.0 (C:\Users\aberg\work\rust\src\libtest)
error[E0460]: found possibly newer version of crate `std` which `getopts` depends on
--> src\libtest\lib.rs:36:1
|
36 | extern crate getopts;
| ^^^^^^^^^^^^^^^^^^^^^
|
= note: perhaps that crate needs to be recompiled?
= note: the following crate versions were found:
crate `std`: \\?\C:\Users\aberg\work\rust\build\x86_64-pc-windows-msvc\stage0-sysroot\lib\rustlib\x86_64-pc-windows-msvc\lib\libstd-d7a80ca2ae113c97.rlib
crate `std`: \\?\C:\Users\aberg\work\rust\build\x86_64-pc-windows-msvc\stage0-sysroot\lib\rustlib\x86_64-pc-windows-msvc\lib\std-d7a80ca2ae113c97.dll
crate `getopts`: \\?\C:\Users\aberg\work\rust\build\x86_64-pc-windows-msvc\stage0-test\x86_64-pc-windows-msvc\release\deps\libgetopts-ae40a96de5f5d144.rlib
error: aborting due to previous error
For more information about this error, try `rustc --explain E0460`.
error: Could not compile `test`.
To learn more, run the command again with --verbose.
command did not execute successfully: "C:\\Users\\aberg\\work\\rust\\build\\x86_64-pc-windows-msvc\\stage0\\bin\\cargo.exe" "build" "--target" "x86_64-pc-windows-msvc" "-j" "12" "--release" "--manifest-path" "C:\\Users\\aberg\\work\\rust\\src/libtest/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101
failed to run: C:\Users\aberg\work\rust\build\bootstrap\debug\bootstrap test --stage 0 --no-doc src/libstd
Build completed unsuccessfully in 0:00:20
```
---
Since you wrote the linked PRs and might remember looking at related problems:
r? @alexcrichton
Update std/lib.rs docs to reflect Rust 2018 usage
Fixes#56544
This paragraph was written for Rust 2015. Since 2018 has been stable for a while I think we can update it.
Per review comments, this commit switches out the backing
type for Instant on windows to a Duration. Tests all pass,
and the code's a lot simpler (plus it should be portable now,
with the exception of the QueryPerformanceWhatever functions).
Right now we do unit conversions between PerfCounter measurements
and nanoseconds for every add/sub we do between Durations and Instants
on Windows machines. This leads to goofy behavior, like this snippet
failing:
```
let now = Instant::now();
let offset = Duration::from_millis(5);
assert_eq!((now + offset) - now, (now - now) + offset);
```
with precision problems like this:
```
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `4.999914ms`,
right: `5ms`', src\main.rs:6:5
```
To fix it, this changeset does the unit conversion once, when we
measure the clock, and all the subsequent math in u64 nanoseconds.
It also adds an exact associativity test to the `sys/time.rs`
test suite to make sure we don't regress on this in the future.
OSX: fix#57534 registering thread dtors while running thread dtors
r? @alexcrichton
- "fast" `thread_local` destructors get run even on the main thread
- "fast" `thread_local` dtors, can initialize other `thread_local`'s
One corner case where this fix doesn't work, is when a C++ `thread_local` triggers the initialization of a rust `thread_local`.
I did not add any std::thread specific flag to indicate that the thread is currently exiting, which would be checked before registering a new dtor (I didn't really know where to stick that). I think this does the trick tho!
Let me know if anything needs tweaking/fixing/etc.
resolves this for macos: https://github.com/rust-lang/rust/issues/28129
fixes: https://github.com/rust-lang/rust/issues/57534
Move spin_loop_hint to core::hint module
As mentioned in #55002. The new name is kept unstable to decide whether the function should have `_hint` in its name.
Upgrade x86_64-fortanix-unknown-sgx platform support to tier 2
## Overview
1. This PR upgrades x86_64-fortanix-unknown-sgx platform support to tier 2 (std only) by setting up build automation for this target.
1. For supporting unwinding, this target needs to link to a port of LLVM's libunwind (more details could be found in #56979), which will be distributed along with the Rust binaries (similar to the extra musl objects)
### Building and copying libunwind:
We have added a new build script (`build-x86_64-fortanix-unknown-sgx-toolchain.sh`) that will run while the container is built. This will build `libunwind.a` from git source.
While the container is built, the persistent volumes where obj/ gets created aren't yet mapped. As a workaround, we copy the built `libunwind.a` to `obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-fortanix-unknown-sgx/lib/` after x.py runs.
If any reviewer knows of a better solution, please do tell.
r? @Mark-Simulacrum
Fix poor worst case performance of set intersection
Specifically, intersection of asymmetrically sized sets when the large set is on the left. See also the [latest answer on stackoverflow](https://stackoverflow.com/questions/35439376/python-set-intersection-is-faster-then-rust-hashset-intersection).
Also applied to the union member, where the effect is much less but still measurable.
Formatted the changed code only, does not increase the error count reported by tidy check, and tried to adhere to the spirit of the unit tests.
Remove the `connect_timeout_unroutable` test.
It requires an unreachable IP address, but there is no such thing, and
this has caused it to fail for multiple people.
Fixes#44698, fixes#50065.
r? @sfackler
std: Render large exit codes as hex on Windows
On Windows process exit codes are never signals but rather always 32-bit
integers. Most faults like segfaults and such end up having large
integers used to represent them, like STATUS_ACCESS_VIOLATION being
0xC0000005. Currently, however, when an `ExitStatus` is printed this
ends up getting rendered as 3221225477 which is somewhat more difficult
to debug.
This commit adds a branch in `Display for ExitStatus` on Windows which
handles exit statuses where the high bit is set and prints those exit
statuses as hex instead of with decimals. This will hopefully preserve
the current display for small exit statuses (like `exit code: 22`), but
assist in quickly debugging segfaults/access violations/etc. I've
found at least that the hex codes are easier to search for than decimal.
I wasn't able to find any official documentation saying that all system
exit codes have the high bit set, but I figure it's a good enough
heuristic for now.
Fix undefined behavior
From the [`MaybeUninit::get_mut` docs](https://doc.rust-lang.org/std/mem/union.MaybeUninit.html):
> It is up to the caller to guarantee that the the MaybeUninit really is in an initialized state, otherwise this will immediately cause undefined behavior.
r? @joshtriplett
Supporting backtrace for x86_64-fortanix-unknown-sgx.
# Overview
Implementing following functions required by `libstd/sys_common` to support `backtrace`:
```
1. unwind_backtrace
2. trace_fn
3. resolve_symname
```
# Description:
The changes here are quite similar to the Cloudabi target `src/libstd/sys/cloudabi/backtrace.rs`
The first 2 functions are implemented via calls to libunwind.a that is linked to the `x86_64-fortanix-unknown-sgx` (#56979), we have not implemented functionality needed by `resolve_symname` (or `dladdr`) to reduce SGX TCB. Rather, we print the function address (relative to enclave image base) in `resolve_symname` which can be later translated to correct symbol name (say, via `addr2line`).
# Note:
For `x86_64-fortanix-unknown-sgx`, the `RUST_BACKTRACE` environment has to be set from within the program running in an enclave.
cc: @jethrogb
r? @alexcrichton
Fixed the link to the ? operator
I'm working on updating all broken links, but figured I'd break up the pull requests so they are easier to review, versus just one big pull request.
On Windows process exit codes are never signals but rather always 32-bit
integers. Most faults like segfaults and such end up having large
integers used to represent them, like STATUS_ACCESS_VIOLATION being
0xC0000005. Currently, however, when an `ExitStatus` is printed this
ends up getting rendered as 3221225477 which is somewhat more difficult
to debug.
This commit adds a branch in `Display for ExitStatus` on Windows which
handles exit statuses where the high bit is set and prints those exit
statuses as hex instead of with decimals. This will hopefully preserve
the current display for small exit statuses (like `exit code: 22`), but
assist in quickly debugging segfaults/access violations/etc. I've
found at least that the hex codes are easier to search for than decimal.
I wasn't able to find any official documentation saying that all system
exit codes have the high bit set, but I figure it's a good enough
heuristic for now.
std: Force `Instant::now()` to be monotonic
This commit is an attempt to force `Instant::now` to be monotonic
through any means possible. We tried relying on OS/hardware/clock
implementations, but those seem buggy enough that we can't rely on them
in practice. This commit implements the same hammer Firefox recently
implemented (noted in #56612) which is to just keep whatever the lastest
`Instant::now()` return value was in memory, returning that instead of
the OS looks like it's moving backwards.
Closes#48514Closes#49281
cc #51648
cc #56560Closes#56612Closes#56940
This commit is an attempt to force `Instant::now` to be monotonic
through any means possible. We tried relying on OS/hardware/clock
implementations, but those seem buggy enough that we can't rely on them
in practice. This commit implements the same hammer Firefox recently
implemented (noted in #56612) which is to just keep whatever the lastest
`Instant::now()` return value was in memory, returning that instead of
the OS looks like it's moving backwards.
Closes#48514Closes#49281
cc #51648
cc #56560Closes#56612Closes#56940
Mazdak Farrokhzad