abi: add a rust-preserve-none calling convention
This is the conceptual opposite of the rust-cold calling convention and is particularly useful in combination with the new `explicit_tail_calls` feature.
For relatively tight loops implemented with tail calling (`become`) each of the function with the regular calling convention is still responsible for restoring the initial value of the preserved registers. So it is not unusual to end up with a situation where each step in the tail call loop is spilling and reloading registers, along the lines of:
foo:
push r12
; do things
pop r12
jmp next_step
This adds up quickly, especially when most of the clobberable registers are already used to pass arguments or other uses.
I was thinking of making the name of this ABI a little less LLVM-derived and more like a conceptual inverse of `rust-cold`, but could not come with a great name (`rust-cold` is itself not a great name: cold in what context? from which perspective? is it supposed to mean that the function is rarely called?)
add `simd_splat` intrinsic
Add `simd_splat` which lowers to the LLVM canonical splat sequence.
```llvm
insertelement <N x elem> poison, elem %x, i32 0
shufflevector <N x elem> v0, <N x elem> poison, <N x i32> zeroinitializer
```
Right now we try to fake it using one of
```rust
fn splat(x: u32) -> u32x8 {
u32x8::from_array([x; 8])
}
```
or (in `stdarch`)
```rust
fn splat(value: $elem_type) -> $name {
#[derive(Copy, Clone)]
#[repr(simd)]
struct JustOne([$elem_type; 1]);
let one = JustOne([value]);
// SAFETY: 0 is always in-bounds because we're shuffling
// a simd type with exactly one element.
unsafe { simd_shuffle!(one, one, [0; $len]) }
}
```
Both of these can confuse the LLVM optimizer, producing sub-par code. Some examples:
- https://github.com/rust-lang/rust/issues/60637
- https://github.com/rust-lang/rust/issues/137407
- https://github.com/rust-lang/rust/issues/122623
- https://github.com/rust-lang/rust/issues/97804
---
As far as I can tell there is no way to provide a fallback implementation for this intrinsic, because there is no `const` way of evaluating the number of elements (there might be issues beyond that, too). So, I added implementations for all 4 backends.
Both GCC and const-eval appear to have some issues with simd vectors containing pointers. I have a workaround for GCC, but haven't yet been able to make const-eval work. See the comments below.
Currently this just adds the intrinsic, it does not actually use it anywhere yet.
This is the conceptual opposite of the rust-cold calling convention and
is particularly useful in combination with the new `explicit_tail_calls`
feature.
For relatively tight loops implemented with tail calling (`become`) each
of the function with the regular calling convention is still responsible
for restoring the initial value of the preserved registers. So it is not
unusual to end up with a situation where each step in the tail call loop
is spilling and reloading registers, along the lines of:
foo:
push r12
; do things
pop r12
jmp next_step
This adds up quickly, especially when most of the clobberable registers
are already used to pass arguments or other uses.
I was thinking of making the name of this ABI a little less LLVM-derived
and more like a conceptual inverse of `rust-cold`, but could not come
with a great name (`rust-cold` is itself not a great name: cold in what
context? from which perspective? is it supposed to mean that the
function is rarely called?)
`c_variadic`: impl `va_copy` and `va_end` as Rust intrinsics
tracking issue: https://github.com/rust-lang/rust/issues/44930
Implement `va_copy` as (the rust equivalent of) `memcpy`, which is the behavior of all current LLVM targets. By providing our own implementation, we can guarantee its behavior. These guarantees are important for implementing c-variadics in e.g. const-eval.
Discussed in [#t-compiler/const-eval > c-variadics in const-eval](https://rust-lang.zulipchat.com/#narrow/channel/146212-t-compiler.2Fconst-eval/topic/c-variadics.20in.20const-eval/with/565509704).
I've also updated the comment for `Drop` a bit. The background here is that the C standard requires that `va_end` is used in the same function (and really, in the same scope) as the corresponding `va_start` or `va_copy`. That is because historically `va_start` would start a scope, which `va_end` would then close. e.g.
https://softwarepreservation.computerhistory.org/c_plus_plus/cfront/release_3.0.3/source/incl-master/proto-headers/stdarg.sol
```c
#define va_start(ap, parmN) {\
va_buf _va;\
_vastart(ap = (va_list)_va, (char *)&parmN + sizeof parmN)
#define va_end(ap) }
#define va_arg(ap, mode) *((mode *)_vaarg(ap, sizeof (mode)))
```
The C standard still has to consider such implementations, but for Rust they are irrelevant. Hence we can use `Clone` for `va_copy` and `Drop` for `va_end`.
Replace Rvalue::NullaryOp by a variant in mir::Operand.
Based on https://github.com/rust-lang/rust/pull/148151
This PR fully removes the MIR `Rvalue::NullaryOp`. After rust-lang/rust#148151, it was only useful for runtime checks like `ub_checks`, `contract_checks` and `overflow_checks`.
These are "runtime" checks, boolean constants that may only be `true` in codegen. It depends on a rustc flag passed to codegen, so we need to represent those flags cross-crate.
This PR replaces those runtime checks by special variants in MIR `ConstValue`. This allows code that expects constants to manipulate those as such, even if we may not always be able to evaluate them to actual scalars.
All usages of `memory_index` start by calling `invert_bijective_mapping`, so
storing the inverted mapping directly saves some work and simplifies the code.
Simplify how inline asm handles `MaybeUninit`
This is just better, but this is also allows it to handle changes from https://github.com/rust-lang/rust/pull/149614 (i.e. `ManuallyDrop` containing `MaybeDangle`).
Overhaul filename handling for cross-compiler consistency
This PR overhauls the way we handle filenames in the compiler and `rmeta` in order to achieve achieve cross-compiler consistency (ie. having the same path no matter if the filename was created in the current compiler session or is coming from `rmeta`).
This is required as some parts of the compiler rely on consistent paths for the soundness of generated code (see rust-lang/rust#148328).
In order to achieved consistency multiple steps are being taken by this PR:
- by making `RealFileName` immutable
- by only having `SourceMap::to_real_filename` create `RealFileName`
- currently `RealFileName` can be created from any `Path` and are remapped afterwards, which creates consistency issue
- by also making `RealFileName` holds it's working directory, embeddable name and the remapped scopes
- this removes the need for a `Session`, to know the current(!) scopes and cwd, which is invalid as they may not be equal to the scopes used when creating the filename
In order for `SourceMap::to_real_filename` to know which scopes to apply `FilePathMapping` now takes the current remapping scopes to apply, which makes `FileNameDisplayPreference` and company useless and are removed.
This PR is split-up in multiple commits (unfortunately not atomic), but should help review the changes.
Unblocks https://github.com/rust-lang/rust/pull/147611
Fixes https://github.com/rust-lang/rust/issues/148328
Make `--print=backend-has-zstd` work by default on any backend
Using a defaulted `CodegenBackend` method that querying for zstd support should automatically print a safe value of `false` on any backend that doesn't specifically indicate the presence or absence of zstd.
This should fix the compiletest failures reported in https://github.com/rust-lang/rust/pull/149666#discussion_r2597881482, which can occur when LLVM is not the default codegen backend.
Remove -Zoom=panic
There are major questions remaining about the reentrancy that this allows. It doesn't have any users on github outside of a single project that uses it in a panic=abort project to show backtraces. It can still be emulated through `#[alloc_error_handler]` or `set_alloc_error_hook` depending on if you use the standard library or not. And finally it makes it harder to do various improvements to the allocator shim.
With this PR the sole remaining symbol in the allocator shim that is not effectively emulating weak symbols is the symbol that prevents skipping the allocator shim on stable even when it would otherwise be empty because libstd + `#[global_allocator]` is used.
Closes https://github.com/rust-lang/rust/issues/43596
Fixes https://github.com/rust-lang/rust/issues/126683
Using a defaulted `CodegenBackend` method that querying for zstd support should
automatically print a safe value of `false` on any backend that doesn't
specifically indicate the presence or absence of zstd.
There are major questions remaining about the reentrancy that this
allows. It doesn't have any users on github outside of a single project
that uses it in a panic=abort project to show backtraces. It
can still be emulated through #[alloc_error_handler] or
set_alloc_error_hook depending on if you use the standard library or
not. And finally it makes it harder to do various improvements to the
allocator shim.
Add `overflow_checks` intrinsic
This adds an intrinsic which allows code in a pre-built library to inherit the overflow checks option from a crate depending on it. This enables code in the standard library to explicitly change behavior based on whether `overflow_checks` are enabled, regardless of the setting used when standard library was compiled.
This is very similar to the `ub_checks` intrinsic, and refactors the two to use a common mechanism.
The primary use case for this is to allow the new `RangeFrom` iterator to yield the maximum element before overflowing, as requested [here](https://github.com/rust-lang/rust/issues/125687#issuecomment-2151118208). This PR includes a working `IterRangeFrom` implementation based on this new intrinsic that exhibits the desired behavior.
[Prior discussion on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Ability.20to.20select.20code.20based.20on.20.60overflow_checks.60.3F)
Forbid ShallowInitBox after box deref elaboration.
MIR currently contains a `ShallowInitBox` rvalue. Its principal usage is to allow for in-place initialization of boxes. Having it is necessary for drop elaboration to be correct with that in-place initialization.
As part of analysis->runtime MIR lowering, we canonicalize deref of boxes to use the stored raw pointer. But we did not perform the same change to the construction of the box.
This PR replaces `ShallowInitBox` by the pointer manipulation it represents.
Alternatives:
- fully remove `ShallowInitBox` and implement `Box` in-place initialization differently;
- remove the `ElaborateBoxDeref` pass and keep dereferencing `Box` in runtime MIR.
the `#[track_caller]` shim should not inherit `#[no_mangle]`
fixes https://github.com/rust-lang/rust/issues/143162
builds on https://github.com/rust-lang/rust/pull/143293 which introduced a mechanism to strip attributes from shims.
cc `@Jules-Bertholet` `@workingjubilee` `@bjorn3`
---
Summary:
This PR fixes an interaction between `#[track_caller]`, `#[no_mangle]`, and casting to a function pointer.
A function annotated with `#[track_caller]` internally has a hidden extra argument for the panic location. The `#[track_caller]` attribute is only allowed on `extern "Rust"` functions. When a function is annotated with both `#[no_mangle]` and `#[track_caller]`, the exported symbol has the signature that includes the extra panic location argument. This works on stable rust today:
```rust
extern "Rust" {
#[track_caller]
fn rust_track_caller_ffi_test_tracked() -> &'static Location<'static>;
}
mod provides {
use std::panic::Location;
#[track_caller] // UB if we did not have this!
#[no_mangle]
fn rust_track_caller_ffi_test_tracked() -> &'static Location<'static> {
Location::caller()
}
}
```
When a `#[track_caller]` function is converted to a function pointer, a shim is added to drop the additional argument. So this is a valid program:
```rust
#[track_caller]
fn foo() {}
fn main() {
let f = foo as fn();
f();
}
```
The issue arises when `foo` is additionally annotated with `#[no_mangle]`, the generated shim currently inherits this attribute, also exporting a symbol named `foo`, but one without the hidden panic location argument. The linker rightfully complains about a duplicate symbol.
The solution of this PR is to have the generated shim drop the `#[no_mangle]` attribute.
bors