Add SIMD funnel shift and round-to-even intrinsics
This PR adds 3 new SIMD intrinsics
- `simd_funnel_shl` - funnel shift left
- `simd_funnel_shr` - funnel shift right
- `simd_round_ties_even` (vector version of `round_ties_even_fN`)
TODO (future PR): implement `simd_fsh{l,r}` in miri, cg_gcc and cg_clif (it is surprisingly hard to implement without branches, the common tricks that rotate uses doesn't work because we have 2 elements now. e.g, the `-n&31` trick used by cg_gcc to implement rotate doesn't work with this because then `fshl(a, b, 0)` will be `a | b`)
[#t-compiler > More SIMD intrinsics](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/More.20SIMD.20intrinsics/with/522130286)
`@rustbot` label T-compiler T-libs A-intrinsics F-core_intrinsics
r? `@workingjubilee`
Insert checks for enum discriminants when debug assertions are enabled
Similar to the existing null-pointer and alignment checks, this checks for valid enum discriminants on creation of enums through unsafe transmutes. Essentially this sanitizes patterns like the following:
```rust
let val: MyEnum = unsafe { std::mem::transmute<u32, MyEnum>(42) };
```
An extension of this check will be done in a follow-up that explicitly sanitizes for extern enum values that come into Rust from e.g. C/C++.
This check is similar to Miri's capabilities of checking for valid construction of enum values.
This PR is inspired by saethlin@'s PR
https://github.com/rust-lang/rust/pull/104862. Thank you so much for keeping this code up and the detailed comments!
I also pair-programmed large parts of this together with vabr-g@.
r? `@saethlin`
Similar to the existing nullpointer and alignment checks, this checks
for valid enum discriminants on creation of enums through unsafe
transmutes. Essentially this sanitizes patterns like the following:
```rust
let val: MyEnum = unsafe { std::mem::transmute<u32, MyEnum>(42) };
```
An extension of this check will be done in a follow-up that explicitly
sanitizes for extern enum values that come into Rust from e.g. C/C++.
This check is similar to Miri's capabilities of checking for valid
construction of enum values.
This PR is inspired by saethlin@'s PR
https://github.com/rust-lang/rust/pull/104862. Thank you so much for
keeping this code up and the detailed comments!
I also pair-programmed large parts of this together with vabr-g@.
Change __rust_no_alloc_shim_is_unstable to be a function
This fixes a long sequence of issues:
1. A customer reported that building for Arm64EC was broken: #138541
2. This was caused by a bug in my original implementation of Arm64EC support, namely that only functions on Arm64EC need to be decorated with `#` but Rust was decorating statics as well.
3. Once I corrected Rust to only decorate functions, I started linking failures where the linker couldn't find statics exported by dylib dependencies. This was caused by the compiler not marking exported statics in the generated DEF file with `DATA`, thus they were being exported as functions not data.
4. Once I corrected the way that the DEF files were being emitted, the linker started failing saying that it couldn't find `__rust_no_alloc_shim_is_unstable`. This is because the MSVC linker requires the declarations of statics imported from other dylibs to be marked with `dllimport` (whereas it will happily link to functions imported from other dylibs whether they are marked `dllimport` or not).
5. I then made a change to ensure that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport`, but the MSVC linker started emitting warnings that `__rust_no_alloc_shim_is_unstable` was marked as `dllimport` but was declared in an obj file. This is a harmless warning which is a performance hint: anything that's marked `dllimport` must be indirected via an `__imp` symbol so I added a linker arg in the target to suppress the warning.
6. A customer then reported a similar warning when using `lld-link` (<https://github.com/rust-lang/rust/pull/140176#issuecomment-2872448443>). I don't think it was an implementation difference between the two linkers but rather that, depending on the obj that the declaration versus uses of `__rust_no_alloc_shim_is_unstable` landed in we would get different warnings, so I suppressed that warning as well: #140954.
7. Another customer reported that they weren't using the Rust compiler to invoke the linker, thus these warnings were breaking their build: <https://github.com/rust-lang/rust/pull/140176#issuecomment-2881867433>. At that point, my original change was reverted (#141024) leaving Arm64EC broken yet again.
Taking a step back, a lot of these linker issues arise from the fact that `__rust_no_alloc_shim_is_unstable` is marked as `extern "Rust"` in the standard library and, therefore, assumed to be a foreign item from a different crate BUT the Rust compiler may choose to generate it either in the current crate, some other crate that will be statically linked in OR some other crate that will by dynamically imported.
Worse yet, it is impossible while building a given crate to know if `__rust_no_alloc_shim_is_unstable` will statically linked or dynamically imported: it might be that one of its dependent crates is the one with an allocator kind set and thus that crate (which is compiled later) will decide depending if it has any dylib dependencies or not to import `__rust_no_alloc_shim_is_unstable` or generate it. Thus, there is no way to know if the declaration of `__rust_no_alloc_shim_is_unstable` should be marked with `dllimport` or not.
There is a simple fix for all this: there is no reason `__rust_no_alloc_shim_is_unstable` must be a static. It needs to be some symbol that must be linked in; thus, it could easily be a function instead. As a function, there is no need to mark it as `dllimport` when dynamically imported which avoids the entire mess above.
There may be a perf hit for changing the `volatile load` to be a `tail call`, so I'm happy to change that part back (although I question what the codegen of a `volatile load` would look like, and if the backend is going to try to use load-acquire semantics).
Build with this change applied BEFORE #140176 was reverted to demonstrate that there are no linking issues with either MSVC or MinGW: <https://github.com/rust-lang/rust/actions/runs/15078657205>
Incidentally, I fixed `tests/run-make/no-alloc-shim` to work with MSVC as I needed it to be able to test locally (FYI for #128602)
r? `@bjorn3`
cc `@jieyouxu`
Sized Hierarchy: Part I
This patch implements the non-const parts of rust-lang/rfcs#3729. It introduces two new traits to the standard library, `MetaSized` and `PointeeSized`. See the RFC for the rationale behind these traits and to discuss whether this change makes sense in the abstract.
These traits are unstable (as is their constness), so users cannot refer to them without opting-in to `feature(sized_hierarchy)`. These traits are not behind `cfg`s as this would make implementation unfeasible, there would simply be too many `cfg`s required to add the necessary bounds everywhere. So, like `Sized`, these traits are automatically implemented by the compiler.
RFC 3729 describes changes which are necessary to preserve backwards compatibility given the introduction of these traits, which are implemented and as follows:
- `?Sized` is rewritten as `MetaSized`
- `MetaSized` is added as a default supertrait for all traits w/out an explicit sizedness supertrait already.
There are no edition migrations implemented in this, as these are primarily required for the constness parts of the RFC and prior to stabilisation of this (and so will come in follow-up PRs alongside the const parts). All diagnostic output should remain the same (showing `?Sized` even if the compiler sees `MetaSized`) unless the `sized_hierarchy` feature is enabled.
Due to the use of unstable extern types in the standard library and rustc, some bounds in both projects have had to be relaxed already - this is unfortunate but unavoidable so that these extern types can continue to be used where they were before. Performing these relaxations in the standard library and rustc are desirable longer-term anyway, but some bounds are not as relaxed as they ideally would be due to the inability to relax `Deref::Target` (this will be investigated separately).
It is hoped that this is implemented such that it could be merged and these traits could exist "under the hood" without that being observable to the user (other than in any performance impact this has on the compiler, etc). Some details might leak through due to the standard library relaxations, but this has not been observed in test output.
**Notes:**
- Any commits starting with "upstream:" can be ignored, as these correspond to other upstream PRs that this is based on which have yet to be merged.
- This best reviewed commit-by-commit. I've attempted to make the implementation easy to follow and keep similar changes and test output updates together.
- Each commit has a short description describing its purpose.
- This patch is large but it's primarily in the test suite.
- I've worked on the performance of this patch and a few optimisations are implemented so that the performance impact is neutral-to-minor.
- `PointeeSized` is a different name from the RFC just to make it more obvious that it is different from `std::ptr::Pointee` but all the names are yet to be bikeshed anyway.
- `@nikomatsakis` has confirmed [that this can proceed as an experiment from the t-lang side](https://rust-lang.zulipchat.com/#narrow/channel/435869-project-goals/topic/SVE.20and.20SME.20on.20AArch64.20.28goals.23270.29/near/506196491)
- FCP in https://github.com/rust-lang/rust/pull/137944#issuecomment-2912207485Fixesrust-lang/rust#79409.
r? `@ghost` (I'll discuss this with relevant teams to find a reviewer)
Apply ABI attributes on return types in `rustc_codegen_cranelift`
- The [x86-64 System V ABI standard](https://gitlab.com/x86-psABIs/x86-64-ABI/-/jobs/artifacts/master/raw/x86-64-ABI/abi.pdf?job=build) doesn't sign/zero-extend integer arguments or return types.
- But the de-facto standard as implemented by Clang and GCC is to sign/zero-extend arguments to 32 bits (but not return types).
- Additionally, Apple targets [sign/zero-extend both arguments and return values to 32 bits](https://developer.apple.com/documentation/xcode/writing-64-bit-intel-code-for-apple-platforms#Pass-arguments-to-functions-correctly).
- However, the `rustc_target` ABI adjustment code currently [unconditionally extends both arguments and return values to 32 bits](https://github.com/rust-lang/rust/blame/e703dff8fe220b78195c53478e83fb2f68d8499c/compiler/rustc_target/src/callconv/x86_64.rs#L240) on all targets.
- This doesn't cause a miscompilation when compiling with LLVM as LLVM will ignore the `signext`/`zeroext` attribute when applied to return types on non-Apple x86-64 targets.
- Cranelift, however, does not have a similar special case, requiring `rustc` to set the argument extension attribute correctly.
- However, `rustc_codegen_cranelift` doesn't currently apply ABI attributes to return types at all, meaning `rustc_codegen_cranelift` will currently miscompile `i8`/`u8`/`i16`/`u16` returns on x86-64 Apple targets as those targets require sign/zero-extension of return types.
This PR fixes the bug(s) by making the `rustc_target` x86-64 System V ABI only mark return types as sign/zero-extended on Apple platforms, while also making `rustc_codegen_cranelift` apply ABI attributes to return types. The RISC-V and s390x C ABIs also require sign/zero extension of return types, so this will fix those targets when building with `rustc_codegen_cranelift` too.
r? `````@bjorn3`````
Move metadata object generation for dylibs to the linker code
This deduplicates some code between codegen backends and may in the future allow adding extra metadata that is only known at link time.
Prerequisite of https://github.com/rust-lang/rust/issues/96708.
bjorn3