Weaken `assert_dep_node_not_yet_allocated_in_current_session` for multiple threads
This changes `assert_dep_node_not_yet_allocated_in_current_session` to not panic if the node is already marked green. Another thread may manage to mark it green if there was uncolored node when we initially tried to mark it green.
Remove some unnecessary `try`-related type annotations
I left a few, like
```rust
let result: Result<_, ModError<'_>> = try {
```
where it felt like seeing it might still be useful for the reader.
Feel free to push back on any of these changes if you think they should be left alone.
skip codegen for intrinsics with big fallback bodies if backend does not need them
This hopefully fixes the perf regression from https://github.com/rust-lang/rust/pull/148478. I only added the intrinsics with big fallback bodies to the list; it doesn't seem worth the effort of going through the entire list.
Fixes https://github.com/rust-lang/rust/issues/149945
Cc @scottmcm @bjorn3
This has the nice side-effect of eliminating `rustc_codegen_ssa`'s
dependency on `rustc_query_system`. (Indeed, looking through such
dependencies was how I found this.)
I left a few, like
```rust
let result: Result<_, ModError<'_>> = try {
```
where it felt like seeing it might still be useful for the reader.
Feel free to push back on any of these changes if you think seeing the type would be better.
This makes cdylibs compile to working Emscripten dynamic libraries without passing extra
RUSTFLAGS. This was previously approved as PR 98358 but there were CI failures that I
never got around to fixing.
add `simd_splat` intrinsic
Add `simd_splat` which lowers to the LLVM canonical splat sequence.
```llvm
insertelement <N x elem> poison, elem %x, i32 0
shufflevector <N x elem> v0, <N x elem> poison, <N x i32> zeroinitializer
```
Right now we try to fake it using one of
```rust
fn splat(x: u32) -> u32x8 {
u32x8::from_array([x; 8])
}
```
or (in `stdarch`)
```rust
fn splat(value: $elem_type) -> $name {
#[derive(Copy, Clone)]
#[repr(simd)]
struct JustOne([$elem_type; 1]);
let one = JustOne([value]);
// SAFETY: 0 is always in-bounds because we're shuffling
// a simd type with exactly one element.
unsafe { simd_shuffle!(one, one, [0; $len]) }
}
```
Both of these can confuse the LLVM optimizer, producing sub-par code. Some examples:
- https://github.com/rust-lang/rust/issues/60637
- https://github.com/rust-lang/rust/issues/137407
- https://github.com/rust-lang/rust/issues/122623
- https://github.com/rust-lang/rust/issues/97804
---
As far as I can tell there is no way to provide a fallback implementation for this intrinsic, because there is no `const` way of evaluating the number of elements (there might be issues beyond that, too). So, I added implementations for all 4 backends.
Both GCC and const-eval appear to have some issues with simd vectors containing pointers. I have a workaround for GCC, but haven't yet been able to make const-eval work. See the comments below.
Currently this just adds the intrinsic, it does not actually use it anywhere yet.
codegen: clarify some variable names around function calls
I looked at rust-lang/rust#145932 to try to understand how it works, and quickly got lost in the variable names -- what refers to the caller, what to the callee? So here's my attempt at making those more clear. Hopefully the new names are correct.^^
Cc @JamieCunliffe
Port `#![crate_type]` to the attribute parser
Tracking issue: https://github.com/rust-lang/rust/issues/131229
~~Note that the actual parsing that is used in the compiler session is unchanged, as it must happen very early on; this just ports the validation logic.~~
Also added `// tidy-alphabetical-start` to `check_attr.rs` to make it a bit less conflict-prone
mGCA: Make trait object types with type-level associated consts dyn compatible
Under feature `min_generic_const_args` (mGCA) (rust-lang/rust#132980), render traits with non-parametrized type-level associated constants (i.e., `#[type_const]` ones) dyn compatible but force the user to specify all type-level associated consts in the trait object type via bindings (either directly, via supertrait bounds and/or behind trait aliases) just like associated types, their sibling.
Fixesrust-lang/rust#130300 (feature request).
Fixesrust-lang/rust#136063 (bug).
Fixesrust-lang/rust#137260 (bug).
Fixesrust-lang/rust#137514 (bug).
While I'm accounting for most illegal `Self` references via const projections & params, I'm intentionally ignoring RUST-123140 (and duplicates) in this PR which is to be tackled some other time.
Additional context: Crate `rustc-demangle` had to be updated to fix v0 demangling. I've patched it in PR https://github.com/rust-lang/rustc-demangle/pull/87 which was was released in version 0.1.27 via PR https://github.com/rust-lang/rustc-demangle/pull/88.
compiler: Temporarily re-export `assert_matches!` to reduce stabilization churn
https://github.com/rust-lang/rust/pull/137487 proposes to stabilize `feature(assert_matches)`, while simultaneously moving the `assert_matches!` and `debug_assert_matches!` macros out of the `std::assert_matches` submodule and exporting them directly from the `std` crate root instead.
Unfortunately, that moving step would cause a lot of `cfg(bootstrap)` churn and noise in the compiler, because the compiler imports and uses those macros in dozens of places.
This PR therefore aims to reduce the overall compiler churn, by temporarily adjusting the compiler to always use `assert_matches!` via a re-export from `rustc_data_structures`. That way, the stabilization itself would only need to add `cfg(bootstrap)` to that single re-export (plus the associated `#![feature(assert_matches)]` attributes), greatly reducing the immediate impact on the compiler.
Once `assert_matches!` is stable in the stage0 bootstrap compiler, we can go back and delete the re-export, and adjust the rest of the compiler to import directly from `std` instead.
Add amdgpu_dispatch_ptr intrinsic
There is an ongoing discussion in rust-lang/rust#150452 about using address spaces from the Rust language in some way.
As that discussion will likely not conclude soon, this PR adds one rustc_intrinsic with an addrspacecast to unblock getting basic information like launch and workgroup size and make it possible to implement something like `core::gpu`.
Add a rustc intrinsic `amdgpu_dispatch_ptr` to access the kernel dispatch packet on amdgpu.
The HSA kernel dispatch packet contains important information like the launch size and workgroup size.
The Rust intrinsic lowers to the `llvm.amdgcn.dispatch.ptr` LLVM intrinsic, which returns a `ptr addrspace(4)`, plus an addrspacecast to `addrspace(0)`, so it can be returned as a Rust reference.
The returned pointer/reference is valid for the whole program lifetime, and is therefore `'static`.
The return type of the intrinsic (`&'static ()`) does not mention the struct so that rustc does not need to know the exact struct type. An alternative would be to define the struct as lang item or add a generic argument to the function.
Is this ok or is there a better way (also, should it return a pointer instead of a reference)?
Short version:
```rust
#[cfg(target_arch = "amdgpu")]
pub fn amdgpu_dispatch_ptr() -> *const ();
```
Tracking issue: rust-lang/rust#135024
It's described as a "backwards compatibility hack to keep the diff
small". Removing it requires only a modest amount of churn, and the
resulting code is clearer without the invisible derefs.
Add a rustc intrinsic `amdgpu_dispatch_ptr` to access the kernel
dispatch packet on amdgpu.
The HSA kernel dispatch packet contains important information like the
launch size and workgroup size.
The Rust intrinsic lowers to the `llvm.amdgcn.dispatch.ptr` LLVM
intrinsic, which returns a `ptr addrspace(4)`, plus an addrspacecast to
`addrspace(0)`, so it can be returned as a Rust reference.
The returned pointer/reference is valid for the whole program lifetime,
and is therefore `'static`.
The return type of the intrinsic (`*const ()`) does not mention the
struct so that rustc does not need to know the exact struct type.
An alternative would be to define the struct as lang item or add a
generic argument to the function.
Short version:
```rust
#[cfg(target_arch = "amdgpu")]
pub fn amdgpu_dispatch_ptr() -> *const ();
```
Jana Dönszelmann