fix va_list test by adding a llvmir signext check
s390x has no option to directly pass 32bit values therefor i32 parameters need an optional llvmir signext attribute.
Restrict spe_acc to PowerPC SPE targets
Update the tests, add powerpc-*-gnuspe testing, and create a distinct clobber_abi list for PowerPC SPE targets.
Note, the SPE target does not have vector, vector-scalar, or floating-point specific registers.
r? ```@Amanieu```
Add `ilog10` result range hints
This PR adds hints that the return value of `T::ilog10` will never exceed `T::MAX.ilog10()`.
This works because `ilog10` is a monotonically nondecreasing function, the maximum return value is reached at the max input value.
Assume the returned value in `.filter(…).count()`
Similar to how this helps in `slice::Iter::position`, LLVM sometimes loses track of how high this can get, so for `TrustedLen` iterators tell it what the upper bound is.
`c_variadic`: make `VaList` abi-compatible with C
tracking issue: https://github.com/rust-lang/rust/issues/44930
related PR: rust-lang/rust#144529
On some platforms, the C `va_list` type is actually a single-element array of a struct (on other platforms it is just a pointer). In C, arrays passed as function arguments expirience array-to-pointer decay, which means that C will pass a pointer to the array in the caller instead of the array itself, and modifications to the array in the callee will be visible to the caller (this does not match Rust by-value semantics). However, for `va_list`, the C standard explicitly states that it is undefined behaviour to use a `va_list` after it has been passed by value to a function (in Rust parlance, the `va_list` is moved, not copied). This matches Rust's pass-by-value semantics, meaning that when the C `va_list` type is a single-element array of a struct, the ABI will match C as long as the Rust type is always be passed indirectly.
In the old implementation, this ABI was achieved by having two separate types: `VaList` was the type that needed to be used when passing a `VaList` as a function parameter, whereas `VaListImpl` was the actual `va_list` type that was correct everywhere else. This however is quite confusing, as there are lots of footguns: it is easy to cause bugs by mixing them up (e.g. the C function `void foo(va_list va)` was equivalent to the Rust `fn foo(va: VaList)` whereas the C function `void bar(va_list* va)` was equivalent to the Rust `fn foo(va: *mut VaListImpl)`, not `fn foo(va: *mut VaList)` as might be expected); also converting from `VaListImpl` to `VaList` with `as_va_list()` had platform specific behaviour: on single-element array of a struct platforms it would return a `VaList` referencing the original `VaListImpl`, whereas on other platforms it would return a cioy,
In this PR, there is now just a single `VaList` type (renamed from `VaListImpl`) which represents the C `va_list` type and will just work in all positions. Instead of having a separate type just to make the ABI work, rust-lang/rust#144529 adds a `#[rustc_pass_indirectly_in_non_rustic_abis]` attribute, which when applied to a struct will force the struct to be passed indirectly by non-Rustic calling conventions. This PR then implements the `VaList` rework, making use of the new attribute on all platforms where the C `va_list` type is a single-element array of a struct.
Cleanup of the `VaList` API and implementation is also included in this PR: since it was decided it was OK to experiment with Rust requiring that not calling `va_end` is not undefined behaviour (https://github.com/rust-lang/rust/issues/141524#issuecomment-3028383594), I've removed the `with_copy` method as it was redundant to the `Clone` impl (the `Drop` impl of `VaList` is a no-op as `va_end` is a no-op on all known platforms).
Previous discussion: rust-lang/rust#141524 and [t-compiler > c_variadic API and ABI](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/c_variadic.20API.20and.20ABI)
Tracking issue: https://github.com/rust-lang/rust/issues/44930
r? `@joshtriplett`
library: Rename `IterRange*` to `Range*Iter`
There is a weak convention in the ecosystem that `IterFoos` is an iterator yielding items of type `Foo` (e.g. `bitflags` `IterNames`, `hashbrown` `IterBuckets`), while `FooIter` is an iterator over `Foo` from an `.iter()` or `.into_iter()` method (e.g. `memchr` `OneIter`, `regex` `SetMatchesIter`). Rename `IterRange`, `IterRangeInclusive`, and `IterRangeFrom` to `RangeIter`, `RangeInclusiveIter`, and `RangeInclusiveIter` to match this.
Tracking issue: https://github.com/rust-lang/rust/issues/125687 (`new_range_api`)
Update the tests, add powerpc-*-gnuspe testing, and create a distinct
clobber_abi list for PowerPC SPE targets.
Note, the SPE target does not have vector, vector-scalar, or
floating-point specific registers.
Rollup of 9 pull requests
Successful merges:
- rust-lang/rust#147224 (Emscripten: Turn wasm-eh on by default)
- rust-lang/rust#149405 (Recover on misspelled item keyword)
- rust-lang/rust#149443 (Tidying up UI tests [6/N])
- rust-lang/rust#149524 (Move attribute safety checking to attribute parsing)
- rust-lang/rust#149593 (powf, powi: point out SNaN non-determinism)
- rust-lang/rust#149605 (Use branch name instead of HEAD when unshallowing)
- rust-lang/rust#149612 (Apply the `bors` environment also to the `outcome` job)
- rust-lang/rust#149623 (Don't require a normal tool build of clippy/rustfmt when running their test steps)
- rust-lang/rust#149627 (Point to the item that is incorrectly annotated with `#[diagnostic::on_const]`)
r? `@ghost`
`@rustbot` modify labels: rollup
This register is only supported on the *powerpc*spe targets. It is
only recognized by LLVM. gcc does not accept this as a clobber, nor
does it support these targets.
This is a volatile register, thus it is included with clobber_abi.
There is a weak convention in the ecosystem that `IterFoos` is an
iterator yielding items of type `Foo` (e.g. `bitflags` `IterNames`,
`hashbrown` `IterBuckets`), while `FooIter` is an iterator over `Foo`
from an `.iter()` or `.into_iter()` method (e.g. `memchr` `OneIter`,
`regex` `SetMatchesIter`). Rename `IterRange`, `IterRangeInclusive`, and
`IterRangeFrom` to `RangeIter`, `RangeInclusiveIter`, and
`RangeInclusiveIter` to match this.
Tracking issue: RUST-125687 (`new_range_api`)
Similar to how this helps in `slice::Iter::position`, LLVM sometimes loses track of how high this can get, so for `TrustedLen` iterators tell it what the upper bound is.
Fix issue with callsite inline attribute not being applied sometimes.
If the calling function had more target features enabled than the callee than the attribute wasn't being applied as the arguments for the check had been swapped round. Also includes target features that are part of the global set as the warning was checking those but when adding the attribute they were not checked.
Add a codegen-llvm test to check that the attribute is actually applied as previously only the warning was being checked.
Tracking issue: rust-lang/rust#145574
Forbid `CHECK: br` and `CHECK-NOT: br` in codegen tests (suggest `br {{.*}}` instead)
`// CHECK-NOT: br` is fragile to false positives in mangled symbol names, while `// CHECK-NOT: br {{.*}}` is not. Remove and forbid the former in codegen tests, and suggest the latter.
cc https://github.com/rust-lang/rust/pull/149125#issuecomment-3560003847 where this caused a CI failure due to a v0 mangled symbol containing `br` in a disambiguator/crate hash/something.
If the calling function had more target features enabled than the
callee than the attribute wasn't being applied as the arguments for
the check had been swapped round. Also includes target features that
are part of the global set as the warning was checking those but when
adding the attribute they were not checked.
Add a codegen-llvm test to check that the attribute is actually
applied as previously only the warning was being checked.
Fix MaybeUninit codegen using GVN
This is an alternative to https://github.com/rust-lang/rust/pull/142837, based on https://github.com/rust-lang/rust/pull/146355#discussion_r2421651968.
The general approach I took here is to aggressively propagate anything that is entirely uninitialized. GVN generally takes the approach of only synthesizing small types, but we need to generate large consts to fix the codegen issue.
I also added a special case to MIR dumps for this where now an entirely uninit const is printed as `const <uninit>`, because otherwise we end up with extremely verbose dumps of the new consts.
After GVN though, we still end up with a lot of MIR that looks like this:
```
StorageLive(_1);
_1 = const <uninit>;
_2 = &raw mut _1;
```
Which will break tests/codegen-llvm/maybeuninit-rvo.rs with the naive lowering. I think the ideal fix here is to somehow omit these `_1 = const <uninit>` assignments that come directly after a StorageLive, but I'm not sure how to do that. For now at least, ignoring such assignments (even if they don't come right after a StorageLive) in codegen seems to work.
Note that since GVN is based on synthesizing a `ConstValue` which has a defined layout, this scenario still gets deoptimized by LLVM.
```rust
#![feature(rustc_attrs)]
#![crate_type = "lib"]
use std::mem::MaybeUninit;
#[unsafe(no_mangle)]
pub fn oof() -> [[MaybeUninit<u8>; 8]; 8] {
#[rustc_no_mir_inline]
pub fn inner<T: Copy>() -> [[MaybeUninit<T>; 8]; 8] {
[[MaybeUninit::uninit(); 8]; 8]
}
inner()
}
```
This case can be handled correctly if enough inlining has happened, or it could be handled by post-mono GVN. Synthesizing `UnevaluatedConst` or some other special kind of const seems dubious.
Turn moves into copies after copy propagation
Previously copy propagation presumed that there is further unspecified distinction between move operands and copy operands in assignments and propagated moves from assignments into terminators. This is inconsistent with current operational semantics.
Turn moves into copies after copy propagation to preserve existing behavior.
Fixes https://github.com/rust-lang/rust/issues/137936.
Fixes https://github.com/rust-lang/rust/issues/146423.
r? `@cjgillot`
Previously copy propagation presumed that there is further unspecified
distinction between move operands and copy operands in assignments and
propagated moves from assignments into terminators. This is inconsistent
with current operational semantics.
Turn moves into copies after copy propagation to preserve existing behavior.
Fixes https://github.com/rust-lang/rust/issues/137936.
Fixes https://github.com/rust-lang/rust/issues/146423.
Rust's current mangling scheme depends on compiler internals; loses
information about generic parameters (and other things) which makes for
a worse experience when using external tools that need to interact with
Rust symbol names; is inconsistent; and can contain `.` characters
which aren't universally supported. Therefore, Rust has defined its own
symbol mangling scheme which is defined in terms of the Rust language,
not the compiler implementation; encodes information about generic
parameters in a reversible way; has a consistent definition; and
generates symbols that only use the characters `A-Z`, `a-z`, `0-9`, and
`_`.
Support for the new Rust symbol mangling scheme has been added to
upstream tools that will need to interact with Rust symbols (e.g.
debuggers).
This commit changes the default symbol mangling scheme from the legacy
scheme to the new Rust mangling scheme.
Signed-off-by: David Wood <david.wood@huawei.com>
use funnel shift as fallback impl for rotating shifts
That lets us remove this gnarly implementation from Miri and const-eval.
However, `rotate_left`/`rotate_right` are stable as const fn, so to do this we have to `rustc_allow_const_fn_unstable` a bunch of const trait stuff. Is that a bad idea? Cc `@oli-obk` `@fee1-dead`
The overflow-checks codegen test was failing on riscv64gc target
because the FileCheck pattern did not account for ABI extension
attributes. RISC-V LP64 ABI requires integer types smaller than
XLEN (64-bit) to be zero-extended or sign-extended to register width.
For u8 parameters, RISC-V generates:
i8 noundef zeroext %a, i8 noundef zeroext %b
While x86_64 and aarch64 generate:
i8 noundef %a, i8 noundef %b
The original CHECK pattern only matched the format without the
`zeroext` attribute, causing test failures on RISC-V.
This patch makes the zeroext attribute optional in the FileCheck
pattern using `{{( zeroext)?}}`, allowing the test to pass on
architectures that add ABI extension attributes (e.g., RISC-V).
Test results before fix:
- x86_64-unknown-linux-gnu: 3 passed
- aarch64-unknown-linux-gnu: 3 passed
- riscv64gc-unknown-linux-gnu: 1 passed, 2 failed
Test results after fix:
- x86_64-unknown-linux-gnu: 3 passed
- aarch64-unknown-linux-gnu: 3 passed
- riscv64gc-unknown-linux-gnu: 3 passed
Add `overflow_checks` intrinsic
This adds an intrinsic which allows code in a pre-built library to inherit the overflow checks option from a crate depending on it. This enables code in the standard library to explicitly change behavior based on whether `overflow_checks` are enabled, regardless of the setting used when standard library was compiled.
This is very similar to the `ub_checks` intrinsic, and refactors the two to use a common mechanism.
The primary use case for this is to allow the new `RangeFrom` iterator to yield the maximum element before overflowing, as requested [here](https://github.com/rust-lang/rust/issues/125687#issuecomment-2151118208). This PR includes a working `IterRangeFrom` implementation based on this new intrinsic that exhibits the desired behavior.
[Prior discussion on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Ability.20to.20select.20code.20based.20on.20.60overflow_checks.60.3F)
This implements a new unstable compiler flag `-Zannotate-moves` that makes
move and copy operations visible in profilers by creating synthetic debug
information. This is achieved with zero runtime cost by manipulating debug
info scopes to make moves/copies appear as calls to `compiler_move<T, SIZE>`
and `compiler_copy<T, SIZE>` marker functions in profiling tools.
This allows developers to identify expensive move/copy operations in their
code using standard profiling tools, without requiring specialized tooling
or runtime instrumentation.
The implementation works at codegen time. When processing MIR operands
(`Operand::Move` and `Operand::Copy`), the codegen creates an `OperandRef`
with an optional `move_annotation` field containing an `Instance` of the
appropriate profiling marker function. When storing the operand,
`store_with_annotation()` wraps the store operation in a synthetic debug
scope that makes it appear inlined from the marker.
Two marker functions (`compiler_move` and `compiler_copy`) are defined
in `library/core/src/profiling.rs`. These are never actually called -
they exist solely as debug info anchors.
Operations are only annotated if the type:
- Meets the size threshold (default: 65 bytes, configurable via
`-Zannotate-moves=SIZE`)
- Has a non-scalar backend representation (scalars use registers,
not memcpy)
This has a very small size impact on object file size. With the default
limit it's well under 0.1%, and even with a very small limit of 8 bytes
it's still ~1.5%. This could be enabled by default.
Add alignment parameter to `simd_masked_{load,store}`
This PR adds an alignment parameter in `simd_masked_load` and `simd_masked_store`, in the form of a const-generic enum `core::intrinsics::simd::SimdAlign`. This represents the alignment of the `ptr` argument in these intrinsics as follows
- `SimdAlign::Unaligned` - `ptr` is unaligned/1-byte aligned
- `SimdAlign::Element` - `ptr` is aligned to the element type of the SIMD vector (default behavior in the old signature)
- `SimdAlign::Vector` - `ptr` is aligned to the SIMD vector type
The main motive for this is stdarch - most vector loads are either fully aligned (to the vector size) or unaligned (byte-aligned), so the previous signature doesn't cut it.
Now, stdarch will mostly use `SimdAlign::Unaligned` and `SimdAlign::Vector`, whereas portable-simd will use `SimdAlign::Element`.
- [x] `cg_llvm`
- [x] `cg_clif`
- [x] `miri`/`const_eval`
## Alternatives
Using a const-generic/"const" `u32` parameter as alignment (and we error during codegen if this argument is not a power of two). This, although more flexible than this, has a few drawbacks
- If we use an const-generic argument, then portable-simd somehow needs to pass `align_of::<T>()` as the alignment, which isn't possible without GCE
- "const" function parameters are just an ugly hack, and a pain to deal with in non-LLVM backends
We can remedy the problem with the const-generic `u32` parameter by adding a special rule for the element alignment case (e.g. `0` can mean "use the alignment of the element type), but I feel like this is not as expressive as the enum approach, although I am open to suggestions
cc `@workingjubilee` `@RalfJung` `@BoxyUwU`
Chris Denton