Introduces `BackendRepr::ScalableVector` corresponding to scalable
vector types annotated with `repr(scalable)` which lowers to a scalable
vector type in LLVM.
Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
misc coercion cleanups and handle safety correctly
r? lcnr
### "remove normalize call"
Fixesrust-lang/rust#132765
If the normalization fails we would sometimes get a `TypeError` containing inference variables created inside of the probe used by coercion. These would then get leaked out causing ICEs in diagnostics logic
### "leak check and lub for closure<->closure coerce-lubs of same defids"
Fixes https://github.com/rust-lang/trait-system-refactor-initiative/issues/233
```rust
fn peculiar() -> impl Fn(u8) -> u8 {
return |x| x + 1
}
```
the `|x| x + 1` expr has a type of `Closure(?31t)` which we wind up inferring the RPIT to. The `CoerceMany` `ret_coercion` for the whole `peculiar` typeck has an expected type of `RPIT` (unnormalized). When we type check the `return |x| x + 1` expr we go from the never type to `Closure(?31t)` which then participates in the `ret_coercion` giving us a `coerce-lub(RPIT, Closure(?31t))`.
Normalizing `RPIT` gives us some `Closure(?50t)` where `?31t` and `?50t` have been unified with `?31t` as the root var. `resolve_vars_if_possible` doesn't resolve infer vars to their roots so these wind up with different structural identities so the fast path doesn't apply and we fall back to coercing to a `fn` ptr. cc rust-lang/rust#147193 which also fixes this
New solver probably just gets more inference variables here because canonicalization + generally different approach to normalization of opaques. Idk :3
### FCP worthy stuffy
there are some other FCP worthy things but they're in my FCP comment which also contains some analysis of the breaking nature of the previously listed changes in this PR: https://github.com/rust-lang/rust/pull/148602#issuecomment-3503497467
Compute jump threading opportunities in a single pass
The current implementation of jump threading walks MIR CFG backwards from each `SwitchInt` terminator. This PR replaces this by a single postorder traversal of MIR. In theory, we could do a full fixpoint dataflow analysis, but this has low returns as we forbid threading through a loop header.
The second commit in this PR modifies the carried state to a lighter data structure. The current implementation uses some kind of `IndexVec<ValueIndex, &[Condition]>`. This is needlessly heavy, as the state rarely ever carries more than a few `Condition`s. The first commit replaces this state with a simpler `&[Condition]`, and puts the corresponding `ValueIndex` inside `Condition`.
The three later commits are perf tweaks.
The sixth commit is the main change. Instead of carrying the goto target inside the condition, we maintain a set of conditions associated with each block, and their consequences in following blocks. Think: if this condition is fulfilled in this block, then that condition is fulfilled in that block. This makes the threading algorithm much easier to implement, without the extra bookkeeping of `ThreadingOpportunity` we had.
Later commits modify that algorithm to shrink the set of duplicated blocks. By propagating fulfilled conditions down the CFG, and trimming costly threads.
Fix MaybeUninit codegen using GVN
This is an alternative to https://github.com/rust-lang/rust/pull/142837, based on https://github.com/rust-lang/rust/pull/146355#discussion_r2421651968.
The general approach I took here is to aggressively propagate anything that is entirely uninitialized. GVN generally takes the approach of only synthesizing small types, but we need to generate large consts to fix the codegen issue.
I also added a special case to MIR dumps for this where now an entirely uninit const is printed as `const <uninit>`, because otherwise we end up with extremely verbose dumps of the new consts.
After GVN though, we still end up with a lot of MIR that looks like this:
```
StorageLive(_1);
_1 = const <uninit>;
_2 = &raw mut _1;
```
Which will break tests/codegen-llvm/maybeuninit-rvo.rs with the naive lowering. I think the ideal fix here is to somehow omit these `_1 = const <uninit>` assignments that come directly after a StorageLive, but I'm not sure how to do that. For now at least, ignoring such assignments (even if they don't come right after a StorageLive) in codegen seems to work.
Note that since GVN is based on synthesizing a `ConstValue` which has a defined layout, this scenario still gets deoptimized by LLVM.
```rust
#![feature(rustc_attrs)]
#![crate_type = "lib"]
use std::mem::MaybeUninit;
#[unsafe(no_mangle)]
pub fn oof() -> [[MaybeUninit<u8>; 8]; 8] {
#[rustc_no_mir_inline]
pub fn inner<T: Copy>() -> [[MaybeUninit<T>; 8]; 8] {
[[MaybeUninit::uninit(); 8]; 8]
}
inner()
}
```
This case can be handled correctly if enough inlining has happened, or it could be handled by post-mono GVN. Synthesizing `UnevaluatedConst` or some other special kind of const seems dubious.
Turn moves into copies after copy propagation
Previously copy propagation presumed that there is further unspecified distinction between move operands and copy operands in assignments and propagated moves from assignments into terminators. This is inconsistent with current operational semantics.
Turn moves into copies after copy propagation to preserve existing behavior.
Fixes https://github.com/rust-lang/rust/issues/137936.
Fixes https://github.com/rust-lang/rust/issues/146423.
r? `@cjgillot`
Skip unused variables warning for unreachable code
Fixesrust-lang/rust#148373
These warnings are not reported on stable branch, but are now reported on the beta.
I tried another solution to record whether a `local` is reachable in `find_dead_assignments`, but the code in this PR seems simpler.
r? `@cjgillot`
Fix the issue of unused assignment from MIR liveness checking
Fixesrust-lang/rust#148960Fixesrust-lang/rust#148418
r? ``@davidtwco``
cc ``@cjgillot``
My first try on MIR related code, so it may not be the best fix.
Allow unnormalized types in drop elaboration
I work on a [rustc driver](https://github.com/AeneasVerif/charon) that aims to extract the full polymorphic MIR of a crate. Currently the one thing I can't get is drop glue because it fails on unnormalizable types like the fields of `Cow` or `NonZero`.
This PR removes the one check for unnormalized types in drop elaboration. It's a `span_delay_bug` so only there to help catch mistakes. I think that's fine to remove? If something downstream relies on types being normalized the right approach is for MIR validation to check for normalized types, not that one random check.
autodiff rlib handling
As I learned recently, we now apparently support rlib builds already in some cases.
With the last hint from saethlin this seems to now cover all cases. To be sure I'll add a few more testcases before I mark it as ready.
Once this PR lands, we should to the best of my knowledge, support autodiff in almost code locations, only vtable/dyn ptr remain unsupported for now.
r? ghost
closes: https://github.com/rust-lang/rust/issues/148856
closes: https://github.com/rust-lang/rust/issues/137520
Previously copy propagation presumed that there is further unspecified
distinction between move operands and copy operands in assignments and
propagated moves from assignments into terminators. This is inconsistent
with current operational semantics.
Turn moves into copies after copy propagation to preserve existing behavior.
Fixes https://github.com/rust-lang/rust/issues/137936.
Fixes https://github.com/rust-lang/rust/issues/146423.
This is an incremental step towards making the expansion tree central to
coverage mapping creation, which will be needed for proper expansion region
support.
Show packed field alignment in mir_transform_unaligned_packed_ref
Fixesrust-lang/rust#147528
I left the expected padding for the field out of the error message so the message would be the same on all platforms. It also isn't always possible to know the expected alignment, so this makes the message simpler.
Add `overflow_checks` intrinsic
This adds an intrinsic which allows code in a pre-built library to inherit the overflow checks option from a crate depending on it. This enables code in the standard library to explicitly change behavior based on whether `overflow_checks` are enabled, regardless of the setting used when standard library was compiled.
This is very similar to the `ub_checks` intrinsic, and refactors the two to use a common mechanism.
The primary use case for this is to allow the new `RangeFrom` iterator to yield the maximum element before overflowing, as requested [here](https://github.com/rust-lang/rust/issues/125687#issuecomment-2151118208). This PR includes a working `IterRangeFrom` implementation based on this new intrinsic that exhibits the desired behavior.
[Prior discussion on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Ability.20to.20select.20code.20based.20on.20.60overflow_checks.60.3F)
Add LLVM realtime sanitizer
This is a new attempt at adding the [LLVM real-time sanitizer](https://clang.llvm.org/docs/RealtimeSanitizer.html) to rust.
Previously this was attempted in https://github.com/rust-lang/rfcs/pull/3766.
Since then the `sanitize` attribute was introduced in https://github.com/rust-lang/rust/pull/142681 and it is a lot more flexible than the old `no_santize` attribute. This allows adding real-time sanitizer without the need for a new attribute, like it was proposed in the RFC. Because i only add a new value to a existing command line flag and to a attribute i don't think an MCP is necessary.
Currently real-time santizer is usable in rust code with the [rtsan-standalone](https://crates.io/crates/rtsan-standalone) crate. This downloads or builds the sanitizer runtime and then links it into the rust binary.
The first commit adds support for more detailed sanitizer information.
The second commit then actually adds real-time sanitizer.
The third adds a warning against using real-time sanitizer with async functions, cloures and blocks because it doesn't behave as expected when used with async functions. I am not sure if this is actually wanted, so i kept it in a seperate commit.
The fourth commit adds the documentation for real-time sanitizer.
Liveness: Cache the set of string constants for suggestions.
Even on a slow diagnostic path, listing all the constants in MIR can be expensive, so cache it.
David Wood