Replace Rvalue::NullaryOp by a variant in mir::Operand.
Based on https://github.com/rust-lang/rust/pull/148151
This PR fully removes the MIR `Rvalue::NullaryOp`. After rust-lang/rust#148151, it was only useful for runtime checks like `ub_checks`, `contract_checks` and `overflow_checks`.
These are "runtime" checks, boolean constants that may only be `true` in codegen. It depends on a rustc flag passed to codegen, so we need to represent those flags cross-crate.
This PR replaces those runtime checks by special variants in MIR `ConstValue`. This allows code that expects constants to manipulate those as such, even if we may not always be able to evaluate them to actual scalars.
Introduces `BackendRepr::ScalableVector` corresponding to scalable
vector types annotated with `repr(scalable)` which lowers to a scalable
vector type in LLVM.
Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
misc coercion cleanups and handle safety correctly
r? lcnr
### "remove normalize call"
Fixesrust-lang/rust#132765
If the normalization fails we would sometimes get a `TypeError` containing inference variables created inside of the probe used by coercion. These would then get leaked out causing ICEs in diagnostics logic
### "leak check and lub for closure<->closure coerce-lubs of same defids"
Fixes https://github.com/rust-lang/trait-system-refactor-initiative/issues/233
```rust
fn peculiar() -> impl Fn(u8) -> u8 {
return |x| x + 1
}
```
the `|x| x + 1` expr has a type of `Closure(?31t)` which we wind up inferring the RPIT to. The `CoerceMany` `ret_coercion` for the whole `peculiar` typeck has an expected type of `RPIT` (unnormalized). When we type check the `return |x| x + 1` expr we go from the never type to `Closure(?31t)` which then participates in the `ret_coercion` giving us a `coerce-lub(RPIT, Closure(?31t))`.
Normalizing `RPIT` gives us some `Closure(?50t)` where `?31t` and `?50t` have been unified with `?31t` as the root var. `resolve_vars_if_possible` doesn't resolve infer vars to their roots so these wind up with different structural identities so the fast path doesn't apply and we fall back to coercing to a `fn` ptr. cc rust-lang/rust#147193 which also fixes this
New solver probably just gets more inference variables here because canonicalization + generally different approach to normalization of opaques. Idk :3
### FCP worthy stuffy
there are some other FCP worthy things but they're in my FCP comment which also contains some analysis of the breaking nature of the previously listed changes in this PR: https://github.com/rust-lang/rust/pull/148602#issuecomment-3503497467
Compute jump threading opportunities in a single pass
The current implementation of jump threading walks MIR CFG backwards from each `SwitchInt` terminator. This PR replaces this by a single postorder traversal of MIR. In theory, we could do a full fixpoint dataflow analysis, but this has low returns as we forbid threading through a loop header.
The second commit in this PR modifies the carried state to a lighter data structure. The current implementation uses some kind of `IndexVec<ValueIndex, &[Condition]>`. This is needlessly heavy, as the state rarely ever carries more than a few `Condition`s. The first commit replaces this state with a simpler `&[Condition]`, and puts the corresponding `ValueIndex` inside `Condition`.
The three later commits are perf tweaks.
The sixth commit is the main change. Instead of carrying the goto target inside the condition, we maintain a set of conditions associated with each block, and their consequences in following blocks. Think: if this condition is fulfilled in this block, then that condition is fulfilled in that block. This makes the threading algorithm much easier to implement, without the extra bookkeeping of `ThreadingOpportunity` we had.
Later commits modify that algorithm to shrink the set of duplicated blocks. By propagating fulfilled conditions down the CFG, and trimming costly threads.
Fix MaybeUninit codegen using GVN
This is an alternative to https://github.com/rust-lang/rust/pull/142837, based on https://github.com/rust-lang/rust/pull/146355#discussion_r2421651968.
The general approach I took here is to aggressively propagate anything that is entirely uninitialized. GVN generally takes the approach of only synthesizing small types, but we need to generate large consts to fix the codegen issue.
I also added a special case to MIR dumps for this where now an entirely uninit const is printed as `const <uninit>`, because otherwise we end up with extremely verbose dumps of the new consts.
After GVN though, we still end up with a lot of MIR that looks like this:
```
StorageLive(_1);
_1 = const <uninit>;
_2 = &raw mut _1;
```
Which will break tests/codegen-llvm/maybeuninit-rvo.rs with the naive lowering. I think the ideal fix here is to somehow omit these `_1 = const <uninit>` assignments that come directly after a StorageLive, but I'm not sure how to do that. For now at least, ignoring such assignments (even if they don't come right after a StorageLive) in codegen seems to work.
Note that since GVN is based on synthesizing a `ConstValue` which has a defined layout, this scenario still gets deoptimized by LLVM.
```rust
#![feature(rustc_attrs)]
#![crate_type = "lib"]
use std::mem::MaybeUninit;
#[unsafe(no_mangle)]
pub fn oof() -> [[MaybeUninit<u8>; 8]; 8] {
#[rustc_no_mir_inline]
pub fn inner<T: Copy>() -> [[MaybeUninit<T>; 8]; 8] {
[[MaybeUninit::uninit(); 8]; 8]
}
inner()
}
```
This case can be handled correctly if enough inlining has happened, or it could be handled by post-mono GVN. Synthesizing `UnevaluatedConst` or some other special kind of const seems dubious.
Turn moves into copies after copy propagation
Previously copy propagation presumed that there is further unspecified distinction between move operands and copy operands in assignments and propagated moves from assignments into terminators. This is inconsistent with current operational semantics.
Turn moves into copies after copy propagation to preserve existing behavior.
Fixes https://github.com/rust-lang/rust/issues/137936.
Fixes https://github.com/rust-lang/rust/issues/146423.
r? `@cjgillot`
Skip unused variables warning for unreachable code
Fixesrust-lang/rust#148373
These warnings are not reported on stable branch, but are now reported on the beta.
I tried another solution to record whether a `local` is reachable in `find_dead_assignments`, but the code in this PR seems simpler.
r? `@cjgillot`
Fix the issue of unused assignment from MIR liveness checking
Fixesrust-lang/rust#148960Fixesrust-lang/rust#148418
r? ``@davidtwco``
cc ``@cjgillot``
My first try on MIR related code, so it may not be the best fix.
Allow unnormalized types in drop elaboration
I work on a [rustc driver](https://github.com/AeneasVerif/charon) that aims to extract the full polymorphic MIR of a crate. Currently the one thing I can't get is drop glue because it fails on unnormalizable types like the fields of `Cow` or `NonZero`.
This PR removes the one check for unnormalized types in drop elaboration. It's a `span_delay_bug` so only there to help catch mistakes. I think that's fine to remove? If something downstream relies on types being normalized the right approach is for MIR validation to check for normalized types, not that one random check.
autodiff rlib handling
As I learned recently, we now apparently support rlib builds already in some cases.
With the last hint from saethlin this seems to now cover all cases. To be sure I'll add a few more testcases before I mark it as ready.
Once this PR lands, we should to the best of my knowledge, support autodiff in almost code locations, only vtable/dyn ptr remain unsupported for now.
r? ghost
closes: https://github.com/rust-lang/rust/issues/148856
closes: https://github.com/rust-lang/rust/issues/137520
Previously copy propagation presumed that there is further unspecified
distinction between move operands and copy operands in assignments and
propagated moves from assignments into terminators. This is inconsistent
with current operational semantics.
Turn moves into copies after copy propagation to preserve existing behavior.
Fixes https://github.com/rust-lang/rust/issues/137936.
Fixes https://github.com/rust-lang/rust/issues/146423.
This is an incremental step towards making the expansion tree central to
coverage mapping creation, which will be needed for proper expansion region
support.
Show packed field alignment in mir_transform_unaligned_packed_ref
Fixesrust-lang/rust#147528
I left the expected padding for the field out of the error message so the message would be the same on all platforms. It also isn't always possible to know the expected alignment, so this makes the message simpler.
bors