Renumber return local after state transform
The current implementation of `StateTransform` renames `_0` before analyzing liveness. This is inconsistent, as a `return` terminator hardcodes a read of `_0`.
This PR proposes to perform such rename *after* analyzing the body, in fact after the whole transform. The implementation is not much more complicated.
Validate CopyForDeref and DerefTemps better and remove them from runtime MIR
(split from my WIP rust-lang/rust#145344)
This PR:
- Removes `Rvalue::CopyForDeref` and `LocalInfo::DerefTemp` from runtime MIR
- Using a new mir pass `EraseDerefTemps`
- `CopyForDeref(x)` is turned into `Use(Copy(x))`
- `DerefTemp` is turned into `Boring`
- Not sure if this part is actually necessary, it made more sense in rust-lang/rust#145344 with `DerefTemp` storing actual data that I wanted to keep from having to be kept in sync with the rest of the body in runtime MIR
- Checks in validation that `CopyForDeref` and `DerefTemp` are only used together
- Removes special handling for `CopyForDeref` from many places
- Removes `CopyForDeref` from `custom_mir` reverting rust-lang/rust#111587
- In runtime MIR simple copies can be used instead
- In post cleanup analysis MIR it was already wrong to use due to the lack of support for creating `DerefTemp` locals
- Possibly this should be its own PR?
- Adds an argument to `deref_finder` to avoid creating new `DerefTemp`s and `CopyForDeref` in runtime MIR.
- Ideally we would just avoid making intermediate derefs instead of fixing it at the end of a pass / during shim building
- Removes some usages of `deref_finder` that I found out don't actually do anything
r? oli-obk
Remove StatementKind::Deinit.
It is a remnant from the time we deaggreated MIR.
Now, it is only constructed by the `LargeEnums` MIR pass, which is disabled by default.
Regression test for const promotion with Option<Ordering>
https://rust.godbolt.org/z/EjxqE8WcTFixesrust-lang/rust#139093
Add a regression test to ensure that comparing `Option<Ordering>` to
`Some(Ordering::Equal)` does not trigger unnecessary const promotion
in MIR.
Previously, inlined constants like `Some(Ordering::Equal)` would get
promoted, leading to more complex MIR and redundant LLVM IR checks.
This test verifies that both the direct form and the `let`-binding form
now generate equivalent, simplified MIR.
r? cjgillot
Prevent downstream `impl DerefMut for Pin<LocalType>`
The safety requirements for [`PinCoerceUnsized`](https://doc.rust-lang.org/stable/std/pin/trait.PinCoerceUnsized.html) are essentially that the type does not have a malicious `Deref` or `DerefMut` impl. However, the `Pin` type is fundamental, so the end-user can provide their own implementation of `DerefMut` for `Pin<&SomeLocalType>`, so it's possible for `Pin` to have a malicious `DerefMut` impl. This unsoundness is known as rust-lang/rust#85099.
Unfortunately, this means that the implementation of `PinCoerceUnsized` for `Pin` is currently unsound. To fix that, modify the impl so that it becomes impossible for downstream crates to provide their own implementation of `DerefMut` for `Pin` by abusing a hidden struct that is not fundamental.
This PR is a breaking change, but it fixesrust-lang/rust#85099. The PR supersedes rust-lang/rust#144896.
r? lcnr
Introduce debuginfo to statements in MIR
The PR introduces support for debug information within dead statements. Currently, only the reference statement is supported, which is sufficient to fix rust-lang/rust#128081.
I don't modify Stable MIR, as I don't think we need debug information when using it.
This PR represents the debug information for the dead reference statement via `#dbg_value`. For example, `let _foo_b = &foo.b` becomes `#dbg_value(ptr %foo, !22, !DIExpression(DW_OP_plus_uconst, 4, DW_OP_stack_value), !26)`. You can see this here: https://rust.godbolt.org/z/d43js6adv.
The general principle for handling debug information is to never provide less debug information than the optimized LLVM IR.
The current rules for dropping debug information in this PR are:
- If the LLVM IR cannot represent a reference address, it's replaced with poison or simply dropped. For example, see: https://rust.godbolt.org/z/shGqPec8W. I'm using poison in all such cases now.
- All debuginfos is dropped when merging multiple successor BBs. An example is available here: https://rust.godbolt.org/z/TE1q3Wq6M.
I doesn't drop debuginfos in `MatchBranchSimplification`, because LLVM also pick one branch for it.
Extending `#[rustc_force_inline]` to be applicable to inherent methods
`#[rustc_force_inline]` is an internal-only attribute similar to `#[inline(always)]` but which emits an error if inlining cannot occur. rustc_force_inline uses the MIR inliner to do this and has limitations on where it can be applied to ensure that an error is always emitted if inlining can't happen (e.g. it can't be applied to trait methods because calls to those can't always be resolved).
`#[rustc_force_inline]` is motivated by AArch64 pointer authentication intrinsics where it is vital for the security properties of these intrinsics that they do not exist in standalone functions that could be used as gadgets in an exploit (if they could, then you could sign whatever pointers you want, for example, which is bad, but if you force inlining, then you can't jump to a reusable function containing only these instructions).
Since its initial implementation, `#[rustc_force_inline]` could only be applied to free functions. This can be relaxed to also allow inherent methods while still preserving the desired properties. In a work-in-progress patch for manual pointer authentication intrinsics, it is useful to introduce types with inherent methods that would need to be force inlined.
r? `@saethlin`
Enable DestinationPropagation by default
This PR proposes to perform destination propagation on MIR. Most of the pass was fully rewritten by `@JakobDegen` in rust-lang/rust#96451.
This pass is quite heavy, as it needs to perform and save the results of a full liveness dataflow analysis. This accounts for ~50% of the pass' runtime.
Perf sees a few decent savings in later llvm passes, but also sizeable régressions when there are no savings to balance this pass' runtime.
Remove Rvalue::Len again.
Now that we have `RawPtrKind::FakeForPtrMetadata`, we can reimplement `Rvalue::Len` using `PtrMetadata(&raw const (fake) place)`.
r? ``@scottmcm``
Reimplement DestinationPropagation according to live ranges.
This PR reimplements DestinationPropagation as a problem of merging live-ranges of locals. We merge locals that have disjoint live-ranges. This allows merging several locals in the same round by updating live range information.
Live ranges are mainly computed using the `MaybeLiveLocals` analysis. The subtlety is that we split each statement and terminator in 2 positions. The first position is the regular statement. The second position is a shadow, which is always more live. It encodes partial writes and dead writes as a local being live for half a statement. This half statement ensures that writes conflict with another local's writes and regular liveness.
r? `@Amanieu`
dianqk