In its first step of computing transmutability, `rustc_transmutability`
constructs a byte-level representation of type layout (`Tree`). Previously, this
representation was computed for ADTs by inspecting the ADT definition and
performing our own layout computations. This process was error-prone, verbose,
and limited our ability to analyze many types (particularly default-repr types).
In this PR, we instead construct `Tree`s from `rustc_target::abi::Layout`s. This
helps ensure that layout optimizations are reflected our analyses, and increases
the kinds of types we can now analyze, including:
- default repr ADTs
- transparent unions
- `UnsafeCell`-containing types
Overall, this PR expands the expressvity of `rustc_transmutability` to be much
closer to the transmutability analysis performed by miri. Future PRs will work
to close the remaining gaps (e.g., support for `Box`, raw pointers, `NonZero*`,
coroutines, etc.).
We can (and will) support analyzing the transmutability of types
whose layouts aren't completely specified by its repr. This change
ensures that the error messages remain sensible after this support
lands.
- Make sure that the most specific Reason is the one that bubbles up when we
are folding over the `Answer` tree. `Reason::DstIsBitIncompatible` is the
least specific, so that should be used only when there isn't anything else
available.
- Small fixes where we used the wrong Reason variant.
- Tiny cleanups
This patch enables support for references in Safe Transmute, by generating
nested obligations during trait selection. Specifically, when we call
`confirm_transmutability_candidate(...)`, we now recursively traverse the
`rustc_transmute::Answer` tree and create obligations for all the `Answer`
variants, some of which include multiple nested `Answer`s.
Also, to handle recursive types, enable support for coinduction for the Safe
Transmute trait (`BikeshedIntrinsicFrom`) by adding the `#[rustc_coinduction]`
annotation.
Also fix some small logic issues when reducing the `or` and `and` combinations
in `rustc_transmute`, so that we don't end up with additional redundant
`Answer`s in the tree.
Co-authored-by: Jack Wrenn <jack@wrenn.fyi>
This patch updates the error reporting when Safe Transmute is not
possible between 2 types by including the reason.
Also, fix some small bugs that occur when computing the `Answer` for
transmutability.
Jack Wrenn