Fix ICE: Scalar layout for non-primitive non-enum type unsafe binder
`UnsafeBinder` uses the inner layout, but the debug layout check still looked at the outer type. Check the inner type first so this does not ICE.
Tracking issue: rust-lang/rust#130516Closes: rust-lang/rust#154426
Change `ItemKind::Trait` to a field variant.
This changes `ItemKind::Trait` from an octuple(!!) to an enum variant with fields. Their names were chosen to match up with existing usage and minimize renaming.
I'm leaning towards renaming `ident` to `name` as well; let me know if that's desired.
Refactor FnDecl and FnSig non-type fields into a new wrapper type
#### Why this Refactor?
This PR is part of an initial cleanup for the [arg splat experiment](https://github.com/rust-lang/rust/issues/153629), but it's a useful refactor by itself.
It refactors the non-type fields of `FnDecl`, `FnSig`, and `FnHeader` into a new packed wrapper types, based on this comment in the `splat` experiment PR:
https://github.com/rust-lang/rust/pull/153697#discussion_r3004637413
It also refactors some common `FnSig` creation settings into their own methods. I did this instead of creating a struct with defaults.
#### Relationship to `splat` Experiment
I don't think we can use functional struct updates (`..default()`) to create `FnDecl` and `FnSig`, because we need the bit-packing for the `splat` experiment.
Bit-packing will avoid breaking "type is small" assertions for commonly used types when `splat` is added.
This PR packs these types:
- ExternAbi: enum + `unwind` variants (38) -> 6 bits
- ImplicitSelfKind: enum variants (5) -> 3 bits
- lifetime_elision_allowed, safety, c_variadic: bool -> 1 bit
#### Minor Changes
Fixes some typos, and applies rustfmt to clippy files that got skipped somehow.
Instead of just using regular struct lowering for these types, which
results in an incorrect ABI (e.g. returning indirectly), use
`BackendRepr::ScalableVector` which will lower to the correct type and
be passed in registers.
This also enables some simplifications for generating alloca of scalable
vectors and greater re-use of `scalable_vector_parts`.
A LLVM codegen test demonstrating the changed IR this generates is
included in the next commit alongside some intrinsics that make these
tuples usable.
The existing names have bugged me for a while. Changes:
- `CycleError` -> `Cycle`. Because we normally use "error" for a `Diag`
with `Level::Error`, and this type is just a precursor to that. Also,
many existing locals of this type are already named `cycle`.
- `CycleError::cycle` -> `Cycle::frames`. Because it is a sequence of
frames, and we want to avoid `Cycle::cycle` (and `cycle.cycle`).
- `cycle_error` -> `find_and_handle_cycle`. Because that's what it does.
The existing name is just a non-descript noun phrase.
- `mk_cycle` -> `handle_cycle`. Because it doesn't make the cycle; the
cycle is already made. It handles the cycle, which involves (a)
creating the error, and (b) handling the error.
- `report_cycle` -> `create_cycle_error`. Because that's what it does:
creates the cycle error (i.e. the `Diag`).
- `value_from_cycle_error` -> `handle_cycle_error_fn`. Because most
cases don't produce a value, they just emit an error and quit.
And the `_fn` suffix is for consistency with other vtable fns.
- `from_cycle_error` -> `handle_cycle_error`. A similar story for this
module.
Overhaul `ensure_ok`
The interaction of `ensure_ok` and the `return_result_from_ensure_ok` query modifier is weird and hacky. This PR cleans it up. Details in the individual commits.
r? @Zalathar
`ensure_ok` provides a special, more efficient way of calling a query
when its return value isn't needed. But there is a complication: if the
query is marked with the `return_result_from_ensure_ok` modifier, then
it will return `Result<(), ErrorGuaranteed`. This is clunky and feels
tacked on. It's annoying to have to add a modifier to a query to declare
information present in its return type, and it's confusing that queries
called via `ensure_ok` have different return types depending on the
modifier.
This commit:
- Eliminates the `return_result_from_ensure_ok` modifier. The proc macro
now looks at the return type and detects if it matches `Result<_,
ErrorGuarantee>`. If so, it adds the modifier
`returns_error_guaranteed`. (Aside: We need better terminology to
distinguish modifiers written by the user in a `query` declaration
(e.g. `cycle_delayed_bug`) from modifiers added by the proc macro
(e.g. `cycle_error_handling`.))
- Introduces `ensure_result`, which replaces the use of `ensure_ok` for
queries that return `Result<_, ErrorGuarantee>`. As a result,
`ensure_ok` can now only be used for the "ignore the return value"
case.
Instead of defaulting to `None` it now defaults to `Align::ONE` i.e.
no alignment restriction. Codegen test changes are due to us now skipping
`align 1` annotations (they are useless; not skipping them makes all the
raw pointers gain an `align 1` annotation which doesn't seem any good)
Make `size`/`align` always correct rather than conditionally on the
`safe` field. This makes it less error prone and easier to work with for
`MaybeDangling` / potential future pointer kinds like `Aligned<_>`.
As a performance optimization, we altogether avoid ever invoking the
query that computes an Instance fn's fully optimized ABI in cases where
the build will not actually deduce parameter attributes.
`fn_abi_of_instance` can look at function bodies to deduce codegen
optimization attributes (namely `ArgAttribute::ReadOnly` and
`ArgAttribute::CapturesNone`) of indirectly-passed parameters. This can
lead to cycles when performed during typeck, when such attributes are
irrelevant.
This patch breaks a subquery out from `fn_abi_of_instance`,
`fn_abi_of_instance_no_deduced_attrs`, which returns the ABI before such
parameter attributes are deduced; and that new subquery is used in CTFE
instead.
This variant was a fallback value used to continue analysis after a
`Representability` error, but it's no longer needed thanks to the
previous commit. This means `Representability` can now be reduced to a
unit type that exists just so `FromCycleError` can exist for the
`representability` query. (There is potential for additional cleanups
here, but those are beyond the scope of this PR.)
This means the `representability`/`representability_adt_ty` queries no
longer have a meaningful return value, i.e. they are check-only queries.
So they now have a `check_` prefix added. And the `rtry!` macro is no
longer needed.
perf(codegen): Restore `noundef` On `PassMode::Cast` Args In Rust ABI
### Summary:
#### Problem:
Small aggregate arguments passed via `PassMode::Cast` in the Rust ABI (e.g. `[u32; 2]` cast to `i64`) are missing `noundef` in the emitted LLVM IR, even when the type contains no uninit bytes:
```rust
#[no_mangle]
pub fn f(v: [u32; 2]) -> u32 { v[0] }
```
```llvm
; expected: define i32 @f(i64 noundef %0)
; actual: define i32 @f(i64 %0) ← noundef missing
```
This blocks LLVM from applying optimizations that require value-defined semantics on function arguments.
#### Root Cause:
`adjust_for_rust_abi` calls `arg.cast_to(Reg::Integer)`, which internally creates a `CastTarget` with `ArgAttributes::new()` — always empty. Any validity attribute that was present before the cast is silently dropped.
This affects all `PassMode::Cast` arguments and return values in the Rust ABI: plain arrays, newtype wrappers, and any `BackendRepr::Memory` type small enough to fit in a register.
A prior attempt (rust-lang/rust#127210) used `Ty`/`repr` attributes to detect padding.
#### Solution:
After `adjust_for_rust_abi`, iterate all `PassMode::Cast` args and the return value. For each, call `layout_is_noundef` on the original layout; if it returns `true`, set `NoUndef` on the `CastTarget`'s `attrs`.
`layout_is_noundef` uses only the computed layout — `BackendRepr`, `FieldsShape`, `Variants`, `Scalar::is_uninit_valid()` — and never touches `Ty` or repr attributes. **Anything it cannot prove returns `false`.**
Covered cases:
- `Scalar` / `ScalarPair` (both halves initialized, fields contiguous)
- `FieldsShape::Array` (element type recursively uninit-free)
- `FieldsShape::Arbitrary` with `Variants::Single` (fields cover `0..size` with no gaps, each recursively uninit-free) — handles newtype wrappers, multi-field structs, single-variant enums, `repr(transparent)`, `repr(C)` wrappers
Conservatively excluded with FIXMEs:
- Multi-variant enums (per-variant padding analysis needed)
- Foreign-ABI casts (cast target may exceed layout size, needs a size guard)
### Changes:
- `compiler/rustc_ty_utils/src/abi.rs`: add restoration loop after `adjust_for_rust_abi`; add `layout_is_noundef` and `fields_cover_layout`.
- `tests/codegen-llvm/abi-noundef-cast.rs`: new FileCheck test covering arrays, newtype wrappers (`repr(Rust)`, `repr(transparent)`, `repr(C)`), multi-field structs, single-variant enums, return values, and negative cases (`MaybeUninit`, struct with trailing padding).
- `tests/codegen-llvm/debuginfo-dse.rs`: update one CHECK pattern — `Aggregate_4xi8` (`struct { i8, i8, i8, i8 }`) now correctly gets `noundef`.
Fixesrust-lang/rust#123183.
r? @RalfJung
refactor(mgca): Change `DefKind::Const` and `DefKind::AssocConst` to have a `is_type_const` flag
Addresses rust-lang/rust#152940
- Changed `DefKind::Const` and `DefKind::AssocConst` to have a `is_type_const` flag.
- changed `is_type_const` query to check for this flag
- removed `is_rhs_type_const` query
r? @BoxyUwU
* refactor: add `is_type_const` flag to `DefKind::Const` and `AssocConst`
* refactor(cleanup) remove the `rhs_is_type_const` query
* style: fix formatting
* refactor: refactor stuff in librustdoc for new Const and AssocConst
* refactor: refactor clippy for the changes
* chore: formatting
* fix: fix test
* fix: fix suggestions
* Update context.rs
Co-authored-by: Boxy <rust@boxyuwu.dev>
* changed AssocKind::Const to store data about being a type const
`adjust_for_rust_abi` was casting small aggregates to an integer register
without propagating `noundef`, causing a performance regression (#123183)
— LLVM could no longer assume the bits were fully defined.
Add `layout_is_noundef` (conservative) + `fields_are_noundef` helper,
then use `cast_to_with_attrs` to forward `NoUndef` when proven:
Scalar → `!is_uninit_valid()`
ScalarPair → both scalars valid + `s1.size + s2.size == layout.size`
(size equality rejects layouts with inter-scalar padding)
Array → recurse into element; empty arrays unconditionally noundef
Arbitrary → `Variants::Single` required; walk fields in offset order —
any gap, non-noundef field, or trailing pad returns false
Union / Primitive / Simd → false (conservative)
Bless `pass-indirectly-attr.stderr` and `debuginfo-dse.rs` for the new
attribute on Cast args. Add `tests/codegen-llvm/abi-noundef-cast.rs`
covering positive Cast cases (arrays, plain structs, single-variant enum)
and negative Cast cases (MaybeUninit, multi-variant enum, field/pair gap,
trailing padding).
Fixes#123183.
Co-authored-by: Ralf Jung <post@ralfj.de>
Implement RFC 3678: Final trait methods
Tracking: https://github.com/rust-lang/rust/issues/131179
This PR is based on rust-lang/rust#130802, with some minor changes and conflict resolution.
Futhermore, this PR excludes final methods from the vtable of a dyn Trait.
And some excerpt from the original PR description:
> Implements the surface part of https://github.com/rust-lang/rfcs/pull/3678.
>
> I'm using the word "method" in the title, but in the diagnostics and the feature gate I used "associated function", since that's more accurate.
cc @joshtriplett
Oli Scherer