Refactor FnDecl and FnSig non-type fields into a new wrapper type
#### Why this Refactor?
This PR is part of an initial cleanup for the [arg splat experiment](https://github.com/rust-lang/rust/issues/153629), but it's a useful refactor by itself.
It refactors the non-type fields of `FnDecl`, `FnSig`, and `FnHeader` into a new packed wrapper types, based on this comment in the `splat` experiment PR:
https://github.com/rust-lang/rust/pull/153697#discussion_r3004637413
It also refactors some common `FnSig` creation settings into their own methods. I did this instead of creating a struct with defaults.
#### Relationship to `splat` Experiment
I don't think we can use functional struct updates (`..default()`) to create `FnDecl` and `FnSig`, because we need the bit-packing for the `splat` experiment.
Bit-packing will avoid breaking "type is small" assertions for commonly used types when `splat` is added.
This PR packs these types:
- ExternAbi: enum + `unwind` variants (38) -> 6 bits
- ImplicitSelfKind: enum variants (5) -> 3 bits
- lifetime_elision_allowed, safety, c_variadic: bool -> 1 bit
#### Minor Changes
Fixes some typos, and applies rustfmt to clippy files that got skipped somehow.
delegation: fix cycles during delayed lowering
This PR forces lowering of delayed owners after `hir_crate_items`, as some diagnostics use `hir_crate_items` which results in query cycle which is then hangs calling `def_path_str` again and again. Fixesrust-lang/rust#154169. Part of rust-lang/rust#118212.
r? @petrochenkov
ty-aware delayed AST -> HIR lowering
This PR implements a prototype of ty-aware delayed AST -> HIR lowering. Part of rust-lang/rust#118212.
r? @petrochenkov
# Motivation
When lowering delegation in perfect scenario we would like to access the ty-level information, in particular, queries like `generics_of`, `type_of`, `fn_sig` for proper HIR generation with less hacks. For example, we do not want to generate more lifetimes than needed, because without ty-level queries we do not know which delegee's lifetimes are late-bound. Next, consider recursive delegations, for their proper support without ty we would have to either duplicate generics inheritance code in AST -> HIR lowering or create stubs for parts of the HIR and materialize them later. We already use those queries when interacting with external crates, however when dealing with compilation of a local crate we should use resolver for similar purposes. Finally, access to ty-level queries is expected to help supporting delegation to inherent impls, as we can not resolve such calls during AST -> HIR stage.
# Benefits
We eliminate almost all code that uses resolver in delegation lowering:
- Attributes inheritance is done without copying attributes from AST at resolve stage
- Fn signatures are obtained from `tcx.fn_sig`
- Param counts are also obtained from `tcx.fn_sig`
- `is_method` function now uses `tcx.associated_item` instead of resolver
- Generics are now inherited through `get_external_generics` that uses `tcx.generics_of`. Generics for recursive delegations should also work
- `DelegationIds` that stored paths for recursive delegations is removed, we now use only `delegee_id`
- Structs that were used for storing delegation-related information in resolver are almost fully removed
- `ast_index` is no more used
# Next steps
- Remove creating generic params through AST cloning, proper const types propagation
- Inherent impls
# High level design overview
## Queries
We store ids of delayed items to lower in `Crate` struct. During first stage of lowering, owners that correspond to delayed items are filled with `MaybeOwner::Phantom`.
Next, we define two new queries: `lower_delayed_owner`, `delayed_owner` and a function `force_delayed_owners_lowering`.
The first query is used when lowering known (which is in `delayed_ids`) delayed owner.
The second is fed with children that were obtained during lowering of a delayed owner (note that the result of lowering of a single `LocalDefId` is not a single `MaybeOwner`, its a list of `(LocalDefId, MaybeOwner)` where the first `MaybeOwner` corresponds to delayed `LocalDefId` and others to children that were obtained during lowering (i.e. generic params)). By default `delayed_owner` returns `MaybeOwner::Phantom`. As we do not want to predict the whole list of children which will be obtained after lowering of a single delayed item, we need to store those children somewhere. There are several options:
- Make the return type of `lower_delayed_owner` to be `FxIndexMap<LocalDefId, MaybeOwner>` and search children here. Search will be either linear or we can introduce a map somewhere which will track parent-child relations between a single delayed `LocalDefId` and its children.
- Try to make query that will lower all delayed items in a loop and return a complete map of all delayed `LocalDefIds` and their children. In this case there will be problems with delayed items that require information about other delayed items.
By using proposed queries we handle the second concern, and in case of acyclic dependencies between delayed ids it automatically works, moreover we use queries as cache for delayed `MaybeOwners`. The only invariant here is that queries which are invoked during delayed AST -> HIR lowering of some `LocalDefId` should not in any way access children of other, yet unlowered, delayed `LocalDefId`, they should firstly materialize it.
The `force_delayed_owners_lowering` forces lowering of all delayed items and now integrated in `hir_crate_items` query.
## Resolver for lowering
> ~Currently the `resolver_for_lowering` is stolen in `lower_to_hir` function, however we want to prolong its life until all delayed `LocalDefIds` are materialized. For this purpose we borrow `resolver_for_lowering` in `lower_to_hir` and drop it after forcing materialization of all delayed `LocalDefId` in `rustc_interface::run_required_analyses`.~
We split resolver for lowering into two parts: the first part is a readonly part that came to us from resolve, the second part is a mutable part that can be used to add or overwrite values in the readonly part. Such splitted resolver is used during delayed lowering, as we can't steal it.
## AST index
Lowering uses an AST index. It is created in `lower_to_hir` function and it references parts of AST. We want to avoid reindexing AST on every delayed `LocalDefId` lowering, however now it is not clear how to properly do it. As delayed HIR lowering is used only for delegation unstable feature it should not affect other use-cases of the compiler. But it will be reworked sooner or later.
Abstract AST lowering resolver
This PR adds new methods for `ResolverAstLoweringExt` for future use in rust-lang/rust#153489 and abstracts resolver in lowering through generic parameters.
> ~This PR splits resolver for AST lowering into two parts: mutable and readonly. This will allow us to use borrowed resolver in rust-lang/rust#153489, when we will not steal but borrow resolver from resolve stage.~
Second step for rust-lang/rust#153489.
r? @petrochenkov
Implement AST -> HIR generics propagation in delegation
This PR adds support for generics propagation during AST -> HIR lowering and is a part of rust-lang/rust#118212.
# High-level design overview
## Motivation
The task is to generate generics for delegations (i.e. in this context we assume a function that is created for `reuse` statements) during AST -> HIR lowering. Then we want to propagate those generated params to generated method call (or default call) in delegation. This will help to solve issues like the following:
```rust
mod to_reuse {
pub fn consts<const N: i32>() -> i32 {
N
}
}
reuse to_reuse::consts;
//~^ ERROR type annotations needed
// DESUGARED CURRENT:
#[attr = Inline(Hint)]
fn consts() -> _ { to_reuse::consts() }
// DESUGARED DESIRED:
#[attr = Inline(Hint)]
fn consts<const N: i32>() -> _ { to_reuse::consts::<N>() }
```
Moreover, user can specify generic args in `reuse`, we need to propagate them (works now) and inherit signature with substituted generic args:
```rust
mod to_reuse {
pub fn foo<T>(t: T) -> i32 {
0
}
}
reuse to_reuse::foo::<i32>;
//~^ ERROR mismatched types
fn main() {
foo(123);
}
error[E0308]: mismatched types
--> src/main.rs:24:17
|
19 | pub fn foo<T>(t: T) -> i32 {
| - found this type parameter
...
24 | reuse to_reuse::foo::<i32>;
| ^^^
| |
| expected `i32`, found type parameter `T`
| arguments to this function are incorrect
|
= note: expected type `i32`
found type parameter `T`
```
In this case we want the delegation to have signature that have one `i32` parameter (not `T` parameter).
Considering all other cases, for now we want to preserve existing behavior, which was almost fully done (at this stage there are changes in behavior of delegations with placeholders and late-bound lifetimes).
## Main approach overview
The main approach is as follows:
- We determine generic params of delegee parent (now only trait can act as a parent as delegation to inherent impls is not yet supported) and delegee function,
- Based on presence of user-specified args in `reuse` statement (i.e. `reuse Trait::<'static, i32, 123>::foo::<String>`) we either generate delegee generic params or not. If not, then we should include user-specified generic args into the signature of delegation,
- The general order of generic params generation is as following:
`[DELEGEE PARENT LIFETIMES, DELEGEE LIFETIMES, DELEGEE PARENT TYPES AND CONSTS, DELEGEE TYPES AND CONSTS]`,
- There are two possible generic params orderings (they differ only in a position of `Self` generic param):
- When Self is after lifetimes, this happens only in free to trait delegation scenario, as we need to generate implicit Self param of the delegee trait,
- When Self is in the beginning and we should not generate Self param, this is basically all other cases if there is an implicit Self generic param in delegation parent.
- Considering propagation, we do not propagate lifetimes for child, as at AST -> HIR lowering stage we can not know whether the lifetime is late-bound or early bound, so for now we do not propagate them at all. There is one more hack with child lifetimes, for the same reason we create predicates of kind `'a: 'a` in order to preserve all lifetimes in HIR, so for now we can generate more lifetimes params then needed. This will be partially fixed in one of next pull requests.
## Implementation details
- We obtain AST generics either from AST of a current crate if delegee is local or from external crate through `generics_of` of `tcx`. Next, as we want to generate new generic params we generate new node ids for them, remove default types and then invoke already existent routine for lowering AST generic params into HIR,
- If there are user-specified args in either parent or child parts of the path, we save HIR ids of those segments and pass them to `hir_analysis` part, where user-specified args are obtained, then lowered through existing API and then used during signature and predicates inheritance,
- If there are no user-specified args then we propagate generic args that correspond to generic params during generation of delegation,
- During signature inheritance we know whether parent or child generic args were specified by the user, if so, we should merge them with generic params (i.e. cases when parent args are specified and child args are not: `reuse Trait::<String>::foo`), next we use those generic args and mapping for delegee parent and child generic params into those args in order to fold delegee signature and delegee predicates.
## Tests
New tests were developed and can be found in `ast-hir-engine` folder, those tests cover all cases of delegation with different number of lifetimes, types, consts in generic params and different user-specified args cases (parent and child, parent/child only, none).
## Edge cases
There are some edge cases worth mentioning.
### Free to trait delegation.
Consider this example:
```rust
trait Trait<'a, T, const N: usize> {
fn foo<'x: 'x, A, B>(&self) {}
}
reuse Trait::foo;
```
As we are reusing from trait and delegee has `&self` param it means that delegation must have `Self` generic param:
```rust
fn foo<'a, 'x, Self, T, const N: usize, A, B>(self) {}
```
We inherit predicates from Self implicit generic param in `Trait`, thus we can pass to delegation anything that implements this trait. Now, consider the case when user explicitly specifies parent generic args.
```rust
reuse Trait::<'static, String, 1>::foo;
```
In this case we do not need to generate parent generic params, but we still need to generate `Self` in delegation (`DelegationGenerics::SelfAndUserSpecified` variant):
```rust
fn foo<'x, Self, A, B>(self) {}
```
User-specified generic arguments should be used to replace parent generic params in delegation, so if we had param of type `T` in `foo`, during signature inheritance we should replace it with user-specified `String` type.
### impl trait delegation
When we delegate from impl trait to something, we want the delegation to have signature that matches signature in trait. For this reason we already resolve delegation not to the actual delegee but to the trait method in order to inherit its signature. That is why when processing user-specified args when the caller kind is `impl trait` (`FnKind::AssocTraitImpl`), we discard parent user-specified args and replace them with those that are specified in trait header. In future we will also discard `child_args` but we need proper error handling for this case, so it will be addressed in one of future pull requests that are approximately specified in "Nearest future work" section.
## Nearest future work (approximate future pull requests):
- Late-bound lifetimes
- `impl Trait` params in functions
- Proper propagation of parent generics when generating method call
- ~Fix diagnostics duplication during lowering of user-specified types~
- Support for recursive delegations
- Self types support `reuse <u8 as Trait<_>>::foo as generic_arguments2`
- Decide what to do with infer args `reuse Trait::<_, _>::foo::<_>`
- Proper error handling when there is a mismatch between actual and expected args (impl trait case)
r? @petrochenkov
add field representing types
*[View all comments](https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/rust/pull/152730)*
> [!NOTE]
> This is a rewrite of #146307 by using a lang item instead of a custom `TyKind`. We still need a `hir::TyKind::FieldOf` variant, because resolving the field name cannot be done before HIR construction. The advantage of doing it this way is that we don't need to make any changes to types after HIR (including symbol mangling). At the very beginning of this feature implementation, I tried to do it using a lang item, but then quickly abandoned the approach, because at that time I was still intending to support nested fields.
Here is a [range-diff](https://triagebot.infra.rust-lang.org/gh-range-diff/rust-lang/rust/605f49b27444a738ea4032cb77e3bdc4eb811bab..d15f5052095b3549111854a2555dd7026b0a729e/605f49b27444a738ea4032cb77e3bdc4eb811bab..f5f42d1e03495dbaa23671c46b15fccddeb3492f) between the two PRs
---
# Add Field Representing Types (FRTs)
This PR implements the first step of the field projection lang experiment (Tracking Issue: rust-lang/rust#145383). Field representing types (FRTs) are a new kind of type. They can be named through the use of the `field_of!` macro with the first argument being the type and the second the name of the field (or variant and field in the case of an enum). No nested fields are supported.
FRTs natively implement the `Field` trait that's also added in this PR. It exposes information about the field such as the type of the field, the type of the base (i.e. the type that contains the field) and the offset within that base type. Only fields of non-packed structs are supported, fields of enums an unions have unique types for each field, but those do not implement the `Field` trait.
This PR was created in collaboration with @dingxiangfei2009, it wouldn't have been possible without him, so huge thanks for mentoring me!
I updated my library solution for field projections to use the FRTs from `core` instead of creating my own using the hash of the name of the field. See the [Rust-for-Linux/field-projection `lang-experiment` branch](https://github.com/Rust-for-Linux/field-projection/tree/lang-experiment).
## API added to `core::field`
```rust
pub unsafe trait Field {
type Base;
type Type;
const OFFSET: usize;
}
pub macro field_of($Container:ty, $($fields:expr)+ $(,)?);
```
Along with a perma-unstable type that the compiler uses in the expansion of the macro:
```rust
#[unstable(feature = "field_representing_type_raw", issue = "none")]
pub struct FieldRepresentingType<T: ?Sized, const VARIANT: u32, const FIELD: u32> {
_phantom: PhantomData<T>,
}
```
## Explanation of Field Representing Types (FRTs)
FRTs are used for compile-time & trait-level reflection for fields of structs & tuples. Each struct & tuple has a unique compiler-generated type nameable through the `field_of!` macro. This type natively contains information about the field such as the outermost container, type of the field and its offset. Users may implement additional traits on these types in order to record custom information (for example a crate may define a [`PinnableField` trait](https://github.com/Rust-for-Linux/field-projection/blob/lang-experiment/src/marker.rs#L9-L23) that records whether the field is structurally pinned).
They are the foundation of field projections, a general operation that's generic over the fields of a struct. This genericism needs to be expressible in the trait system. FRTs make this possible, since an operation generic over fields can just be a function with a generic parameter `F: Field`.
> [!NOTE]
> The approach of field projections has changed considerably since this PR was opened. In the end we might not need FRTs, so this API is highly experimental.
FRTs should act as though they were defined as `struct MyStruct_my_field<StructGenerics>;` next to the struct. So it should be local to the crate defining the struct so that one can implement any trait for the FRT from that crate. The `Field` traits should be implemented by the compiler & populated with correct information (`unsafe` code needs to be able to rely on them being correct).
## TODOs
There are some `FIXME(FRTs)` scattered around the code:
- Diagnostics for `field_of!` can be improved
- `tests/ui/field_representing_types/nonexistent.rs`
- `tests/ui/field_representing_types/non-struct.rs`
- `tests/ui/field_representing_types/offset.rs`
- `tests/ui/field_representing_types/not-field-if-packed.rs`
- `tests/ui/field_representing_types/invalid.rs`
- Simple type alias already seem to work, but might need some extra work in `compiler/rustc_hir_analysis/src/hir_ty_lowering/mod.rs`
r? @oli-obk
If late lifetime resolution fails for whatever reason, forward to RBV
the guarantee that an error was emitted - thereby eliminating the need
for a "hack" to suppress subsequent/superfluous error diagnostics.
Revert, but without type const.
Update symbol for feature err, then update suggestion output, and lastly update tests that change because of those.
Update these new tests with the correct syntax, and few existing tests with the new outputs the merge with main added.
Fix for tidyfmt and some errors when manually resolving a merge conflicts.
Update these tests to use update error messages and type const syntax.
Update comments and error message to use new syntax instead of old type_const attribute.
Remove the type_const attribute
update some more tests to use the new syntax.
Update these test cases.
update feature gate test
Change gate logic for `mgca_type_const_syntax` to work also if `min_generic_const_args` is enabled.
Create a new feature gate that checks for the feature before expansion.
Make rustfmt handle the `type const` syntax correctly.
Add a convience method to check if a RhsKind is type const.
Rename `Const` discriminant to `Body` for `ConstItemRhsKind`
Give the `TraitItemKind` flag an enum instead of a simple bool to better describe what the flag is for.
Update formatting for these match statements.
Update clippy test to use type const syntax.
Update test to use type const syntax.
update rustfmt to match ast items.
Update clippy to match ast and hir items.
Few more test cases that used old attribute, instead of 'type const'
Update to match the output from the feature gate checks.
tidyfmt adjustments.
Update the is_type_const, so I can constrain record!(..) in encoder.rs
Update conditional compilation test.
Move the feature gate to after expansion to allow for cfg(...) to work.
Update some more tests to use the new syntax.
Update type const tests in associated-const-bindings to use new syntax.
Don't check based off the attribute, but the item here.
Update some tests outside of the const_generics folder that were using #[type_const]
update the tests in associated consts that use #[type_const] to use type const
Update these mgca tests with the type const syntax.
Add a flag to TraitItemKind for detecting type const for now. Maybe later change ItemConstRhs to have optional consts but that touches a lot more lines of code.
Don't need into for these now that it's a query.
Add is_type_const query to handle foreign def ids.
update this test to use type const syntax.
Fix logic here, we only want to lower if there is expression in this case.
Update built-in macros to use ConstItemRhsKind
Update more instance of the old ConstItemRhs.
Rename ConstItemKind to ConstItemRhsKind, I noticed there is a typed called ConstantItemKind, so add the Rhs to the name to avoid confusion.
Update lower to use ConstItemKind
Add an other helper method to check if the rhs kinda has an expr.
Update item parse to use ConstItemKind enum.
Felt the field name could a be little clear when editing a few other things.
Change the ConstItem struct see know if we have a type const or regular const.
Make sure this syntax is properly feature gated.
Dont strip const blocks in array lengths
r? oli-obk
mGCA now handles const blocks by *always* handling them during `lower_expr_to_const_arg_direct` instead of *sometimes* stripping them out at parse time. This is just generally a lot clearer/nicer but also means parsing isn't lossy which is just straight up wrong.
We now use `MgcaDisambiguation::Direct` for const blocks because we "directly" represent a const block as `hir::ConstArgKind::Anon` :> The only time that an anon const for const generics uses `MgcaDisambiguation::AnonConst` is for unbraced literals.
Once we properly support literals in `hir::ConstArgKind` (see rust-lang/rust#152139rust-lang/rust#152001) then `MgcaDisambiguation` can be renamed to `AnonConstKind` with `TypeSystem` and `NonTypeSystem` variants. We can also get rid of `mgca_direct_lit_hack`. I expect this to be a very nice cleanup :)
Fixesrust-lang/rustfmt#6788
The diff relating to passing spans around is to avoid a bunch of mGCA diagnostics changing from `const {}` to `{}`. I'm not entirely sure why this was happening.
cc @rust-lang/rustfmt
How do I run the tests in the rustfmt repo from here? `x test rustfmt` only seems to run like 100 tests and doesn't result in a `target/issue-6788.rs` getting created. I've verified locally that this formats correctly though
make attrs actually use `Target::GenericParam`
currently attributes lower `GenericParam` -> `Target::Param` this PR fixes this, so that `GenericParam` is lowered to `Target::GenericParam`
r? @JonathanBrouwer
bors