Allow applying autodiff macros to trait functions.
It will use enzyme to generate a default derivative implementation, which can be overwritten by the user.
closes: https://github.com/rust-lang/rust/issues/153329
r? @JonathanBrouwer
ty-aware delayed AST -> HIR lowering
This PR implements a prototype of ty-aware delayed AST -> HIR lowering. Part of rust-lang/rust#118212.
r? @petrochenkov
# Motivation
When lowering delegation in perfect scenario we would like to access the ty-level information, in particular, queries like `generics_of`, `type_of`, `fn_sig` for proper HIR generation with less hacks. For example, we do not want to generate more lifetimes than needed, because without ty-level queries we do not know which delegee's lifetimes are late-bound. Next, consider recursive delegations, for their proper support without ty we would have to either duplicate generics inheritance code in AST -> HIR lowering or create stubs for parts of the HIR and materialize them later. We already use those queries when interacting with external crates, however when dealing with compilation of a local crate we should use resolver for similar purposes. Finally, access to ty-level queries is expected to help supporting delegation to inherent impls, as we can not resolve such calls during AST -> HIR stage.
# Benefits
We eliminate almost all code that uses resolver in delegation lowering:
- Attributes inheritance is done without copying attributes from AST at resolve stage
- Fn signatures are obtained from `tcx.fn_sig`
- Param counts are also obtained from `tcx.fn_sig`
- `is_method` function now uses `tcx.associated_item` instead of resolver
- Generics are now inherited through `get_external_generics` that uses `tcx.generics_of`. Generics for recursive delegations should also work
- `DelegationIds` that stored paths for recursive delegations is removed, we now use only `delegee_id`
- Structs that were used for storing delegation-related information in resolver are almost fully removed
- `ast_index` is no more used
# Next steps
- Remove creating generic params through AST cloning, proper const types propagation
- Inherent impls
# High level design overview
## Queries
We store ids of delayed items to lower in `Crate` struct. During first stage of lowering, owners that correspond to delayed items are filled with `MaybeOwner::Phantom`.
Next, we define two new queries: `lower_delayed_owner`, `delayed_owner` and a function `force_delayed_owners_lowering`.
The first query is used when lowering known (which is in `delayed_ids`) delayed owner.
The second is fed with children that were obtained during lowering of a delayed owner (note that the result of lowering of a single `LocalDefId` is not a single `MaybeOwner`, its a list of `(LocalDefId, MaybeOwner)` where the first `MaybeOwner` corresponds to delayed `LocalDefId` and others to children that were obtained during lowering (i.e. generic params)). By default `delayed_owner` returns `MaybeOwner::Phantom`. As we do not want to predict the whole list of children which will be obtained after lowering of a single delayed item, we need to store those children somewhere. There are several options:
- Make the return type of `lower_delayed_owner` to be `FxIndexMap<LocalDefId, MaybeOwner>` and search children here. Search will be either linear or we can introduce a map somewhere which will track parent-child relations between a single delayed `LocalDefId` and its children.
- Try to make query that will lower all delayed items in a loop and return a complete map of all delayed `LocalDefIds` and their children. In this case there will be problems with delayed items that require information about other delayed items.
By using proposed queries we handle the second concern, and in case of acyclic dependencies between delayed ids it automatically works, moreover we use queries as cache for delayed `MaybeOwners`. The only invariant here is that queries which are invoked during delayed AST -> HIR lowering of some `LocalDefId` should not in any way access children of other, yet unlowered, delayed `LocalDefId`, they should firstly materialize it.
The `force_delayed_owners_lowering` forces lowering of all delayed items and now integrated in `hir_crate_items` query.
## Resolver for lowering
> ~Currently the `resolver_for_lowering` is stolen in `lower_to_hir` function, however we want to prolong its life until all delayed `LocalDefIds` are materialized. For this purpose we borrow `resolver_for_lowering` in `lower_to_hir` and drop it after forcing materialization of all delayed `LocalDefId` in `rustc_interface::run_required_analyses`.~
We split resolver for lowering into two parts: the first part is a readonly part that came to us from resolve, the second part is a mutable part that can be used to add or overwrite values in the readonly part. Such splitted resolver is used during delayed lowering, as we can't steal it.
## AST index
Lowering uses an AST index. It is created in `lower_to_hir` function and it references parts of AST. We want to avoid reindexing AST on every delayed `LocalDefId` lowering, however now it is not clear how to properly do it. As delayed HIR lowering is used only for delegation unstable feature it should not affect other use-cases of the compiler. But it will be reworked sooner or later.
Fix whitespace after fragment specifiers in macro pretty printing
When a macro-generating-macro captures fragment specifier tokens (like `$x:ident`) as `tt` metavariables and replays them before a keyword (like `where`), the pretty printer concatenates them into an invalid fragment specifier (e.g. `$x:identwhere` instead of `$x:ident where`).
This happens because `tt` captures preserve the original token spacing. When the fragment specifier name (e.g. `ident`) was originally the last token before a closing delimiter, it retains `JointHidden` spacing. The `print_tts` function only checks `space_between` for `Spacing::Alone` tokens, so `JointHidden` tokens skip the space check entirely, causing adjacent identifier-like tokens to merge.
The fix adds a check in `print_tts` to insert a space between adjacent identifier-like tokens (identifiers and keywords) regardless of the original spacing, preventing them from being concatenated into invalid tokens.
This is similar to the existing `space_between` mechanism that prevents token merging for `Spacing::Alone` tokens, extended to also handle `Joint`/`JointHidden` cases where two identifier-like tokens would merge.
## Example
**before** (`rustc 1.96.0-nightly (3b1b0ef4d 2026-03-11)`):
```rust
#![feature(prelude_import)]
#![no_std]
extern crate std;
#[prelude_import]
use ::std::prelude::rust_2015::*;
//@ pretty-mode:expanded
//@ pp-exact:macro-fragment-specifier-whitespace.pp
// Test that fragment specifier names in macro definitions are properly
// separated from the following keyword/identifier token when pretty-printed.
// This is a regression test for a bug where `$x:ident` followed by `where`
// was pretty-printed as `$x:identwhere` (an invalid fragment specifier).
macro_rules! outer {
($d:tt $($params:tt)*) =>
{
#[macro_export] macro_rules! inner
{ ($($params)* where $d($rest:tt)*) => {}; }
};
}
#[macro_export]
macro_rules! inner { ($x:identwhere $ ($rest : tt)*) => {}; }
fn main() {}
```
**after** (this branch):
```rust
#![feature(prelude_import)]
#![no_std]
extern crate std;
#[prelude_import]
use ::std::prelude::rust_2015::*;
//@ pretty-mode:expanded
//@ pp-exact:macro-fragment-specifier-whitespace.pp
// Test that fragment specifier names in macro definitions are properly
// separated from the following keyword/identifier token when pretty-printed.
// This is a regression test for a bug where `$x:ident` followed by `where`
// was pretty-printed as `$x:identwhere` (an invalid fragment specifier).
macro_rules! outer {
($d:tt $($params:tt)*) =>
{
#[macro_export] macro_rules! inner
{ ($($params)* where $d($rest:tt)*) => {}; }
};
}
#[macro_export]
macro_rules! inner { ($x:ident where $ ($rest : tt)*) => {}; }
fn main() {}
```
Notice the `$x:identwhere` in the before — an invalid fragment specifier that causes a hard parse error. The after correctly separates it as `$x:ident where`.
Insert space after float literal ending with `.` in pretty printer
The pretty printer was collapsing unsuffixed float literals (like `0.`) with adjacent dot tokens, producing ambiguous or invalid output:
- `0. ..45.` (range) became `0...45.`
- `0. ..=360.` (inclusive range) became `0...=360.`
- `0. .to_string()` (method call) became `0..to_string()`
The fix inserts a space after an unsuffixed float literal ending with `.` when it appears as the start expression of a range operator or the receiver of a method call or field access, preventing the trailing dot from merging with the following `.`, `..`, or `..=` token. This is skipped when the expression is already parenthesized, since the closing `)` naturally provides separation.
## Example
**before** (`rustc 1.96.0-nightly (3b1b0ef4d 2026-03-11)`):
```rust
#![feature(prelude_import)]
#![no_std]
extern crate std;
#[prelude_import]
use ::std::prelude::rust_2015::*;
//@ pp-exact
fn main() {
let _ = 0...45.;
let _ = 0...=360.;
let _ = 0...;
let _ = 0..to_string();
}
```
**after** (this branch):
```rust
#![feature(prelude_import)]
#![no_std]
extern crate std;
#[prelude_import]
use ::std::prelude::rust_2015::*;
//@ pp-exact
fn main() {
let _ = 0. ..45.;
let _ = 0. ..=360.;
let _ = 0. ..;
let _ = 0. .to_string();
}
```
Notice `0...45.` (ambiguous — looks like deprecated `...` inclusive range) becomes `0. ..45.` (clear: float `0.` followed by range `..`). Similarly `0..to_string()` (parsed as range) becomes `0. .to_string()` (method call on float).
The pretty printer was collapsing unsuffixed float literals (like `0.`)
with adjacent dot tokens, producing ambiguous or invalid output:
- `0. ..45.` (range) became `0...45.`
- `0. ..=360.` (inclusive range) became `0...=360.`
- `0. .to_string()` (method call) became `0..to_string()`
The fix inserts a space after an unsuffixed float literal ending with
`.` when it appears as the start expression of a range operator or the
receiver of a method call or field access, preventing the trailing dot
from merging with the following `.`, `..`, or `..=` token. This is
skipped when the expression is already parenthesized, since the closing
`)` naturally provides separation.
Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>
When a macro-generating-macro captures fragment specifier tokens (like
`$x:ident`) as `tt` metavariables and replays them before a keyword
(like `where`), the pretty printer concatenates them into an invalid
fragment specifier (e.g. `$x:identwhere` instead of `$x:ident where`).
This happens because `tt` captures preserve the original token spacing.
When the fragment specifier name (e.g. `ident`) was originally the last
token before a closing delimiter, it retains `JointHidden` spacing. The
`print_tts` function only checks `space_between` for `Spacing::Alone`
tokens, so `JointHidden` tokens skip the space check entirely, causing
adjacent identifier-like tokens to merge.
The fix adds a check in `print_tts` to insert a space between adjacent
identifier-like tokens (identifiers and keywords) regardless of the
original spacing, preventing them from being concatenated into invalid
tokens.
This is similar to the existing `space_between` mechanism that prevents
token merging for `Spacing::Alone` tokens, extended to also handle
`Joint`/`JointHidden` cases where two identifier-like tokens would merge.
Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>
The AST pretty printer was dropping parentheses around or-patterns
when they appeared inside `@` bindings, `&` references, or `box`
patterns. For example:
- `v @ (1 | 2 | 3)` was printed as `v @ 1 | 2 | 3`
- `&(1 | 2 | 3)` was printed as `&1 | 2 | 3`
- `box (1 | 2 | 3)` was printed as `box 1 | 2 | 3`
Since `|` has the lowest precedence among pattern operators, all of
these are parsed incorrectly without parentheses — e.g. `v @ 1 | 2 | 3`
becomes `(v @ 1) | 2 | 3`, binding `v` only to the first alternative.
This caused E0408 ("variable not bound in all patterns") when the
expanded output was fed back to the compiler, affecting crates like
html5ever and wgpu-core that use macros expanding to or-patterns
after `@`.
The fix adds a `print_pat_paren_if_or` helper that wraps `PatKind::Or`
subpatterns in parentheses, and uses it in the `@`, `&`, and `box`
printing arms. This is similar in spirit to the existing `FixupContext`
parenthesization approach used for expression printing.
Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>
The AST pretty printer strips braces from single-item `use` sub-groups,
simplifying `use foo::{Bar}` to `use foo::Bar`. However, when the single
item is `self`, this produces `use foo::self` which is not valid Rust
(E0429) — the grammar requires `use foo::{self}`.
This affects both `stringify!` and `rustc -Zunpretty=expanded`, causing
`cargo expand` output to be unparseable when a crate uses `use
path::{self}` imports (a common pattern in the ecosystem).
The fix checks whether the single nested item's path starts with `self`
before stripping braces. If so, the braces are preserved.
Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>
The AST pretty printer was dropping parentheses around `Fn` trait
bounds in `dyn`/`impl` types when additional `+` bounds were present.
For example:
dyn (FnMut(&mut T) -> &mut dyn ResourceLimiter) + Send + Sync
was pretty-printed as:
dyn FnMut(&mut T) -> &mut dyn ResourceLimiter + Send + Sync
Without parens, `+ Send + Sync` binds to the inner `dyn ResourceLimiter`
instead of the outer type, producing invalid Rust.
The parser already tracks parentheses via `PolyTraitRef.parens`, but
`print_poly_trait_ref` never checked this field. This adds `popen()`
and `pclose()` calls when `parens == Parens::Yes`.
Signed-off-by: Andrew V. Teylu <andrew.teylu@vector.com>
Implement AST -> HIR generics propagation in delegation
This PR adds support for generics propagation during AST -> HIR lowering and is a part of rust-lang/rust#118212.
# High-level design overview
## Motivation
The task is to generate generics for delegations (i.e. in this context we assume a function that is created for `reuse` statements) during AST -> HIR lowering. Then we want to propagate those generated params to generated method call (or default call) in delegation. This will help to solve issues like the following:
```rust
mod to_reuse {
pub fn consts<const N: i32>() -> i32 {
N
}
}
reuse to_reuse::consts;
//~^ ERROR type annotations needed
// DESUGARED CURRENT:
#[attr = Inline(Hint)]
fn consts() -> _ { to_reuse::consts() }
// DESUGARED DESIRED:
#[attr = Inline(Hint)]
fn consts<const N: i32>() -> _ { to_reuse::consts::<N>() }
```
Moreover, user can specify generic args in `reuse`, we need to propagate them (works now) and inherit signature with substituted generic args:
```rust
mod to_reuse {
pub fn foo<T>(t: T) -> i32 {
0
}
}
reuse to_reuse::foo::<i32>;
//~^ ERROR mismatched types
fn main() {
foo(123);
}
error[E0308]: mismatched types
--> src/main.rs:24:17
|
19 | pub fn foo<T>(t: T) -> i32 {
| - found this type parameter
...
24 | reuse to_reuse::foo::<i32>;
| ^^^
| |
| expected `i32`, found type parameter `T`
| arguments to this function are incorrect
|
= note: expected type `i32`
found type parameter `T`
```
In this case we want the delegation to have signature that have one `i32` parameter (not `T` parameter).
Considering all other cases, for now we want to preserve existing behavior, which was almost fully done (at this stage there are changes in behavior of delegations with placeholders and late-bound lifetimes).
## Main approach overview
The main approach is as follows:
- We determine generic params of delegee parent (now only trait can act as a parent as delegation to inherent impls is not yet supported) and delegee function,
- Based on presence of user-specified args in `reuse` statement (i.e. `reuse Trait::<'static, i32, 123>::foo::<String>`) we either generate delegee generic params or not. If not, then we should include user-specified generic args into the signature of delegation,
- The general order of generic params generation is as following:
`[DELEGEE PARENT LIFETIMES, DELEGEE LIFETIMES, DELEGEE PARENT TYPES AND CONSTS, DELEGEE TYPES AND CONSTS]`,
- There are two possible generic params orderings (they differ only in a position of `Self` generic param):
- When Self is after lifetimes, this happens only in free to trait delegation scenario, as we need to generate implicit Self param of the delegee trait,
- When Self is in the beginning and we should not generate Self param, this is basically all other cases if there is an implicit Self generic param in delegation parent.
- Considering propagation, we do not propagate lifetimes for child, as at AST -> HIR lowering stage we can not know whether the lifetime is late-bound or early bound, so for now we do not propagate them at all. There is one more hack with child lifetimes, for the same reason we create predicates of kind `'a: 'a` in order to preserve all lifetimes in HIR, so for now we can generate more lifetimes params then needed. This will be partially fixed in one of next pull requests.
## Implementation details
- We obtain AST generics either from AST of a current crate if delegee is local or from external crate through `generics_of` of `tcx`. Next, as we want to generate new generic params we generate new node ids for them, remove default types and then invoke already existent routine for lowering AST generic params into HIR,
- If there are user-specified args in either parent or child parts of the path, we save HIR ids of those segments and pass them to `hir_analysis` part, where user-specified args are obtained, then lowered through existing API and then used during signature and predicates inheritance,
- If there are no user-specified args then we propagate generic args that correspond to generic params during generation of delegation,
- During signature inheritance we know whether parent or child generic args were specified by the user, if so, we should merge them with generic params (i.e. cases when parent args are specified and child args are not: `reuse Trait::<String>::foo`), next we use those generic args and mapping for delegee parent and child generic params into those args in order to fold delegee signature and delegee predicates.
## Tests
New tests were developed and can be found in `ast-hir-engine` folder, those tests cover all cases of delegation with different number of lifetimes, types, consts in generic params and different user-specified args cases (parent and child, parent/child only, none).
## Edge cases
There are some edge cases worth mentioning.
### Free to trait delegation.
Consider this example:
```rust
trait Trait<'a, T, const N: usize> {
fn foo<'x: 'x, A, B>(&self) {}
}
reuse Trait::foo;
```
As we are reusing from trait and delegee has `&self` param it means that delegation must have `Self` generic param:
```rust
fn foo<'a, 'x, Self, T, const N: usize, A, B>(self) {}
```
We inherit predicates from Self implicit generic param in `Trait`, thus we can pass to delegation anything that implements this trait. Now, consider the case when user explicitly specifies parent generic args.
```rust
reuse Trait::<'static, String, 1>::foo;
```
In this case we do not need to generate parent generic params, but we still need to generate `Self` in delegation (`DelegationGenerics::SelfAndUserSpecified` variant):
```rust
fn foo<'x, Self, A, B>(self) {}
```
User-specified generic arguments should be used to replace parent generic params in delegation, so if we had param of type `T` in `foo`, during signature inheritance we should replace it with user-specified `String` type.
### impl trait delegation
When we delegate from impl trait to something, we want the delegation to have signature that matches signature in trait. For this reason we already resolve delegation not to the actual delegee but to the trait method in order to inherit its signature. That is why when processing user-specified args when the caller kind is `impl trait` (`FnKind::AssocTraitImpl`), we discard parent user-specified args and replace them with those that are specified in trait header. In future we will also discard `child_args` but we need proper error handling for this case, so it will be addressed in one of future pull requests that are approximately specified in "Nearest future work" section.
## Nearest future work (approximate future pull requests):
- Late-bound lifetimes
- `impl Trait` params in functions
- Proper propagation of parent generics when generating method call
- ~Fix diagnostics duplication during lowering of user-specified types~
- Support for recursive delegations
- Self types support `reuse <u8 as Trait<_>>::foo as generic_arguments2`
- Decide what to do with infer args `reuse Trait::<_, _>::foo::<_>`
- Proper error handling when there is a mismatch between actual and expected args (impl trait case)
r? @petrochenkov
For the attribute `FooBar` the parser is generally called `FooBarParser`
and the kind is called `AttributeKind::FooBar`. This commit renames some
cases that don't match that pattern. The most common cases:
- Adding `Rustc` to the front of the parser name for a `rustc_*`
attribute.
- Adding `Parser` to the end of a parser name.
- Slight word variations, e.g. `Deprecation` instead of `Deprecated`,
`Pointer` instead of `Ptr`, `Stability` instead of `Stable`.
Currently all core and std macros are automatically added to the prelude
via #[macro_use]. However a situation arose where we want to add a new macro
`assert_matches` but don't want to pull it into the standard prelude for
compatibility reasons. By explicitly exporting the macros found in the core and
std crates we get to decide on a per macro basis and can later add them via
the rust_20xx preludes.
Support syntax for one-line trait reuse
This PR adds support for reusing the whole trait with a one-line reuse syntax and is part of the delegation feature rust-lang/rust#118212:
```rust
trait T {
fn foo(&self);
}
struct S;
impl T for S { ... }
struct Wrapper(S);
reuse impl T for Wrapper { self.0 }
```
The core idea is that we already have support for glob reuse, so in this scenario we want to transform one-line reuse into a trait impl block with a glob reuse in the following way:
```rust
//Before
reuse impl T for Wrapper { self.0 }
//After
impl T for Wrapper {
reuse T::* { self.0 }
}
```
It seems like this task can be solved during parsing stage, when we encountered a one-line trait reuse, we can expand into this impl block right away, and the code which was already written to expand glob delegations will take care about the rest. We will copy trait path into glob reuse path.
The implementation of the transformation reuses already existing methods for `impl` parsing, however, we do not parse inner `impl` items, instead we parse "inner items" as delegation body. Thus, we do not have to deal with generics, consts, unsafe and other `impl` related features.
Other syntax possibility is trying to shorten one-line reuse by replacing `impl` keyword with `reuse` keyword:
```rust
reuse T for Wrapper { self.0 }
```
In this case implementation may become more complicated, and the syntax more confusing, as keywords such as `const` or `unsafe` will precede `reuse`, and there are also generics:
```rust
unsafe reuse<T1, T2> T for Wrapper { self.0 }
```
In the first (currently implemented) version reuse is placed in the beginning of the item, and it is clear that we will reuse trait implementation, while in the second, shorter version, the `reuse` keyword may be lost in generics and keywords that may precede `impl`.
r? ``@petrochenkov``
148725 moved the default to being homogeneous; this adds heterogeneous ones back under an obvious-bikeshed syntax so people can experiment with that as well.
Essentially resolves 149025 by letting them move to this syntax instead.
`c_variadic`: Add future-incompatibility warning for `...` arguments without a pattern outside of `extern` blocks
This PR makes `...` arguments without a pattern in non-foreign functions (such as the argument in `unsafe extern "C" fn f(...) {}`) a future-compatibility warning; making this error would be consistent with how `unsafe extern "C" fn f(u32) {}` is handled. Allowing `...` arguments without a pattern in non-foreign functions is a source of confusion for programmers coming from C, where the `...` parameter is never named and instead calling `va_start` is required; disallowing `...` arguments without a pattern also improves the overall consistency of the Rust language by matching the treatment of other arguments without patterns. `...` arguments without a pattern in `extern` blocks (such as `unsafe extern "C" { fn f(...); }`) continue to compile without warnings after this PR, as they are already stable and heavily used (and don't cause the mentioned confusion as they are just being used in function declarations).
As all the syntax gating for `c_variadic` has been done post-expansion, this is technically a breaking change. In particular, code like this has compiled on stable since Rust 1.35.0:
```rust
#[cfg(any())] // Equivalent to the more recent #[cfg(false)]
unsafe extern "C" fn bar(_: u32, ...) {}
```
Since this is more or less a stability hole and a Crater run shows only the `binrw` crate is using this, I think it would be ok to break this. This will require a lang FCP.
The idea of rejecting `...` pre-expansion was first raised here https://github.com/rust-lang/rust/pull/143546#issuecomment-3043142052.
Tracking issue: rust-lang/rust#44930
cc `@folkertdev` `@workingjubilee`
r? `@joshtriplett`
Use `splice` to avoid shifting the other items twice.
Put `extern crate std;` first so it's already resolved when we resolve `::std::prelude::rust_20XX`.
Implement asymmetrical precedence for closures and jumps
I have been through a series of asymmetrical precedence designs in Syn, and finally have one that I like and is worth backporting into rustc. It is based on just 2 bits of state: `next_operator_can_begin_expr` and `next_operator_can_continue_expr`.
Asymmetrical precedence is the thing that enables `(return 1) + 1` to require parentheses while `1 + return 1` does not, despite `+` always having stronger precedence than `return` [according to the Rust Reference](https://doc.rust-lang.org/1.83.0/reference/expressions.html#expression-precedence). This is facilitated by `next_operator_can_continue_expr`.
Relatedly, it is the thing that enables `(return) - 1` to require parentheses while `return + 1` does not, despite `+` and `-` having exactly the same precedence. This is facilitated by `next_operator_can_begin_expr`.
**Example:**
```rust
macro_rules! repro {
($e:expr) => {
$e - $e;
$e + $e;
};
}
fn main() {
repro!{return}
repro!{return 1}
}
```
`-Zunpretty=expanded` **Before:**
```console
fn main() {
(return) - (return);
(return) + (return);
(return 1) - (return 1);
(return 1) + (return 1);
}
```
**After:**
```console
fn main() {
(return) - return;
return + return;
(return 1) - return 1;
(return 1) + return 1;
}
```
Jonathan Brouwer