mirror of
https://github.com/rust-lang/rust.git
synced 2026-05-30 04:56:25 +03:00
bors
94b4de0e07
Attach tokens to all AST types used in `Nonterminal` We perform token capturing when we have outer attributes (for nonterminals that support attributes - e.g. `Stmt`), or when we parse a `Nonterminal` for a `macro_rules!` argument. The full list of `Nonterminals` affected by this PR is: * `NtBlock` * `NtStmt` * `NtTy` * `NtMeta` * `NtPath` * `NtVis` * `NtLiteral` Of these nonterminals, only `NtStmt` and `NtLiteral` (which is actually just an `Expr`), support outer attributes - the rest only ever have token capturing perform when they match a `macro_rules!` argument. This makes progress towards solving https://github.com/rust-lang/rust/issues/43081 - we now collect tokens for everything that might need them. However, we still need to handle `#[cfg]`, inner attributes, and misc pretty-printing issues (e.g. #75734) I've separated the changes into (mostly) independent commits, which could be split into individual PRs for each `Nonterminal` variant. The purpose of having them all in one PR is to do a single Crater run for all of them. Most of the changes in this PR are trivial (adding `tokens: None` everywhere we construct the various AST structs). The significant changes are: * `ast::Visibility` is changed from `type Visibility = Spanned<VisibilityKind>` to a `struct Visibility { kind, span, tokens }`. * `maybe_collect_tokens` is made generic, and used for both `ast::Expr` and `ast::Stmt`. * Some of the statement-parsing functions are refactored so that we can capture the trailing semicolon. * `Nonterminal` and `Expr` both grew by 8 bytes, as some of the structs which are stored inline (rather than behind a `P`) now have an `Option<TokenStream>` field. Hopefully the performance impact of doing this is negligible.