Take commandline arguments into account for incr. comp.
Implements the conservative strategy described in https://github.com/rust-lang/rust/issues/33727.
From now one, every time a new commandline option is added, one has to specify if it influences the incremental compilation cache. I've tried to implement this as automatic as possible: One just has to added either the `[TRACKED]` or the `[UNTRACKED]` marker next to the field. The `Options`, `CodegenOptions`, and `DebuggingOptions` definitions in `session::config` show plenty of examples.
The PR removes some cruft from `session::config::Options`, mostly unnecessary copies of flags also present in `DebuggingOptions` or `CodeGenOptions` in the same struct.
One notable removal is the `cfg` field that contained the values passed via `--cfg` commandline arguments. I chose to remove it because (1) its content is only a subset of what later is stored in `hir::Crate::config` and it's pretty likely that reading the cfgs from `Options` would not be what you wanted, and (2) we could not incorporate it into the dep-tracking hash of the `Options` struct because of how the test framework works, leaving us with a piece of untracked but vital data.
It is now recommended (just as before) to access the crate config via the `krate()` method in the HIR map.
Because the `cfg` field is not present in the `Options` struct any more, some methods in the `CompilerCalls` trait now take the crate config as an explicit parameter -- which might constitute a breaking change for plugin authors.
Implemented a smarter TokenStream concatenation system
The new algorithm performs 'aggressive compacting' during concatenation as follows:
- If the nodes' combined total total length is less than 32, we copy both of
them into a new vector and build a new leaf node.
- If one node is an internal node and the other is a 'small' leaf (length<32),
we recur down the internal node on the appropriate side.
- Otherwise, we construct a new internal node that points to them as left and
right.
This should produce notably better behavior than the current concatenation implementation.
macros: Make metavariables hygienic
This PR makes metavariables hygienic. For example, consider:
```rust
macro_rules! foo {
($x:tt) => { // Suppose that this token tree argument is always a metavariable.
macro_rules! bar { ($x:expr, $y:expr) => { ($x, $y) } }
}
}
fn main() {
foo!($z); // This currently compiles.
foo!($y); // This is an error today but compiles after this PR.
}
```
Today, the `macro_rules! bar { ... }` definition is only valid when the metavariable passed to `foo` is not `$y` (since it unhygienically conflicts with the `$y` in the definition of `bar`) or `$x` (c.f. #35450).
After this PR, the definition of `bar` is always valid (and `bar!(a, b)` always expands to `(a, b)` as expected).
This can break code that was allowed in #34925 (landed two weeks ago). For example,
```rust
macro_rules! outer {
($t:tt) => {
macro_rules! inner { ($i:item) => { $t } }
}
}
outer!($i); // This `$i` should not interact with the `$i` in the definition of `inner!`.
inner!(fn main() {}); // After this PR, this is an error ("unknown macro variable `i`").
```
Due to the severe limitations on nested `macro_rules!` before #34925, this is not a breaking change for stable/beta.
Fixes#35450.
r? @nrc
Implement `impl Trait` in return type position by anonymization.
This is the first step towards implementing `impl Trait` (cc #34511).
`impl Trait` types are only allowed in function and inherent method return types, and capture all named lifetime and type parameters, being invariant over them.
No lifetimes that are not explicitly named lifetime parameters are allowed to escape from the function body.
The exposed traits are only those listed explicitly, i.e. `Foo` and `Clone` in `impl Foo + Clone`, with the exception of "auto traits" (like `Send` or `Sync`) which "leak" the actual contents.
The implementation strategy is anonymization, i.e.:
```rust
fn foo<T>(xs: Vec<T>) -> impl Iterator<Item=impl FnOnce() -> T> {
xs.into_iter().map(|x| || x)
}
// is represented as:
type A</*invariant over*/ T> where A<T>: Iterator<Item=B<T>>;
type B</*invariant over*/ T> where B<T>: FnOnce() -> T;
fn foo<T>(xs: Vec<T>) -> A<T> {
xs.into_iter().map(|x| || x): $0 where $0: Iterator<Item=$1>, $1: FnOnce() -> T
}
```
`$0` and `$1` are resolved (to `iter::Map<vec::Iter<T>, closure>` and the closure, respectively) and assigned to `A` and `B`, after checking the body of `foo`. `A` and `B` are *never* resolved for user-facing type equality (typeck), but always for the low-level representation and specialization (trans).
The "auto traits" exception is implemented by collecting bounds like `impl Trait: Send` that have failed for the obscure `impl Trait` type (i.e. `A` or `B` above), pretending they succeeded within the function and trying them again after type-checking the whole crate, by replacing `impl Trait` with the real type.
While passing around values which have explicit lifetime parameters (of the function with `-> impl Trait`) in their type *should* work, regionck appears to assign inference variables in *way* too many cases, and never properly resolving them to either explicit lifetime parameters, or `'static`.
We might not be able to handle lifetime parameters in `impl Trait` without changes to lifetime inference, but type parameters can have arbitrary lifetimes in them from the caller, so most type-generic usecases (or not generic at all) should not run into this problem.
cc @rust-lang/lang
Commandline arguments influence whether incremental compilation
can use its compilation cache and thus their changes relative to
previous compilation sessions need to be taking into account. This
commit makes sure that one has to specify for every commandline
argument whether it influences incremental compilation or not.
Properly feature gate all unstable ABIs
Fixes https://github.com/rust-lang/rust/issues/34900
[breaking-change]
r? @pnkfelix
---
Function-visiting machinery for AST/HIR is surprisingly error-prone, it's *very* easy to miss some cases or visit something twice while writing a visitor. This is the true problem behind https://github.com/rust-lang/rust/issues/34900. I'll try to restructure these visitors a bit and send one more PR later.
Better attribute and metaitem encapsulation throughout the compiler
This PR refactors most (hopefully all?) of the `MetaItem` interactions outside of `libsyntax` (and a few inside) to interact with MetaItems through the provided traits instead of directly creating / destruct / matching against them. This is a necessary first step to eventually converting `MetaItem`s to internally use `TokenStream` representations (which will make `MetaItem` interactions much nicer for macro writers once the new macro system is in place).
r? @nrc
Enable reuse of `.o` files if nothing has changed
This PR completes a first "spike" for incremental compilation by enabling us to reuse `.o` files when nothing has changed. When in incr. mode, we will save `.o` files into the temporary directory, then copy them back out again if they are still valid. The code is still a bit rough but it does seem to work. =)
r? @michaelwoerister
Fixes#34036Fixes#34037Fixes#34038
macros: fix bug in `stmt` matchers
Today, `stmt` matchers stop too early when parsing expression statements that begin with non-braced macro invocations. For example,
```rust
fn main() {
macro_rules! m { ($s:stmt;) => { $s } }
id!(vec![].push(0););
//^ Before this PR, the `stmt` matcher only consumes "vec![]", so this is an error.
//| After this PR, the `stmt` matcher consumes "vec![].push(0)", so this compiles.
}
```
This change is backwards compatible due to the follow set for `stmt`.
r? @eddyb
Added tokenstream parser procedure
A tiny PR that simply adds a procedure for parsing `TokenStream`s to the parser in `src/libsyntax`. This is to ease using `TokenStream`s with the current (old) procedural macro system.
Simplify librustc_errors
This is part 2 of the error crate refactor, starting with #34403.
In this refactor, I focused on slimming down the error crate to fewer moving parts. As such, I've removed quite a few parts and replaced the with simpler, straight-line code. Specifically, this PR:
* Removes BasicEmitter
* Remove emit from emitter, leaving emit_struct
* Renames emit_struct to emit
* Removes CoreEmitter and focuses on a single Emitter
* Implements the latest changes to error format RFC (#1644)
* Removes (now-unused) code in emitter.rs and snippet.rs
* Moves more tests to the UI tester, removing some duplicate tests in the process
There is probably more that could be done with some additional refactoring, but this felt like it was getting to a good state.
r? @alexcrichton cc: @Manishearth (as there may be breaking changes in stuff I removed/changed)
Better error message for inner attribute following doc comment
Before it was always just "an inner attribute is not permitted in this context", whereas now we add a special case for when an inner attr follows an outer attr. If the outer attr is a doc comment, then the error is "an inner attr is not permitted following a doc comment", and otherwise it's "an inner attr is not permitted following an outer attribute". In all other cases it's still "an inner attribute is not permitted in this context".
Note that the public API and behaviour of `parse_attribute` is unchanged. Also, all new names are very open to bikeshedding -- they're arguably clunky.
Fixes#34516. cc @brson
Simplify the macro hygiene algorithm
This PR removes renaming from the hygiene algorithm and treats differently marked identifiers as unequal.
This change makes the scope of identifiers in `macro_rules!` items empty. That is, identifiers in `macro_rules!` definitions do not inherit any semantics from the `macro_rules!`'s scope.
Since `macro_rules!` macros are items, the scope of their identifiers "should" be the same as that of other items; in particular, the scope should contain only items. Since all items are unhygienic today, this would mean the scope should be empty.
However, the scope of an identifier in a `macro_rules!` statement today is the scope that the identifier would have if it replaced the `macro_rules!` (excluding anything unhygienic, i.e. locals only).
To continue to support this, this PR tracks the scope of each `macro_rules!` and uses it in `resolve` to ensure that an identifier expanded from a `macro_rules!` gets a chance to resolve to the locals in the `macro_rules!`'s scope.
This PR is a pure refactoring. After this PR,
- `syntax::ext::expand` is much simpler.
- We can expand macros in any order without causing problems for hygiene (needed for macro modularization).
- We can deprecate or remove today's `macro_rules!` scope easily.
- Expansion performance improves by 25%, post-expansion memory usage decreases by ~5%.
- Expanding a block is no longer quadratic in the number of `let` statements (fixes#10607).
r? @nrc
bors