Commit Graph

7233 Commits

Author SHA1 Message Date
bors 9ff8ec8fdf Auto merge of #50204 - Manishearth:approx-enum, r=estebank
Use enum for approximate suggestions

r? @nrc @killercup
2018-04-30 05:38:06 +00:00
bors 78bcd9b5fe Auto merge of #50092 - abonander:issue-49934, r=petrochenkov
Warn on pointless #[derive] in more places

This fixes the regression in #49934 and ensures that unused `#[derive]` invocations on statements, expressions and generic type parameters survive to trip the `unused_attributes` lint. There is a separate warning hardcoded for `#[derive]` on macro invocations since linting (even the early-lint pass) occurs after expansion. This also adds regression tests for some nodes that were already warning properly.

closes #49934
2018-04-30 00:18:49 +00:00
Austin Bonander f16d2ff7ec Warn on pointless #[derive] in more places
This fixes the regression in #49934 and ensures that unused `#[derive]`s on statements, expressions and generic type parameters survive to trip the `unused_attributes` lint. For `#[derive]` on macro invocations it has a hardcoded warning since linting occurs after expansion. This also adds regression testing for some nodes that were already warning properly.

closes #49934
2018-04-29 16:01:41 -07:00
bors 774a6a3c4b Auto merge of #50317 - varkor:repr-align-assign, r=nagisa
Improve error message for #[repr(align=x)]

Before:
```
error[E0552]: unrecognized representation hint
 --> src/main.rs:1:8
  |
1 | #[repr(align="8")]
  |        ^^^^^^^^^
```
After:
```
error[E0693]: incorrect `repr(align)` attribute format
 --> src/main.rs:1:8
  |
2 | #[repr(align="8")]
  |        ^^^^^^^^^ help: use parentheses instead: `align(8)`
```

Fixes #50314.
2018-04-29 21:55:50 +00:00
varkor a815f753bc Add error when using repr(align=x) instead of repr(align(x)) 2018-04-29 18:42:43 +01:00
bors ff65726ebf Auto merge of #50155 - est31:label_expressions, r=petrochenkov
'label can start expressions

```Rust
let foo = 'label: loop { break 'label 42; };
```

is valid Rust code.
2018-04-28 10:08:56 +00:00
kennytm 8b36d9abaa Rollup merge of #49968 - christianpoveda:stabilize_dyn, r=nikomatsakis
Stabilize dyn trait

This PR stabilizes RFC 2113. I followed the [stabilization guide](https://forge.rust-lang.org/stabilization-guide.html).

Related issue: https://github.com/rust-lang/rust/issues/49218
2018-04-28 03:32:12 +08:00
Christian Poveda 72a8eb92b0 fixed rustc version for dyn_trait 2018-04-27 10:05:57 -05:00
Christian Poveda 199ee32773 stop requiring the feature-gate to use dyn_trait 2018-04-27 10:04:02 -05:00
Irina Popa a131c518ad Fixed tidy errors. 2018-04-26 17:49:24 +03:00
Irina Popa 04fa0e7bb3 rustc_target: move in syntax::abi and flip dependency. 2018-04-26 17:49:16 +03:00
Manish Goregaokar 4e2cd4104a Approximate -> Applicability 2018-04-25 14:55:25 -07:00
Manish Goregaokar b9c44ebd3f Use enum for approximate suggestions 2018-04-24 15:42:27 -07:00
bors 190a6c41cf Auto merge of #50096 - alexcrichton:less-simd-warnings, r=michaelwoerister
Tweak some warnings around #[target_feature]

This commit fixes up some issues discovered when getting the `stdsimd` crate's CI compiling again.
2018-04-24 20:59:15 +00:00
bors 2a6200a5c8 Auto merge of #49911 - rcoh:master, r=nikomatsakis
Don't allow #[should_panic] with non-() tests

Adds (removes) support for `#[should_panic]` when the test is non-`()`
2018-04-24 10:44:22 +00:00
kennytm 7c552a299d Rollup merge of #49985 - zackmdavis:0, r=estebank
don't see issue #0

The unstable-feature attribute requires an issue (neglecting it is
E0547), which gets used in the error messages. Unfortunately, there are
some cases where "0" is apparently used a placeholder where no issue
exists, directing the user to see the (nonexistent) issue #0. (It would
have been better to either let `issue` be optional—compare to how issue
is an `Option<u32>` in the feature-gate declarations in
libsyntax/feature-gate.rs—or actually require that an issue be created.)
Rather than endeavoring to change how `#[unstable]` works at this time
(given competing contributor and reviewer priorities), this simple patch
proposes the less-ambitious solution of just not adding the "(see
issue)" note when the number is zero.

Resolves #49983.
2018-04-24 11:57:07 +08:00
bors 0135bf647c Auto merge of #49368 - matthewjasper:feature-gate-where-clause, r=nikomatsakis
Feature gate where clauses on associated types

Fixes #49365. Requires crater: these have been usable since 1.24.
2018-04-23 19:29:25 +00:00
Russell Cohen 14e5e0e9c9 Don't allow #[should_panic] with non-() tests 2018-04-23 07:13:29 -07:00
est31 40f2ca2c95 'label can start expressions
let foo = 'label: loop { break 'label 42; };

is valid Rust code.
2018-04-23 11:17:59 +02:00
bors 2c57826b92 Auto merge of #48946 - PramodBisht:issues/48636, r=estebank
Doc comments present after a particular syntax error cause an unhelpful error message to be output.

fixed: #48636

r? @estebank
2018-04-23 01:36:51 +00:00
bors ff48277add Auto merge of #50152 - petrochenkov:nooverhyg, r=alexcrichton
parser: Do not override syntactic context for dummy spans

Fixes https://github.com/rust-lang/rust/issues/50061

https://github.com/rust-lang/rust/commit/e2afefd80bf779bc3c6f697a3c6cc3a476993602 seemingly did everything right, but uncovered a preexisting bug.
2018-04-22 16:58:12 +00:00
Pramod Bisht 1bed654053 1) Addresses #48636
2) Changed position of help message, incase comma is missing
3) added few missing spaces and handled span_suggestion for vscode
4) updated stderr file
2018-04-22 11:43:25 +05:30
bors bbdd1cf744 Auto merge of #49757 - GuillaumeGomez:never-search, r=QuietMisdreavus
Add specific never search

Fixes #49529.

r? @QuietMisdreavus
2018-04-22 02:18:41 +00:00
Vadim Petrochenkov 6a4e0b3fae parser: Do not override syntactic context for dummy spans 2018-04-22 04:33:30 +03:00
Guillaume Gomez 48ab422be4 Add tracking issue number for doc alias feature 2018-04-22 00:30:48 +02:00
bors aa7ce896f2 Auto merge of #50121 - pnkfelix:revert-stabilization-of-never-type-et-al, r=alexcrichton
Revert stabilization of never_type (!) et al

Fix #49691

I *think* this correctly adopts @nikomatsakis 's desired fix of:
 * reverting stabilization of `!` and `TryFrom`, and
 * returning to the previous fallback semantics (i.e. it is once again dependent on whether the crate has opted into `#[feature(never_type)]`,
 * **without** attempting to put back in the previous future-proofing warnings regarding the change in fallback semantics.

(I'll be away from computers for a week starting now, so any updates to this PR should be either pushed into it, or someone else should adopt the task of polishing this fix and put up their own PR.)
2018-04-21 21:14:53 +00:00
Guillaume Gomez 57bcabc108 Generate alias file 2018-04-21 22:02:53 +02:00
bors 222551f3f3 Auto merge of #50120 - alexcrichton:more-proc-macro-gates, r=petrochenkov
rustc: Tweak custom attribute capabilities

This commit starts to lay some groundwork for the stabilization of custom
attribute invocations and general procedural macros. It applies a number of
changes discussed on [internals] as well as a [recent issue][issue], namely:

* The path used to specify a custom attribute must be of length one and cannot
  be a global path. This'll help future-proof us against any ambiguities and
  give us more time to settle the precise syntax. In the meantime though a bare
  identifier can be used and imported to invoke a custom attribute macro. A new
  feature gate, `proc_macro_path_invoc`, was added to gate multi-segment paths
  and absolute paths.

* The set of items which can be annotated by a custom procedural attribute has
  been restricted. Statements, expressions, and modules are disallowed behind
  two new feature gates: `proc_macro_expr` and `proc_macro_mod`.

* The input to procedural macro attributes has been restricted and adjusted.
  Today an invocation like `#[foo(bar)]` will receive `(bar)` as the input token
  stream, but after this PR it will only receive `bar` (the delimiters were
  removed). Invocations like `#[foo]` are still allowed and will be invoked in
  the same way as `#[foo()]`. This is a **breaking change** for all nightly
  users as the syntax coming in to procedural macros will be tweaked slightly.

* Procedural macros (`foo!()` style) can only be expanded to item-like items by
  default. A separate feature gate, `proc_macro_non_items`, is required to
  expand to items like expressions, statements, etc.

Closes #50038

[internals]: https://internals.rust-lang.org/t/help-stabilize-a-subset-of-macros-2-0/7252
[issue]: https://github.com/rust-lang/rust/issues/50038
2018-04-21 13:50:58 +00:00
bors 9af69fe232 Auto merge of #50080 - klnusbaum:edition_49591, r=Manishearth
add --edition option

This adds an official `edition` flag to the rust compiler
2018-04-21 05:28:21 +00:00
Alex Crichton 79630d4fdf rustc: Tweak custom attribute capabilities
This commit starts to lay some groundwork for the stabilization of custom
attribute invocations and general procedural macros. It applies a number of
changes discussed on [internals] as well as a [recent issue][issue], namely:

* The path used to specify a custom attribute must be of length one and cannot
  be a global path. This'll help future-proof us against any ambiguities and
  give us more time to settle the precise syntax. In the meantime though a bare
  identifier can be used and imported to invoke a custom attribute macro. A new
  feature gate, `proc_macro_path_invoc`, was added to gate multi-segment paths
  and absolute paths.

* The set of items which can be annotated by a custom procedural attribute has
  been restricted. Statements, expressions, and modules are disallowed behind
  two new feature gates: `proc_macro_expr` and `proc_macro_mod`.

* The input to procedural macro attributes has been restricted and adjusted.
  Today an invocation like `#[foo(bar)]` will receive `(bar)` as the input token
  stream, but after this PR it will only receive `bar` (the delimiters were
  removed). Invocations like `#[foo]` are still allowed and will be invoked in
  the same way as `#[foo()]`. This is a **breaking change** for all nightly
  users as the syntax coming in to procedural macros will be tweaked slightly.

* Procedural macros (`foo!()` style) can only be expanded to item-like items by
  default. A separate feature gate, `proc_macro_non_items`, is required to
  expand to items like expressions, statements, etc.

Closes #50038

[internals]: https://internals.rust-lang.org/t/help-stabilize-a-subset-of-macros-2-0/7252
[issue]: https://github.com/rust-lang/rust/issues/50038
2018-04-20 19:56:16 -07:00
Felix S. Klock II fadabd6fbb Revert stabilization of feature(never_type).
This commit is just covering the feature gate itself and the tests
that made direct use of `!` and thus need to opt back into the
feature.

A follow on commit brings back the other change that motivates the
revert: Namely, going back to the old rules for falling back to `()`.
2018-04-20 18:09:28 +02:00
bors 85f5dd489e Auto merge of #50052 - nnethercote:char_lit, r=Mark-Simulacrum
Avoid allocating when parsing \u{...} literals.

`char_lit` uses an allocation in order to ignore '_' chars in \u{...}
literals. This patch changes it to not do that by processing the chars
more directly.

This improves various rustc-perf benchmark measurements by up to 6%,
particularly regex, futures, clap, coercions, hyper, and encoding.

rustc-perf results, on a stage 2 build with jemalloc disabled:

<details>

```
regex-check
	avg: -5.4%	min: -6.5%	max: -2.7%
futures-check
	avg: -3.5%	min: -5.3%	max: -1.7%
regex-opt
	avg: -2.0%	min: -5.1%	max: -0.2%
regex
	avg: -2.3%	min: -5.0%	max: -0.6%
futures-opt
	avg: -3.0%	min: -4.8%	max: -1.1%
futures
	avg: -3.1%	min: -4.8%	max: -1.3%
clap-rs-check
	avg: -1.8%	min: -3.5%	max: -0.9%
coercions-check
	avg: -2.0%	min: -3.3%	max: -1.0%
hyper-check
	avg: -2.2%	min: -3.1%	max: -1.3%
hyper
	avg: -1.3%	min: -2.4%	max: -0.3%
hyper-opt
	avg: -0.9%	min: -2.3%	max: -0.1%
coercions
	avg: -1.1%	min: -2.2%	max: -0.4%
encoding-check
	avg: -1.7%	min: -2.2%	max: -0.9%
clap-rs-opt
	avg: -0.7%	min: -2.2%	max: 0.0%
coercions-opt
	avg: -1.2%	min: -2.1%	max: -0.3%
clap-rs
	avg: -0.8%	min: -1.9%	max: -0.4%
encoding-opt
	avg: -1.0%	min: -1.9%	max: -0.3%
encoding
	avg: -1.1%	min: -1.9%	max: -0.4%
piston-image-check
	avg: -0.7%	min: -1.3%	max: -0.3%
inflate-opt
	avg: -0.3%	min: -0.9%	max: -0.0%
piston-image
	avg: -0.3%	min: -0.8%	max: -0.1%
piston-image-opt
	avg: -0.3%	min: -0.7%	max: -0.1%
syn-check
	avg: -0.3%	min: -0.6%	max: -0.1%
deep-vector
	avg: 0.1%	min: -0.1%	max: 0.5%
syn-opt
	avg: -0.1%	min: -0.4%	max: 0.0%
html5ever
	avg: -0.2%	min: -0.4%	max: -0.0%
deep-vector-check
	avg: 0.0%	min: -0.3%	max: 0.3%
syn
	avg: -0.2%	min: -0.3%	max: -0.1%
html5ever-check
	avg: -0.3%	min: -0.3%	max: -0.2%
issue-46449-check
	avg: -0.1%	min: -0.2%	max: 0.2%
html5ever-opt
	avg: -0.0%	min: -0.2%	max: 0.1%
deep-vector-opt
	avg: -0.0%	min: -0.2%	max: 0.1%
issue-46449-opt
	avg: -0.0%	min: -0.2%	max: 0.1%
unify-linearly-check
	avg: -0.0%	min: -0.2%	max: 0.1%
helloworld-check
	avg: 0.0%	min: -0.0%	max: 0.2%
parser-check
	avg: -0.0%	min: -0.2%	max: 0.0%
inflate
	avg: 0.0%	min: -0.0%	max: 0.1%
tokio-webpush-simple-check
	avg: -0.1%	min: -0.1%	max: -0.0%
regression-31157-check
	avg: 0.0%	min: -0.1%	max: 0.1%
issue-46449
	avg: 0.0%	min: -0.1%	max: 0.1%
tuple-stress-opt
	avg: 0.0%	min: -0.0%	max: 0.1%
tuple-stress-check
	avg: -0.0%	min: -0.1%	max: 0.1%
tuple-stress
	avg: 0.0%	min: -0.0%	max: 0.1%
deeply-nested-check
	avg: 0.0%	min: -0.0%	max: 0.1%
regression-31157
	avg: -0.0%	min: -0.1%	max: 0.1%
deeply-nested-opt
	avg: -0.0%	min: -0.1%	max: 0.1%
parser-opt
	avg: -0.0%	min: -0.1%	max: 0.0%
parser
	avg: 0.1%	min: 0.0%	max: 0.1%
tokio-webpush-simple
	avg: -0.0%	min: -0.1%	max: 0.1%
regression-31157-opt
	avg: -0.0%	min: -0.1%	max: 0.1%
helloworld-opt
	avg: 0.0%	min: -0.0%	max: 0.1%
unify-linearly-opt
	avg: 0.0%	min: -0.0%	max: 0.1%
unused-warnings-check
	avg: 0.0%	min: 0.0%	max: 0.1%
tokio-webpush-simple-opt
	avg: -0.0%	min: -0.1%	max: 0.0%
helloworld
	avg: -0.0%	min: -0.0%	max: 0.1%
unused-warnings
	avg: 0.0%	min: -0.0%	max: 0.0%
deeply-nested
	avg: -0.0%	min: -0.0%	max: -0.0%
unused-warnings-opt
	avg: 0.0%	min: -0.0%	max: 0.0%
unify-linearly
	avg: 0.0%	min: -0.0%	max: 0.0%
inflate-check
	avg: 0.0%	min: -0.0%	max: 0.0%
```

</details>
2018-04-20 10:40:25 +00:00
Kurtis Nusbaum 320fdaa942 add EDITIONS_NAME_LIST, make edition tracked, enforce that only stable editions are allowed to be used on non-nightly builds 2018-04-19 21:03:21 -07:00
Alex Crichton 81a643731d rustc: Blanket whitelist #[target_feature]
This commit transitions the `target_feature` attribute from `Normal` to
`Whitelisted`. Discovered in #50095 the fact of whether this attribute is used
or not is dependent on typechecking running and executing `check_name`, but
incremental compilation doesn't currently account for this, meaning that the
attribute ends up being flagged as unused when it shouldn't be.

I was a little too ambitious it seems hoping that `Normal` could be used, so
instead this transitions to `Whitelisted` to be the same as other codegen
attributes like `#[inline]`

Closes #50095
2018-04-19 17:54:20 -07:00
Kurtis Nusbaum 51f51109ce add --edition option 2018-04-19 13:57:01 -07:00
Alex Crichton e9348738fc proc_macro: Stay on the "use the cache" path more
Discovered in #50061 we're falling off the "happy path" of using a stringified
token stream more often than we should. This was due to the fact that a
user-written token like `0xf` is equality-different from the stringified token
of `15` (despite being semantically equivalent).

This patch updates the call to `eq_unspanned` with an even more awful solution,
`probably_equal_for_proc_macro`, which ignores the value of each token and
basically only compares the structure of the token stream, assuming that the AST
doesn't change just one token at a time.

While this is a step towards fixing #50061 there is still one regression
from #49154 which needs to be fixed.
2018-04-18 19:36:48 -07:00
Nicholas Nethercote 9f145022ef Avoid allocating when parsing \u{...} literals.
`char_lit` uses an allocation in order to ignore '_' chars in \u{...}
literals. This patch changes it to not do that by processing the chars
more directly.

This improves various rustc-perf benchmark measurements by up to 6%,
particularly regex, futures, clap, coercions, hyper, and encoding.
2018-04-19 09:17:40 +10:00
bors c8fa49f83b Auto merge of #50006 - rcoh:reorder-compiler-builtins, r=oli-obk
Reorder injection of std to get better compilation error

Per #49851, reorder injection imports to get a better error message.

r? @oli-obk
2018-04-18 17:07:21 +00:00
bors 3dfda16525 Auto merge of #49993 - nnethercote:shrink-Token, r=alexcrichton
Change the hashcounts in raw `Lit` variants from usize to u16.

This reduces the size of `Token` from 32 bytes to 24 bytes on 64-bit
platforms.
2018-04-18 14:44:54 +00:00
bors 186db76159 Auto merge of #49664 - alexcrichton:stable-simd, r=BurntSushi
Stabilize x86/x86_64 SIMD

This commit stabilizes the SIMD in Rust for the x86/x86_64 platforms. Notably
this commit is stabilizing:

* The `std::arch::{x86, x86_64}` modules and the intrinsics contained inside.
* The `is_x86_feature_detected!` macro in the standard library
* The `#[target_feature(enable = "...")]` attribute
* The `#[cfg(target_feature = "...")]` matcher

Stabilization of the module and intrinsics were primarily done in
rust-lang-nursery/stdsimd#414 and the two attribute stabilizations are done in
this commit. The standard library is also tweaked a bit with the new way that
stdsimd is integrated.

Note that other architectures like `std::arch::arm` are not stabilized as part
of this commit, they will likely stabilize in the future after they've been
implemented and fleshed out. Similarly the `std::simd` module is also not being
stabilized in this commit, only `std::arch`. Finally, nothing related to `__m64`
is stabilized in this commit either (MMX), only SSE and up types and intrinsics
are stabilized.

Closes #29717
Closes #44839
Closes #48556
2018-04-17 03:57:22 +00:00
Alex Crichton 1217d70465 Separately gate each target_feature feature
Use an explicit whitelist for what features are actually stable and can be
enabled.
2018-04-16 13:58:42 -07:00
Russell Cohen 084d2bc479 Reorder injection of std to get better compilation error 2018-04-16 12:28:30 -07:00
kennytm 6c3e1d7555 Remove underscore_lifetimes and match_default_bindings from active feature list
These are already stabilized in 1.26.
2018-04-17 03:28:25 +08:00
Alex Crichton 598d836fff Stabilize x86/x86_64 SIMD
This commit stabilizes the SIMD in Rust for the x86/x86_64 platforms. Notably
this commit is stabilizing:

* The `std::arch::{x86, x86_64}` modules and the intrinsics contained inside.
* The `is_x86_feature_detected!` macro in the standard library
* The `#[target_feature(enable = "...")]` attribute
* The `#[cfg(target_feature = "...")]` matcher

Stabilization of the module and intrinsics were primarily done in
rust-lang-nursery/stdsimd#414 and the two attribute stabilizations are done in
this commit. The standard library is also tweaked a bit with the new way that
stdsimd is integrated.

Note that other architectures like `std::arch::arm` are not stabilized as part
of this commit, they will likely stabilize in the future after they've been
implemented and fleshed out. Similarly the `std::simd` module is also not being
stabilized in this commit, only `std::arch`. Finally, nothing related to `__m64`
is stabilized in this commit either (MMX), only SSE and up types and intrinsics
are stabilized.

Closes #29717
Closes #44839
Closes #48556
2018-04-16 07:25:10 -07:00
bors d6ba1b9b02 Auto merge of #49719 - mark-i-m:no_sep, r=petrochenkov
Update `?` repetition disambiguation.

**Do not merge** (yet)

This is a test implementation of some ideas from discussion in https://github.com/rust-lang/rust/issues/48075 . This PR
- disallows `?` repetition from taking a separator, since the separator is never used.
- disallows the use of `?` as a separator. This allows patterns like `$(a)?+` to match `+` and `a+` rather than `a?a?a`. This is a _breaking change_, but maybe that's ok? Perhaps a crater run is the right approach?

cc @durka @alexreg @nikomatsakis
2018-04-16 00:06:10 +00:00
Zack M. Davis e77110e1f6 don't see issue #0
The unstable-feature attribute requires an issue (neglecting it is
E0547), which gets used in the error messages. Unfortunately, there are
some cases where "0" is apparently used a placeholder where no issue
exists, directing the user to see the (nonexistent) issue #0. (It would
have been better to either let `issue` be optional—compare to how issue
is an `Option<u32>` in the feature-gate declarations in
libsyntax/feature-gate.rs—or actually require that an issue be created.)
Rather than endeavoring to change how `#[unstable]` works at this time
(given competing contributor and reviewer priorities), this simple patch
proposes the less-ambitious solution of just not adding the "(see
issue)" note when the number is zero.

Resolves #49983.
2018-04-15 11:12:33 -07:00
bors d4d43e2483 Auto merge of #48173 - GuillaumeGomez:error-codes-libsyntax_ext, r=estebank
Add error codes for libsyntax_ext

I intend to add error codes for `libsyntax_ext` as well. However, they cannot be used at stage 0 directly so I thought it might be possible to enable them at the stage 1 only so we can have access to the macros. However, the error code registration seems to not work this way. Currently I get the following error:

```
error: used diagnostic code E0660 not registered
  --> libsyntax_ext/asm.rs:93:25
   |
93 |                         span_err!(cx, sp, E0660, "malformed inline assembly");
   |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: used diagnostic code E0661 not registered
   --> libsyntax_ext/asm.rs:151:33
    |
151 | /                                 span_err!(cx, sp, E0661,
152 | |                                           "output operand constraint lacks '=' or '+'");
    | |________________________________________________________________________________________^
    |
    = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: aborting due to 2 previous errors

error: Could not compile `syntax_ext`.
```

If anyone has an idea, I'd gladly take it. I'm trying to figure this out on my side as well. I also opened this PR to know if it was worth it to continue (maybe we don't want this?).

Anyway, any answer for both questions is very welcome!

cc @rust-lang/compiler
2018-04-15 01:26:11 +00:00
Guillaume Gomez 44c686113f Add error codes for libsyntax_ext 2018-04-14 17:25:35 +02:00
kennytm 95b7e6fe92 Rollup merge of #49852 - alexcrichton:fix-more-proc-macros, r=nrc
proc_macro: Avoid cached TokenStream more often

This commit adds even more pessimization to use the cached `TokenStream` inside
of an AST node. As a reminder the `proc_macro` API requires taking an arbitrary
AST node and transforming it back into a `TokenStream` to hand off to a
procedural macro. Such functionality isn't actually implemented in rustc today,
so the way `proc_macro` works today is that it stringifies an AST node and then
reparses for a list of tokens.

This strategy unfortunately loses all span information, so we try to avoid it
whenever possible. Implemented in #43230 some AST nodes have a `TokenStream`
cache representing the tokens they were originally parsed from. This
`TokenStream` cache, however, has turned out to not always reflect the current
state of the item when it's being tokenized. For example `#[cfg]` processing or
macro expansion could modify the state of an item. Consequently we've seen a
number of bugs (#48644 and #49846) related to using this stale cache.

This commit tweaks the usage of the cached `TokenStream` to compare it to our
lossy stringification of the token stream. If the tokens that make up the cache
and the stringified token stream are the same then we return the cached version
(which has correct span information). If they differ, however, then we will
return the stringified version as the cache has been invalidated and we just
haven't figured that out.

Closes #48644
Closes #49846
2018-04-14 15:21:19 +08:00
Vadim Petrochenkov 7e1f73beb6 macros: Do not match on "complex" nonterminals requiring AST comparisons 2018-04-14 02:28:39 +03:00