Improve diagnose for unconditional panic when resource limit
Improve diagnostic message for similar issue rust-lang/rust#115021.
When `parallel_compiler=true`, the Rust compiler frontend sets `-Z threads` to match the number of cores, which is reasonable and common. However, in a constrained environment or with an excessive number of cores (such as 377 mentioned below 😑), it could consume all resources and cause a panic.
Setting a default maximum for `-Z threads` in a parallel compiler is challenging. However, the panic error message can guide the user to check the system limit and explicitly lower the thread count according to their needs.
```
14:55:47 thread 'main' panicked at /rustc/f1586001ace26df7cafeb6534eaf76fb2c5513e5/compiler/rustc_interface/src/util.rs:216:18:
14:55:47 called `Result::unwrap()` on an `Err` value: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }
...
14:55:47 note: compiler flags: --crate-type lib -C opt-level=z -C embed-bitcode=no -C linker=/cache/84996/rust-sdk/target/shim/aarch64-unknown-linux-ohos/clang -Z unstable-options -C symbol-mangling-version=v0 -Z panic-in-drop=abort -C codegen-units=1 -C debuginfo=1 -C embed-bitcode=yes -Z threads=377 -C link-arg=-Wl,--build-id=sha1 -Z binary-dep-depinfo
```
This prevents a warning about ABI relevant target features not being set
on x86 and arm. In addition it is required for miri to report correct
features in is_*_feature_detected!() if miri switches to the dummy backend.
Add a dummy codegen backend
This allows building a rustc capable of running the frontend without any backend present. While this may not seem all that useful, it allows running the frontend of rustc to report errors or running miri to interpret a program without any backend present. This is useful when you are trying to say run miri in the browser as upstream LLVM can't be compiled for wasm yet. Or to run rustc itself in miri like I did a while ago and caught some UB.
This allows building a rustc capable of running the frontend without any
backend present. While this may not seem all that useful, it allows
running the frontend of rustc to report errors or running miri to
interpret a program without any backend present. This is useful when you
are trying to say run miri in the browser as upstream LLVM can't be
compiled for wasm yet. Or to run rustc itself in miri like I did a while
ago and caught some UB.
Bazel requires knowledge of outputs from actions at analysis time,
including file or directory name. In order to work around the lack of
predictable output name for dwo files, we group the dwo files in a
subdirectory of --out-dir as a post-processing step before returning
control to bazel. Unfortunately some debugging workflows rely on
directly opening the dwo file rather than loading the merged dwp file,
and our trick of moving the files breaks those users. We can't just
hardlink the file or copy it, because with remote build execution we
wouldn't end up with the un-moved file copied back to the developer's
workstation. As a fix, we add this unstable flag that causes dwo files
to be written to a build-system-controllable location, which then lets
bazel hoover up the dwo files, but the objects also have the correct
path for the dwo files.
Refactor lint buffering to avoid requiring a giant enum
Lint buffering currently relies on a giant enum `BuiltinLintDiag` containing all the lints that might potentially get buffered. In addition to being an unwieldy enum in a central crate, this also makes `rustc_lint_defs` a build bottleneck: it depends on various types from various crates (with a steady pressure to add more), and many crates depend on it.
Having all of these variants in a separate crate also prevents detecting when a variant becomes unused, which we can do with a dedicated type defined and used in the same crate.
Refactor this to use a dyn trait, to allow using `LintDiagnostic` types directly.
Because the existing `BuiltinLintDiag` requires some additional types in order to decorate some variants, which are only available later in `rustc_lint`, use an enum `DecorateDiagCompat` to handle both the `dyn LintDiagnostic` case and the `BuiltinLintDiag` case.
---
With the infrastructure in place, use it to migrate three of the enum variants to use `LintDiagnostic` directly, as a proof of concept and to demonstrate that the net result is a reduction in code size and a removal of a boilerplate-heavy layer of indirection.
Also remove an unused `BuiltinLintDiag` variant.
Lint buffering currently relies on a giant enum `BuiltinLintDiag`
containing all the lints that might potentially get buffered. In
addition to being an unwieldy enum in a central crate, this also makes
`rustc_lint_defs` a build bottleneck: it depends on various types from
various crates (with a steady pressure to add more), and many crates
depend on it.
Having all of these variants in a separate crate also prevents detecting
when a variant becomes unused, which we can do with a dedicated type
defined and used in the same crate.
Refactor this to use a dyn trait, to allow using `LintDiagnostic` types
directly.
This requires boxing, but all of this is already on the slow path
(emitting an error).
Because the existing `BuiltinLintDiag` requires some additional types in
order to decorate some variants, which are only available later in
`rustc_lint`, use an enum `DecorateDiagCompat` to handle both the `dyn
LintDiagnostic` case and the `BuiltinLintDiag` case.
Before this change we had two different ways to attempt to locate the
sysroot which are inconsistently used:
* get_or_default_sysroot which tries to locate based on the 0th cli
argument and if that doesn't work falls back to locating it using the
librustc_driver.so location and returns a single path.,
* sysroot_candidates which takes the former and additionally does
another attempt at locating using librustc_driver.so except without
linux multiarch handling and then returns both paths.,
The latter was originally introduced to be able to locate the codegen
backend back when cg_llvm was dynamically linked even for a custom
driver when the --sysroot passed in does not contain a copy of cg_llvm.
Back then get_or_default_sysroot did not attempt to locate the sysroot
based on the location of librustc_driver.so yet. Because that is now
done, the only case where removing sysroot_candidates can break things
is if you have a custom driver inside what looks like a sysroot
including the lib/rustlib directory, but which is missing some parts of
the full sysroot like eg rust-lld.
Add a jobserver proxy to ensure at least one token is always held
This adds a jobserver proxy to ensure at least one token is always held by `rustc`. Currently with `-Z threads` `rustc` can temporarily give up all its tokens, causing `cargo` to spawn additional `rustc` instances beyond the job limit.
The current behavior causes an issue with `cargo fix` which has a global lock preventing concurrent `rustc` instances, but it also holds a jobserver token, causing a deadlock when `rustc` gives up its token. That is fixed by this PR.
Fixes https://github.com/rust-lang/rust/issues/67385.
Fixes https://github.com/rust-lang/rust/issues/133873.
Fixes https://github.com/rust-lang/rust/issues/140093.
Support for `f16` and `f128` is varied across targets, backends, and
backend versions. Eventually we would like to reach a point where all
backends support these approximately equally, but until then we have to
work around some of these nuances of support being observable.
Introduce the `cfg_target_has_reliable_f16_f128` internal feature, which
provides the following new configuration gates:
* `cfg(target_has_reliable_f16)`
* `cfg(target_has_reliable_f16_math)`
* `cfg(target_has_reliable_f128)`
* `cfg(target_has_reliable_f128_math)`
`reliable_f16` and `reliable_f128` indicate that basic arithmetic for
the type works correctly. The `_math` versions indicate that anything
relying on `libm` works correctly, since sometimes this hits a separate
class of codegen bugs.
These options match configuration set by the build script at [1]. The
logic for LLVM support is duplicated as-is from the same script. There
are a few possible updates that will come as a follow up.
The config introduced here is not planned to ever become stable, it is
only intended to replace the build scripts for `std` tests and
`compiler-builtins` that don't have any way to configure based on the
codegen backend.
MCP: https://github.com/rust-lang/compiler-team/issues/866
Closes: https://github.com/rust-lang/compiler-team/issues/866
[1]: https://github.com/rust-lang/rust/blob/555e1d0386f024a8359645c3217f4b3eae9be042/library/std/build.rs#L84-L186
Currently it is called twice, once with `allow_unstable` set to true and
once with it set to false. This results in some duplicated work. Most
notably, for the LLVM backend, `LLVMRustHasFeature` is called twice for
every feature, and it's moderately slow. For very short running
compilations on platforms with many features (e.g. a `check` build of
hello-world on x86) this is a significant fraction of runtime.
This commit changes `target_features_cfg` so it is only called once, and
it now returns a pair of feature sets. This halves the number of
`LLVMRustHasFeature` calls.
`rustc_span::symbol` defines some things that are re-exported from
`rustc_span`, such as `Symbol` and `sym`. But it doesn't re-export some
closely related things such as `Ident` and `kw`. So you can do `use
rustc_span::{Symbol, sym}` but you have to do `use
rustc_span::symbol::{Ident, kw}`, which is inconsistent for no good
reason.
This commit re-exports `Ident`, `kw`, and `MacroRulesNormalizedIdent`,
and changes many `rustc_span::symbol::` qualifiers in `compiler/` to
`rustc_span::`. This is a 200+ net line of code reduction, mostly
because many files with two `use rustc_span` items can be reduced to
one.
Delete the `cfg(not(parallel))` serial compiler
Since it's inception a long time ago, the parallel compiler and its cfgs have been a maintenance burden. This was a necessary evil the allow iteration while not degrading performance because of synchronization overhead.
But this time is over. Thanks to the amazing work by the parallel working group (and the dyn sync crimes), the parallel compiler has now been fast enough to be shipped by default in nightly for quite a while now.
Stable and beta have still been on the serial compiler, because they can't use `-Zthreads` anyways.
But this is quite suboptimal:
- the maintenance burden still sucks
- we're not testing the serial compiler in nightly
Because of these reasons, it's time to end it. The serial compiler has served us well in the years since it was split from the parallel one, but it's over now.
Let the knight slay one head of the two-headed dragon!
#113349
Note that the default is still 1 thread, as more than 1 thread is still fairly broken.
cc `@onur-ozkan` to see if i did the bootstrap field removal correctly, `@SparrowLii` on the sync parts
Since it's inception a long time ago, the parallel compiler and its cfgs
have been a maintenance burden. This was a necessary evil the allow
iteration while not degrading performance because of synchronization
overhead.
But this time is over. Thanks to the amazing work by the parallel
working group (and the dyn sync crimes), the parallel compiler has now
been fast enough to be shipped by default in nightly for quite a while
now.
Stable and beta have still been on the serial compiler, because they
can't use `-Zthreads` anyways.
But this is quite suboptimal:
- the maintenance burden still sucks
- we're not testing the serial compiler in nightly
Because of these reasons, it's time to end it. The serial compiler has
served us well in the years since it was split from the parallel one,
but it's over now.
Let the knight slay one head of the two-headed dragon!
bjorn3