only run --all-targets in stage0 for Std
Repro'd the issue with `python3 x.py check --stage 1 library/std` and tested the fix with the same command.
r? `@jyn514`
I tried to implement [this solution](https://github.com/rust-lang/rust/issues/98720#issuecomment-1184435462), but didn't have any luck. I don't think I fully understood what needed to be done. However, I would love to be mentored on it since it would be a more correct solution, and I can learn more about how bootstrap works.
- use `path` instead of `paths`
- don't mark rust-analyzer as an optional tool
- print the cargo command that's run in the proc-macro-test build script
this originally was part of a change to fix `test --stage 0 rust-analyzer`,
but I'm going to leave that for a separate PR so it's easier to review.
Support `-A`, `-W`, `-D` and `-F` when running `./x.py clippy`
Resolves#97059
This PR adds support for `-A`, `-W`, `-D` and `-F` when running `./x.py clippy`.
Add `x {check,build,doc} {compiler,library}` aliases.
While working on https://github.com/rust-lang/rust/pull/95503, I realized that it will interfere with existing command lines:
Currently people run `x build library/std` expecting it to "add all library crates to the sysroot",
but after that change, it will *only* build `libstd` and its dependencies (and add them to the sysroot), not libtest or libproc_macro.
That will work for local testing in most cases, but could be confusing. Even if not, though, I think `x build library` is more clear about what actually happens than the current `x build library/std`.
The intended end goal is something like:
- For check/build/doc, we have library + compiler aliases, which correspond to basically "most possible" for that piece. This is the intended path of entry (rather than library/test or similar as today) for when you just want the thing to work -- for example, getting a compiler that is "crates.io-compatible" would be roughly `x.py build library`). #95504
- Specific crate invocations build up to that crate, which means that if you don't care about tests you probably want x.py build library/proc_macro or library/std for faster build times. #95503
Note that this is already implemented today for the `doc` command and seems to work pretty well in practice.
I plan to change the dev-guide and various instructions in the README to `build library` once this is merged.
`@rustbot` label +A-rustbuild
Bootstrap already allows selecting these in `PathSet::has`, which allows
any string that matches the end of a full path.
I found these by adding `assert!(path.exists())` in `StepDescription::paths`.
I think ideally we wouldn't have any aliases that aren't paths, but I've held
off on enforcing that here since it may be controversial, I'll open a separate PR.
While working on https://github.com/rust-lang/rust/pull/95503,
I realized that this will interfere with existing command lines:
Currently people run `x build library/std` expecting it to be added to the sysroot,
but after that change, it will *only* build `libstd` without making it available
for the toolchain.
It's debatable whether that's a breaking change that will be accepted; if so, this PR is absolutely
necessary to make sure there's a command for "build the standard library and add it to the sysroot".
Even if not, though, I think `x build library` is more clear about what actually happens than the
current `x build library/std`.
For consistency, also add support for `compiler` and all other command variants. Note that `doc
compiler` was already supported, so in a sense this is just fixing an existing inconsistency.
I plan to change the dev-guide and various instructions in the README to `build library` once this is merged.
Libgccjit codegen
This PR introduces a subtree for a gcc-based codegen backend to the repository, per decision in https://github.com/rust-lang/compiler-team/issues/442. We do not yet expect to ship this backend on nightly or run tests in CI, but we do verify that the backend checks (i.e., `cargo check`) successfully.
Work is expected to progress primarily in https://github.com/rust-lang/rustc_codegen_gcc, with semi-regular upstreaming, like with other subtrees.
Now that Cargo deduplicates diagnostics from different targets, this doesn't flood the console with
duplicate errors.
Note that this doesn't add `--all-targets` in `Builder::cargo` directly because `impl Step for Std`
actually wants to omit `--all-targets` the first time while it's still building libtest.
When passed `--all-targets`, this warns that the option isn't needed, but still continues to compile.
This only updates the submodules the first time they're needed, instead
of unconditionally the first time you run x.py.
Ideally, this would move *all* submodules and not exclude some tools and
backtrace. Unfortunately, cargo requires all `Cargo.toml` files in the
whole workspace to be present to build any crate.
On my machine, this takes the time for an initial submodule clone (for
`x.py --help`) from 55.70 to 15.87 seconds.
This uses exactly the same logic as the LLVM update used, modulo some
minor cleanups:
- Use a local variable for `src.join(relative_path)`
- Remove unnecessary arrays for `book!` macro and make the macro simpler to use
- Add more comments
This fixes the following error:
```
error: found crates (`serde_derive` and `serde_derive`) with colliding StableCrateId values.
--> /home/joshua/.local/lib/cargo/registry/src/github.com-1ecc6299db9ec823/cargo_metadata-0.8.2/src/lib.rs:162:1
|
162 | extern crate serde_derive;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
```
This is a bug in resolve (https://github.com/rust-lang/rust/issues/56935)
but it will be difficult to fix in the near future. This works around it
in the meantime by not copying serde_derive and other dependencies to
the sysroot when they're built for other tools. This rebuilds the
dependencies slightly more often than necessary, but avoids the crate
conflicts.
This can be reverted once #56935 is fixed.
- Add rustfmt to `x.py check`
- Update Cargo.lock
- Remove rustfmt from the toolstate list
- Make rustfmt an in-tree tool
- Give an error on `x.py test rustfmt` if rustfmt fails to build or if tests fail
- Don't call `save_toolstate` when testing rustfmt
This works by unconditionally passing -Z unstable-options to the
compiler. This has no affect in practice since bootstrap doesn't use
`deny(rustc::internal)`.
This was introduced as part of the MVP for `download-rustc`.
Unfortunately, it doesn't work very well:
- Steps are ignored by default, which makes it easy to leave out a step
that should be built. For example, the MVP forgot to enable any tests,
so it was *only* possible to build locally.
- It didn't work correctly even when it was enabled: calling
`builder.ensure()` would completely ignore the constant and rebuild the
step anyway. This has no obvious fix since `ensure()` has to return a
`Step::Output`.
Instead, this handles `download-rustc` in `impl Step for Rustc` and
`impl Step for Std`, which to my knowledge are the only build steps that
don't first go through `impl Step for Sysroot` (`Rustc` is used for
the `rustc-dev` component).
See https://github.com/rust-lang/rust/pull/79540#discussion_r563350075
and https://github.com/rust-lang/rust/issues/81930 for further context.
Here are some example runs with these changes and `download-rustc`
enabled:
```
$ x.py build src/tools/clippy
Building stage1 tool clippy-driver (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 1m 09s
Building stage1 tool cargo-clippy (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.11s
$ x.py test src/tools/clippy
Updating only changed submodules
Submodules updated in 0.01 seconds
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Building stage1 tool clippy-driver (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.09s
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.28s
Finished release [optimized] target(s) in 15.26s
Running build/x86_64-unknown-linux-gnu/stage1-tools/x86_64-unknown-linux-gnu/release/deps/clippy_driver-8b407b140e0aa91c
test result: ok. 592 passed; 0 failed; 3 ignored; 0 measured; 0 filtered out
$ x.py build src/tools/rustdoc
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 41.28s
Build completed successfully in 0:00:41
$ x.py test src/test/rustdoc-ui
Building stage0 tool compiletest (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.12s
Building rustdoc for stage1 (x86_64-unknown-linux-gnu)
Finished release [optimized] target(s) in 0.10s
test result: ok. 105 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 8.15s
$ x.py build compiler/rustc
Finished dev [unoptimized + debuginfo] target(s) in 0.09s
Build completed successfully in 0:00:00
```
Note a few things:
- Clippy depends on stage1 rustc-dev artifacts, but rustc didn't have to
be recompiled. Instead, the artifacts were copied automatically.
- All steps are always enabled. There is no danger of forgetting a step,
since only the entrypoints have to handle `download-rustc`.
- Building the compiler (`compiler/rustc`) automatically does no work.
- Use the same compiler for stage0 and stage1. This should be fixed at
some point (so bootstrap isn't constantly rebuilt).
- Make sure `x.py build` and `x.py check` work.
- Use `git merge-base` to determine the most recent commit to download.
- Copy stage0 to the various sysroots in `Sysroot`, and delegate to
Sysroot in Assemble. Leave all other code unchanged.
- Rename date -> key
This can also be a commit hash, so 'date' is no longer a good name.
- Add the commented-out option to config.toml.example
- Disable all steps by default when `download-rustc` is enabled
Most steps don't make sense when downloading a compiler, because they'll
be pre-built in the sysroot. Only enable the ones that might be useful,
in particular Rustdoc and all `check` steps.
At some point, this should probably enable other tools, but rustdoc is
enough to test out `download-rustc`.
- Don't print 'Skipping' twice in a row
Bootstrap forcibly enables a dry run if it isn't already set, so
previously it would print the message twice:
```
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
Skipping bootstrap::compile::Std because it is not enabled for `download-rustc`
```
Now it correctly only prints once.
## Future work
- Add FIXME about supporting beta commits
- Debug logging will never work. This should be fixed.
This silences the following clippy lints in ./x.py clippy:
many_single_char_names (there are a lot of warnings caused by stdarch)
collapsible_if (can reduce readability)
type_complexity
missing_safety_doc (there are almost 3K warnings issued)
too_many_arguments
needless_lifetimes (people want 'tcx lifetimes etc)
wrong_self_convention (warns about from_..(), to_..(), into_..().. fns that do or do not take self by reference.
Support enable/disable sanitizers/profiler per target
This PR add options under `[target.*]` of `config.toml` which can enable or disable sanitizers/profiler runtime for corresponding target.
If these options are empty, the global options under `[build]` will take effect.
Fix#78329
Here's the error for rustdoc:
```
Checking rustdoc artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
error: no library targets found in package `rustdoc-tool`
```
This matches Cargo behavior and avoids the (somewhat expensive) double checking,
as well as the unfortunate duplicate error messages (#76822,
rust-lang/cargo#5128).
Often when modifying compiler code you'll miss that you've changed an API used
by unit tests, since x.py check didn't previously catch that.
It's also useful to have this for editing with rust-analyzer and similar tooling
where editing tests previously didn't notify you of errors in test files.
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
Oli Scherer