Add rust-dev component to support rustc development
This is preparatory work for permitting rustc developers to use CI-built LLVM rather than building it locally. Unlike distro-built LLVM, CI built LLVM is essentially guaranteed to behave perfectly for local development -- it is fully up to date, and carries all necessary patches.
This is a separate PR from #76349 because it needs to land before that one, since we want a master build with the full CI LLVM to be available for easier testing.
This currently includes libLLVM, llvm-config, and FileCheck, but will perhaps
expand to more tooling overtime. It should be considered entirely unstable and
may change at any time.
This is generally a good idea, and will help with being able to build bootstrap
without Python over time as it means we can "just" build with cargo +beta build
rather than needing the user to set environment variables. This is a minor step,
but a necessary one on that road.
Often when modifying compiler code you'll miss that you've changed an API used
by unit tests, since x.py check didn't previously catch that.
It's also useful to have this for editing with rust-analyzer and similar tooling
where editing tests previously didn't notify you of errors in test files.
Allow --bess ing expect-tests in tools
I haven't tried this, but I think this should do the trick, as `RustdocCrate` is a special step in bootstrap, which uses `tool_caro`
r? @ghost
Set ninja=true by default
Ninja substantially improves LLVM build time. On a 96-way system, using
Make took 248s, and using Ninja took 161s, a 35% improvement.
We already require a variety of tools to build Rust. If someone wants to
build without Ninja (for instance, to minimize the set of packages
required to bootstrap a new target), they can easily set `ninja=false`
in `config.toml`. Our defaults should help people build Rust (and LLVM)
faster, to speed up development.
Ninja substantially improves LLVM build time. On a 96-way system, using
Make took 248s, and using Ninja took 161s, a 35% improvement.
We already require a variety of tools to build Rust. If someone wants to
build without Ninja (for instance, to minimize the set of packages
required to bootstrap a new target), they can easily set `ninja=false`
in `config.toml`. Our defaults should help people build Rust (and LLVM)
faster, to speed up development.
Introduce expect snapshot testing library into rustc
Snapshot testing is a technique for writing maintainable unit tests.
Unlike usual `assert_eq!` tests, snapshot tests allow
to *automatically* upgrade expected values on test failure.
In a sense, snapshot tests are inline-version of our beloved
UI-tests.
Example:
A particular library we use, `expect_test` provides an `expect!`
macro, which creates a sort of self-updating string literal (by using
`file!` macro). Self-update is triggered by setting `UPDATE_EXPECT`
environmental variable (this info is printed during the test failure).
This library was extracted from rust-analyzer, where we use it for
most of our tests.
There are some other, more popular snapshot testing libraries:
* https://github.com/mitsuhiko/insta
* https://github.com/aaronabramov/k9
The main differences of `expect` are:
* first-class snapshot objects (so, tests can be written as functions,
rather than as macros)
* focus on inline-snapshots (but file snapshots are also supported)
* restricted feature set (only `assert_eq` and `assert_debug_eq`)
* no extra runtime (ie, no `cargo insta`)
See rust-analyzer/rust-analyzer#5101 for a
an extended comparison.
It is unclear if this testing style will stick with rustc in the long
run. At the moment, rustc is mainly tested via integrated UI tests.
But in the library-ified world, unit-tests will become somewhat more
important (that's why use use `rustc_lexer` library-ified library as
an example in this PR). Given that the cost of removal shouldn't be
too high, it probably makes sense to just see if this flies!
Fix windows-gnu host cross-compilation
Fixes https://github.com/rust-lang/rust/issues/64218
Also turns out it's faster to run Linux virtual machine on Windows and cross-compile `./x.py dist` than doing it on Windows directly...
Snapshot testing is a technique for writing maintainable unit tests.
Unlike usual `assert_eq!` tests, snapshot tests allow
to *automatically* upgrade expected values on test failure.
In a sense, snapshot tests are inline-version of our beloved
UI-tests.
Example:
A particular library we use, `expect_test` provides an `expect!`
macro, which creates a sort of self-updating string literal (by using
`file!` macro). Self-update is triggered by setting `UPDATE_EXPECT`
environmental variable (this info is printed during the test failure).
This library was extracted from rust-analyzer, where we use it for
most of our tests.
There are some other, more popular snapshot testing libraries:
* https://github.com/mitsuhiko/insta
* https://github.com/aaronabramov/k9
The main differences of `expect` are:
* first-class snapshot objects (so, tests can be written as functions,
rather than as macros)
* focus on inline-snapshots (but file snapshots are also supported)
* restricted feature set (only `assert_eq` and `assert_debug_eq`)
* no extra runtime (ie, no `cargo insta`)
See https://github.com/rust-analyzer/rust-analyzer/pull/5101 for a
an extended comparison.
It is unclear if this testing style will stick with rustc in the long
run. At the moment, rustc is mainly tested via integrated UI tests.
But in the library-ified world, unit-tests will become somewhat more
important (that's why use use `rustc_lexer` library-ified library as
an example in this PR). Given that the cost of removal shouldn't be
too high, it probably makes sense to just see if this flies!
Adjust installation place for compiler docs
This avoids conflicts when installing with rustup; rustup does not currently
support overlapping installations.
r? @matthiaskrgr
cmake-rs@8141f0e changed the logic for handling asm compiler flags.
This change was pulled in with the cmake 0.1.42 -> 0.1.44 update.
This introduced two new flags to the LLVM build, breaking it:
"-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64"
"-DCMAKE_ASM_COMPILER=/usr/bin/cc"
This patch should resolve the breakage by handling it in bootstrap.
Add option to use the new symbol mangling in rustc/std
I don't know if this causes problems in some cases -- maybe it should be on by default for at least rustc. I've never encountered problems with it other than tools not supporting it, though.
cc @nnethercote
r? @eddyb
Add sanitizer support on FreeBSD
Restarting #47337. Everything is better now, no more weird llvm problems, well not everything:
Unfortunately, the sanitizers don't have proper support for versioned symbols (https://github.com/google/sanitizers/issues/628), so `libc`'s usage of `stat@FBSD_1.0` and so on explodes, e.g. in calling `std::fs::metadata`.
Building std (now easy thanks to cargo `-Zbuild-std`) and libc with `freebsd12/13` config via the `LIBC_CI=1` env variable is a good workaround…
```
LIBC_CI=1 RUSTFLAGS="-Z sanitizer=address" cargo +san-test -Zbuild-std run --target x86_64-unknown-freebsd --verbose
```
…*except* std won't build because there's no `st_lspare` in the ino64 version of the struct, so an std patch is required:
```diff
--- i/src/libstd/os/freebsd/fs.rs
+++ w/src/libstd/os/freebsd/fs.rs
@@ -66,8 +66,6 @@ pub trait MetadataExt {
fn st_flags(&self) -> u32;
#[stable(feature = "metadata_ext2", since = "1.8.0")]
fn st_gen(&self) -> u32;
- #[stable(feature = "metadata_ext2", since = "1.8.0")]
- fn st_lspare(&self) -> u32;
}
#[stable(feature = "metadata_ext", since = "1.1.0")]
@@ -136,7 +134,4 @@ impl MetadataExt for Metadata {
fn st_flags(&self) -> u32 {
self.as_inner().as_inner().st_flags as u32
}
- fn st_lspare(&self) -> u32 {
- self.as_inner().as_inner().st_lspare as u32
- }
}
```
I guess std could like.. detect that `libc` isn't built for the old ABI, and replace the implementation of `st_lspare` with a panic?
Fix crate-version with rustdoc in bootstrap.
Cargo will now automatically use the `--crate-version` flag (see https://github.com/rust-lang/cargo/pull/8509). Cargo has special handling to avoid passing the flag if it is passed in via RUSTDOCFLAGS, but the `rustdoc` wrapper circumvents that check. This causes a problem because rustdoc will fail if the flag is passed in twice. Fix this by using RUSTDOCFLAGS.
This will be necessary when 1.47 is promoted to beta, but should be safe to do now.
Set CMAKE_SYSTEM_NAME when cross-compiling
Configure CMAKE_SYSTEM_NAME when cross-compiling in `configure_cmake`,
to tell CMake about target system. Previously this was done only for
LLVM step and now applies more generally to steps using cmake.
Helps with #74576.
Configure CMAKE_SYSTEM_NAME when cross-compiling in `configure_cmake`,
to tell CMake about target system. Previously this was done only for
LLVM step and now applies more generally to steps using cmake.
Make rust.use-lld config option work with non MSVC targets
Builds fine and passes tests on Linux.
Not overriding `use-lld` by `linker` makes sense on those platforms since very old GCC versions don't understand `-fuse-ld=lld`. This allows pointing to newer GCC or Clang that will know how to call LLD.
Tyler Mandry
