Don't pass --test-args to `python -m unitest`
The args for unittest and cargo test are mutually incompatible. Suggest that people use `python -m unittest ...` manually instead.
This also changes `bootstrap_test.py` to be easier to run standalone; see the commit for details.
r? `@clubby789` cc https://github.com/rust-lang/rust/pull/112281#discussion_r1248849172
The args for unittest and cargo test are mutually incompatible. Suggest that people use `python -m unittest ...` manually instead.
This also changes `bootstrap_test.py` to be easier to run standalone; see the commit for details.
Convert `run-make/coverage-reports` tests to use a custom compiletest mode
I was frustrated by the fact that most of the coverage tests are glued together with makefiles and shell scripts, so I tried my hand at converting most of them over to a newly-implemented `run-coverage` mode/suite in compiletest.
This ~~*mostly*~~ resolves#85009, ~~though I've left a small number of the existing tests as-is because they would require more work to fix/support~~.
---
I had time to go back and add support for the more troublesome tests that I had initially skipped over, so this PR now manages to completely get rid of `run-make/coverage-reports`.
---
The patches are arranged as follows:
- Declare the new mode/suite in bootstrap
- Small changes to compiletest that will be used by the new mode
- Implement the new mode in compiletest
- Migrate most of the tests over
- Add more code to bootstrap and compiletest to support the remaining tests
- Migrate the remaining tests (with some temporary hacks to avoid re-blessing them)
- Remove the temporary hacks and re-bless the migrated tests
- Remove the unused remnants of `run-make/coverage-reports`
Switch some more Steps to `builder.msg`
In particular, this hides compiletest suites under a dropdown - some builders are configured with `verbose-tests`, which makes the suites take up many thousands of lines.
before:
```
Documenting stage1 cargo (x86_64-unknown-linux-gnu)
Check compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
```
after:
```
::group::Documenting stage1 cargo (x86_64-unknown-linux-gnu)
::group::Testing stage1 compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu)
```
found this from https://github.com/rust-lang-ci/rust/actions/runs/5366993848/jobs/9736792611
r? `@oli-obk`
Test the cargo args generated by bootstrap.py
I recommend reviewing this commit-by-commit using the instructions in https://rustc-dev-guide.rust-lang.org/git.html#moving-large-sections-of-code.
- Test cargo arguments passed by bootstrap.py
This moves a lot of code around, but the logic itself is not too terribly complicated.
- Move almost all logic in `def bootstrap` to the `RustBuild` class, to avoid mixing setting configuration with running commands
- Update various doctests to the new (more complete) RustBuild config. In particular, don't pretend that `bin_root` supports `build` being unset.
- Change `parse_args` not to use a global, to allow testing it
- Set BUILD_DIR appropriately so bootstrap.py doesn't panic because cargo isn't found
- Allow passing arguments to `bootstrap_test.py`
Previous, it used the built-in test runner, which doesn't support options unless they're manually passed in the script.
- Fix progress messages for configure in bootstrap_test.py
Before it would unconditionally print `configure-args = []`.
r? `@albertlarsan68` cc https://github.com/rust-lang/rust/pull/112089https://github.com/rust-lang/rust/pull/111979#issuecomment-1568525699
In particular, this hides compiletest suites under a dropdown - some builders are configured with `verbose-tests`, which makes the suites take up many thousands of lines.
before:
```
Documenting stage1 cargo (x86_64-unknown-linux-gnu)
Check compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
```
after:
```
::group::Documenting stage1 cargo (x86_64-unknown-linux-gnu)
::group::Testing stage1 compiletest suite=rustdoc-js mode=js-doc-test (x86_64-unknown-linux-gnu)
```
The old code actually replaced `.json` with `.bak` (so, `settings.bak`),
rather than appending `.bak` as claimed (`settings.json.bak`).
`Path::set_extension` can instead be used with dots:
> The new extension may contain dots and will be used in its entirety,
> but only the part after the final dot will be reflected in
> self.extension.
This moves a lot of code around, but the logic itself is not too terribly complicated.
- Move almost all logic in `def bootstrap` to the `RustBuild` class, to avoid mixing setting configuration with running commands
- Update various doctests to the new (more complete) RustBuild config. In particular, don't pretend that `bin_root` supports `build` being unset.
- Change `parse_args` not to use a global, to allow testing it
- Set BUILD_DIR appropriately so bootstrap.py doesn't panic because cargo isn't found
Don't try to auto-bless 32-bit `mir-opt` tests on ARM Mac hosts
#112418 added special support for automatically blessing 32-bit output on 64-bit hosts, for the subset of `mir-opt` tests that are pointer-width-dependent.
This relies on the 64-bit host having some corresponding 32-bit target that can be built “easily”. For most 64-bit hosts this is fine, but ARM Macs don't have a corresponding 32-bit target. (There have never been 32-bit ARM Macs, and ARM Macs don't have the libraries needed for building `i686-apple-darwin`.)
There is an entry for `("i686-apple-darwin", "aarch64-apple-darwin")` in the list of corresponding 32-bit platforms, but this doesn't actually work on ARM Macs. Instead, the bootstrap invocation fails to build the necessary 32-bit target support, and nothing gets tested or blessed.
According to [this Zulip thread](https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Can't.20bless.20any.20mir-opt.20tests.20on.20aarch64.20Mac/near/367109789), that mapping was only added because the author assumed it would work. But since it doesn't actually work on ARM Macs, the solution is to just remove that mapping.
With the mapping removed, ARM Macs still can't auto-bless 32-bit output (they will see a warning instead), but at least they can now bless the output of `mir-opt` tests that don't care about pointer width.
Blessing 32-bit tests on 64-bit hosts relies on having a corresponding 32-bit
target that can be built "easily" on those hosts.
ARM Macs don't have a corresponding 32-bit target, so trying to build one is
usually going to fail.
Add support for targets without unwinding in `mir-opt`, and improve `--bless` for it
The main goal of this PR is to add support for targets without unwinding support in the `mir-opt` test suite, by adding the `EMIT_MIR_FOR_EACH_PANIC_STRATEGY` comment. Similarly to 32bit vs 64bit, when that comment is present, blessed output files will have the `.panic-unwind` or `.panic-abort` suffix, and the right one will be chosen depending on the target's panic strategy.
The `EMIT_MIR_FOR_EACH_PANIC_STRATEGY` comment replaced all the `ignore-wasm32` comments in the `mir-opt` test suite, as those comments were added due to `wasm32` being a target without unwinding support. The comment was also added on other tests that were only executed on x86 but were still panic strategy dependent.
The `mir-opt` suite was then blessed, which caused a ton of churn as most of the existing output files had to be renamed and (mostly) duplicated with the abort strategy.
---
After [asking on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/mir-opt.20tests.20and.20panic.3Dabort), the main concern about this change is it'd make blessing the `mir-opt` suite even harder, as you'd need to both bless it with an unwinding target and an aborting target. This exacerbated the current situation, where you'd need to bless it with a 32bit and a 64bit target already.
Because of that, this PR also makes significant enhancements to `--bless` for the `mir-opt` suite, where it will automatically bless the suite four times with different targets, while requiring minimal cross-compilation.
To handle the 32bit vs 64bit blessing, there is now an hardcoded list of target mapping between 32bit and 64bit. The goal of the list is to find a related target that will *probably* work without requiring additional cross-compilation toolchains on the system. If a mapping is found, bootstrap will bless the suite with both targets, otherwise just with the current target.
To handle the panic strategy blessing (abort vs unwind), I had to resort to what I call "synthetic targets". For each of the target we're blessing (so either the current one, or a 32bit and a 64bit depending on the previous paragraph), bootstrap will extract the JSON spec of the target and change it to include `"panic-strategy": "abort"`. It will then build the standard library with this synthetic target, and bless the `mir-opt` suite with it.
As a result of these changes, blessing the `mir-opt` suite will actually bless it two or four times with different targets, ensuring all possible variants are actually blessed.
---
This PR is best reviewed commit-by-commit.
r? `@jyn514`
cc `@saethlin` `@oli-obk`
Matthias Krüger