mirror of
https://github.com/rust-lang/rust.git
synced 2026-04-27 18:57:42 +03:00
Jonathan Brouwer
4c6755ca61
Allow `./x test` to run tests without doc tests and without benchmarks # Problem For Ferrocene we would like to run only the `coretests` and `alloctests` test suites when measuring code coverage. Running `corebenches` and `allocbenches` would alter the numbers, which is not compliant with the certification. This is currently not possible in bootstrap. By default `./x test` runs unit, integration and doc tests. `./x test --doc` only runs doc tests. So far, so good. The problem is that while `./x test --no-doc` stops bootstrap from executing doc tests, it surprisingly starts executing benchmarks (next to examples and bins as well). It basically behaves like `cargo test --all-targets`. # Solution This PR renames the existing `--no-doc` flag to `--all-targets` and keeps the same behaviour as before. Unfortunately it is not possible to internally switch from `cargo --bins --examples --tests --benches` to `cargo --all-targets` because it will fail e.g. `./x test library/alloc --all-targets` with an error like "use of unstable library feature `test`". Additionally, this PR add a `./x test --tests` flag (equivalent to `cargo test --tests`) that only executes unit and integration tests. Both changes we are doing in https://github.com/ferrocene/ferrocene anyways, but believe that they may be useful for other people as well and therefore would like to contribute them. Note that this PR does _not_ change the behaviour of either `./x test` or `./x test --doc`. ## Note on `test = true` While this change enables bootstrap to run tests without doc tests and without benchmarks, executing `./x test library/core library/alloc --tests` will still build and execute `corebenches` and `allocbenches`. What?! 😱 Why all of this effort to enable it then? Good question! The reason they are still executed is, that they are marked with `test = true` in their respective `Cargo.toml` ([corebenches](https://github.com/rust-lang/rust/blob/3f9853562c73af38a5e6af8b0da1b2734a327e19/library/coretests/Cargo.toml#L24), [allocbenches](https://github.com/rust-lang/rust/blob/3f9853562c73af38a5e6af8b0da1b2734a327e19/library/alloctests/Cargo.toml#L32)). @bjorn3 mentioned that it is important for upstream that those benchmarks are executed by default, even if no `./x --all-targets` is passed. This is perfectly possible with this PR. Benchmarks marked with `test = true` will be executed when calling either of `./x test`, `./x test --tests` or `./x --all-targets`. Therefore this PR does not include a commit to change that. We will just do this change in https://github.com/ferrocene/ferrocene. ## Open questions ### Error message I added one commit that adds an error message if a user passes `--no-doc` and points them to `--all-targets` and `--tests`. I think it might be nice to have, but if you think it is not necessary, I can remove it. # How to test this change You can see the change in action by running `./x test library/alloc --tests` and `./x test library/alloc --all-targets`. The first will execute `alloctests` and `allocbenches` (which is marked with `test = true`), while the second will additionally run `benches/vec_deque_append.rs` (which is _not_ marked with `test = true`).