Mirrors/rust
1
0
Fork 0
mirror of https://github.com/rust-lang/rust.git synced 2026-04-27 18:57:42 +03:00
Code Issues Packages Projects Releases Wiki Activity

sembr src/tests/adding.md

Browse Source
This commit is contained in:
Tshepang Mbambo
2026-04-08 08:52:00 +02:00
parent 9c0a07250f
commit da97a7f7bb
1 changed files with 53 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/doc/rustc-dev-guide/src/tests/adding.md
+53 -36
View File
@@ -2,12 +2,13 @@
**In general, we expect every PR that fixes a bug in rustc to come accompanied
by a regression test of some kind.** This test should fail in `main` but pass
after the PR. These tests are really useful for preventing us from repeating the
after the PR.
These tests are really useful for preventing us from repeating the
mistakes of the past.
The first thing to decide is which kind of test to add. This will depend on the
nature of the change and what you want to exercise. Here are some rough
guidelines:
The first thing to decide is which kind of test to add.
This will depend on the nature of the change and what you want to exercise.
Here are some rough guidelines:
- The majority of compiler tests are done with [compiletest].
- The majority of compiletest tests are [UI](ui.md) tests in the [`tests/ui`]
@@ -24,14 +25,17 @@ guidelines:
`library/${crate}tests/lib.rs`.
- If the code is part of an isolated system, and you are not testing compiler
output, consider using a [unit or integration test](intro.md#package-tests).
- Need to run rustdoc? Prefer a `rustdoc` or `rustdoc-ui` test. Occasionally
you'll need `rustdoc-js` as well.
- Need to run rustdoc?
Prefer a `rustdoc` or `rustdoc-ui` test.
Occasionally you'll need `rustdoc-js` as well.
- Other compiletest test suites are generally used for special purposes:
- Need to run gdb or lldb? Use the `debuginfo` test suite.
- Need to inspect LLVM IR or MIR IR? Use the `codegen` or `mir-opt` test
suites.
- Need to inspect the resulting binary in some way? Or if all the other test
suites are too limited for your purposes? Then use `run-make`.
- Need to run gdb or lldb?
Use the `debuginfo` test suite.
- Need to inspect LLVM IR or MIR IR?
Use the `codegen` or `mir-opt` test suites.
- Need to inspect the resulting binary in some way?
Or if all the other test suites are too limited for your purposes?
Then use `run-make`.
- Use `run-make-cargo` if you need to exercise in-tree `cargo` in conjunction
with in-tree `rustc`.
- Check out the [compiletest] chapter for more specialized test suites.
@@ -47,14 +51,16 @@ modified several years later, how can we make it easier for them?).
## UI test walkthrough
The following is a basic guide for creating a [UI test](ui.md), which is one of
the most common compiler tests. For this tutorial, we'll be adding a test for an
async error message.
the most common compiler tests.
For this tutorial, we'll be adding a test for an async error message.
### Step 1: Add a test file
The first step is to create a Rust source file somewhere in the [`tests/ui`]
tree. When creating a test, do your best to find a good location and name (see
[Test organization](ui.md#test-organization) for more). Since naming is the
tree.
When creating a test, do your best to find a good location and name (see
[Test organization](ui.md#test-organization) for more).
Since naming is the
hardest part of development, everything should be downhill from here!
Let's place our async test at `tests/ui/async-await/await-without-async.rs`:
@@ -77,19 +83,23 @@ A few things to notice about our test:
- The top should start with a short comment that [explains what the test is
for](#explanatory_comment).
- The `//@ edition:2018` comment is called a [directive](directives.md) which
provides instructions to compiletest on how to build the test. Here we need to
provides instructions to compiletest on how to build the test.
Here we need to
set the edition for `async` to work (the default is edition 2015).
- Following that is the source of the test. Try to keep it succinct and to the
point. This may require some effort if you are trying to minimize an example
- Following that is the source of the test.
Try to keep it succinct and to the point.
This may require some effort if you are trying to minimize an example
from a bug report.
- We end this test with an empty `fn main` function. This is because the default
- We end this test with an empty `fn main` function.
This is because the default
for UI tests is a `bin` crate-type, and we don't want the "main not found"
error in our test. Alternatively, you could add `#![crate_type="lib"]`.
error in our test.
Alternatively, you could add `#![crate_type="lib"]`.
### Step 2: Generate the expected output
The next step is to create the expected output snapshots from the compiler. This
can be done with the `--bless` option:
The next step is to create the expected output snapshots from the compiler.
This can be done with the `--bless` option:
```sh
./x test tests/ui/async-await/await-without-async.rs --bless
@@ -99,8 +109,8 @@ This will build the compiler (if it hasn't already been built), compile the
test, and place the output of the compiler in a file called
`tests/ui/async-await/await-without-async.stderr`.
However, this step will fail! You should see an error message, something like
this:
However, this step will fail!
You should see an error message, something like this:
> error: /rust/tests/ui/async-await/await-without-async.rs:7: unexpected
> error: '7:10: 7:16: `await` is only allowed inside `async` functions and
@@ -112,7 +122,8 @@ annotations in the source file.
### Step 3: Add error annotations
Every error needs to be annotated with a comment in the source with the text of
the error. In this case, we can add the following comment to our test file:
the error.
In this case, we can add the following comment to our test file:
```rust,ignore
fn bar() {
@@ -136,7 +147,8 @@ It should now pass, yay!
### Step 4: Review the output
Somewhat hand-in-hand with the previous step, you should inspect the `.stderr`
file that was created to see if it looks like how you expect. If you are adding
file that was created to see if it looks like how you expect.
If you are adding
a new diagnostic message, now would be a good time to also consider how readable
the message looks overall, particularly for people new to Rust.
@@ -161,9 +173,9 @@ You may notice some things look a little different than the regular compiler
output.
- The `$DIR` removes the path information which will differ between systems.
- The `LL` values replace the line numbers. That helps avoid small changes in
the source from triggering large diffs. See the
[Normalization](ui.md#normalization) section for more.
- The `LL` values replace the line numbers.
That helps avoid small changes in the source from triggering large diffs.
See the [Normalization](ui.md#normalization) section for more.
Around this stage, you may need to iterate over the last few steps a few times
to tweak your test, re-bless the test, and re-review the output.
@@ -171,8 +183,10 @@ to tweak your test, re-bless the test, and re-review the output.
### Step 5: Check other tests
Sometimes when adding or changing a diagnostic message, this will affect other
tests in the test suite. The final step before posting a PR is to check if you
have affected anything else. Running the UI suite is usually a good start:
tests in the test suite.
The final step before posting a PR is to check if you
have affected anything else.
Running the UI suite is usually a good start:
```sh
./x test tests/ui
@@ -188,16 +202,19 @@ You may also need to re-bless the output with the `--bless` flag.
## Comment explaining what the test is about
The first comment of a test file should **summarize the point of the test**, and
highlight what is important about it. If there is an issue number associated
with the test, include the issue number.
highlight what is important about it.
If there is an issue number associated with the test, include the issue number.
This comment doesn't have to be super extensive. Just something like "Regression
This comment doesn't have to be super extensive.
Just something like "Regression
test for #18060: match arms were matching in the wrong order." might already be
enough.
These comments are very useful to others later on when your test breaks, since
they often can highlight what the problem is. They are also useful if for some
they often can highlight what the problem is.
They are also useful if for some
reason the tests need to be refactored, since they let others know which parts
of the test were important. Often a test must be rewritten because it no longer
of the test were important.
Often a test must be rewritten because it no longer
tests what it was meant to test, and then it's useful to know what it *was*
meant to test exactly.