Forbid *-pass directives in incremental tests

Incremental tests that would have used the `check-pass`, `build-pass`, or
`run-pass` directives should instead use a revision name starting with
`cpass`/`bpass`/`rpass` as appropriate.
This commit is contained in:
Zalathar
2026-04-21 17:32:53 +10:00
parent edbb86264d
commit 87ec57f3a1
3 changed files with 7 additions and 14 deletions
@@ -70,11 +70,11 @@ See [Controlling pass/fail expectations](ui.md#controlling-passfail-expectations
| Directive | Explanation | Supported test suites | Possible values |
|-----------------------------|---------------------------------------------|-------------------------------------------|-----------------|
| `check-pass` | Building (no codegen) should pass | `ui`, `crashes`, `incremental` | N/A |
| `check-pass` | Building (no codegen) should pass | `ui`, `crashes` | N/A |
| `check-fail` | Building (no codegen) should fail | `ui`, `crashes` | N/A |
| `build-pass` | Building should pass | `ui`, `crashes`, `codegen`, `incremental` | N/A |
| `build-pass` | Building should pass | `ui`, `crashes`, `codegen` | N/A |
| `build-fail` | Building should fail | `ui`, `crashes` | N/A |
| `run-pass` | Program must exit with code `0` | `ui`, `crashes`, `incremental` | N/A |
| `run-pass` | Program must exit with code `0` | `ui`, `crashes` | N/A |
| `run-fail` | Program must exit with code `1..=127` | `ui`, `crashes` | N/A |
| `run-crash` | Program must crash | `ui` | N/A |
| `run-fail-or-crash` | Program must `run-fail` or `run-crash` | `ui` | N/A |
@@ -90,9 +90,9 @@ comparison](ui.md#output-comparison) and [Rustfix tests](ui.md#rustfix-tests) fo
| Directive | Explanation | Supported test suites | Possible values |
|-----------------------------------|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------|-----------------------------------------------------------------------------------------|
| `check-run-results` | Check run test binary `run-{pass,fail}` output snapshot | `ui`, `crashes`, `incremental` if `run-pass` | N/A |
| `error-pattern` | Check that output contains a specific string | `ui`, `crashes`, `incremental` if `run-pass` | String |
| `regex-error-pattern` | Check that output contains a regex pattern | `ui`, `crashes`, `incremental` if `run-pass` | Regex |
| `check-run-results` | Check run test binary `run-{pass,fail}` output snapshot | `ui`, `crashes`, `incremental` | N/A |
| `error-pattern` | Check that output contains a specific string | `ui`, `crashes`, `incremental` | String |
| `regex-error-pattern` | Check that output contains a regex pattern | `ui`, `crashes`, `incremental` | Regex |
| `check-stdout` | Check `stdout` against `error-pattern`s from running test binary[^check_stdout] | `ui`, `crashes`, `incremental` | N/A |
| `normalize-stderr-32bit` | Normalize actual stderr (for 32-bit platforms) with a rule `"<raw>" -> "<normalized>"` before comparing against snapshot | `ui`, `incremental` | `"<RAW>" -> "<NORMALIZED>"`, `<RAW>`/`<NORMALIZED>` is regex capture and replace syntax |
| `normalize-stderr-64bit` | Normalize actual stderr (for 64-bit platforms) with a rule `"<raw>" -> "<normalized>"` before comparing against snapshot | `ui`, `incremental` | `"<RAW>" -> "<NORMALIZED>"`, `<RAW>`/`<NORMALIZED>` is regex capture and replace syntax |
-7
View File
@@ -439,13 +439,6 @@ fn update_pass_mode(&mut self, ln: &DirectiveLine<'_>, config: &Config) {
(TestMode::Ui, _) => (),
(TestMode::Crashes, _) => (),
(TestMode::Codegen, "build-pass") => (),
(TestMode::Incremental, _) => {
// FIXME(Zalathar): This only detects forbidden directives that are
// declared _after_ the incompatible `//@ revisions:` directive(s).
if self.revisions.iter().any(|r| !r.starts_with("bfail")) {
panic!("`{s}` directive is only supported in `bfail` incremental tests")
}
}
(mode, _) => panic!("`{s}` directive is not supported in `{mode}` tests"),
};
let pass_mode = if config.parse_name_directive(ln, "check-pass") {
+1 -1
View File
@@ -333,7 +333,7 @@ fn should_compile_successfully(&self, pm: Option<PassMode>) -> bool {
{
true
} else if revision.starts_with("bfail") {
pm.is_some()
false
} else {
panic!("revision name must begin with `cpass`, `bfail`, `bpass`, or `rpass`");
}