Commit Graph

4626 Commits

Author SHA1 Message Date
liushuyu 343a0bf5c2 bootstrap: add test cases for beta revision extraction 2023-05-20 18:33:17 -06:00
liushuyu 50e38e928e Read beta version from the version file ...
... if building from a source tarball
2023-05-20 17:43:18 -06:00
bors 6bacf5a544 Auto merge of #111798 - GuillaumeGomez:rollup-t3bus8o, r=GuillaumeGomez
Rollup of 5 pull requests

Successful merges:

 - #111450 (Use `OpaqueTypeKey` in query response)
 - #111726 (Migrate GUI colors test to original CSS color format)
 - #111746 (Merge some query impl modules into one)
 - #111765 (Migrate GUI colors test to original CSS color format)
 - #111771 (add `--remote-time` flag to curl for bootstrap)

r? `@ghost`
`@rustbot` modify labels: rollup
2023-05-20 16:07:54 +00:00
Guillaume Gomez a6ebaf8425 Rollup merge of #111771 - liushuyu:ubuntu/rep-stage0, r=ozkanonur
add `--remote-time` flag to curl for bootstrap

This pull request sets the timestamp of the downloaded stage0 binary according to the server-reported timestamp (if possible).
This allows make_orig-dl_tarball.sh to be more reproducible on the filesystem.
2023-05-20 15:37:25 +02:00
bors 4eb5225cdf Auto merge of #111413 - workingjubilee:bump-object-0-31-1, r=MarkSimulacrum
Bump object and thorin-dwp

Required to fix watchOS breakage.
2023-05-20 13:19:37 +00:00
Dylan DPC fa11c9e8ca Rollup merge of #111606 - jyn514:nightly-diagnostics, r=lcnr
very minor cleanups

- add `must_use` to `early_error_no_abort`

  this was already being used at its only callsite, but this ensures that new code remembers to use it if it's called in the future. found this while investigating https://github.com/rust-lang/rust/issues/110090.

- remove outdated and incorrect comment in `builder.rs`. `doc_rust_lang_org_channel` doesn't exist in rustdoc, it gets it from an env var instead: https://github.com/rust-lang/rust/blob/b275d2c30b6e88cc48747f349f7137076d450658/src/librustdoc/clean/utils.rs#L569-L573
2023-05-20 12:20:59 +05:30
Ximin Luo 0d7d2cae2f Set the timestamp of downloaded stage0 files ...
... using server-reported timestamp.
This allows us to track changes to the downloaded artifact more easily
and in a more reproducible manner.

Co-authored-by: Zixing Liu <zixing.liu@canonical.com>
2023-05-19 18:49:50 -06:00
jyn 04265621f9 very minor cleanups
- add `must_use` to `early_error_no_abort`

  this was already being used at its only callsite, but this ensures
that new code remembers to use it if it's called in the future.

- remove outdated and incorrect comment in `builder.rs`.
  `doc_rust_lang_org_channel` doesn't exist in rustdoc, it gets it from
an env var instead.
2023-05-18 08:06:47 -05:00
Dylan DPC f5a0c6356b Rollup merge of #111685 - Kobzol:bootstrap-typo, r=clubby789
Fix typo in bootstrap command description

Unless `duild` is some clever name for `dist build`, this is probably a typo.
2023-05-18 17:37:08 +05:30
Dylan DPC 57172a0675 Rollup merge of #111160 - chbaker0:update-serde, r=Mark-Simulacrum
Update serde in workspace and non-synced dependencies

The main workspace, bootstrap, cargo, miri, and rust-analyzer all lock serde to different versions. It's preferable to share the same version where possible, so update it.

Rustfmt is synced from another repository and has its own Cargo.lock, but since it's added to the overall workspace it should respect the version here.

Cargo is already at the latest version. Miri and rust-analyzer would require upstream updates.
2023-05-18 10:52:33 +05:30
Jakub Beránek 90678ccff5 Fix typo in bootstrap command description 2023-05-17 17:35:02 +02:00
yukang 5e2969e7d5 comment feedback 2023-05-14 17:41:53 +08:00
yukang 46489040e6 fix 2023-05-14 12:44:16 +08:00
yukang 3c1c2c03fc skip rustc version check in dry_run 2023-05-14 11:35:03 +08:00
yukang 69c7d82f42 Make sure the build.rustc version is either the same or 1 apart 2023-05-14 11:34:59 +08:00
bors 2e18605af2 Auto merge of #111388 - clubby789:clap-complete, r=jyn514
Generate shell completions for bootstrap with Clap

Now that #110693 has been merged, we can look at generating shell completions for x.py with `clap_complete`. Leaving this as draft for now as I'm not sure of the best way to integration the completion generator. Additionally, the generated completions for zsh are completely broken (will need to be resolved upstream, it doesn't seem to handle subcommands + global arguments well).
I don't have Fish installed and would be interested to know how well completions work there.

Alternative to #107827
2023-05-14 00:09:05 +00:00
Ralf Jung 18fd17d2cd run miri tests with MIR opts in rustc CI 2023-05-12 15:07:24 +02:00
Jubilee Young 7156ff67be Bump object and thorin-dwp
object -> 0.31.1
thorin-dwp -> 0.6.0

Required to fix watchOS breakage.
2023-05-10 21:36:22 -07:00
clubby789 a348f8aea5 Generate shell completions for bootstrap with Clap 2023-05-10 21:41:38 +01:00
Matthias Krüger efe697e133 Rollup merge of #111242 - wangkirin:support_rpath_independent_config, r=albertlarsan68
support set `rpath` option  for each target independently

Currently  the `rpath` option is a global config and it's effect on all targets.
But sometimes when developers edit the rustc code and try to release rust toolchains themselves, they may not want to add `rpath` in all targets  to avoid dynamically linked shared object library privilege escalation attack.
This PR supports set `rpath` option  for each target independently .
Common developers are not aware of the existence of this configuration  option and do not affect the existing development process. This configuration option takes effect only after developers explicitly sets .

r? ``@albertlarsan68``
2023-05-09 20:49:32 +02:00
bors 7e7483d26e Auto merge of #110152 - ChrisDenton:windows-sys, r=thomcc
Start using `windows sys` for Windows FFI bindings in std

Switch to using windows-sys for FFI. In order to avoid some currently contentious issues, this uses windows-bindgen to generate a smaller set of bindings instead of using the full crate.

Unlike the windows-sys crate, the generated bindings uses `*mut c_void` for handle types instead of `isize`. This to sidestep opsem concerns about mixing pointer types and integers between languages. Note that `SOCKET` remains defined as an integer but instead of being a usize, it's changed to fit the [standard library definition](https://github.com/rust-lang/rust/blob/a41fc00eaf352541008965fec0dee811e44373b3/library/std/src/os/windows/raw.rs#L12-L16):

```rust
#[cfg(target_pointer_width = "32")]
pub type SOCKET = u32;
#[cfg(target_pointer_width = "64")]
pub type SOCKET = u64;
```

The generated bindings also customizes the `#[link]` imports. I hope to switch to using raw-dylib but I don't want to tie that too closely with the switch to windows-sys.

---

Changes outside of the bindings are, for the most part, fairly minimal (e.g. some differences in `*mut` vs. `*const` or a few types differ). One issue is that our own bindings sometimes mix in higher level types, like `BorrowedHandle`. This is pretty adhoc though.
2023-05-09 05:20:41 +00:00
bors c86e7fb60f Auto merge of #111342 - Dylan-DPC:rollup-b5p6wzy, r=Dylan-DPC
Rollup of 7 pull requests

Successful merges:

 - #110297 (Make `(try_)subst_and_normalize_erasing_regions` take `EarlyBinder`)
 - #110827 (Fix lifetime suggestion for type aliases with objects in them)
 - #111022 (Use smaller ints for bitflags)
 - #111056 (Fix some suggestions where a `Box<T>` is expected.)
 - #111262 (Further normalize msvc-non-utf8-ouput)
 - #111265 (Make generics_of has_self on RPITITs delegate to the opaque)
 - #111323 (Give a more helpful error when running the rustc shim directly)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
2023-05-08 07:35:30 +00:00
bors ea0c22ea4f Auto merge of #106621 - ozkanonur:enable-elided-lifetimes-for-doctests, r=Mark-Simulacrum
enable `rust_2018_idioms` lint group for doctests

With this change, `rust_2018_idioms` lint group will be enabled for compiler/libstd doctests.

Resolves #106086
Resolves #99144

Signed-off-by: ozkanonur <work@onurozkan.dev>
2023-05-08 04:50:28 +00:00
Wang Qilin bb4976ab2d supplement CHANGELOG for add rpath in target section 2023-05-08 09:13:42 +08:00
Collin Baker 1d640e5338 Update serde in workspace and non-synced dependencies
The main workspace, bootstrap, cargo, miri, and rust-analyzer all lock
serde to different versions. It's preferable to share the same version
where possible, so update it.

Rustfmt is synced from another repository and has its own Cargo.lock,
but since it's added to the overall workspace it should respect the
version here.

Cargo is already at the latest version. Miri and rust-analyzer would
require upstream updates.
2023-05-07 16:47:15 -04:00
jyn 654f56e086 Give a more helpful error when running the rustc shim directly
cc https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Building.20.60coretests.60.20by.20hand
2023-05-07 11:20:16 -05:00
bors 70a779cb7d Auto merge of #110693 - clubby789:x-clap-take-2, r=Mark-Simulacrum
Migrate bootstrap to Clap-based argument parsing

Supercedes #108083

I chose to re-do the work rather than rebase the onto the large changes since the original PR. If it's preferred I can instead force-push the original PR to this version.

cc `@jyn514` `@albertlarsan68`
2023-05-07 02:36:15 +00:00
clubby789 32e27cc607 Migrate bootstrap to Clap-based arguments 2023-05-06 22:15:49 +01:00
ozkanonur 4e7c14fe9f enable rust_2018_idioms for doctests
Signed-off-by: ozkanonur <work@onurozkan.dev>
2023-05-07 00:12:29 +03:00
Chris Denton f9b3d6a525 Generate windows-sys bindings 2023-05-05 20:48:16 +01:00
Wang Qilin 23e443a5fa support enable rpath in each target independently 2023-05-05 17:46:10 +08:00
Yuki Okushi 650dc01a64 Rollup merge of #111223 - jyn514:free-args, r=clubby789
Use `free-args` consistently in bootstrap

Previously, this was only passed to miri and compiletest. Extended it to all other tests and binaries as well.

cc https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Running.20a.20single.20doctest

r? `@clubby789`
2023-05-05 12:46:28 +09:00
Yuki Okushi 5d85652684 Rollup merge of #111187 - krasimirgg:llvm-runtimes, r=jyn514
bootstrap: add llvm-project/runtimes to the sources

This is needed to build libunwind for LLVM 16:
https://discourse.llvm.org/t/runtimes-removed-support-for-llvm-enable-projects-in-libcxx-libcxxabi-and-libunwind/65707

Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/missing.20llvm-project.2Fruntimes.20in.20beta.20srcs

Checked by running `x.py dist` and verifying the produced source archive (build/dist/rustc-1.71.0-dev-src.tar.gz) contains the new directory.
2023-05-05 12:46:27 +09:00
jyn 0b6a79efbd Use free-args consistently in bootstrap
Previously, this was only passed to miri and compiletest. Extended it to
all other tests and binaries as well.
2023-05-04 17:35:46 -05:00
Matthias Krüger 75e8f87673 Rollup merge of #111201 - krasimirgg:add_gitmodules, r=jyn514
bootstrap: add .gitmodules to the sources

The bootstrap builder now expects this file to exist: https://github.com/rust-lang/rust/blob/6f8c0557e0b73c73a8a7163a15f4a5a3feca7d5c/src/bootstrap/builder.rs#L494

Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/missing.20llvm-project.2Fruntimes.20in.20beta.20srcs
2023-05-04 19:18:22 +02:00
Matthias Krüger 0098cd4e83 Rollup merge of #108865 - Zoxc:library-dummy-crate, r=jyn514
Add a `sysroot` crate to represent the standard library crates

This adds a dummy crate named `sysroot` to represent the standard library target instead of using the `test` crate. This allows the removal of `proc_macro` as a dependency of `test` allowing these 2 crates to build in parallel saving around 9 seconds locally.
2023-05-04 19:18:17 +02:00
Krasimir Georgiev 6530b1bd53 bootstrap: add .gitmodules to the sources
The bootstrap builder now expects this file to exist:
https://github.com/rust-lang/rust/blob/6f8c0557e0b73c73a8a7163a15f4a5a3feca7d5c/src/bootstrap/builder.rs#L494
2023-05-04 13:07:10 +00:00
Krasimir Georgiev 025c60310a bootstrap: add llvm-project/runtimes to the sources
This is needed to build libunwind for LLVM 16:
https://discourse.llvm.org/t/runtimes-removed-support-for-llvm-enable-projects-in-libcxx-libcxxabi-and-libunwind/65707

Doesn't work: with this, running `python3 x.py dist` produces a tar
archive that still doesn't contain the runtimes subdirectory?
2023-05-04 10:42:22 +00:00
Dylan DPC a47fa34558 Rollup merge of #111069 - ozkanonur:remove-pointless-fixme, r=albertlarsan68
remove pointless `FIXME` in `bootstrap::download`

The suggestion given by `FIXME` to use `CompilerMetadata` for `download_toolchain` in `bootstrap::download` can result in more confusion. This is because `stamp_key` is not always a date; it can also be a commit hash. Additionally, unlike in `download_beta_toolchain`, in the `download_ci_rustc` function, `version` and `commit` values are calculated separately.
2023-05-04 00:17:26 +05:30
Albert Larsan 30119498be Make x.py work again in most (all?) cases
Wrap all of x.py in `if __name__ == '__main__':` to avoid problems with `multiprocessing`
Make the pool sizing better
2023-05-01 16:20:45 +00:00
ozkanonur bc68de94c8 remove pointless FIXME in bootstrap::download
The suggestion given by `FIXME` to use `CompilerMetadata` for
`download_toolchain` in `bootstrap::download` can result in more
confusion. This is because `stamp_key` is not always a date; it
can also be a commit hash. Additionally, unlike in `download_beta_toolchain`,
in the `download_ci_rustc` function, `version` and `commit` values
are calculated separately.

Signed-off-by: ozkanonur <work@onurozkan.dev>
2023-05-01 18:36:30 +03:00
bors 9ecda8de85 Auto merge of #110427 - Nilstrieb:parallel-bootstrap-startup, r=albertlarsan68
Parallelize initial Rust download in bootstrap

Parallelize the initial download of Rust in `bootstrap.py`

`time ./x.py --help` after `rm -r build`
Before: 33s
After: 27s
2023-04-30 20:02:00 +00:00
Nilstrieb a98968ee0e Parallelize initial rust extraction
This is quite slow and embarassingly parallel, even in python.

This speeds up the initial bootstrap build by about 5-10s.
2023-04-30 19:58:47 +00:00
Matthias Krüger a42261086f Rollup merge of #110999 - clubby789:bootstrap-stderr, r=Mark-Simulacrum
Output some bootstrap messages on stderr

Fixes #110995
2023-04-30 16:25:48 +02:00
Matthias Krüger 9b10f985f9 Rollup merge of #110996 - loongarch-rs:fix-bootstrap, r=ozkanonur
bootstrap: Fix compile error: unused-mut

Compile errors:

```
   Compiling bootstrap v0.0.0 (/home/hev/rust/rust/src/bootstrap)
error: variable does not need to be mutable
    --> config.rs:1312:17
     |
1312 |             let mut build_target = config
     |                 ----^^^^^^^^^^^^
     |                 |
     |                 help: remove this `mut`
     |
     = note: `-D unused-mut` implied by `-D warnings`

error: could not compile `bootstrap` (lib) due to previous error
```
2023-04-30 16:25:47 +02:00
Matthias Krüger b64d4c2f26 Rollup merge of #110118 - jyn514:download-error, r=Mark-Simulacrum
download-rustc: Give a better error message if artifacts can't be dowloaded

It should be very rare in practice to happen; people would need to both have `download-ci-llvm` disabled and `download-rustc` enabled. I think it may be more common if we start turning this on by default, though.

Helps with https://github.com/rust-lang/rust/issues/81930.

Before:
```
downloading https://ci-artifacts.rust-lang.org/rustc-builds/bf5cad8e775fb326465e5c1b98693e5d259da156/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404
```

After:
```
downloading https://ci-artifacts.rust-lang.org/rustc-builds/bf5cad8e775fb326465e5c1b98693e5d259da156/rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz
curl: (22) The requested URL returned error: 404

error: failed to download pre-built rustc from CI

note: old builds get deleted after a certain time
help: if trying to compile an old commit of rustc, disable `download-rustc` in config.toml:

[rust]
download-rustc = false
```
2023-04-30 16:25:46 +02:00
clubby789 8c8d198d59 Output some bootstrap messages on stderr 2023-04-29 22:00:21 +01:00
bors 87b1f891ea Auto merge of #110576 - jyn514:unify-test-args, r=ozkanonur
bootstrap: Unify test argument handling

Fixes #104198. Does *not* help with https://github.com/rust-lang/rust/issues/80124 because I couldn't figure out a reasonable way to omit `--lib` only for `panic_abort` and not other `std` dependencies.

- Remove unnecessary `test_kind` field and `TestKind` struct. These are just subsets of the existing `builder.kind` / `Kind` struct.
- Add a new `run_cargo_test` function which handles passing arguments to cargo based on `builder.config`
- Switch all Steps in `mod test` to `run_cargo_test` where possible
- Combine several steps into one `CrateBootstrap` step. These tests all do the same thing, just with different crate names.
- Fix `x test --no-doc`. This is much simpler after the refactors mentioned earlier, but I'm happy to split it into a separate PR if desired. Before, this would panic a lot because steps forgot to pass `--lib`.
2023-04-29 18:55:28 +00:00
WANG Rui 489925fcaf bootstrap: Fix compile error: unused-mut 2023-04-30 00:58:09 +08:00
jyn 78a709348d Fix x test --no-deps
- Use `cargo metadata` to determine whether a crate has a library
  package or not
- Collect metadata for all workspaces, not just the root workspace and
  cargo
- Don't pass `--lib` for crates without a library
- Use `run_cargo_test` for rust-installer
- Don't build documentation in `lint-docs` if `--no-doc` is passed
2023-04-29 11:51:57 -05:00