rustbuild: allow dynamically linking LLVM
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Update: There are now four static/shared scenarios that can happen
for the supported LLVM versions:
- 3.9+: By default use `llvm-config --link-static`
- 3.9+ and `--enable-llvm-link-shared`: Use `--link-shared` instead.
- 3.8: Use `llvm-config --shared-mode` and go with its answer.
- 3.7: Just assume static, maintaining the status quo.
rustbuild: Allow configuration of python interpreter
Add a configuration key to `config.toml`, read it from `./configure`, and add
auto-detection if none of those were specified.
Closes#35760
The makefiles and `mklldeps.py` called `llvm-config --shared-mode` to
find out if LLVM defaulted to shared or static libraries, and just went
with that. But under rustbuild, `librustc_llvm/build.rs` was assuming
that LLVM should be static, and even forcing `--link-static` for 3.9+.
Now that build script also uses `--shared-mode` to learn the default,
which should work better for pre-3.9 configured for dynamic linking, as
it wasn't possible back then to choose differently via `llvm-config`.
Further, the configure script now has a new `--enable-llvm-link-shared`
option, which allows one to manually override `--link-shared` on 3.9+
instead of forcing static.
Add llvm debuginfo configure option
CC @nnethercote @Mark-Simulacrum
We add a new configure option, `--enable-llvm-debuginfo`, to do exactly what you'd think.
Re: #31033Fixes#37738
If a rule is flagged with `default(true)` then the pseudo-rule `default:foo`
will include that. If a rule is also flagged with `.host(true)`, however, then
the rule shouldn't be included for targets that aren't in the host array. This
adds a filter to ensure we don't pull in host rules for targets by accident.
update_with_config_mk() needs to read the new llvm debuginfo config
option from config.mk. Other than that, rustbuild already supports
LLVM's RelWithDebInfo build type.
compile-test: allow overriding nodejs binary location
Add a command-line argument to manually specify which nodejs binary should be used,
which disables the default search.
Original work done by @tari.
Fixes#34188.
rustbuild: enable an initial local cargo
This allows the initial build of src/bootstrap itself to use a local
cargo taken from `configure --local-rust-root`. It was already finding
rustc this way, but was always downloading cargo since it didn't know
where to find it.
It now matches the same logic that `config.rs` will use for stage0,
where both rustc and cargo are taken from `CFG_LOCAL_RUST_ROOT`.
Move all Linux/OSX CI infastructure to Travis
This commit configures our `.travis.yml` to test the full suite of tests we have
on Buildbot right now. A whole mess of docker images are added to the `src/ci`
directory which represent all the build environments for each configuration.
Each of these environments is then configured in `.travis.yml` to run on the
auto branch.
Note that the full matrix of tests aren't intended to be run on all PRs.
Instead, we continue to run only one entry in the matrix, forcing all others to
finish quickly. Only the `auto` branch should run the full matrix of builds.
Also note that the infrastructure hasn't quite been allocated yet to the
rust-lang/rust repository, so everything is disabled for now except for the one
build that happens on PRs. Once that infrastructure is allocated though we can
enable this and let it fly!
Notable modifications from the current test suite today:
* Android tests are run in rustbuild instead of the makefiles, for whatever
reason I couldn't get the makefiles to work on Travis.
* A debuginfo test was updated to work with the current version of the Android
NDK.
* Some dependencies in `mk/tests.mk` were fixed to allow running tests in
parallel.
This commit configures our `.travis.yml` to test the full suite of tests we have
on Buildbot right now. A whole mess of docker images are added to the `src/ci`
directory which represent all the build environments for each configuration.
Each of these environments is then configured in `.travis.yml` to run on the
auto branch.
Note that the full matrix of tests aren't intended to be run on all PRs.
Instead, we continue to run only one entry in the matrix, forcing all others to
finish quickly. Only the `auto` branch should run the full matrix of builds.
Also note that the infrastructure hasn't quite been allocated yet to the
rust-lang/rust repository, so everything is disabled for now except for the one
build that happens on PRs. Once that infrastructure is allocated though we can
enable this and let it fly!
Notable modifications from the current test suite today:
* Android tests are run in rustbuild instead of the makefiles, for whatever
reason I couldn't get the makefiles to work on Travis.
* A debuginfo test was updated to work with the current version of the Android
NDK.
* Some dependencies in `mk/tests.mk` were fixed to allow running tests in
parallel.
This allows the initial build of src/bootstrap itself to use a local
cargo taken from `configure --local-rust-root`. It was already finding
rustc this way, but was always downloading cargo since it didn't know
where to find it.
It now matches the same logic that `config.rs` will use for stage0,
where both rustc and cargo are taken from `CFG_LOCAL_RUST_ROOT`.
Vendor all rustbuild dependencies in this repo
This commit vendors all crates.io dependencies into the rust-lang/rust repository using the `cargo-vendor` tool. This is done in an effort to make rustbuild distro-ready by ensuring that our source tarballs are self-contained units which don't need extraneous network downloads.
A new `src/vendor` directory is created with all vendored crates, and Cargo, when using rustbuild, is configured to use this directory. Over time we can deduplicate this directory with the actual src tree (e.g. src/librustc_serialize, src/liblibc, src/libgetopts, ...). For now though that's left to a separate commit.
A few changes are included here:
* The `winapi` and `url` dependencies were dropped. The source code for these
projects is pretty weighty, and we're about to vendor them, so let's not
commit to that intake just yet. If necessary we can vendor them later but for
now it shouldn't be necessary.
* The `--frozen` flag is now always passed to Cargo, obviating the need for
tidy's `cargo_lock` check.
* Tidy was updated to not check the vendor directory
Closes#34687
There is a *little* code duplication, but primarily for sake of "match
exhaustiveness". Let's blame Linux/MIPS for not exposing endianness
explicitly in `uname -m` (that's user-space interface and as such is
frozen).
Currently the build won't work as we have to wait for a new stage0 for
the MIPS host compilers, but this paves the way to self-hosted Rust on
MIPS. The cross-compiled MIPS binaries are confirmed to work on the
Loongson 3A2000 (MIPS64r2-compatible) so we have plenty of confidence
that they'll work on other MIPS platforms too, as Linux/MIPS user-space
ABI is consistent across machines of the same bitness.
This commit is a rewrite of the user-facing interface to the rustbuild build
system. The intention here is to make it much easier to compile/test the project
without having to remember weird rule names and such. An overall view of the new
interface is:
# build everything
./x.py build
# document everyting
./x.py doc
# test everything
./x.py test
# test libstd
./x.py test src/libstd
# build libcore stage0
./x.py build src/libcore --stage 0
# run stage1 run-pass tests
./x.py test src/test/run-pass --stage 1
The `src/bootstrap/bootstrap.py` script is now aliased as a top-level `x.py`
script. This `x` was chosen to be both short and easily tab-completable (no
collisions in that namespace!). The build system now accepts a "subcommand" of
what to do next, the main ones being build/doc/test.
Each subcommand then receives an optional list of arguments. These arguments are
paths in the source repo of what to work with. That is, if you want to test a
directory, you just pass that directory as an argument.
The purpose of this rewrite is to do away with all of the arcane renames like
"rpass" is the "run-pass" suite, "cfail" is the "compile-fail" suite, etc. By
simply working with directories and files it's much more intuitive of how to run
a test (just pass it as an argument).
The rustbuild step/dependency management was also rewritten along the way to
make this easy to work with and define, but that's largely just a refactoring of
what was there before.
The *intention* is that this support is extended for arbitrary files (e.g.
`src/test/run-pass/my-test-case.rs`), but that isn't quite implemented just yet.
Instead directories work for now but we can follow up with stricter path
filtering logic to plumb through all the arguments.
Enable line number debuginfo in releases
This commit enables by default passing the `-C debuginfo=1` argument to the
compiler for the stable, beta, and nightly release channels. A new configure
option was also added, `--enable-debuginfo-lines`, to enable this behavior in
developer builds as well.
Closes#36452
This commit enables by default passing the `-C debuginfo=1` argument to the
compiler for the stable, beta, and nightly release channels. A new configure
option was also added, `--enable-debuginfo-lines`, to enable this behavior in
developer builds as well.
Closes#36452
A new point-release shouldn't change any language semantics, so a local
stage0 that matches MAJOR.MINOR version should still be considered a
local-rebuild as far as `--cfg stageN` features go.
e.g. `1.14.0` should be considered a local-rebuild for any `1.14.X`.
(Bootstrap keys used to be an issue too, until #37265.)
Allow bootstrapping without a key. Fixes#36548
This will make it easier for packagers to bootstrap rustc when they happen
to have a bootstrap compiler with a slightly different version number.
It's not ok for anything other than the build system to set this environment variable.
r? @alexcrichton
This will make it easier for packagers to bootstrap rustc when they happen
to have a bootstrap compiler with a slightly different version number.
It's not ok for anything other than the build system to set this environment variable.
We hope to move to AppVeyor in the near future off of Buildbot + EC2. This adds
an `appveyor.yml` configuration file which is ready to run builds on the auto
branch. This is also accompanied with a few minor fixes to the build system and
such to accomodate AppVeyor.
The intention is that we're not switching over to AppVeyor entirely just yet,
but rather we'll watch the builds for a week or so. If everything checks out
then we'll start gating on AppVeyor instead of Buildbot!
configure: Add options for separate musl roots
This allows using the `./configure` script to enable rustbuild to compile
multiple musl targets at once. We'll hopefully use this soon on our bots to
produce a bunch of targets.
As the entry point for building the Rust compiler, a good user experience hinges
on this compiling quickly to get to the meat of the problem. To that end use
`#[cfg]`-specific dependencies to avoid building Windows crates on Unix and drop
the `regex` crate for now which was easily replacable with some string
searching.
This updates the commit to use workspaces to use `cargo metadata` instead of
hardcoded lists about what to test. This should help us be resilient to updates
in the future on behalf of the crate DAG and minimize the amount of files that
need to be touched.
This involves hacking the code used to run cargo test on various
packages, because it reads Cargo.lock to determine which packages should
be tested. This change implements a blacklist, since that will catch new
crates when they are added in the future.
bors