autodiff: emit an error if we fail to find libEnzyme
Tested manually by moving libEnzyme-21.so away. We should adjust the error msg. once we have the component up.
It's the first usage within rustc of this experimental feature, but afaik we're open to dogfooding those for test purpose, right?
r? ``@Kobzol``
Introduces `BackendRepr::ScalableVector` corresponding to scalable
vector types annotated with `repr(scalable)` which lowers to a scalable
vector type in LLVM.
Co-authored-by: Jamie Cunliffe <Jamie.Cunliffe@arm.com>
Externally implementable items
Supersedes https://github.com/rust-lang/rust/pull/140010
Tracking issue: https://github.com/rust-lang/rust/issues/125418
Getting started:
```rust
#![feature(eii)]
#[eii(eii1)]
pub fn decl1(x: u64)
// body optional (it's the default)
{
println!("default {x}");
}
// in another crate, maybe
#[eii1]
pub fn decl2(x: u64) {
println!("explicit {x}");
}
fn main() {
decl1(4);
}
```
- tiny perf regression, underlying issue makes multiple things in the compiler slow, not just EII, planning to solve those separately.
- No codegen_gcc support, they don't have bindings for weak symbols yet but could
- No windows support yet for weak definitions
This PR merges the implementation of EII for just llvm + not windows, doesn't yet contain like a new panic handler implementation or alloc handler. With this implementation, it would support implementing the panic handler in terms of EII already since it requires no default implementation so no weak symbols
The PR has been open in various forms for about a year now, but I feel that having some implementation merged to build upon
Overhaul filename handling for cross-compiler consistency
This PR overhauls the way we handle filenames in the compiler and `rmeta` in order to achieve achieve cross-compiler consistency (ie. having the same path no matter if the filename was created in the current compiler session or is coming from `rmeta`).
This is required as some parts of the compiler rely on consistent paths for the soundness of generated code (see rust-lang/rust#148328).
In order to achieved consistency multiple steps are being taken by this PR:
- by making `RealFileName` immutable
- by only having `SourceMap::to_real_filename` create `RealFileName`
- currently `RealFileName` can be created from any `Path` and are remapped afterwards, which creates consistency issue
- by also making `RealFileName` holds it's working directory, embeddable name and the remapped scopes
- this removes the need for a `Session`, to know the current(!) scopes and cwd, which is invalid as they may not be equal to the scopes used when creating the filename
In order for `SourceMap::to_real_filename` to know which scopes to apply `FilePathMapping` now takes the current remapping scopes to apply, which makes `FileNameDisplayPreference` and company useless and are removed.
This PR is split-up in multiple commits (unfortunately not atomic), but should help review the changes.
Unblocks https://github.com/rust-lang/rust/pull/147611
Fixes https://github.com/rust-lang/rust/issues/148328
This commit refactors `SourceMap` and most importantly `RealFileName` to
make it self-contained in order to achieve cross-compiler consistency.
This is achieved:
- by making `RealFileName` immutable
- by only having `SourceMap::to_real_filename` create `RealFileName`
- by also making `RealFileName` holds it's working directory,
it's embeddable name and the remapped scopes
- by making most `FileName` and `RealFileName` methods take a scope as
an argument
In order for `SourceMap::to_real_filename` to know which scopes to apply
`FilePathMapping` now takes the current remapping scopes to apply, which
makes `FileNameDisplayPreference` and company useless and are removed.
The scopes type `RemapPathScopeComponents` was moved from
`rustc_session::config` to `rustc_span`.
The previous system for scoping the local/remapped filenames
`RemapFileNameExt::for_scope` is no longer useful as it's replaced by
methods on `FileName` and `RealFileName`.
Make `--print=backend-has-zstd` work by default on any backend
Using a defaulted `CodegenBackend` method that querying for zstd support should automatically print a safe value of `false` on any backend that doesn't specifically indicate the presence or absence of zstd.
This should fix the compiletest failures reported in https://github.com/rust-lang/rust/pull/149666#discussion_r2597881482, which can occur when LLVM is not the default codegen backend.
Remove -Zoom=panic
There are major questions remaining about the reentrancy that this allows. It doesn't have any users on github outside of a single project that uses it in a panic=abort project to show backtraces. It can still be emulated through `#[alloc_error_handler]` or `set_alloc_error_hook` depending on if you use the standard library or not. And finally it makes it harder to do various improvements to the allocator shim.
With this PR the sole remaining symbol in the allocator shim that is not effectively emulating weak symbols is the symbol that prevents skipping the allocator shim on stable even when it would otherwise be empty because libstd + `#[global_allocator]` is used.
Closes https://github.com/rust-lang/rust/issues/43596
Fixes https://github.com/rust-lang/rust/issues/126683
Using a defaulted `CodegenBackend` method that querying for zstd support should
automatically print a safe value of `false` on any backend that doesn't
specifically indicate the presence or absence of zstd.
Add perma-unstable `--print=backend-has-zstd` for use by compiletest
Tests for `-Zdebuginfo-compression=zstd` need to be skipped if LLVM was built without support for zstd compression.
Currently, compiletest relies on messy and fragile heuristics to detect whether the compiler's LLVM was built with zstd support. But the compiler itself already knows whether LLVM has zstd or not, so it's easier for compiletest to just ask the compiler.
---
Originally I was intending for this to be a `--print=debuginfo-compression` flag that would print out a list of values supported by `-Zdebuginfo-compression=`. I got that working locally, but it was more complex than I was happy with (in both rustc and compiletest), so I decided to cut scope and instead add a very narrow perma-unstable print request instead.
There is always a circularity hazard whenever we ask the compiler-under-test for information about how to test it. But in this case, the underlying compiler code is fairly simple, whereas the previous heuristics were inherently messy and unreliable anyway.
Tests for `-Zdebuginfo-compression=zstd` need to be skipped if LLVM was built
without support for zstd compression.
Currently, compiletest relies on messy and fragile heuristics to detect whether
the compiler's LLVM was built with zstd support. But the compiler itself
already knows whether LLVM has zstd or not, so it's easier for compiletest to
just ask the compiler.
This register is only supported on the *powerpc*spe targets. It is
only recognized by LLVM. gcc does not accept this as a clobber, nor
does it support these targets.
This is a volatile register, thus it is included with clobber_abi.
Fix issue with callsite inline attribute not being applied sometimes.
If the calling function had more target features enabled than the callee than the attribute wasn't being applied as the arguments for the check had been swapped round. Also includes target features that are part of the global set as the warning was checking those but when adding the attribute they were not checked.
Add a codegen-llvm test to check that the attribute is actually applied as previously only the warning was being checked.
Tracking issue: rust-lang/rust#145574
There are major questions remaining about the reentrancy that this
allows. It doesn't have any users on github outside of a single project
that uses it in a panic=abort project to show backtraces. It
can still be emulated through #[alloc_error_handler] or
set_alloc_error_hook depending on if you use the standard library or
not. And finally it makes it harder to do various improvements to the
allocator shim.
If the calling function had more target features enabled than the
callee than the attribute wasn't being applied as the arguments for
the check had been swapped round. Also includes target features that
are part of the global set as the warning was checking those but when
adding the attribute they were not checked.
Add a codegen-llvm test to check that the attribute is actually
applied as previously only the warning was being checked.
automate gpu offloading - part 1
Automates step 1 from the rustc-dev-guide offload section:
https://rustc-dev-guide.rust-lang.org/offload/usage.html#compile-instructions
`"clang-offload-packager" "-o" "host.out" "--image=file=device.bc,triple=amdgcn-amd-amdhsa,arch=gfx90a,kind=openmp"`
Verified on an MI 250X
cc `@jhuber6,` `@kevinsala,` `@jdoerfert,` `@Sa4dUs`
r? oli-obk
Error if an autodiff user does not set lto=fat
Based on your feedback, I started to provide a nice error message for a lack of `lto=fat`, instead of us forcing it.
In a next step, we should replace `RUSTFLAGS="-Zautodiff=Enable"` with another Cargo.toml setting, as discussed here: https://github.com/rust-lang/rust/issues/147487#issuecomment-3446558644
As another improvement, we should also figure out why rlib builds do not properly obey the fat=lto setting.
```````@bjorn3```````
use funnel shift as fallback impl for rotating shifts
That lets us remove this gnarly implementation from Miri and const-eval.
However, `rotate_left`/`rotate_right` are stable as const fn, so to do this we have to `rustc_allow_const_fn_unstable` a bunch of const trait stuff. Is that a bad idea? Cc `@oli-obk` `@fee1-dead`
To make debugger stepping intuitive with `-Copt-level=0`. See the
adjusted `basic-stepping.rs` test.
This is kind of a revert of bd0aae92dc, except we don't revert it,
we just make it conditional on `opt-level`. That commit regressed
`basic-stepping.rs`, but it was not noticed since that test did not
exist back then. I have retroactively bisected to find that out.
It seems messy to sprinkle if-cases inside of the
`write_operand_repeatedly()` so make the whole function conditional.
The test that bd0aae92dc added in
`tests/codegen/issues/issue-111603.rs` already use `-Copt-level=3`, so
we don't need to adjust the compiler flags for it to keep passing.
Add LLVM realtime sanitizer
This is a new attempt at adding the [LLVM real-time sanitizer](https://clang.llvm.org/docs/RealtimeSanitizer.html) to rust.
Previously this was attempted in https://github.com/rust-lang/rfcs/pull/3766.
Since then the `sanitize` attribute was introduced in https://github.com/rust-lang/rust/pull/142681 and it is a lot more flexible than the old `no_santize` attribute. This allows adding real-time sanitizer without the need for a new attribute, like it was proposed in the RFC. Because i only add a new value to a existing command line flag and to a attribute i don't think an MCP is necessary.
Currently real-time santizer is usable in rust code with the [rtsan-standalone](https://crates.io/crates/rtsan-standalone) crate. This downloads or builds the sanitizer runtime and then links it into the rust binary.
The first commit adds support for more detailed sanitizer information.
The second commit then actually adds real-time sanitizer.
The third adds a warning against using real-time sanitizer with async functions, cloures and blocks because it doesn't behave as expected when used with async functions. I am not sure if this is actually wanted, so i kept it in a seperate commit.
The fourth commit adds the documentation for real-time sanitizer.
Add -Zannotate-moves for profiler visibility of move/copy operations (codegen)
**Note:** this is an alternative implementation of https://github.com/rust-lang/rust/pull/147206; rather than being a MIR transform, it adds the annotations closer to codegen. It's functionally the same but the implementation is lower impact and it could be more correct.
---
This implements a new unstable compiler flag `-Zannotate-moves` that makes move and copy operations visible in profilers by creating synthetic debug information. This is achieved with zero runtime cost by manipulating debug info scopes to make moves/copies appear as calls to `compiler_move<T, SIZE>` and `compiler_copy<T, SIZE>` marker functions in profiling tools.
This allows developers to identify expensive move/copy operations in their code using standard profiling tools, without requiring specialized tooling or runtime instrumentation.
The implementation works at codegen time. When processing MIR operands (`Operand::Move` and `Operand::Copy`), the codegen creates an `OperandRef` with an optional `move_annotation` field containing an `Instance` of the appropriate profiling marker function. When storing the operand, `store_with_annotation()` wraps the store operation in a synthetic debug scope that makes it appear inlined from the marker.
Two marker functions (`compiler_move` and `compiler_copy`) are defined in `library/core/src/profiling.rs`. These are never actually called - they exist solely as debug info anchors.
Operations are only annotated if:
- We're generating debug info and the feature is enabled.
- Meets the size threshold (default: 65 bytes, configurable via `-Zannotate-moves=SIZE`), and is non-zero
- Has a memory representation
This has a very small size impact on object file size. With the default limit it's well under 0.1%, and even with a very small limit of 8 bytes it's still ~1.5%. This could be enabled by default.
Offload device
LLVM's offload functionality usually expects an extra dyn_ptr argument. We could avoid it,b ut likely gonna need it very soon in one of the follow-up PRs (e.g. to request shared memory). So we might as well already add it.
This PR adds a %dyn_ptr ptr to GPUKernel ABI functions, if the offload feature is enabled.
WIP
r? ```@ghost```
Jacob Pratt