hwaddress: automatically add `-Ctarget-feature=+tagged-globals`
Note that since HWAddressSanitizer is/should be a target modifier, we do not have to worry about whether this LLVM target feature changes the ABI.
Fixes: rust-lang/rust#148185
Use convergent attribute to funcs for GPU targets
On targets with convergent operations, we need to add the convergent attribute to all functions that run convergent operations. Following clang, we can conservatively apply the attribute to all functions when compiling for such a target and rely on LLVM optimizing away the attribute in cases where it is not necessary.
This affects the amdgpu and nvptx targets.
cc @kjetilkjeka, @kulst for nvptx
cc @ZuseZ4
r? @nnethercote, as you already reviewed this in the other PR
Split out from rust-lang/rust#149637, the part here should be uncontroversial.
Because the things in this module aren't MIR and don't use anything
from `rustc_middle::mir`. Also, modules that use `mono` often don't use
anything else from `rustc_middle::mir`.
Various LTO cleanups
* Move some special casing of thin local LTO into a single location.
* Move lto_import_only_modules handling for fat LTO earlier. There is no reason to keep it separate until right before pass the LTO modules to the codegen backend. For thin LTO this introduces `ThinLtoInput` to correctly handle incr comp caching.
* Remove the `Linker` type from cg_llvm. It previously helped deduplicate code for `-Zcombine-cgus`, but that flag no longer exists.
Part of https://github.com/rust-lang/compiler-team/issues/908
Abstract over the existing `simd_cast` intrinsic to implement a new
`sve_cast` intrinsic - this is better than allowing scalable vectors to
be used with all of the generic `simd_*` intrinsics.
Clang changed to representing tuples of scalable vectors as
structs rather than as wide vectors (that is, scalable vector types
where the `N` part of the `<vscale x N x ty>` type was multiplied by
the number of vectors). rustc mirrored this in the initial implementation
of scalable vectors.
Earlier versions of our patches used the wide vector representation and
our intrinsic patches used the legacy
`llvm.aarch64.sve.tuple.{create,get,set}{2,3,4}` intrinsics for creating
these tuples/getting/setting the vectors, which were only supported
due to LLVM's `AutoUpgrade` pass converting these intrinsics into
`llvm.vector.insert`. `AutoUpgrade` only supports these legacy intrinsics
with the wide vector representation.
With the current struct representation, Clang has special handling in
codegen for generating `insertvalue`/`extractvalue` instructions for
these operations, which must be replicated by rustc's codegen for our
intrinsics to use. This patch implements new intrinsics in
`core::intrinsics::scalable` (mirroring the structure of
`core::intrinsics::simd`) which rustc lowers to the appropriate
`insertvalue`/`extractvalue` instructions.
Instead of just using regular struct lowering for these types, which
results in an incorrect ABI (e.g. returning indirectly), use
`BackendRepr::ScalableVector` which will lower to the correct type and
be passed in registers.
This also enables some simplifications for generating alloca of scalable
vectors and greater re-use of `scalable_vector_parts`.
A LLVM codegen test demonstrating the changed IR this generates is
included in the next commit alongside some intrinsics that make these
tuples usable.
specifically this emits an error when
- a custom target is used
- `RiscV32 if self.llvm_abiname == "ilp32e"` this abi is used for 32-bit
embedded targets, and clang/llvm document that the ABI may change in
the future.
this enables packed-stack just as -mpacked-stack in clang and gcc.
packed-stack is needed on s390x for kernel development.
Co-authored-by: Ralf Jung <post@ralfj.de>
simd_fmin/fmax: make semantics and name consistent with scalar intrinsics
This is the SIMD version of https://github.com/rust-lang/rust/pull/153343: change the documented semantics of the SIMD float min/max intrinsics to that of the scalar intrinsics, and also make the name consistent. The overall semantic change this amounts to is that we restrict the non-determinism: the old semantics effectively mean "when one input is an SNaN, the result non-deterministically is a NaN or the other input"; the new semantics say that in this case the other input must be returned. For all other cases, old and new semantics are equivalent. This means all users of these intrinsics that were correct with the old semantics are still correct: the overall set of possible behaviors has become smaller, no new possible behaviors are being added.
In terms of providers of this API:
- Miri, GCC, and cranelift already implement the new semantics, so no changes are needed.
- LLVM is adjusted to use `minimumnum nsz` instead of `minnum`, thus giving us the new semantics.
In terms of consumers of this API:
- Portable SIMD almost certainly wants to match the scalar behavior, so this is strictly a bugfix here.
- Stdarch mostly stopped using the intrinsic, except on nvptx, where arguably the new semantics are closer to what we actually want than the old semantics (https://github.com/rust-lang/stdarch/issues/2056).
Q: Should there be an `f` in the intrinsic name to indicate that it is for floats? E.g., `simd_fminimum_number_nsz`?
Also see https://github.com/rust-lang/rust/issues/153395.
Merge `fabsf16/32/64/128` into `fabs::<F>`
Following [a small conversation on Zulip](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/Float.20intrinsics/with/521501401) (and because I'd be interested in starting to contribute on Rust), I thought I'd give a try at merging the float intrinsics :)
This PR just merges `fabsf16`, `fabsf32`, `fabsf64`, `fabsf128`, as it felt like an easy first target.
Notes:
- I'm opening the PR for one intrinsic as it's probably easier if the shift is done one intrinsic at a time, but let me know if you'd rather I do several at a time to reduce the number of PRs.
- Currently this PR increases LOCs, despite being an attempt at simplifying the intrinsics/compilers. I believe this increase is a one time thing as I had to define new functions and move some things around, and hopefully future PRs/commits will reduce overall LoCs
- `fabsf32` and `fabsf64` are `#[rustc_intrinsic_const_stable_indirect]`, while `fabsf16` and `fabsf128` aren't; because `f32`/`f64` expect the function to be const, the generic version must be made indirectly stable too. We'd need to check with T-lang this change is ok; the only other intrinsics where there is such a mismatch is `minnum`, `maxnum` and `copysign`.
- I haven't touched libm because I'm not familiar with how it works; any guidance would be welcome!
debuginfo: emit DW_TAG_call_site entries
Set `FlagAllCallsDescribed` on function definition DIEs so LLVM emits DW_TAG_call_site entries, letting debuggers and analysis tools track tail calls.
Add `-Zsanitize=kernel-hwaddress`
The Linux kernel has a config option called `CONFIG_KASAN_SW_TAGS` that enables `-fsanitize=kernel-hwaddress`. This is not supported by Rust.
One slightly awkward detail is that `#[sanitize(address = "off")]` applies to both `-Zsanitize=address` and `-Zsanitize=kernel-address`. Probably it was done this way because both are the same LLVM pass. I replicated this logic here for hwaddress, but it might be undesirable.
Note that `#[sanitize(kernel_hwaddress = "off")]` could be supported as an annotation on statics, but since it's also missing for `#[sanitize(hwaddress = "off")]`, I did not add it.
MCP: https://github.com/rust-lang/compiler-team/issues/975
Tracking issue: https://github.com/rust-lang/rust/issues/154171
cc @rcvalle @maurer @ojeda
Unknown -> Unsupported compression algorithm
Both zstd and zlib are *known* compression algorithms, they just may not be supported by the backend. We shouldn't mislead users into e.g. thinking they made a typo.
cc https://github.com/rust-lang/rust/issues/120953
Rename `target.abi` to `target.cfg_abi` and enum-ify llvm_abiname
See [Zulip](https://rust-lang.zulipchat.com/#narrow/channel/131828-t-compiler/topic/De-spaghettifying.20ABI.20controls/with/578893542) for more context. Discussed a bit in https://github.com/rust-lang/rust/pull/153769#discussion_r2934399038 too.
This renames `target.abi` to `target.cfg_abi` to make it less likely that someone will use it to determine things about the actual ccABI, i.e. the calling convention used on the target. `target.abi` does not control that calling convention, it just *sometimes* informs the user about that calling convention (and also about other aspects of the ABI).
Also turn llvm_abiname into an enum to make it more natural to match on.
Cc @workingjubilee @madsmtm
[BPF] add target feature allows-misaligned-mem-access
This PR adds the allows-misaligned-mem-access target feature to the BPF target. The feature can enable misaligned memory access support in the LLVM backend, aligning Rust’s BPF target behavior with the corresponding LLVM update introduced in [llvm/llvm-project#167013](https://github.com/llvm/llvm-project/pull/167013) (included in LLVM 22).
Both zstd and zlib are *known* compression algorithms, they just may not
be supported by the backend. We shouldn't mislead users into e.g.
thinking they made a typo.
On targets with convergent operations, we need to add the convergent
attribute to all functions that run convergent operations. Following
clang, we can conservatively apply the attribute to all functions when
compiling for such a target and rely on LLVM optimizing away the
attribute in cases where it is not necessary.
This affects the amdgpu and nvptx targets.
LLVM does not support host feature detection (only host cpu
detection) on apple platforms. As such, the returned feature
string will be empty. Adding this empty string to the target-features
attribute results in a verifier error on LLVM 22.
Fix this by not adding the empty string to the target features.
The reason why this was not caught by the target-cpu-native test
is that it requires a function that adds *some* target features,
otherwise the attribute is omitted entirely. We achieve this with
a somewhat peculiar construction that enables `neon` if it's
already enabled. (This is to avoid enabling it on softfloat targets.)
It's defined in `rustc_span::source_map` which doesn't make any sense
because it has nothing to do with source maps. This commit moves it to
the crate root, a more sensible spot for something this basic.
sayantn