Enable outline-atomics by default on more AArch64 platforms

[tgross35]

The baseline Armv8.0 ISA doesn't have atomics instructions, but in
practice most hardware is at least Armv8.1-A (2014), which includes
single-instruction atomics as part of the LSE feature. As a performance
optimization for these cases, GCC and LLVM have the -moutline-atomics flag
to turn atomic operations into calls to symbols like __aarch64_cas1_acq.
These can do runtime feature detection and use the LSE instructions if
available, falling back to more portable load-exclusive/store-exclusive
loops.

Since the recent 3b50253 ("compiler-builtins: plumb LSE support
for aarch64 on linux") our builtins support this LSE optimization, and
since 6936bb9 ("Dynamically enable LSE for aarch64 rust provided
intrinsics"), std will set the flag as part of its startup code. The first
commit in this PR configures this to work on all platforms built with
outline-atomics, not just Linux.

Thus, enable outline-atomics by default on Android, FreeBSD, OpenBSD,
Windows, Fuchsia, and Apple platforms that don't have LSE in the baseline.
The feature is already enabled on Linux. Platform-specific details are
included in each commit message.

The current implementation can still be accessed by setting
-Ctarget-feature=-outline-atomics. Setting -Ctarget-feature=+lse or
a relevant CPU will use the single-instruction atomics without the call
overhead. https://rust.godbolt.org/z/dsdrzszoe

Link: https://learn.arm.com/learning-paths/servers-and-cloud-computing/lse/intro/
Original Clang outline-atomics benchmarks: https://reviews.llvm.org/D91157#2435844

try-job: aarch64-apple
try-job: aarch64-msvc-*
try-job: arm-android
try-job: dist-android
try-job: dist-aarch64-windows-gnullvm
try-job: dist-aarch64-apple
try-job: dist-aarch64-msvc
try-job: dist-various-*
try-job: dist-x86_64-freebsd
try-job: test-various

[rustbot]

r? @ChrisDenton

rustbot has assigned @ChrisDenton.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

These commits modify compiler targets.
(See the Target Tier Policy.)

[tgross35]

Whoops

r? @ghost

[tgross35]

@bors2 try

[rust-bors]

☀️ Try build successful (CI)
Build commit: 09a8dba (09a8dba552fc10cf0f87e6af03c30037287492e7, parent: 9cd272dc85320e85a8c83a1a338870de52c005f3)

[tgross35]

@bors2 try

[rustbot]

r? @davidtwco

rustbot has assigned @davidtwco.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[tgross35]

Cc target maintainers:

• Apple @madsmtm
• Android @maurer @mgeisler
• FreeBSD @asomers
• Fuchsia @erickt, @Nashenas88
• OpenBSD @semarie
• Windows @ChrisDenton @dpaoliello @wesleywiser

Zulip discussion: #t-compiler > outline-atomics on non-Linux targets

[madsmtm]

Looked through it as best I could, but this is a fair bit outside my comfort zone.

Is there a test or two I could run to verify whether this works on-device?

View changes since this review

[madsmtm]

I'm pretty sure you need something like:

#[cfg_attr(target_vendor = "apple", unsafe(link_section = "__DATA,__mod_init_func,mod_init_funcs"))]

For this to work on Apple platforms.

[tgross35]

I believe LLVM turns init_array into whatever is applicable for the platform, but that would indeed be good to confirm. We do use init_array in one other location

rust/library/std/src/sys/args/unix.rs

Line 120 in 6c699a3

#[unsafe(link_section = ".init_array.00099")]

[madsmtm]

I know it's pre-existing, but why the 90 in .init_array.90? Is that for a specific ordering?

[tgross35]

90 is a priority, and per https://maskray.me/blog/2021-11-07-init-ctors-init-array it should be <= 100 (though I have no idea if that limit is actually meaningful). I don't think there is any significance to this number other than being less than the 99 value used in the args constructor.

I'll add a comment.

[madsmtm]

Shouldn't there also be a cfg(not(target_feature = "lse")) here, or am I misunderstanding how this should work?

[tgross35]

That's probably fair: the C implementations don't seem to gate this, but it would be nice to save the (small) startup time. The only downside I can think of would be if you build std with LSE then link applications that don't enable LSE (meaning they always get the slow fallback without checking), but I imagine this is rare.

[tgross35]

Is there a test or two I could run to verify whether this works on-device?

It's easy enough to check that outline atomics are used, just inspecting the code used by https://rust.godbolt.org/z/PWqeKP3W5. Verifying that feature detection works so LSE gets used is trickier though. The easiest thing is to call something that will use out-of-line atomics (like the example from that godbolt) and step through with a debugger, and ensure it is hitting the atomic instruction at this bit of code https://github.com/rust-lang/compiler-builtins/blob/a63d089c673aa9397d583c3cef506ad457c5f403/compiler-builtins/src/aarch64_linux.rs#L146-L148 rather than going to the fallback. Alternatively add an extern static or something that binds to the mangled name of HAVE_LSE_ATOMICS in that module, and just print to make sure it's nonzero.

(requires our compiler-builtins, would be similar but slightly different if using the C versions)

[rust-bors]

☀️ Try build successful (CI)
Build commit: 7041613 (704161326f1460ec409add8d318ad1976f8f3eb5, parent: 6c699a37235700ab749e3f14147fe41d49c056e8)