Add LLVM atomic memcpy intrinsics, expose in core/std

[joshlf]

Expose LLVM's "element wise atomic memory intrinsics". In particular:

• Expose llvm.memcpy.element.unordered.atomic as atomic_element_unordered_copy_memory_nonoverlapping
• Expose llvm.memmove.element.unordered.atomic as atomic_element_unordered_copy_memory
• Expose llvm.memset.element.unordered.atomic as atomic_element_unordered_set_memory

Expose these through functions in the std::ptr module. Each function is implemented by the equivalent intrinsic or by a loop of relaxed atomic operations if the intrinsic is not available on the target platform (TODO: Given this platform-specific behavior, can this also be exposed in core::ptr?)

• copy_nonoverlapping_atomic_unordered, backed by atomic_element_unordered_copy_memory_nonoverlapping
• copy_atomic_unordered, backed by atomic_element_unordered_copy_memory
• write_atomic_unordered, backed by atomic_element_unordered_set_memory

Previously discussed on the internals forum here.

Folks with the authority to approve this: Let me know if this is OK to move forward; I'd like to post it to This Week in Rust's CFP.

[joshlf]

@alexcrichton Do you have thoughts on this? Want to make sure I get the go-ahead before I send somebody on TWiR's CFP down a dead end.

[alexcrichton]

I suspect there'll be opinions on stabilization, but seems fine to add as unstable at least to me!

[joshlf]

OK cool, thanks!

[tmccombs]

I'd like to take a stab at implementing this.

[joshlf]

Awesome, thanks @tmccombs ! This will unblock a lot of important work :)

[joshlf]

@alexcrichton On the subject of exposing in core::ptr: Is the ban on platform-specific logic in core a general one, or just for API-visible elements? If it's the latter, then we should be able to implement the facade in core::ptr as well.

[alexcrichton]

Currently it's largely just a "we strongly discorage platform-specific APIs" in libcore, there's already platform-specific pieces to the implementation, especially some math-related pieces on MSVC

[tarcieri]

I saw there was some discussions around use cases on #59155 so I thought I'd bring up mine (and perhaps you can all cross-check me that this would be the proper solution).

I maintain a crate for zeroizing memory (for e.g. wiping cryptographic keys/plaintexts) which presently relies on undefined behavior - namely mixed volatile / non-volatile memory accesses (in conjunction with compiler and hardware fences):

https://github.com/iqlusioninc/crates/tree/develop/zeroize

I'm presently (ab)using core::ptr::write_volatile (and relying on undefined LLVM behavior / implementation details) in conjunction with core::atomic::fence.

What I'd really like is an atomic memset-style operation, i.e. llvm.memset.element.unordered.atomic, which can write repeated copies of a value atomically. The closest thing that presently exists is volatile_set_memory, however that comes with undefined behavior when volatile and non-volatile operations are mixed.

[RalfJung]

undefined behavior when volatile and non-volatile operations are mixed.

I don't know why you think that mixing volatile and non-volatile accesses is UB. I see no issue with that per se.

That said, atomic memset might still be useful.

[tarcieri]

It is explicitly documented as UB:

https://doc.rust-lang.org/core/ptr/fn.write_volatile.html

Memory accessed with read_volatile or write_volatile should not be accessed with non-volatile operations.

Here is a thread on the subject of using volatile writes for zeroing sensitive memory:

https://internals.rust-lang.org/t/volatile-and-sensitive-memory/3188