[docs][AtomicExpandPass] Document the alternate lowering strategy for part-word atomicrmw/cmpxchg

[asb]

Summary

D47882, D48130 and D48131 introduce a new lowering strategy for part-word atomicrmw/cmpxchg and uses it to lower these operations for the RISC-V target. Rather than having AtomicExpandPass produce the LL/SC loop in the IR level, it instead calculates the necessary mask values and inserts a target-specific intrinsic, which is lowered at a much later stage (after register allocation). This ensures that architecture-specific restrictions for forward-progress in LL/SC loops can be guaranteed.

This patch documents this new AtomicExpandPass functionality. See the previous llvm-dev RFC for more info.

[jyknight]

I'd prefer it to more strongly indicate that the IR-level LL/SC lowering is broken and should not be used in the future, and hopefully will be removed once targets have migrated off of it.

But since there is not yet universal agreement for doing that, this is fine for the moment.

docs/Atomics.rst

474 ↗

(On Diff #165975)

I'd insert a "However, " here.

[asb]

Update to add a 'However' has suggested by @jyknight. Thanks for the review.

I'd also prefer to stronger statements to encourage the use of late-stage lowering, but as you say there's more work to be done in terms of getting community support for that.

I'll hold off on committing until D48131 lands, as that contains the masked cmpxchg expansion implementation that is referenced.

llvm/trunk/docs/Atomics.rst

 * large loads/stores -> ll-sc/cmpxchg
   by overriding ``shouldExpandAtomicStoreInIR()``/``shouldExpandAtomicLoadInIR()``
 * strong atomic accesses -> monotonic accesses + fences by overriding
   ``shouldInsertFencesForAtomic()``, ``emitLeadingFence()``, and
   ``emitTrailingFence()``
 * atomic rmw -> loop with cmpxchg or load-linked/store-conditional
   by overriding ``expandAtomicRMWInIR()``
 * expansion to __atomic_* libcalls for unsupported sizes.
-
-For an example of all of these, look at the ARM backend.
+* part-word atomicrmw/cmpxchg -> target-specific intrinsic by overriding
+  ``shouldExpandAtomicRMWInIR``, ``emitMaskedAtomicRMWIntrinsic``,
+  ``shouldExpandAtomicCmpXchgInIR``, and ``emitMaskedAtomicCmpXchgIntrinsic``.
+
+For an example of these look at the ARM (first five lowerings) or RISC-V (last
+lowering) backend.
+
+AtomicExpandPass supports two strategies for lowering atomicrmw/cmpxchg to
+load-linked/store-conditional (LL/SC) loops. The first expands the LL/SC loop
+in IR, calling target lowering hooks to emit intrinsics for the LL and SC
+operations. However, many architectures have strict requirements for LL/SC
+loops to ensure forward progress, such as restrictions on the number and type
+of instructions in the loop. It isn't possible to enforce these restrictions
+when the loop is expanded in LLVM IR, and so affected targets may prefer to
+expand to LL/SC loops at a very late stage (i.e. after register allocation).
+AtomicExpandPass can help support lowering of part-word atomicrmw or cmpxchg
+using this strategy by producing IR for any shifting and masking that can be
+performed outside of the LL/SC loop.
 
 Libcalls: __atomic_*
 ====================
 
 There are two kinds of atomic library calls that are generated by LLVM. Please
 note that both sets of library functions somewhat confusingly share the names of
 builtin functions defined by clang. Despite this, the library functions are
 not directly related to the builtins: it is *not* the case that ``__atomic_*``