[RISCV] Codegen for i8, i16, and i32 atomicrmw with RV32A

[asb]

Summary

Introduce a new RISCVExpandPseudoInsts pass to expand atomic pseudo-instructions after register allocation. This is necessary in order to ensure that register spills aren't introduced between LL and SC, thus breaking the forward progress guarantee for the operation. AArch64 does something similar for CmpXchg (though only at O0), and Mips is moving towards this approach (see D31287). See also this mailing list post from James Knight, which summarises the issues with lowering to ll/sc in IR or pre-RA.

See the accompanying RFC thread for an overview of the lowering strategy.

[jyknight]

The problem with min and max is that you must not have a branch within the ll/sc loop. But, they can be done branchless with not too many instructions, so ISTM we should just implement them that way.

I think the following will work -- it computes b ^ ((a^b) & -(a < b)). The & -(a < b) either returns the LHS (if comparison is true) or returns 0 (if comparison is false), so you either get b^a^b, or b.

min(a0, a1) -> a0:

slt     a2, a0, a1
neg     a2, a2
xor     a0, a0, a1
and     a0, a0, a2
xor     a0, a1, a0

And I think just use sltu instead of slt for umin/umax, and reverse the args of the slt for max/umax.

lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

179 ↗

(On Diff #150330)

I'd note that all of the code which is OUTSIDE the ll/sc loop doesn't really need to be implemented within this pseudo. I'm not sure if it's worthwhile extracting that or not, but it could theoretically have benefit, due to being able to potentially reuse e.g. the address math.

Have you considered making such a split?

lib/Target/RISCV/RISCVTargetMachine.cpp

104 ↗

(On Diff #150330)

Is it okay to do this pre-scheduling? Are we guaranteed that the scheduler won't do something invalid here? (I'm not sure the set of transforms it's capable of at the moment.)

[asb]

In D47882#1125698, @jyknight wrote:

The problem with min and max is that you must not have a branch within the ll/sc loop. But, they can be done branchless with not too many instructions, so ISTM we should just implement them that way.

I think the following will work -- it computes b ^ ((a^b) & -(a < b)). The & -(a < b) either returns the LHS (if comparison is true) or returns 0 (if comparison is false), so you either get b^a^b, or b.

I did consider the branchless approach. However forward branches _are_ part of the architectural guarantee of LR/SC forward progress:

In the standard A extension, certain constrained LR/SC sequences are guaranteed to succeed
eventually. The static code for the LR/SC sequence plus the code to retry the sequence in case
of failure must comprise at most 16 integer instructions placed sequentially in memory. For the
sequence to be guaranteed to eventually succeed, the dynamic code executed between the LR and
SC instructions can only contain other instructions from the base “I” subset, excluding loads, stores,
backward jumps or taken backward branches, FENCE, FENCE.I, and SYSTEM instructions. The
code to retry a failing LR/SC sequence can contain backward jumps and/or branches to repeat the
LR/SC sequence, but otherwise has the same constraints. The SC must be to the same address
and of the same data size as the latest LR executed. LR/SC sequences that do not meet these
constraints might complete on some attempts on some implementations, but there is no guarantee
of eventual success.

lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

179 ↗

(On Diff #150330)

Yes, I've thought about this quite a bit. The ideal is to expand everything other than the LL+SC+branch at the IR level. I had been thinking about a new inlineasm-like 'black box' construct that would be expanded by target-dependent code only at the very last moment, when lowering to MCInst. But when I started sketching it out in more detail with the aim of incorporating it as an option in an RFC on atomics lowering I started thinking that there's not much benefit in this versus expanding pseudo-instructions in preEmitPass2. Targets can guarantee that runs after everything else that might cause problems (such as the machine outliner). I'll play with this some more.

lib/Target/RISCV/RISCVTargetMachine.cpp

104 ↗

(On Diff #150330)

preEmitPass2 seems the ideal point. Although the machine outliner could be taught to leave instructions between LLC/SC alone, it's probably better to ensure that this lowering happens at the latest possible point.

[jyknight]

In D47882#1126302, @asb wrote:

can only contain other instructions from the base “I” subset, excluding loads, stores,
backward jumps or taken backward branches,

Oh, nice! That changed since the last time I read it in 2016; it used to prohibit -all- taken branches, not just backwards ones.

lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

179 ↗

(On Diff #150330)

I agree -- adding an intrinsic, say "llvm.riscv.atomicrmw.masked.llsc.loop", and then teaching AtomicExpandPass to emit that for the inner loop, seems like it should be fine without adding any new concepts to LLVM.

That's pretty easy for atomicrmw, but not so much for cmpxchg, if you want to avoid extra conditionals and jumps on the way out and also support the separate success/failure fences. But this review is for atomicrmw, so we can ignore that for now. (But: there's already work ongoing to allow for "calls" which have multiple successors, which seems like it'll help to fix that issue once it lands.)

I'd be fine with you landing this, and going back and rework it to expand the prefix/suffix in AtomicExpandPass in followup changes.

[asb]

Updates:

• Expand masked AMOs to IR for address calculation and masking + an intrinsic for the core LL/SC loop. Introduce a new hook in TargetLowering to allow this
• Support for part-word atomicrmw max/min/umax/umin
• Use MaskedMerge rather than calculating an inverse mask
• Expand atomicrmw sub i32 to (AMOADD_W GPR:$addr, (SUB X0, GPR:$incr)) with appropriate AQ/RL bits

If there is enthusiasm from other targets, it may be worth moving more of the logic in RISCVISelLowering::emitAtomicRMW to AtomicExpandPass. As evidenced in the follow-up patch which supports 8/16-bit bitwise AMOs with 32-bit AMOs, it is useful to retain the ability to provide custom expansion logic.

I've authored an RFC which details the strategy which I'll post as soon as I've completed uploading the patch-set.

[asb]

The main update in this patch revision is moving more of the IR generation for the masked atomic operation to AtomicExpandPass, reusing the createMaskInstrs helper used by expandPartWordAtomicRMW and expandPartWordCmpXchg. We now introduce AtomicExpansionKind::MaskedIntrinsic rather than AtomicExpansionKind::Custom.

No other backend is forced to move towards this lowering strategy, but it should now be possible to do so with minimal code duplication.

[jyknight]

include/llvm/CodeGen/TargetLowering.h

1561 ↗

(On Diff #159859)

s/Custom/Masked/

lib/CodeGen/AtomicExpandPass.cpp

888 ↗

(On Diff #159859)

For the OR, XOR, and AND operations, there's no need to implement or use a masked intrinsic at all -- the normal 32bit atomic will do the job fine given ValOperand_Shifted as the argument.

For the masked AND, you do need to pass "ValOperand_Shifted | PMV.Inv_Mask" instead, of course.

(I note that I neglected to implement that optimization for AND in the cmpxchg expansion in expandPartwordAtomicRMW.)

lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

120 ↗

(On Diff #159859)

It's not a big deal, but this function and others like it make me think it'd be better to just have one instruction for LR and one for SC, with an argument for the atomic ordering. I assume there's some reason that was not doable?

204 ↗

(On Diff #159859)

Nice -- the sequence in AtomicExpandPass::performMaskedAtomicOp should be updated to do the same thing, I think!

379 ↗

(On Diff #159859)

Doesn't this need to sign-extend the values prior to doing a signed comparison?

[asb]

lib/CodeGen/AtomicExpandPass.cpp

888 ↗

(On Diff #159859)

Yes I recognise this, and D48129 implements the three optimisations you suggest for RISC-V. I've now reworked that patch so it is target-independent and the same i8/i16 -> i32 transformation logic is used regardless of the AtomicExpansionKind. Let me know what you think,

It probably makes sense to rebase this patch so it is dependent on D48129.

[asb]

Rebase on top of D48129 and update to address most review comments.

TODO: Look again at signed atomicrmw min/max.

lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

120 ↗

(On Diff #159859)

Instruction definitions in the RISC-V backend follow the instruction encoding, meaning they are easy to inspect for correctness with respect to the ISA manual and the MC layer support is trivial. We could define a RISCV::PseudoLR that took AtomicOrdering and have logic that selected the appropriate AQ/RL bits, but we'd still be implementing something equivalent to this switch - just moving it elsewhere.

204 ↗

(On Diff #159859)

I've added a TODO to performMaskedAtomicOp

[jyknight]

One comment left unresolved, re sign-extend -- patch LG other than that!

lib/CodeGen/AtomicExpandPass.cpp

888 ↗

(On Diff #159859)

Sorry, I totally forgot about that other patch! Yes, it makes more sense to structure this way around. :)

lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

120 ↗

(On Diff #159859)

Nah, it's fine, not a big deal.

[asb]

Update to sign-extends the values for signed min/max, which is the last known issue with this patch.

Happy to commit?

[asb]

Actually, hold-off on reviewing. I don't think the logic for sign extension is correct when the target memory location isn't 32-bit aligned. Sorry for the noise. I'll add look again and add more tests to cover this.

It's a little tempting to just fail for [u]min/max of i8/i16 given Clang never generates them, but we've got this far now...

[asb]

The patch has been updated so sign-extension is performed correct for part-word signed min/max operations.

All known issues are resolved. Final review would be very welcome.

[jyknight]

Final update looks good as far as I can tell.

Ship it! :)

llvm/trunk/include/llvm/CodeGen/TargetLowering.h

public:
   /// w.r.t. what they should expand to.
   enum class AtomicExpansionKind {
     None,    // Don't expand the instruction.
     LLSC,    // Expand the instruction into loadlinked/storeconditional; used
              // by ARM/AArch64.
     LLOnly,  // Expand the (load) instruction into just a load-linked, which has
              // greater atomic guarantees than a normal load.
     CmpXChg, // Expand the instruction into cmpxchg; used by at least X86.
+    MaskedIntrinsic, // Use a target-specific intrinsic for the LL/SC loop.
   };
 
   /// Enum that specifies when a multiplication should be expanded.
   enum class MulExpansionKind {
     Always,            // Always expand the instruction.
     OnlyLegalOrCustom, // Only expand when the resulting instructions are legal
                        // or custom.
   };
public:
 
   /// Perform a store-conditional operation to Addr. Return the status of the
   /// store. This should be 0 if the store succeeded, non-zero otherwise.
   virtual Value *emitStoreConditional(IRBuilder<> &Builder, Value *Val,
                                       Value *Addr, AtomicOrdering Ord) const {
     llvm_unreachable("Store conditional unimplemented on this target");
   }
 
+  /// Perform a masked atomicrmw using a target-specific intrinsic. This
+  /// represents the core LL/SC loop which will be lowered at a late stage by
+  /// the backend.
+  virtual Value *emitMaskedAtomicRMWIntrinsic(IRBuilder<> &Builder,
+                                              AtomicRMWInst *AI,
+                                              Value *AlignedAddr, Value *Incr,
+                                              Value *Mask, Value *ShiftAmt,
+                                              AtomicOrdering Ord) const {
+    llvm_unreachable("Masked atomicrmw expansion unimplemented on this target");
+  }
+
   /// Inserts in the IR a target-specific intrinsic specifying a fence.
   /// It is called by AtomicExpandPass before expanding an
   ///   AtomicRMW/AtomicCmpXchg/AtomicStore/AtomicLoad
   ///   if shouldInsertFencesForAtomic returns true.
   ///
   /// Inst is the original atomic instruction, prior to other expansions that
   /// may be performed.
   ///

llvm/trunk/include/llvm/IR/Intrinsics.td

 include "llvm/IR/IntrinsicsXCore.td"
 include "llvm/IR/IntrinsicsHexagon.td"
 include "llvm/IR/IntrinsicsNVVM.td"
 include "llvm/IR/IntrinsicsMips.td"
 include "llvm/IR/IntrinsicsAMDGPU.td"
 include "llvm/IR/IntrinsicsBPF.td"
 include "llvm/IR/IntrinsicsSystemZ.td"
 include "llvm/IR/IntrinsicsWebAssembly.td"
+include "llvm/IR/IntrinsicsRISCV.td"

llvm/trunk/include/llvm/IR/IntrinsicsRISCV.td

+//===- IntrinsicsRISCV.td - Defines RISCV intrinsics -------*- tablegen -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines all of the RISCV-specific intrinsics.
+//
+//===----------------------------------------------------------------------===//
+
+let TargetPrefix = "riscv" in {
+
+//===----------------------------------------------------------------------===//
+// Atomics
+
+class MaskedAtomicRMW32Intrinsic
+    : Intrinsic<[llvm_i32_ty],
+                [llvm_anyptr_ty, llvm_i32_ty, llvm_i32_ty, llvm_i32_ty],
+                [IntrArgMemOnly, NoCapture<0>]>;
+
+class MaskedAtomicRMW32WithSextIntrinsic
+    : Intrinsic<[llvm_i32_ty],
+                [llvm_anyptr_ty, llvm_i32_ty, llvm_i32_ty, llvm_i32_ty,
+                 llvm_i32_ty],
+                [IntrArgMemOnly, NoCapture<0>]>;
+
+def int_riscv_masked_atomicrmw_xchg_i32 : MaskedAtomicRMW32Intrinsic;
+def int_riscv_masked_atomicrmw_add_i32  : MaskedAtomicRMW32Intrinsic;
+def int_riscv_masked_atomicrmw_sub_i32  : MaskedAtomicRMW32Intrinsic;
+def int_riscv_masked_atomicrmw_nand_i32 : MaskedAtomicRMW32Intrinsic;
+def int_riscv_masked_atomicrmw_max_i32  : MaskedAtomicRMW32WithSextIntrinsic;
+def int_riscv_masked_atomicrmw_min_i32  : MaskedAtomicRMW32WithSextIntrinsic;
+def int_riscv_masked_atomicrmw_umax_i32 : MaskedAtomicRMW32Intrinsic;
+def int_riscv_masked_atomicrmw_umin_i32 : MaskedAtomicRMW32Intrinsic;
+
+} // TargetPrefix = "riscv"

llvm/trunk/lib/CodeGen/AtomicExpandPass.cpp

private:
     void expandAtomicOpToLLSC(
         Instruction *I, Type *ResultTy, Value *Addr, AtomicOrdering MemOpOrder,
         function_ref<Value *(IRBuilder<> &, Value *)> PerformOp);
     void expandPartwordAtomicRMW(
         AtomicRMWInst *I,
         TargetLoweringBase::AtomicExpansionKind ExpansionKind);
     AtomicRMWInst *widenPartwordAtomicRMW(AtomicRMWInst *AI);
     void expandPartwordCmpXchg(AtomicCmpXchgInst *I);
+    void expandAtomicRMWToMaskedIntrinsic(AtomicRMWInst *AI);
 
     AtomicCmpXchgInst *convertCmpXchgToIntegerType(AtomicCmpXchgInst *CI);
     static Value *insertRMWCmpXchgLoop(
         IRBuilder<> &Builder, Type *ResultType, Value *Addr,
         AtomicOrdering MemOpOrder,
         function_ref<Value *(IRBuilder<> &, Value *)> PerformOp,
         CreateCmpXchgInstFun CreateCmpXchg);
 
case TargetLoweringBase::AtomicExpansionKind::LLSC:
     expandAtomicOpToLLSC(
         LI, LI->getType(), LI->getPointerOperand(), LI->getOrdering(),
         [](IRBuilder<> &Builder, Value *Loaded) { return Loaded; });
     return true;
   case TargetLoweringBase::AtomicExpansionKind::LLOnly:
     return expandAtomicLoadToLL(LI);
   case TargetLoweringBase::AtomicExpansionKind::CmpXChg:
     return expandAtomicLoadToCmpXchg(LI);
-  }
+  default:
     llvm_unreachable("Unhandled case in tryExpandAtomicLoad");
   }
+}
 
 bool AtomicExpand::expandAtomicLoadToLL(LoadInst *LI) {
   IRBuilder<> Builder(LI);
 
   // On some architectures, load-linked instructions are atomic for larger
   // sizes than normal loads. For example, the only 64-bit load guaranteed
   // to be single-copy atomic by ARM is an ldrexd (A3.5.3).
   Value *Val =
case TargetLoweringBase::AtomicExpansionKind::CmpXChg: {
     if (ValueSize < MinCASSize) {
       expandPartwordAtomicRMW(AI,
                               TargetLoweringBase::AtomicExpansionKind::CmpXChg);
     } else {
       expandAtomicRMWToCmpXchg(AI, createCmpXchgInstFun);
     }
     return true;
   }
+  case TargetLoweringBase::AtomicExpansionKind::MaskedIntrinsic: {
+    expandAtomicRMWToMaskedIntrinsic(AI);
+    return true;
+  }
   default:
     llvm_unreachable("Unhandled case in tryExpandAtomicRMW");
   }
 }
 
 namespace {
 
 /// Result values from createMaskInstrs helper.

 
 /// Emit IR to implement a masked version of a given atomicrmw
 /// operation. (That is, only the bits under the Mask should be
 /// affected by the operation)
 static Value *performMaskedAtomicOp(AtomicRMWInst::BinOp Op,
                                     IRBuilder<> &Builder, Value *Loaded,
                                     Value *Shifted_Inc, Value *Inc,
                                     const PartwordMaskValues &PMV) {
+  // TODO: update to use
+  // https://graphics.stanford.edu/~seander/bithacks.html#MaskedMerge in order
+  // to merge bits from two values without requiring PMV.Inv_Mask.
   switch (Op) {
   case AtomicRMWInst::Xchg: {
     Value *Loaded_MaskOut = Builder.CreateAnd(Loaded, PMV.Inv_Mask);
     Value *FinalVal = Builder.CreateOr(Loaded_MaskOut, Shifted_Inc);
     return FinalVal;
   }
   case AtomicRMWInst::Or:
   case AtomicRMWInst::Xor:
void AtomicExpand::expandAtomicOpToLLSC(
   IRBuilder<> Builder(I);
   Value *Loaded =
       insertRMWLLSCLoop(Builder, ResultType, Addr, MemOpOrder, PerformOp);
 
   I->replaceAllUsesWith(Loaded);
   I->eraseFromParent();
 }
 
+void AtomicExpand::expandAtomicRMWToMaskedIntrinsic(AtomicRMWInst *AI) {
+  IRBuilder<> Builder(AI);
+
+  PartwordMaskValues PMV =
+      createMaskInstrs(Builder, AI, AI->getType(), AI->getPointerOperand(),
+                       TLI->getMinCmpXchgSizeInBits() / 8);
+
+  // The value operand must be sign-extended for signed min/max so that the
+  // target's signed comparison instructions can be used. Otherwise, just
+  // zero-ext.
+  Instruction::CastOps CastOp = Instruction::ZExt;
+  AtomicRMWInst::BinOp RMWOp = AI->getOperation();
+  if (RMWOp == AtomicRMWInst::Max || RMWOp == AtomicRMWInst::Min)
+    CastOp = Instruction::SExt;
+
+  Value *ValOperand_Shifted = Builder.CreateShl(
+      Builder.CreateCast(CastOp, AI->getValOperand(), PMV.WordType),
+      PMV.ShiftAmt, "ValOperand_Shifted");
+  Value *OldResult = TLI->emitMaskedAtomicRMWIntrinsic(
+      Builder, AI, PMV.AlignedAddr, ValOperand_Shifted, PMV.Mask, PMV.ShiftAmt,
+      AI->getOrdering());
+  Value *FinalOldResult = Builder.CreateTrunc(
+      Builder.CreateLShr(OldResult, PMV.ShiftAmt), PMV.ValueType);
+  AI->replaceAllUsesWith(FinalOldResult);
+  AI->eraseFromParent();
+}
+
 Value *AtomicExpand::insertRMWLLSCLoop(
     IRBuilder<> &Builder, Type *ResultTy, Value *Addr,
     AtomicOrdering MemOpOrder,
     function_ref<Value *(IRBuilder<> &, Value *)> PerformOp) {
   LLVMContext &Ctx = Builder.getContext();
   BasicBlock *BB = Builder.GetInsertBlock();
   Function *F = BB->getParent();
 

llvm/trunk/lib/Target/RISCV/CMakeLists.txt

 tablegen(LLVM RISCVGenMCPseudoLowering.inc -gen-pseudo-lowering)
 tablegen(LLVM RISCVGenRegisterInfo.inc -gen-register-info)
 tablegen(LLVM RISCVGenSubtargetInfo.inc -gen-subtarget)
 
 add_public_tablegen_target(RISCVCommonTableGen)
 
 add_llvm_target(RISCVCodeGen
   RISCVAsmPrinter.cpp
+  RISCVExpandPseudoInsts.cpp
   RISCVFrameLowering.cpp
   RISCVInstrInfo.cpp
   RISCVISelDAGToDAG.cpp
   RISCVISelLowering.cpp
   RISCVMCInstLower.cpp
   RISCVMergeBaseOffset.cpp
   RISCVRegisterInfo.cpp
   RISCVSubtarget.cpp

llvm/trunk/lib/Target/RISCV/RISCV.h

 // RISC-V back-end.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_RISCV_RISCV_H
 #define LLVM_LIB_TARGET_RISCV_RISCV_H
 
 #include "MCTargetDesc/RISCVBaseInfo.h"
+#include "llvm/Target/TargetMachine.h"
 
 namespace llvm {
 class RISCVTargetMachine;
 class AsmPrinter;
 class FunctionPass;
 class MCInst;
 class MCOperand;
 class MachineInstr;
 class MachineOperand;
 class PassRegistry;
 
 void LowerRISCVMachineInstrToMCInst(const MachineInstr *MI, MCInst &OutMI,
                                     const AsmPrinter &AP);
 bool LowerRISCVMachineOperandToMCOperand(const MachineOperand &MO,
                                          MCOperand &MCOp, const AsmPrinter &AP);
 
 FunctionPass *createRISCVISelDag(RISCVTargetMachine &TM);
 
 FunctionPass *createRISCVMergeBaseOffsetOptPass();
 void initializeRISCVMergeBaseOffsetOptPass(PassRegistry &);
+
+FunctionPass *createRISCVExpandPseudoPass();
+void initializeRISCVExpandPseudoPass(PassRegistry &);
 }
 
 #endif

llvm/trunk/lib/Target/RISCV/RISCVExpandPseudoInsts.cpp

+//===-- RISCVExpandPseudoInsts.cpp - Expand pseudo instructions -----------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains a pass that expands pseudo instructions into target
+// instructions. This pass should be run after register allocation but before
+// the post-regalloc scheduling pass.
+//
+//===----------------------------------------------------------------------===//
+
+#include "RISCV.h"
+#include "RISCVInstrInfo.h"
+#include "RISCVTargetMachine.h"
+
+#include "llvm/CodeGen/LivePhysRegs.h"
+#include "llvm/CodeGen/MachineFunctionPass.h"
+#include "llvm/CodeGen/MachineInstrBuilder.h"
+
+using namespace llvm;
+
+#define RISCV_EXPAND_PSEUDO_NAME "RISCV pseudo instruction expansion pass"
+
+namespace {
+
+class RISCVExpandPseudo : public MachineFunctionPass {
+public:
+  const RISCVInstrInfo *TII;
+  static char ID;
+
+  RISCVExpandPseudo() : MachineFunctionPass(ID) {
+    initializeRISCVExpandPseudoPass(*PassRegistry::getPassRegistry());
+  }
+
+  bool runOnMachineFunction(MachineFunction &MF) override;
+
+  StringRef getPassName() const override { return RISCV_EXPAND_PSEUDO_NAME; }
+
+private:
+  bool expandMBB(MachineBasicBlock &MBB);
+  bool expandMI(MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
+                MachineBasicBlock::iterator &NextMBBI);
+  bool expandAtomicBinOp(MachineBasicBlock &MBB,
+                         MachineBasicBlock::iterator MBBI, AtomicRMWInst::BinOp,
+                         bool IsMasked, int Width,
+                         MachineBasicBlock::iterator &NextMBBI);
+  bool expandAtomicMinMaxOp(MachineBasicBlock &MBB,
+                            MachineBasicBlock::iterator MBBI,
+                            AtomicRMWInst::BinOp, bool IsMasked, int Width,
+                            MachineBasicBlock::iterator &NextMBBI);
+};
+
+char RISCVExpandPseudo::ID = 0;
+
+bool RISCVExpandPseudo::runOnMachineFunction(MachineFunction &MF) {
+  TII = static_cast<const RISCVInstrInfo *>(MF.getSubtarget().getInstrInfo());
+  bool Modified = false;
+  for (auto &MBB : MF)
+    Modified |= expandMBB(MBB);
+  return Modified;
+}
+
+bool RISCVExpandPseudo::expandMBB(MachineBasicBlock &MBB) {
+  bool Modified = false;
+
+  MachineBasicBlock::iterator MBBI = MBB.begin(), E = MBB.end();
+  while (MBBI != E) {
+    MachineBasicBlock::iterator NMBBI = std::next(MBBI);
+    Modified |= expandMI(MBB, MBBI, NMBBI);
+    MBBI = NMBBI;
+  }
+
+  return Modified;
+}
+
+bool RISCVExpandPseudo::expandMI(MachineBasicBlock &MBB,
+                                 MachineBasicBlock::iterator MBBI,
+                                 MachineBasicBlock::iterator &NextMBBI) {
+  switch (MBBI->getOpcode()) {
+  case RISCV::PseudoAtomicLoadNand32:
+    return expandAtomicBinOp(MBB, MBBI, AtomicRMWInst::Nand, false, 32,
+                             NextMBBI);
+  case RISCV::PseudoMaskedAtomicSwap32:
+    return expandAtomicBinOp(MBB, MBBI, AtomicRMWInst::Xchg, true, 32,
+                             NextMBBI);
+  case RISCV::PseudoMaskedAtomicLoadAdd32:
+    return expandAtomicBinOp(MBB, MBBI, AtomicRMWInst::Add, true, 32, NextMBBI);
+  case RISCV::PseudoMaskedAtomicLoadSub32:
+    return expandAtomicBinOp(MBB, MBBI, AtomicRMWInst::Sub, true, 32, NextMBBI);
+  case RISCV::PseudoMaskedAtomicLoadNand32:
+    return expandAtomicBinOp(MBB, MBBI, AtomicRMWInst::Nand, true, 32,
+                             NextMBBI);
+  case RISCV::PseudoMaskedAtomicLoadMax32:
+    return expandAtomicMinMaxOp(MBB, MBBI, AtomicRMWInst::Max, true, 32,
+                                NextMBBI);
+  case RISCV::PseudoMaskedAtomicLoadMin32:
+    return expandAtomicMinMaxOp(MBB, MBBI, AtomicRMWInst::Min, true, 32,
+                                NextMBBI);
+  case RISCV::PseudoMaskedAtomicLoadUMax32:
+    return expandAtomicMinMaxOp(MBB, MBBI, AtomicRMWInst::UMax, true, 32,
+                                NextMBBI);
+  case RISCV::PseudoMaskedAtomicLoadUMin32:
+    return expandAtomicMinMaxOp(MBB, MBBI, AtomicRMWInst::UMin, true, 32,
+                                NextMBBI);
+  }
+
+  return false;
+}
+
+static unsigned getLRForRMW32(AtomicOrdering Ordering) {
+  switch (Ordering) {
+  default:
+    llvm_unreachable("Unexpected AtomicOrdering");
+  case AtomicOrdering::Monotonic:
+    return RISCV::LR_W;
+  case AtomicOrdering::Acquire:
+    return RISCV::LR_W_AQ;
+  case AtomicOrdering::Release:
+    return RISCV::LR_W;
+  case AtomicOrdering::AcquireRelease:
+    return RISCV::LR_W_AQ;
+  case AtomicOrdering::SequentiallyConsistent:
+    return RISCV::LR_W_AQ_RL;
+  }
+}
+
+static unsigned getSCForRMW32(AtomicOrdering Ordering) {
+  switch (Ordering) {
+  default:
+    llvm_unreachable("Unexpected AtomicOrdering");
+  case AtomicOrdering::Monotonic:
+    return RISCV::SC_W;
+  case AtomicOrdering::Acquire:
+    return RISCV::SC_W;
+  case AtomicOrdering::Release:
+    return RISCV::SC_W_RL;
+  case AtomicOrdering::AcquireRelease:
+    return RISCV::SC_W_RL;
+  case AtomicOrdering::SequentiallyConsistent:
+    return RISCV::SC_W_AQ_RL;
+  }
+}
+
+static void doAtomicBinOpExpansion(const RISCVInstrInfo *TII, MachineInstr &MI,
+                                   DebugLoc DL, MachineBasicBlock *ThisMBB,
+                                   MachineBasicBlock *LoopMBB,
+                                   MachineBasicBlock *DoneMBB,
+                                   AtomicRMWInst::BinOp BinOp, int Width) {
+  assert(Width == 32 && "RV64 atomic expansion currently unsupported");
+  unsigned DestReg = MI.getOperand(0).getReg();
+  unsigned ScratchReg = MI.getOperand(1).getReg();
+  unsigned AddrReg = MI.getOperand(2).getReg();
+  unsigned IncrReg = MI.getOperand(3).getReg();
+  AtomicOrdering Ordering =
+      static_cast<AtomicOrdering>(MI.getOperand(4).getImm());
+
+  // .loop:
+  //   lr.w dest, (addr)
+  //   binop scratch, dest, val
+  //   sc.w scratch, scratch, (addr)
+  //   bnez scratch, loop
+  BuildMI(LoopMBB, DL, TII->get(getLRForRMW32(Ordering)), DestReg)
+      .addReg(AddrReg);
+  switch (BinOp) {
+  default:
+    llvm_unreachable("Unexpected AtomicRMW BinOp");
+  case AtomicRMWInst::Nand:
+    BuildMI(LoopMBB, DL, TII->get(RISCV::AND), ScratchReg)
+        .addReg(DestReg)
+        .addReg(IncrReg);
+    BuildMI(LoopMBB, DL, TII->get(RISCV::XORI), ScratchReg)
+        .addReg(ScratchReg)
+        .addImm(-1);
+    break;
+  }
+  BuildMI(LoopMBB, DL, TII->get(getSCForRMW32(Ordering)), ScratchReg)
+      .addReg(AddrReg)
+      .addReg(ScratchReg);
+  BuildMI(LoopMBB, DL, TII->get(RISCV::BNE))
+      .addReg(ScratchReg)
+      .addReg(RISCV::X0)
+      .addMBB(LoopMBB);
+}
+
+static void insertMaskedMerge(const RISCVInstrInfo *TII, DebugLoc DL,
+                              MachineBasicBlock *MBB, unsigned DestReg,
+                              unsigned OldValReg, unsigned NewValReg,
+                              unsigned MaskReg, unsigned ScratchReg) {
+  assert(OldValReg != ScratchReg && "OldValReg and ScratchReg must be unique");
+  assert(OldValReg != MaskReg && "OldValReg and MaskReg must be unique");
+  assert(ScratchReg != MaskReg && "ScratchReg and MaskReg must be unique");
+
+  // We select bits from newval and oldval using:
+  // https://graphics.stanford.edu/~seander/bithacks.html#MaskedMerge
+  // r = oldval ^ ((oldval ^ newval) & masktargetdata);
+  BuildMI(MBB, DL, TII->get(RISCV::XOR), ScratchReg)
+      .addReg(OldValReg)
+      .addReg(NewValReg);
+  BuildMI(MBB, DL, TII->get(RISCV::AND), ScratchReg)
+      .addReg(ScratchReg)
+      .addReg(MaskReg);
+  BuildMI(MBB, DL, TII->get(RISCV::XOR), DestReg)
+      .addReg(OldValReg)
+      .addReg(ScratchReg);
+}
+
+static void doMaskedAtomicBinOpExpansion(
+    const RISCVInstrInfo *TII, MachineInstr &MI, DebugLoc DL,
+    MachineBasicBlock *ThisMBB, MachineBasicBlock *LoopMBB,
+    MachineBasicBlock *DoneMBB, AtomicRMWInst::BinOp BinOp, int Width) {
+  assert(Width == 32 && "RV64 atomic expansion currently unsupported");
+  unsigned DestReg = MI.getOperand(0).getReg();
+  unsigned ScratchReg = MI.getOperand(1).getReg();
+  unsigned AddrReg = MI.getOperand(2).getReg();
+  unsigned IncrReg = MI.getOperand(3).getReg();
+  unsigned MaskReg = MI.getOperand(4).getReg();
+  AtomicOrdering Ordering =
+      static_cast<AtomicOrdering>(MI.getOperand(5).getImm());
+
+  // .loop:
+  //   lr.w destreg, (alignedaddr)
+  //   binop scratch, destreg, incr
+  //   xor scratch, destreg, scratch
+  //   and scratch, scratch, masktargetdata
+  //   xor scratch, destreg, scratch
+  //   sc.w scratch, scratch, (alignedaddr)
+  //   bnez scratch, loop
+  BuildMI(LoopMBB, DL, TII->get(getLRForRMW32(Ordering)), DestReg)
+      .addReg(AddrReg);
+  switch (BinOp) {
+  default:
+    llvm_unreachable("Unexpected AtomicRMW BinOp");
+  case AtomicRMWInst::Xchg:
+    BuildMI(LoopMBB, DL, TII->get(RISCV::ADD), ScratchReg)
+        .addReg(RISCV::X0)
+        .addReg(IncrReg);
+    break;
+  case AtomicRMWInst::Add:
+    BuildMI(LoopMBB, DL, TII->get(RISCV::ADD), ScratchReg)
+        .addReg(DestReg)
+        .addReg(IncrReg);
+    break;
+  case AtomicRMWInst::Sub:
+    BuildMI(LoopMBB, DL, TII->get(RISCV::SUB), ScratchReg)
+        .addReg(DestReg)
+        .addReg(IncrReg);
+    break;
+  case AtomicRMWInst::Nand:
+    BuildMI(LoopMBB, DL, TII->get(RISCV::AND), ScratchReg)
+        .addReg(DestReg)
+        .addReg(IncrReg);
+    BuildMI(LoopMBB, DL, TII->get(RISCV::XORI), ScratchReg)
+        .addReg(ScratchReg)
+        .addImm(-1);
+    break;
+  }
+
+  insertMaskedMerge(TII, DL, LoopMBB, ScratchReg, DestReg, ScratchReg, MaskReg,
+                    ScratchReg);
+
+  BuildMI(LoopMBB, DL, TII->get(getSCForRMW32(Ordering)), ScratchReg)
+      .addReg(AddrReg)
+      .addReg(ScratchReg);
+  BuildMI(LoopMBB, DL, TII->get(RISCV::BNE))
+      .addReg(ScratchReg)
+      .addReg(RISCV::X0)
+      .addMBB(LoopMBB);
+}
+
+bool RISCVExpandPseudo::expandAtomicBinOp(
+    MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
+    AtomicRMWInst::BinOp BinOp, bool IsMasked, int Width,
+    MachineBasicBlock::iterator &NextMBBI) {
+  MachineInstr &MI = *MBBI;
+  DebugLoc DL = MI.getDebugLoc();
+
+  MachineFunction *MF = MBB.getParent();
+  auto LoopMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+  auto DoneMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+
+  // Insert new MBBs.
+  MF->insert(++MBB.getIterator(), LoopMBB);
+  MF->insert(++LoopMBB->getIterator(), DoneMBB);
+
+  // Set up successors and transfer remaining instructions to DoneMBB.
+  LoopMBB->addSuccessor(LoopMBB);
+  LoopMBB->addSuccessor(DoneMBB);
+  DoneMBB->splice(DoneMBB->end(), &MBB, MI, MBB.end());
+  DoneMBB->transferSuccessors(&MBB);
+  MBB.addSuccessor(LoopMBB);
+
+  if (!IsMasked)
+    doAtomicBinOpExpansion(TII, MI, DL, &MBB, LoopMBB, DoneMBB, BinOp, Width);
+  else
+    doMaskedAtomicBinOpExpansion(TII, MI, DL, &MBB, LoopMBB, DoneMBB, BinOp,
+                                 Width);
+
+  NextMBBI = MBB.end();
+  MI.eraseFromParent();
+
+  LivePhysRegs LiveRegs;
+  computeAndAddLiveIns(LiveRegs, *LoopMBB);
+  computeAndAddLiveIns(LiveRegs, *DoneMBB);
+
+  return true;
+}
+
+static void insertSext(const RISCVInstrInfo *TII, DebugLoc DL,
+                       MachineBasicBlock *MBB, unsigned ValReg,
+                       unsigned ShamtReg) {
+  BuildMI(MBB, DL, TII->get(RISCV::SLL), ValReg)
+      .addReg(ValReg)
+      .addReg(ShamtReg);
+  BuildMI(MBB, DL, TII->get(RISCV::SRA), ValReg)
+      .addReg(ValReg)
+      .addReg(ShamtReg);
+}
+
+bool RISCVExpandPseudo::expandAtomicMinMaxOp(
+    MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
+    AtomicRMWInst::BinOp BinOp, bool IsMasked, int Width,
+    MachineBasicBlock::iterator &NextMBBI) {
+  assert(IsMasked == true &&
+         "Should only need to expand masked atomic max/min");
+  assert(Width == 32 && "RV64 atomic expansion currently unsupported");
+
+  MachineInstr &MI = *MBBI;
+  DebugLoc DL = MI.getDebugLoc();
+  MachineFunction *MF = MBB.getParent();
+  auto LoopHeadMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+  auto LoopIfBodyMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+  auto LoopTailMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+  auto DoneMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+
+  // Insert new MBBs.
+  MF->insert(++MBB.getIterator(), LoopHeadMBB);
+  MF->insert(++LoopHeadMBB->getIterator(), LoopIfBodyMBB);
+  MF->insert(++LoopIfBodyMBB->getIterator(), LoopTailMBB);
+  MF->insert(++LoopTailMBB->getIterator(), DoneMBB);
+
+  // Set up successors and transfer remaining instructions to DoneMBB.
+  LoopHeadMBB->addSuccessor(LoopIfBodyMBB);
+  LoopHeadMBB->addSuccessor(LoopTailMBB);
+  LoopIfBodyMBB->addSuccessor(LoopTailMBB);
+  LoopTailMBB->addSuccessor(LoopHeadMBB);
+  LoopTailMBB->addSuccessor(DoneMBB);
+  DoneMBB->splice(DoneMBB->end(), &MBB, MI, MBB.end());
+  DoneMBB->transferSuccessors(&MBB);
+  MBB.addSuccessor(LoopHeadMBB);
+
+  unsigned DestReg = MI.getOperand(0).getReg();
+  unsigned Scratch1Reg = MI.getOperand(1).getReg();
+  unsigned Scratch2Reg = MI.getOperand(2).getReg();
+  unsigned AddrReg = MI.getOperand(3).getReg();
+  unsigned IncrReg = MI.getOperand(4).getReg();
+  unsigned MaskReg = MI.getOperand(5).getReg();
+  bool IsSigned = BinOp == AtomicRMWInst::Min || BinOp == AtomicRMWInst::Max;
+  AtomicOrdering Ordering =
+      static_cast<AtomicOrdering>(MI.getOperand(IsSigned ? 7 : 6).getImm());
+
+  //
+  // .loophead:
+  //   lr.w destreg, (alignedaddr)
+  //   and scratch2, destreg, mask
+  //   mv scratch1, destreg
+  //   [sext scratch2 if signed min/max]
+  //   ifnochangeneeded scratch2, incr, .looptail
+  BuildMI(LoopHeadMBB, DL, TII->get(getLRForRMW32(Ordering)), DestReg)
+      .addReg(AddrReg);
+  BuildMI(LoopHeadMBB, DL, TII->get(RISCV::AND), Scratch2Reg)
+      .addReg(DestReg)
+      .addReg(MaskReg);
+  BuildMI(LoopHeadMBB, DL, TII->get(RISCV::ADDI), Scratch1Reg)
+      .addReg(DestReg)
+      .addImm(0);
+
+  switch (BinOp) {
+  default:
+    llvm_unreachable("Unexpected AtomicRMW BinOp");
+  case AtomicRMWInst::Max: {
+    insertSext(TII, DL, LoopHeadMBB, Scratch2Reg, MI.getOperand(6).getReg());
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGE))
+        .addReg(Scratch2Reg)
+        .addReg(IncrReg)
+        .addMBB(LoopTailMBB);
+    break;
+  }
+  case AtomicRMWInst::Min: {
+    insertSext(TII, DL, LoopHeadMBB, Scratch2Reg, MI.getOperand(6).getReg());
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGE))
+        .addReg(IncrReg)
+        .addReg(Scratch2Reg)
+        .addMBB(LoopTailMBB);
+    break;
+  }
+  case AtomicRMWInst::UMax:
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGEU))
+        .addReg(Scratch2Reg)
+        .addReg(IncrReg)
+        .addMBB(LoopTailMBB);
+    break;
+  case AtomicRMWInst::UMin:
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGEU))
+        .addReg(IncrReg)
+        .addReg(Scratch2Reg)
+        .addMBB(LoopTailMBB);
+    break;
+  }
+
+  // .loopifbody:
+  //   xor scratch1, destreg, incr
+  //   and scratch1, scratch1, mask
+  //   xor scratch1, destreg, scratch1
+  insertMaskedMerge(TII, DL, LoopIfBodyMBB, Scratch1Reg, DestReg, IncrReg,
+                    MaskReg, Scratch1Reg);
+
+  // .looptail:
+  //   sc.w scratch1, scratch1, (addr)
+  //   bnez scratch1, loop
+  BuildMI(LoopTailMBB, DL, TII->get(getSCForRMW32(Ordering)), Scratch1Reg)
+      .addReg(AddrReg)
+      .addReg(Scratch1Reg);
+  BuildMI(LoopTailMBB, DL, TII->get(RISCV::BNE))
+      .addReg(Scratch1Reg)
+      .addReg(RISCV::X0)
+      .addMBB(LoopHeadMBB);
+
+  NextMBBI = MBB.end();
+  MI.eraseFromParent();
+
+  LivePhysRegs LiveRegs;
+  computeAndAddLiveIns(LiveRegs, *LoopHeadMBB);
+  computeAndAddLiveIns(LiveRegs, *LoopIfBodyMBB);
+  computeAndAddLiveIns(LiveRegs, *LoopTailMBB);
+  computeAndAddLiveIns(LiveRegs, *DoneMBB);
+
+  return true;
+}
+
+} // end of anonymous namespace
+
+INITIALIZE_PASS(RISCVExpandPseudo, "riscv-expand-pseudo",
+                RISCV_EXPAND_PSEUDO_NAME, false, false)
+namespace llvm {
+
+FunctionPass *createRISCVExpandPseudoPass() { return new RISCVExpandPseudo(); }
+
+} // end of namespace llvm

llvm/trunk/lib/Target/RISCV/RISCVISelLowering.h

 
 class RISCVTargetLowering : public TargetLowering {
   const RISCVSubtarget &Subtarget;
 
 public:
   explicit RISCVTargetLowering(const TargetMachine &TM,
                                const RISCVSubtarget &STI);
 
+  bool getTgtMemIntrinsic(IntrinsicInfo &Info, const CallInst &I,
+                          MachineFunction &MF,
+                          unsigned Intrinsic) const override;
   bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM, Type *Ty,
                              unsigned AS,
                              Instruction *I = nullptr) const override;
   bool isLegalICmpImmediate(int64_t Imm) const override;
   bool isLegalAddImmediate(int64_t Imm) const override;
   bool isTruncateFree(Type *SrcTy, Type *DstTy) const override;
   bool isTruncateFree(EVT SrcVT, EVT DstVT) const override;
   bool isZExtFree(SDValue Val, EVT VT2) const override;
private:
   SDValue lowerSELECT(SDValue Op, SelectionDAG &DAG) const;
   SDValue lowerVASTART(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
 
   bool IsEligibleForTailCallOptimization(CCState &CCInfo,
     CallLoweringInfo &CLI, MachineFunction &MF,
     const SmallVector<CCValAssign, 16> &ArgLocs) const;
+
+  TargetLowering::AtomicExpansionKind
+  shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const override;
+  virtual Value *emitMaskedAtomicRMWIntrinsic(
+      IRBuilder<> &Builder, AtomicRMWInst *AI, Value *AlignedAddr, Value *Incr,
+      Value *Mask, Value *ShiftAmt, AtomicOrdering Ord) const override;
 };
 }
 
 #endif

llvm/trunk/lib/Target/RISCV/RISCVISelLowering.cpp

if (Subtarget.hasStdExtD()) {
     setLoadExtAction(ISD::EXTLOAD, MVT::f64, MVT::f32, Expand);
     setTruncStoreAction(MVT::f64, MVT::f32, Expand);
   }
 
   setOperationAction(ISD::GlobalAddress, XLenVT, Custom);
   setOperationAction(ISD::BlockAddress, XLenVT, Custom);
   setOperationAction(ISD::ConstantPool, XLenVT, Custom);
 
-  if (Subtarget.hasStdExtA())
+  if (Subtarget.hasStdExtA()) {
     setMaxAtomicSizeInBitsSupported(Subtarget.getXLen());
-  else
+    setMinCmpXchgSizeInBits(32);
+  } else {
     setMaxAtomicSizeInBitsSupported(0);
+  }
 
   setBooleanContents(ZeroOrOneBooleanContent);
 
   // Function alignments (log2).
   unsigned FunctionAlignment = Subtarget.hasStdExtC() ? 1 : 2;
   setMinFunctionAlignment(FunctionAlignment);
   setPrefFunctionAlignment(FunctionAlignment);
 
   // Effectively disable jump table generation.
   setMinimumJumpTableEntries(INT_MAX);
 }
 
 EVT RISCVTargetLowering::getSetCCResultType(const DataLayout &DL, LLVMContext &,
                                             EVT VT) const {
   if (!VT.isVector())
     return getPointerTy(DL);
   return VT.changeVectorElementTypeToInteger();
 }
 
+bool RISCVTargetLowering::getTgtMemIntrinsic(IntrinsicInfo &Info,
+                                             const CallInst &I,
+                                             MachineFunction &MF,
+                                             unsigned Intrinsic) const {
+  switch (Intrinsic) {
+  default:
+    return false;
+  case Intrinsic::riscv_masked_atomicrmw_xchg_i32:
+  case Intrinsic::riscv_masked_atomicrmw_add_i32:
+  case Intrinsic::riscv_masked_atomicrmw_sub_i32:
+  case Intrinsic::riscv_masked_atomicrmw_nand_i32:
+  case Intrinsic::riscv_masked_atomicrmw_max_i32:
+  case Intrinsic::riscv_masked_atomicrmw_min_i32:
+  case Intrinsic::riscv_masked_atomicrmw_umax_i32:
+  case Intrinsic::riscv_masked_atomicrmw_umin_i32:
+    PointerType *PtrTy = cast<PointerType>(I.getArgOperand(0)->getType());
+    Info.opc = ISD::INTRINSIC_W_CHAIN;
+    Info.memVT = MVT::getVT(PtrTy->getElementType());
+    Info.ptrVal = I.getArgOperand(0);
+    Info.offset = 0;
+    Info.align = 4;
+    Info.flags = MachineMemOperand::MOLoad | MachineMemOperand::MOStore |
+                 MachineMemOperand::MOVolatile;
+    return true;
+  }
+}
+
 bool RISCVTargetLowering::isLegalAddressingMode(const DataLayout &DL,
                                                 const AddrMode &AM, Type *Ty,
                                                 unsigned AS,
                                                 Instruction *I) const {
   // No global is ever allowed as a base.
   if (AM.BaseGV)
     return false;
 

 
 Instruction *RISCVTargetLowering::emitTrailingFence(IRBuilder<> &Builder,
                                                     Instruction *Inst,
                                                     AtomicOrdering Ord) const {
   if (isa<LoadInst>(Inst) && isAcquireOrStronger(Ord))
     return Builder.CreateFence(AtomicOrdering::Acquire);
   return nullptr;
 }
+
+TargetLowering::AtomicExpansionKind
+RISCVTargetLowering::shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const {
+  unsigned Size = AI->getType()->getPrimitiveSizeInBits();
+  if (Size == 8 || Size == 16)
+    return AtomicExpansionKind::MaskedIntrinsic;
+  return AtomicExpansionKind::None;
+}
+
+static Intrinsic::ID
+getIntrinsicForMaskedAtomicRMWBinOp32(AtomicRMWInst::BinOp BinOp) {
+  switch (BinOp) {
+  default:
+    llvm_unreachable("Unexpected AtomicRMW BinOp");
+  case AtomicRMWInst::Xchg:
+    return Intrinsic::riscv_masked_atomicrmw_xchg_i32;
+  case AtomicRMWInst::Add:
+    return Intrinsic::riscv_masked_atomicrmw_add_i32;
+  case AtomicRMWInst::Sub:
+    return Intrinsic::riscv_masked_atomicrmw_sub_i32;
+  case AtomicRMWInst::Nand:
+    return Intrinsic::riscv_masked_atomicrmw_nand_i32;
+  case AtomicRMWInst::Max:
+    return Intrinsic::riscv_masked_atomicrmw_max_i32;
+  case AtomicRMWInst::Min:
+    return Intrinsic::riscv_masked_atomicrmw_min_i32;
+  case AtomicRMWInst::UMax:
+    return Intrinsic::riscv_masked_atomicrmw_umax_i32;
+  case AtomicRMWInst::UMin:
+    return Intrinsic::riscv_masked_atomicrmw_umin_i32;
+  }
+}
+
+Value *RISCVTargetLowering::emitMaskedAtomicRMWIntrinsic(
+    IRBuilder<> &Builder, AtomicRMWInst *AI, Value *AlignedAddr, Value *Incr,
+    Value *Mask, Value *ShiftAmt, AtomicOrdering Ord) const {
+  Value *Ordering = Builder.getInt32(static_cast<uint32_t>(AI->getOrdering()));
+  Type *Tys[] = {AlignedAddr->getType()};
+  Function *LrwOpScwLoop = Intrinsic::getDeclaration(
+      AI->getModule(),
+      getIntrinsicForMaskedAtomicRMWBinOp32(AI->getOperation()), Tys);
+
+  // Must pass the shift amount needed to sign extend the loaded value prior
+  // to performing a signed comparison for min/max. ShiftAmt is the number of
+  // bits to shift the value into position. Pass XLen-ShiftAmt-ValWidth, which
+  // is the number of bits to left+right shift the value in order to
+  // sign-extend.
+  if (AI->getOperation() == AtomicRMWInst::Min ||
+      AI->getOperation() == AtomicRMWInst::Max) {
+    const DataLayout &DL = AI->getModule()->getDataLayout();
+    unsigned ValWidth =
+        DL.getTypeStoreSizeInBits(AI->getValOperand()->getType());
+    Value *SextShamt = Builder.CreateSub(
+        Builder.getInt32(Subtarget.getXLen() - ValWidth), ShiftAmt);
+    return Builder.CreateCall(LrwOpScwLoop,
+                              {AlignedAddr, Incr, Mask, SextShamt, Ordering});
+  }
+
+  return Builder.CreateCall(LrwOpScwLoop, {AlignedAddr, Incr, Mask, Ordering});
+}

llvm/trunk/lib/Target/RISCV/RISCVInstrInfo.td

 //
 // Naming convention: For 'generic' pattern classes, we use the naming
 // convention PatTy1Ty2. For pattern classes which offer a more complex
 // expension, prefix the class name, e.g. BccPat.
 //===----------------------------------------------------------------------===//
 
 /// Generic pattern classes
 
-class PatGprGpr<SDPatternOperator OpNode, RVInstR Inst>
+class PatGprGpr<SDPatternOperator OpNode, RVInst Inst>
     : Pat<(OpNode GPR:$rs1, GPR:$rs2), (Inst GPR:$rs1, GPR:$rs2)>;
 class PatGprSimm12<SDPatternOperator OpNode, RVInstI Inst>
     : Pat<(OpNode GPR:$rs1, simm12:$imm12), (Inst GPR:$rs1, simm12:$imm12)>;
 class PatGprUimmLog2XLen<SDPatternOperator OpNode, RVInstIShift Inst>
     : Pat<(OpNode GPR:$rs1, uimmlog2xlen:$shamt),
           (Inst GPR:$rs1, uimmlog2xlen:$shamt)>;
 
 /// Predicates

llvm/trunk/lib/Target/RISCV/RISCVInstrInfoA.td

 
 defm : LdPat<atomic_load_8,  LB>;
 defm : LdPat<atomic_load_16, LH>;
 defm : LdPat<atomic_load_32, LW>;
 
 defm : AtomicStPat<atomic_store_8,  SB, GPR>;
 defm : AtomicStPat<atomic_store_16, SH, GPR>;
 defm : AtomicStPat<atomic_store_32, SW, GPR>;
+
+/// AMOs
+
+multiclass AMOPat<string AtomicOp, string BaseInst> {
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_monotonic"),
+                  !cast<RVInst>(BaseInst)>;
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_acquire"),
+                  !cast<RVInst>(BaseInst#"_AQ")>;
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_release"),
+                  !cast<RVInst>(BaseInst#"_RL")>;
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_acq_rel"),
+                  !cast<RVInst>(BaseInst#"_AQ_RL")>;
+  def : PatGprGpr<!cast<PatFrag>(AtomicOp#"_seq_cst"),
+                  !cast<RVInst>(BaseInst#"_AQ_RL")>;
+}
+
+defm : AMOPat<"atomic_swap_32", "AMOSWAP_W">;
+defm : AMOPat<"atomic_load_add_32", "AMOADD_W">;
+defm : AMOPat<"atomic_load_and_32", "AMOAND_W">;
+defm : AMOPat<"atomic_load_or_32", "AMOOR_W">;
+defm : AMOPat<"atomic_load_xor_32", "AMOXOR_W">;
+defm : AMOPat<"atomic_load_max_32", "AMOMAX_W">;
+defm : AMOPat<"atomic_load_min_32", "AMOMIN_W">;
+defm : AMOPat<"atomic_load_umax_32", "AMOMAXU_W">;
+defm : AMOPat<"atomic_load_umin_32", "AMOMINU_W">;
+
+def : Pat<(atomic_load_sub_32_monotonic GPR:$addr, GPR:$incr),
+          (AMOADD_W GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(atomic_load_sub_32_acquire GPR:$addr, GPR:$incr),
+          (AMOADD_W_AQ GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(atomic_load_sub_32_release GPR:$addr, GPR:$incr),
+          (AMOADD_W_RL GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(atomic_load_sub_32_acq_rel GPR:$addr, GPR:$incr),
+          (AMOADD_W_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
+def : Pat<(atomic_load_sub_32_seq_cst GPR:$addr, GPR:$incr),
+          (AMOADD_W_AQ_RL GPR:$addr, (SUB X0, GPR:$incr))>;
+
+/// Pseudo AMOs
+
+class PseudoAMO : Pseudo<(outs GPR:$res, GPR:$scratch),
+                         (ins GPR:$addr, GPR:$incr, i32imm:$ordering), []> {
+  let Constraints = "@earlyclobber $res,@earlyclobber $scratch";
+  let mayLoad = 1;
+  let mayStore = 1;
+  let hasSideEffects = 0;
+}
+
+def PseudoAtomicLoadNand32 : PseudoAMO;
+// Ordering constants must be kept in sync with the AtomicOrdering enum in 
+// AtomicOrdering.h.
+def : Pat<(atomic_load_nand_32_monotonic GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 2)>;
+def : Pat<(atomic_load_nand_32_acquire GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 4)>;
+def : Pat<(atomic_load_nand_32_release GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 5)>;
+def : Pat<(atomic_load_nand_32_acq_rel GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 6)>;
+def : Pat<(atomic_load_nand_32_seq_cst GPR:$addr, GPR:$incr),
+          (PseudoAtomicLoadNand32 GPR:$addr, GPR:$incr, 7)>;
+
+class PseudoMaskedAMO
+    : Pseudo<(outs GPR:$res, GPR:$scratch),
+             (ins GPR:$addr, GPR:$incr, GPR:$mask, i32imm:$ordering), []> {
+  let Constraints = "@earlyclobber $res,@earlyclobber $scratch";
+  let mayLoad = 1;
+  let mayStore = 1;
+  let hasSideEffects = 0;
+}
+
+class PseudoMaskedAMOMinMax
+    : Pseudo<(outs GPR:$res, GPR:$scratch1, GPR:$scratch2),
+             (ins GPR:$addr, GPR:$incr, GPR:$mask, i32imm:$sextshamt,
+              i32imm:$ordering), []> {
+  let Constraints = "@earlyclobber $res,@earlyclobber $scratch1,"
+                    "@earlyclobber $scratch2";
+  let mayLoad = 1;
+  let mayStore = 1;
+  let hasSideEffects = 0;
+}
+
+class PseudoMaskedAMOUMinUMax
+    : Pseudo<(outs GPR:$res, GPR:$scratch1, GPR:$scratch2),
+             (ins GPR:$addr, GPR:$incr, GPR:$mask, i32imm:$ordering), []> {
+  let Constraints = "@earlyclobber $res,@earlyclobber $scratch1,"
+                    "@earlyclobber $scratch2";
+  let mayLoad = 1;
+  let mayStore = 1;
+  let hasSideEffects = 0;
+}
+
+class PseudoMaskedAMOPat<Intrinsic intrin, Pseudo AMOInst>
+    : Pat<(intrin GPR:$addr, GPR:$incr, GPR:$mask, imm:$ordering),
+          (AMOInst GPR:$addr, GPR:$incr, GPR:$mask, imm:$ordering)>;
+
+class PseudoMaskedAMOMinMaxPat<Intrinsic intrin, Pseudo AMOInst>
+    : Pat<(intrin GPR:$addr, GPR:$incr, GPR:$mask, GPR:$shiftamt,
+           imm:$ordering),
+          (AMOInst GPR:$addr, GPR:$incr, GPR:$mask, GPR:$shiftamt,
+           imm:$ordering)>;
+
+def PseudoMaskedAtomicSwap32 : PseudoMaskedAMO;
+def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_xchg_i32,
+                         PseudoMaskedAtomicSwap32>;
+def PseudoMaskedAtomicLoadAdd32 : PseudoMaskedAMO;
+def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_add_i32,
+                         PseudoMaskedAtomicLoadAdd32>;
+def PseudoMaskedAtomicLoadSub32 : PseudoMaskedAMO;
+def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_sub_i32,
+                         PseudoMaskedAtomicLoadSub32>;
+def PseudoMaskedAtomicLoadNand32 : PseudoMaskedAMO;
+def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_nand_i32,
+                         PseudoMaskedAtomicLoadNand32>;
+def PseudoMaskedAtomicLoadMax32 : PseudoMaskedAMOMinMax;
+def : PseudoMaskedAMOMinMaxPat<int_riscv_masked_atomicrmw_max_i32,
+                               PseudoMaskedAtomicLoadMax32>;
+def PseudoMaskedAtomicLoadMin32 : PseudoMaskedAMOMinMax;
+def : PseudoMaskedAMOMinMaxPat<int_riscv_masked_atomicrmw_min_i32,
+                               PseudoMaskedAtomicLoadMin32>;
+def PseudoMaskedAtomicLoadUMax32 : PseudoMaskedAMOUMinUMax;
+def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_umax_i32,
+                         PseudoMaskedAtomicLoadUMax32>;
+def PseudoMaskedAtomicLoadUMin32 : PseudoMaskedAMOUMinUMax;
+def : PseudoMaskedAMOPat<int_riscv_masked_atomicrmw_umin_i32,
+                         PseudoMaskedAtomicLoadUMin32>;
 } // Predicates = [HasStdExtA]

llvm/trunk/lib/Target/RISCV/RISCVTargetMachine.cpp

 #include "llvm/Support/FormattedStream.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Target/TargetOptions.h"
 using namespace llvm;
 
 extern "C" void LLVMInitializeRISCVTarget() {
   RegisterTargetMachine<RISCVTargetMachine> X(getTheRISCV32Target());
   RegisterTargetMachine<RISCVTargetMachine> Y(getTheRISCV64Target());
+  auto PR = PassRegistry::getPassRegistry();
+  initializeRISCVExpandPseudoPass(*PR);
 }
 
 static std::string computeDataLayout(const Triple &TT) {
   if (TT.isArch64Bit()) {
     return "e-m:e-p:64:64-i64:64-i128:128-n64-S128";
   } else {
     assert(TT.isArch32Bit() && "only RV32 and RV64 are currently supported");
     return "e-m:e-p:32:32-i64:64-n32-S128";
public:
 
   RISCVTargetMachine &getRISCVTargetMachine() const {
     return getTM<RISCVTargetMachine>();
   }
 
   void addIRPasses() override;
   bool addInstSelector() override;
   void addPreEmitPass() override;
+  void addPreEmitPass2() override;
   void addPreRegAlloc() override;
 };
 }
 
 TargetPassConfig *RISCVTargetMachine::createPassConfig(PassManagerBase &PM) {
   return new RISCVPassConfig(*this, PM);
 }
 
 void RISCVPassConfig::addIRPasses() {
   addPass(createAtomicExpandPass());
   TargetPassConfig::addIRPasses();
 }
 
 bool RISCVPassConfig::addInstSelector() {
   addPass(createRISCVISelDag(getRISCVTargetMachine()));
 
   return false;
 }
 
 void RISCVPassConfig::addPreEmitPass() { addPass(&BranchRelaxationPassID); }
 
+void RISCVPassConfig::addPreEmitPass2() {
+  // Schedule the expansion of AMOs at the last possible moment, avoiding the
+  // possibility for other passes to break the requirements for forward
+  // progress in the LR/SC block.
+  addPass(createRISCVExpandPseudoPass());
+}
+
 void RISCVPassConfig::addPreRegAlloc() {
   addPass(createRISCVMergeBaseOffsetOptPass());
 }

llvm/trunk/test/CodeGen/RISCV/atomic-rmw.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=riscv32 -verify-machineinstrs < %s \
 ; RUN:   | FileCheck -check-prefix=RV32I %s
+; RUN: llc -mtriple=riscv32 -mattr=+a -verify-machineinstrs < %s \
+; RUN:   | FileCheck -check-prefix=RV32IA %s
 
 define i8 @atomicrmw_xchg_i8_monotonic(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_exchange_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB0_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB0_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_xchg_i8_acquire(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_exchange_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB1_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB1_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_xchg_i8_release(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_exchange_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB2_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB2_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_xchg_i8_acq_rel(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_exchange_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB3_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB3_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_xchg_i8_seq_cst(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_exchange_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB4_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB4_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_add_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_add_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB5_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB5_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_add_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_add_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB6_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB6_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_add_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_add_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB7_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB7_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_add_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_add_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB8_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB8_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_add_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_add_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB9_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB9_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_sub_i8_monotonic(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_sub_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB10_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB10_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_sub_i8_acquire(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_sub_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB11_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB11_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_sub_i8_release(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_sub_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB12_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB12_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_sub_i8_acq_rel(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_sub_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB13_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB13_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_sub_i8_seq_cst(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_sub_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB14_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB14_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_and_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_and_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    or a1, a3, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_and_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_and_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    or a1, a3, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_and_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_and_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    or a1, a3, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_and_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_and_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    or a1, a3, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_and_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_and_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    or a1, a3, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_nand_i8_monotonic(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_nand_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB20_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB20_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_nand_i8_acquire(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_nand_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB21_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB21_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_nand_i8_release(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_nand_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB22_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB22_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_nand_i8_acq_rel(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_nand_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB23_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB23_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_nand_i8_seq_cst(i8* %a, i8 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_nand_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a3, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB24_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a3
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB24_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_or_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_or_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_or_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_or_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_or_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_or_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_or_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_or_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_or_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_or_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_xor_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_xor_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_xor_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_xor_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_xor_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_xor_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_xor_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_xor_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_xor_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_xor_1
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_max_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB35_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB35_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB35_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB35_3: # in Loop: Header=BB35_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB35_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_max_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB36_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB36_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB36_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB36_3: # in Loop: Header=BB36_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB36_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_max_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB37_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB37_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB37_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB37_3: # in Loop: Header=BB37_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB37_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_max_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB38_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB38_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB38_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB38_3: # in Loop: Header=BB38_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB38_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_max_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB39_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB39_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB39_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB39_3: # in Loop: Header=BB39_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB39_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_min_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB40_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB40_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB40_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB40_3: # in Loop: Header=BB40_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB40_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_min_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB41_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB41_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB41_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB41_3: # in Loop: Header=BB41_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB41_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_min_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB42_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB42_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB42_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB42_3: # in Loop: Header=BB42_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB42_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_min_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB43_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB43_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB43_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB43_3: # in Loop: Header=BB43_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB43_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_min_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 24
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    addi a4, zero, 255
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 24
+; RV32IA-NEXT:    srai a1, a1, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB44_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB44_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB44_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB44_3: # in Loop: Header=BB44_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB44_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_umax_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB45_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a3, a1, .LBB45_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB45_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB45_3: # in Loop: Header=BB45_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB45_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_umax_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB46_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a3, a1, .LBB46_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB46_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB46_3: # in Loop: Header=BB46_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB46_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_umax_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB47_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a3, a1, .LBB47_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB47_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB47_3: # in Loop: Header=BB47_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB47_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_umax_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB48_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a3, a1, .LBB48_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB48_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB48_3: # in Loop: Header=BB48_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB48_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_umax_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB49_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a3, a1, .LBB49_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB49_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB49_3: # in Loop: Header=BB49_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB49_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i8 @atomicrmw_umin_i8_monotonic(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i8_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i8_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB50_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a3, .LBB50_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB50_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB50_3: # in Loop: Header=BB50_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB50_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i8* %a, i8 %b monotonic
   ret i8 %1
 }
 
 define i8 @atomicrmw_umin_i8_acquire(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i8_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i8_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB51_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a3, .LBB51_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB51_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB51_3: # in Loop: Header=BB51_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB51_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i8* %a, i8 %b acquire
   ret i8 %1
 }
 
 define i8 @atomicrmw_umin_i8_release(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i8_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i8_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB52_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a3, .LBB52_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB52_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB52_3: # in Loop: Header=BB52_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB52_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i8* %a, i8 %b release
   ret i8 %1
 }
 
 define i8 @atomicrmw_umin_i8_acq_rel(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i8_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i8_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB53_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a3, .LBB53_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB53_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB53_3: # in Loop: Header=BB53_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB53_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i8* %a, i8 %b acq_rel
   ret i8 %1
 }
 
 define i8 @atomicrmw_umin_i8_seq_cst(i8 *%a, i8 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i8_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i8_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 255
+; RV32IA-NEXT:    sll a6, a3, a2
+; RV32IA-NEXT:    andi a1, a1, 255
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB54_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    and a3, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a3, .LBB54_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB54_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB54_3: # in Loop: Header=BB54_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB54_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i8* %a, i8 %b seq_cst
   ret i8 %1
 }
 
 define i16 @atomicrmw_xchg_i16_monotonic(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_exchange_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB55_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB55_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_xchg_i16_acquire(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_exchange_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB56_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB56_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_xchg_i16_release(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_exchange_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB57_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB57_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_xchg_i16_acq_rel(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_exchange_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB58_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB58_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_xchg_i16_seq_cst(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_exchange_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB59_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    add a5, zero, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB59_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_add_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_add_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB60_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB60_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_add_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_add_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB61_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB61_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_add_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_add_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB62_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB62_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_add_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_add_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB63_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB63_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_add_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_add_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB64_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    add a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB64_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_sub_i16_monotonic(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_sub_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB65_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB65_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_sub_i16_acquire(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_sub_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB66_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB66_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_sub_i16_release(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_sub_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB67_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB67_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_sub_i16_acq_rel(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_sub_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB68_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB68_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_sub_i16_seq_cst(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_sub_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB69_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    sub a5, a4, a1
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB69_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_and_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_and_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    not a2, a2
+; RV32IA-NEXT:    or a1, a2, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_and_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_and_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    not a2, a2
+; RV32IA-NEXT:    or a1, a2, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_and_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_and_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    not a2, a2
+; RV32IA-NEXT:    or a1, a2, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_and_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_and_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    not a2, a2
+; RV32IA-NEXT:    or a1, a2, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_and_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_and_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    not a2, a2
+; RV32IA-NEXT:    or a1, a2, a1
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_nand_i16_monotonic(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_nand_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB75_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB75_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_nand_i16_acquire(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_nand_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB76_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB76_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_nand_i16_release(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_nand_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB77_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB77_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_nand_i16_acq_rel(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_nand_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB78_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB78_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_nand_i16_seq_cst(i16* %a, i16 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_nand_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a2, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB79_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    and a5, a4, a1
+; RV32IA-NEXT:    not a5, a5
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    and a5, a5, a2
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB79_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_or_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_or_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_or_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_or_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_or_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_or_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_or_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_or_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_or_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_or_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_xor_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_xor_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_xor_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_xor_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_xor_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_xor_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_xor_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_xor_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_xor_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_xor_2
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    srl a0, a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_max_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB90_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB90_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB90_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB90_3: # in Loop: Header=BB90_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB90_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_max_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB91_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB91_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB91_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB91_3: # in Loop: Header=BB91_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB91_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_max_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB92_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB92_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB92_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB92_3: # in Loop: Header=BB92_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB92_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_max_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB93_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB93_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB93_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB93_3: # in Loop: Header=BB93_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB93_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_max_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB94_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a4, a1, .LBB94_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB94_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB94_3: # in Loop: Header=BB94_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB94_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_min_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB95_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB95_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB95_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB95_3: # in Loop: Header=BB95_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB95_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_min_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB96_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB96_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB96_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB96_3: # in Loop: Header=BB96_1 Depth=1
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB96_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_min_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB97_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB97_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB97_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB97_3: # in Loop: Header=BB97_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB97_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_min_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB98_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB98_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB98_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB98_3: # in Loop: Header=BB98_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB98_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_min_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    slli a2, a0, 3
+; RV32IA-NEXT:    andi a2, a2, 24
+; RV32IA-NEXT:    addi a3, zero, 16
+; RV32IA-NEXT:    sub a6, a3, a2
+; RV32IA-NEXT:    lui a4, 16
+; RV32IA-NEXT:    addi a4, a4, -1
+; RV32IA-NEXT:    sll a7, a4, a2
+; RV32IA-NEXT:    slli a1, a1, 16
+; RV32IA-NEXT:    srai a1, a1, 16
+; RV32IA-NEXT:    sll a1, a1, a2
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB99_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a5, (a0)
+; RV32IA-NEXT:    and a4, a5, a7
+; RV32IA-NEXT:    mv a3, a5
+; RV32IA-NEXT:    sll a4, a4, a6
+; RV32IA-NEXT:    sra a4, a4, a6
+; RV32IA-NEXT:    bge a1, a4, .LBB99_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB99_1 Depth=1
+; RV32IA-NEXT:    xor a3, a5, a1
+; RV32IA-NEXT:    and a3, a3, a7
+; RV32IA-NEXT:    xor a3, a5, a3
+; RV32IA-NEXT:  .LBB99_3: # in Loop: Header=BB99_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB99_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a5, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_umax_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB100_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a2, a1, .LBB100_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB100_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB100_3: # in Loop: Header=BB100_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB100_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_umax_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB101_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a2, a1, .LBB101_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB101_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB101_3: # in Loop: Header=BB101_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB101_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_umax_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB102_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a2, a1, .LBB102_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB102_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB102_3: # in Loop: Header=BB102_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB102_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_umax_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -48

 ; RV32I-NEXT:    lw s5, 24(sp)
 ; RV32I-NEXT:    lw s4, 28(sp)
 ; RV32I-NEXT:    lw s3, 32(sp)
 ; RV32I-NEXT:    lw s2, 36(sp)
 ; RV32I-NEXT:    lw s1, 40(sp)
 ; RV32I-NEXT:    lw ra, 44(sp)
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB103_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a2, a1, .LBB103_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB103_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB103_3: # in Loop: Header=BB103_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB103_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_umax_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB104_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a2, a1, .LBB104_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB104_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB104_3: # in Loop: Header=BB104_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB104_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i16 @atomicrmw_umin_i16_monotonic(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i16_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i16_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB105_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a2, .LBB105_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB105_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB105_3: # in Loop: Header=BB105_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB105_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i16* %a, i16 %b monotonic
   ret i16 %1
 }
 
 define i16 @atomicrmw_umin_i16_acquire(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i16_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i16_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB106_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a2, .LBB106_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB106_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB106_3: # in Loop: Header=BB106_1 Depth=1
+; RV32IA-NEXT:    sc.w a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB106_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i16* %a, i16 %b acquire
   ret i16 %1
 }
 
 define i16 @atomicrmw_umin_i16_release(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i16_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i16_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB107_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a2, .LBB107_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB107_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB107_3: # in Loop: Header=BB107_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB107_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i16* %a, i16 %b release
   ret i16 %1
 }
 
 define i16 @atomicrmw_umin_i16_acq_rel(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i16_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -48

 ; RV32I-NEXT:    lw s5, 24(sp)
 ; RV32I-NEXT:    lw s4, 28(sp)
 ; RV32I-NEXT:    lw s3, 32(sp)
 ; RV32I-NEXT:    lw s2, 36(sp)
 ; RV32I-NEXT:    lw s1, 40(sp)
 ; RV32I-NEXT:    lw ra, 44(sp)
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i16_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB108_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a2, .LBB108_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB108_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB108_3: # in Loop: Header=BB108_1 Depth=1
+; RV32IA-NEXT:    sc.w.rl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB108_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i16* %a, i16 %b acq_rel
   ret i16 %1
 }
 
 define i16 @atomicrmw_umin_i16_seq_cst(i16 *%a, i16 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i16_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i16_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    lui a2, 16
+; RV32IA-NEXT:    addi a2, a2, -1
+; RV32IA-NEXT:    and a1, a1, a2
+; RV32IA-NEXT:    slli a3, a0, 3
+; RV32IA-NEXT:    andi a3, a3, 24
+; RV32IA-NEXT:    sll a6, a2, a3
+; RV32IA-NEXT:    sll a1, a1, a3
+; RV32IA-NEXT:    andi a0, a0, -4
+; RV32IA-NEXT:  .LBB109_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a4, (a0)
+; RV32IA-NEXT:    and a2, a4, a6
+; RV32IA-NEXT:    mv a5, a4
+; RV32IA-NEXT:    bgeu a1, a2, .LBB109_3
+; RV32IA-NEXT:  # %bb.2: # in Loop: Header=BB109_1 Depth=1
+; RV32IA-NEXT:    xor a5, a4, a1
+; RV32IA-NEXT:    and a5, a5, a6
+; RV32IA-NEXT:    xor a5, a4, a5
+; RV32IA-NEXT:  .LBB109_3: # in Loop: Header=BB109_1 Depth=1
+; RV32IA-NEXT:    sc.w.aqrl a5, a5, (a0)
+; RV32IA-NEXT:    bnez a5, .LBB109_1
+; RV32IA-NEXT:  # %bb.4:
+; RV32IA-NEXT:    srl a0, a4, a3
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i16* %a, i16 %b seq_cst
   ret i16 %1
 }
 
 define i32 @atomicrmw_xchg_i32_monotonic(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_exchange_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoswap.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_xchg_i32_acquire(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_exchange_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoswap.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_xchg_i32_release(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_exchange_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoswap.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_xchg_i32_acq_rel(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_exchange_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoswap.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_xchg_i32_seq_cst(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_exchange_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoswap.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_add_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_add_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoadd.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_add_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_add_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoadd.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_add_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_add_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoadd.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_add_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_add_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoadd.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_add_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_add_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoadd.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_sub_i32_monotonic(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_sub_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    neg a1, a1
+; RV32IA-NEXT:    amoadd.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_sub_i32_acquire(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_sub_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    neg a1, a1
+; RV32IA-NEXT:    amoadd.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_sub_i32_release(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_sub_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    neg a1, a1
+; RV32IA-NEXT:    amoadd.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_sub_i32_acq_rel(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_sub_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    neg a1, a1
+; RV32IA-NEXT:    amoadd.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_sub_i32_seq_cst(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_sub_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    neg a1, a1
+; RV32IA-NEXT:    amoadd.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_and_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_and_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoand.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_and_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_and_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoand.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_and_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_and_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoand.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_and_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_and_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_and_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_and_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoand.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_nand_i32_monotonic(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_nand_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:  .LBB130_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a2, (a0)
+; RV32IA-NEXT:    and a3, a2, a1
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB130_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_nand_i32_acquire(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_nand_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:  .LBB131_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a2, (a0)
+; RV32IA-NEXT:    and a3, a2, a1
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    sc.w a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB131_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_nand_i32_release(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_nand_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:  .LBB132_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w a2, (a0)
+; RV32IA-NEXT:    and a3, a2, a1
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB132_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_nand_i32_acq_rel(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_nand_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:  .LBB133_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aq a2, (a0)
+; RV32IA-NEXT:    and a3, a2, a1
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    sc.w.rl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB133_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_nand_i32_seq_cst(i32* %a, i32 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_nand_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:  .LBB134_1: # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    lr.w.aqrl a2, (a0)
+; RV32IA-NEXT:    and a3, a2, a1
+; RV32IA-NEXT:    not a3, a3
+; RV32IA-NEXT:    sc.w.aqrl a3, a3, (a0)
+; RV32IA-NEXT:    bnez a3, .LBB134_1
+; RV32IA-NEXT:  # %bb.2:
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_or_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_or_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoor.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_or_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_or_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoor.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_or_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_or_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoor.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_or_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_or_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_or_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_or_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_xor_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a2, zero
 ; RV32I-NEXT:    call __atomic_fetch_xor_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoxor.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_xor_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_xor_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoxor.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_xor_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_xor_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoxor.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_xor_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_xor_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_xor_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a2, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_xor_4
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amoxor.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_max_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomax.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_max_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomax.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_max_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomax.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_max_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomax.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_max_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomax.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_min_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomin.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_min_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomin.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_min_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomin.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_min_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomin.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_min_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomin.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_umax_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomaxu.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_umax_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomaxu.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_umax_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomaxu.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_umax_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomaxu.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_umax_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amomaxu.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i32 @atomicrmw_umin_i32_monotonic(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i32_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:  # %bb.4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i32_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amominu.w a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i32* %a, i32 %b monotonic
   ret i32 %1
 }
 
 define i32 @atomicrmw_umin_i32_acquire(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i32_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i32_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amominu.w.aq a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i32* %a, i32 %b acquire
   ret i32 %1
 }
 
 define i32 @atomicrmw_umin_i32_release(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i32_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i32_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amominu.w.rl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i32* %a, i32 %b release
   ret i32 %1
 }
 
 define i32 @atomicrmw_umin_i32_acq_rel(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i32_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i32_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amominu.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i32* %a, i32 %b acq_rel
   ret i32 %1
 }
 
 define i32 @atomicrmw_umin_i32_seq_cst(i32 *%a, i32 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i32_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i32_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    amominu.w.aqrl a0, a1, (a0)
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i32* %a, i32 %b seq_cst
   ret i32 %1
 }
 
 define i64 @atomicrmw_xchg_i64_monotonic(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    call __atomic_exchange_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    mv a3, zero
+; RV32IA-NEXT:    call __atomic_exchange_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_xchg_i64_acquire(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    call __atomic_exchange_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 2
+; RV32IA-NEXT:    call __atomic_exchange_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_xchg_i64_release(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 3
 ; RV32I-NEXT:    call __atomic_exchange_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 3
+; RV32IA-NEXT:    call __atomic_exchange_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_xchg_i64_acq_rel(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    call __atomic_exchange_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 4
+; RV32IA-NEXT:    call __atomic_exchange_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_xchg_i64_seq_cst(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_xchg_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    call __atomic_exchange_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xchg_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 5
+; RV32IA-NEXT:    call __atomic_exchange_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xchg i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_add_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    call __atomic_fetch_add_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    mv a3, zero
+; RV32IA-NEXT:    call __atomic_fetch_add_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_add_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_add_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 2
+; RV32IA-NEXT:    call __atomic_fetch_add_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_add_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_add_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 3
+; RV32IA-NEXT:    call __atomic_fetch_add_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_add_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_add_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 4
+; RV32IA-NEXT:    call __atomic_fetch_add_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_add_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_add_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_add_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_add_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 5
+; RV32IA-NEXT:    call __atomic_fetch_add_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw add i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_sub_i64_monotonic(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    call __atomic_fetch_sub_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    mv a3, zero
+; RV32IA-NEXT:    call __atomic_fetch_sub_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_sub_i64_acquire(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_sub_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 2
+; RV32IA-NEXT:    call __atomic_fetch_sub_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_sub_i64_release(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_sub_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 3
+; RV32IA-NEXT:    call __atomic_fetch_sub_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_sub_i64_acq_rel(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_sub_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 4
+; RV32IA-NEXT:    call __atomic_fetch_sub_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_sub_i64_seq_cst(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_sub_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_sub_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_sub_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 5
+; RV32IA-NEXT:    call __atomic_fetch_sub_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw sub i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_and_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    call __atomic_fetch_and_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    mv a3, zero
+; RV32IA-NEXT:    call __atomic_fetch_and_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_and_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_and_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 2
+; RV32IA-NEXT:    call __atomic_fetch_and_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_and_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_and_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 3
+; RV32IA-NEXT:    call __atomic_fetch_and_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_and_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_and_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 4
+; RV32IA-NEXT:    call __atomic_fetch_and_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_and_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_and_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_and_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_and_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 5
+; RV32IA-NEXT:    call __atomic_fetch_and_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw and i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_nand_i64_monotonic(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    call __atomic_fetch_nand_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    mv a3, zero
+; RV32IA-NEXT:    call __atomic_fetch_nand_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_nand_i64_acquire(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_nand_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 2
+; RV32IA-NEXT:    call __atomic_fetch_nand_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_nand_i64_release(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_nand_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 3
+; RV32IA-NEXT:    call __atomic_fetch_nand_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_nand_i64_acq_rel(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_nand_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 4
+; RV32IA-NEXT:    call __atomic_fetch_nand_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_nand_i64_seq_cst(i64* %a, i64 %b) {
 ; RV32I-LABEL: atomicrmw_nand_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_nand_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_nand_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 5
+; RV32IA-NEXT:    call __atomic_fetch_nand_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw nand i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_or_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    call __atomic_fetch_or_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    mv a3, zero
+; RV32IA-NEXT:    call __atomic_fetch_or_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_or_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_or_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 2
+; RV32IA-NEXT:    call __atomic_fetch_or_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_or_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_or_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 3
+; RV32IA-NEXT:    call __atomic_fetch_or_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_or_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_or_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 4
+; RV32IA-NEXT:    call __atomic_fetch_or_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_or_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_or_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_or_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_or_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 5
+; RV32IA-NEXT:    call __atomic_fetch_or_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw or i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_xor_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    call __atomic_fetch_xor_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    mv a3, zero
+; RV32IA-NEXT:    call __atomic_fetch_xor_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_xor_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    call __atomic_fetch_xor_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 2
+; RV32IA-NEXT:    call __atomic_fetch_xor_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_xor_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 3
 ; RV32I-NEXT:    call __atomic_fetch_xor_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 3
+; RV32IA-NEXT:    call __atomic_fetch_xor_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_xor_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    call __atomic_fetch_xor_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 4
+; RV32IA-NEXT:    call __atomic_fetch_xor_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_xor_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_xor_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp)
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    call __atomic_fetch_xor_8
 ; RV32I-NEXT:    lw ra, 12(sp)
 ; RV32I-NEXT:    addi sp, sp, 16
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_xor_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -16
+; RV32IA-NEXT:    sw ra, 12(sp)
+; RV32IA-NEXT:    addi a3, zero, 5
+; RV32IA-NEXT:    call __atomic_fetch_xor_8
+; RV32IA-NEXT:    lw ra, 12(sp)
+; RV32IA-NEXT:    addi sp, sp, 16
+; RV32IA-NEXT:    ret
   %1 = atomicrmw xor i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_max_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:  .LBB200_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB200_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB200_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB200_4
+; RV32IA-NEXT:    j .LBB200_5
+; RV32IA-NEXT:  .LBB200_3: # in Loop: Header=BB200_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB200_5
+; RV32IA-NEXT:  .LBB200_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB200_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB200_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB200_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB200_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB200_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB200_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB200_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, zero
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB200_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_max_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 2
+; RV32IA-NEXT:  .LBB201_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB201_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB201_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB201_4
+; RV32IA-NEXT:    j .LBB201_5
+; RV32IA-NEXT:  .LBB201_3: # in Loop: Header=BB201_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB201_5
+; RV32IA-NEXT:  .LBB201_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB201_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB201_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB201_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB201_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB201_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB201_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB201_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB201_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_max_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 3
+; RV32IA-NEXT:  .LBB202_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB202_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB202_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB202_4
+; RV32IA-NEXT:    j .LBB202_5
+; RV32IA-NEXT:  .LBB202_3: # in Loop: Header=BB202_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB202_5
+; RV32IA-NEXT:  .LBB202_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB202_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB202_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB202_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB202_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB202_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB202_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB202_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB202_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_max_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -48

 ; RV32I-NEXT:    lw s5, 24(sp)
 ; RV32I-NEXT:    lw s4, 28(sp)
 ; RV32I-NEXT:    lw s3, 32(sp)
 ; RV32I-NEXT:    lw s2, 36(sp)
 ; RV32I-NEXT:    lw s1, 40(sp)
 ; RV32I-NEXT:    lw ra, 44(sp)
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -48
+; RV32IA-NEXT:    sw ra, 44(sp)
+; RV32IA-NEXT:    sw s1, 40(sp)
+; RV32IA-NEXT:    sw s2, 36(sp)
+; RV32IA-NEXT:    sw s3, 32(sp)
+; RV32IA-NEXT:    sw s4, 28(sp)
+; RV32IA-NEXT:    sw s5, 24(sp)
+; RV32IA-NEXT:    sw s6, 20(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    addi s4, sp, 8
+; RV32IA-NEXT:    addi s5, zero, 4
+; RV32IA-NEXT:    addi s6, zero, 2
+; RV32IA-NEXT:  .LBB203_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB203_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB203_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    sw a2, 8(sp)
+; RV32IA-NEXT:    beqz a0, .LBB203_4
+; RV32IA-NEXT:    j .LBB203_5
+; RV32IA-NEXT:  .LBB203_3: # in Loop: Header=BB203_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 8(sp)
+; RV32IA-NEXT:    bnez a0, .LBB203_5
+; RV32IA-NEXT:  .LBB203_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB203_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB203_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB203_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB203_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB203_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB203_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB203_1 Depth=1
+; RV32IA-NEXT:    sw a1, 12(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s6
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 12(sp)
+; RV32IA-NEXT:    lw a2, 8(sp)
+; RV32IA-NEXT:    beqz a0, .LBB203_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s6, 20(sp)
+; RV32IA-NEXT:    lw s5, 24(sp)
+; RV32IA-NEXT:    lw s4, 28(sp)
+; RV32IA-NEXT:    lw s3, 32(sp)
+; RV32IA-NEXT:    lw s2, 36(sp)
+; RV32IA-NEXT:    lw s1, 40(sp)
+; RV32IA-NEXT:    lw ra, 44(sp)
+; RV32IA-NEXT:    addi sp, sp, 48
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_max_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_max_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_max_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 5
+; RV32IA-NEXT:  .LBB204_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB204_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB204_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB204_4
+; RV32IA-NEXT:    j .LBB204_5
+; RV32IA-NEXT:  .LBB204_3: # in Loop: Header=BB204_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB204_5
+; RV32IA-NEXT:  .LBB204_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB204_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB204_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB204_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB204_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB204_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB204_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB204_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB204_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw max i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_min_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:  .LBB205_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB205_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB205_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    j .LBB205_4
+; RV32IA-NEXT:  .LBB205_3: # in Loop: Header=BB205_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB205_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB205_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB205_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB205_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB205_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB205_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB205_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB205_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB205_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB205_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, zero
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB205_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_min_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 2
+; RV32IA-NEXT:  .LBB206_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB206_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB206_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    j .LBB206_4
+; RV32IA-NEXT:  .LBB206_3: # in Loop: Header=BB206_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB206_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB206_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB206_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB206_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB206_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB206_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB206_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB206_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB206_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB206_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB206_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_min_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 3
+; RV32IA-NEXT:  .LBB207_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB207_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB207_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    j .LBB207_4
+; RV32IA-NEXT:  .LBB207_3: # in Loop: Header=BB207_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB207_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB207_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB207_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB207_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB207_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB207_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB207_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB207_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB207_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB207_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB207_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_min_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -48

 ; RV32I-NEXT:    lw s5, 24(sp)
 ; RV32I-NEXT:    lw s4, 28(sp)
 ; RV32I-NEXT:    lw s3, 32(sp)
 ; RV32I-NEXT:    lw s2, 36(sp)
 ; RV32I-NEXT:    lw s1, 40(sp)
 ; RV32I-NEXT:    lw ra, 44(sp)
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -48
+; RV32IA-NEXT:    sw ra, 44(sp)
+; RV32IA-NEXT:    sw s1, 40(sp)
+; RV32IA-NEXT:    sw s2, 36(sp)
+; RV32IA-NEXT:    sw s3, 32(sp)
+; RV32IA-NEXT:    sw s4, 28(sp)
+; RV32IA-NEXT:    sw s5, 24(sp)
+; RV32IA-NEXT:    sw s6, 20(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    addi s4, sp, 8
+; RV32IA-NEXT:    addi s5, zero, 4
+; RV32IA-NEXT:    addi s6, zero, 2
+; RV32IA-NEXT:  .LBB208_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB208_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB208_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    j .LBB208_4
+; RV32IA-NEXT:  .LBB208_3: # in Loop: Header=BB208_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB208_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB208_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 8(sp)
+; RV32IA-NEXT:    bnez a0, .LBB208_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB208_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB208_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB208_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB208_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB208_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB208_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB208_1 Depth=1
+; RV32IA-NEXT:    sw a1, 12(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s6
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 12(sp)
+; RV32IA-NEXT:    lw a2, 8(sp)
+; RV32IA-NEXT:    beqz a0, .LBB208_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s6, 20(sp)
+; RV32IA-NEXT:    lw s5, 24(sp)
+; RV32IA-NEXT:    lw s4, 28(sp)
+; RV32IA-NEXT:    lw s3, 32(sp)
+; RV32IA-NEXT:    lw s2, 36(sp)
+; RV32IA-NEXT:    lw s1, 40(sp)
+; RV32IA-NEXT:    lw ra, 44(sp)
+; RV32IA-NEXT:    addi sp, sp, 48
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_min_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_min_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_min_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 5
+; RV32IA-NEXT:  .LBB209_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB209_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB209_1 Depth=1
+; RV32IA-NEXT:    slt a0, s1, a1
+; RV32IA-NEXT:    j .LBB209_4
+; RV32IA-NEXT:  .LBB209_3: # in Loop: Header=BB209_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB209_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB209_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB209_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB209_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB209_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB209_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB209_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB209_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB209_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB209_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB209_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw min i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_umax_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:  .LBB210_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB210_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB210_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB210_4
+; RV32IA-NEXT:    j .LBB210_5
+; RV32IA-NEXT:  .LBB210_3: # in Loop: Header=BB210_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB210_5
+; RV32IA-NEXT:  .LBB210_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB210_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB210_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB210_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB210_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB210_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB210_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB210_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, zero
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB210_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_umax_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 2
+; RV32IA-NEXT:  .LBB211_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB211_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB211_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB211_4
+; RV32IA-NEXT:    j .LBB211_5
+; RV32IA-NEXT:  .LBB211_3: # in Loop: Header=BB211_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB211_5
+; RV32IA-NEXT:  .LBB211_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB211_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB211_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB211_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB211_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB211_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB211_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB211_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB211_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_umax_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 3
+; RV32IA-NEXT:  .LBB212_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB212_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB212_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB212_4
+; RV32IA-NEXT:    j .LBB212_5
+; RV32IA-NEXT:  .LBB212_3: # in Loop: Header=BB212_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB212_5
+; RV32IA-NEXT:  .LBB212_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB212_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB212_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB212_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB212_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB212_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB212_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB212_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB212_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_umax_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -48

 ; RV32I-NEXT:    lw s5, 24(sp)
 ; RV32I-NEXT:    lw s4, 28(sp)
 ; RV32I-NEXT:    lw s3, 32(sp)
 ; RV32I-NEXT:    lw s2, 36(sp)
 ; RV32I-NEXT:    lw s1, 40(sp)
 ; RV32I-NEXT:    lw ra, 44(sp)
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -48
+; RV32IA-NEXT:    sw ra, 44(sp)
+; RV32IA-NEXT:    sw s1, 40(sp)
+; RV32IA-NEXT:    sw s2, 36(sp)
+; RV32IA-NEXT:    sw s3, 32(sp)
+; RV32IA-NEXT:    sw s4, 28(sp)
+; RV32IA-NEXT:    sw s5, 24(sp)
+; RV32IA-NEXT:    sw s6, 20(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    addi s4, sp, 8
+; RV32IA-NEXT:    addi s5, zero, 4
+; RV32IA-NEXT:    addi s6, zero, 2
+; RV32IA-NEXT:  .LBB213_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB213_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB213_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    sw a2, 8(sp)
+; RV32IA-NEXT:    beqz a0, .LBB213_4
+; RV32IA-NEXT:    j .LBB213_5
+; RV32IA-NEXT:  .LBB213_3: # in Loop: Header=BB213_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 8(sp)
+; RV32IA-NEXT:    bnez a0, .LBB213_5
+; RV32IA-NEXT:  .LBB213_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB213_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB213_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB213_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB213_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB213_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB213_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB213_1 Depth=1
+; RV32IA-NEXT:    sw a1, 12(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s6
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 12(sp)
+; RV32IA-NEXT:    lw a2, 8(sp)
+; RV32IA-NEXT:    beqz a0, .LBB213_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s6, 20(sp)
+; RV32IA-NEXT:    lw s5, 24(sp)
+; RV32IA-NEXT:    lw s4, 28(sp)
+; RV32IA-NEXT:    lw s3, 32(sp)
+; RV32IA-NEXT:    lw s2, 36(sp)
+; RV32IA-NEXT:    lw s1, 40(sp)
+; RV32IA-NEXT:    lw ra, 44(sp)
+; RV32IA-NEXT:    addi sp, sp, 48
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_umax_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umax_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umax_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 5
+; RV32IA-NEXT:  .LBB214_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB214_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB214_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB214_4
+; RV32IA-NEXT:    j .LBB214_5
+; RV32IA-NEXT:  .LBB214_3: # in Loop: Header=BB214_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB214_5
+; RV32IA-NEXT:  .LBB214_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB214_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB214_5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB214_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB214_7
+; RV32IA-NEXT:  # %bb.6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB214_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB214_7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB214_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB214_1
+; RV32IA-NEXT:  # %bb.8: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umax i64* %a, i64 %b seq_cst
   ret i64 %1
 }
 
 define i64 @atomicrmw_umin_i64_monotonic(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i64_monotonic:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i64_monotonic:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:  .LBB215_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB215_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB215_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    j .LBB215_4
+; RV32IA-NEXT:  .LBB215_3: # in Loop: Header=BB215_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB215_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB215_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB215_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB215_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB215_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB215_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB215_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB215_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB215_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB215_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, zero
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB215_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i64* %a, i64 %b monotonic
   ret i64 %1
 }
 
 define i64 @atomicrmw_umin_i64_acquire(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i64_acquire:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i64_acquire:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 2
+; RV32IA-NEXT:  .LBB216_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB216_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB216_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    j .LBB216_4
+; RV32IA-NEXT:  .LBB216_3: # in Loop: Header=BB216_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB216_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB216_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB216_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB216_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB216_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB216_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB216_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB216_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB216_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB216_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB216_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i64* %a, i64 %b acquire
   ret i64 %1
 }
 
 define i64 @atomicrmw_umin_i64_release(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i64_release:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i64_release:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 3
+; RV32IA-NEXT:  .LBB217_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB217_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB217_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    j .LBB217_4
+; RV32IA-NEXT:  .LBB217_3: # in Loop: Header=BB217_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB217_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB217_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB217_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB217_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB217_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB217_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB217_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB217_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB217_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB217_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, zero
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB217_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i64* %a, i64 %b release
   ret i64 %1
 }
 
 define i64 @atomicrmw_umin_i64_acq_rel(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i64_acq_rel:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -48

 ; RV32I-NEXT:    lw s5, 24(sp)
 ; RV32I-NEXT:    lw s4, 28(sp)
 ; RV32I-NEXT:    lw s3, 32(sp)
 ; RV32I-NEXT:    lw s2, 36(sp)
 ; RV32I-NEXT:    lw s1, 40(sp)
 ; RV32I-NEXT:    lw ra, 44(sp)
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i64_acq_rel:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -48
+; RV32IA-NEXT:    sw ra, 44(sp)
+; RV32IA-NEXT:    sw s1, 40(sp)
+; RV32IA-NEXT:    sw s2, 36(sp)
+; RV32IA-NEXT:    sw s3, 32(sp)
+; RV32IA-NEXT:    sw s4, 28(sp)
+; RV32IA-NEXT:    sw s5, 24(sp)
+; RV32IA-NEXT:    sw s6, 20(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    addi s4, sp, 8
+; RV32IA-NEXT:    addi s5, zero, 4
+; RV32IA-NEXT:    addi s6, zero, 2
+; RV32IA-NEXT:  .LBB218_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB218_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB218_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    j .LBB218_4
+; RV32IA-NEXT:  .LBB218_3: # in Loop: Header=BB218_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB218_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB218_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 8(sp)
+; RV32IA-NEXT:    bnez a0, .LBB218_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB218_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB218_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB218_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB218_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB218_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB218_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB218_1 Depth=1
+; RV32IA-NEXT:    sw a1, 12(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s6
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 12(sp)
+; RV32IA-NEXT:    lw a2, 8(sp)
+; RV32IA-NEXT:    beqz a0, .LBB218_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s6, 20(sp)
+; RV32IA-NEXT:    lw s5, 24(sp)
+; RV32IA-NEXT:    lw s4, 28(sp)
+; RV32IA-NEXT:    lw s3, 32(sp)
+; RV32IA-NEXT:    lw s2, 36(sp)
+; RV32IA-NEXT:    lw s1, 40(sp)
+; RV32IA-NEXT:    lw ra, 44(sp)
+; RV32IA-NEXT:    addi sp, sp, 48
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i64* %a, i64 %b acq_rel
   ret i64 %1
 }
 
 define i64 @atomicrmw_umin_i64_seq_cst(i64 *%a, i64 %b) nounwind {
 ; RV32I-LABEL: atomicrmw_umin_i64_seq_cst:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32

 ; RV32I-NEXT:    lw s5, 8(sp)
 ; RV32I-NEXT:    lw s4, 12(sp)
 ; RV32I-NEXT:    lw s3, 16(sp)
 ; RV32I-NEXT:    lw s2, 20(sp)
 ; RV32I-NEXT:    lw s1, 24(sp)
 ; RV32I-NEXT:    lw ra, 28(sp)
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
+;
+; RV32IA-LABEL: atomicrmw_umin_i64_seq_cst:
+; RV32IA:       # %bb.0:
+; RV32IA-NEXT:    addi sp, sp, -32
+; RV32IA-NEXT:    sw ra, 28(sp)
+; RV32IA-NEXT:    sw s1, 24(sp)
+; RV32IA-NEXT:    sw s2, 20(sp)
+; RV32IA-NEXT:    sw s3, 16(sp)
+; RV32IA-NEXT:    sw s4, 12(sp)
+; RV32IA-NEXT:    sw s5, 8(sp)
+; RV32IA-NEXT:    mv s1, a2
+; RV32IA-NEXT:    mv s2, a1
+; RV32IA-NEXT:    mv s3, a0
+; RV32IA-NEXT:    lw a1, 4(a0)
+; RV32IA-NEXT:    lw a2, 0(a0)
+; RV32IA-NEXT:    mv s4, sp
+; RV32IA-NEXT:    addi s5, zero, 5
+; RV32IA-NEXT:  .LBB219_1: # %atomicrmw.start
+; RV32IA-NEXT:    # =>This Inner Loop Header: Depth=1
+; RV32IA-NEXT:    beq a1, s1, .LBB219_3
+; RV32IA-NEXT:  # %bb.2: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB219_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s1, a1
+; RV32IA-NEXT:    j .LBB219_4
+; RV32IA-NEXT:  .LBB219_3: # in Loop: Header=BB219_1 Depth=1
+; RV32IA-NEXT:    sltu a0, s2, a2
+; RV32IA-NEXT:  .LBB219_4: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB219_1 Depth=1
+; RV32IA-NEXT:    xori a0, a0, 1
+; RV32IA-NEXT:    sw a2, 0(sp)
+; RV32IA-NEXT:    bnez a0, .LBB219_6
+; RV32IA-NEXT:  # %bb.5: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB219_1 Depth=1
+; RV32IA-NEXT:    mv a2, s2
+; RV32IA-NEXT:  .LBB219_6: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB219_1 Depth=1
+; RV32IA-NEXT:    mv a3, a1
+; RV32IA-NEXT:    bnez a0, .LBB219_8
+; RV32IA-NEXT:  # %bb.7: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB219_1 Depth=1
+; RV32IA-NEXT:    mv a3, s1
+; RV32IA-NEXT:  .LBB219_8: # %atomicrmw.start
+; RV32IA-NEXT:    # in Loop: Header=BB219_1 Depth=1
+; RV32IA-NEXT:    sw a1, 4(sp)
+; RV32IA-NEXT:    mv a0, s3
+; RV32IA-NEXT:    mv a1, s4
+; RV32IA-NEXT:    mv a4, s5
+; RV32IA-NEXT:    mv a5, s5
+; RV32IA-NEXT:    call __atomic_compare_exchange_8
+; RV32IA-NEXT:    lw a1, 4(sp)
+; RV32IA-NEXT:    lw a2, 0(sp)
+; RV32IA-NEXT:    beqz a0, .LBB219_1
+; RV32IA-NEXT:  # %bb.9: # %atomicrmw.end
+; RV32IA-NEXT:    mv a0, a2
+; RV32IA-NEXT:    lw s5, 8(sp)
+; RV32IA-NEXT:    lw s4, 12(sp)
+; RV32IA-NEXT:    lw s3, 16(sp)
+; RV32IA-NEXT:    lw s2, 20(sp)
+; RV32IA-NEXT:    lw s1, 24(sp)
+; RV32IA-NEXT:    lw ra, 28(sp)
+; RV32IA-NEXT:    addi sp, sp, 32
+; RV32IA-NEXT:    ret
   %1 = atomicrmw umin i64* %a, i64 %b seq_cst
   ret i64 %1
 }