[X86, Peephole] Enable FoldImmediate for X86

[Carrot]

Summary

Enable FoldImmediate for X86 by implementing X86InstrInfo::FoldImmediate.

Also enhanced peephole by deleting identical instructions after FoldImmediate.

This is extracted from D119916.

[craig.topper]

llvm/lib/CodeGen/PeepholeOptimizer.cpp

1434

happenly?

llvm/test/CodeGen/X86/popcnt.ll

621

This defeats the code size optimization in SelectionDAG. See X86InstrInfo.td

// If we have multiple users of an immediate, it's much smaller to reuse         
// the register, rather than encode the immediate in every instruction.          
// This has the risk of increasing register pressure from stretched live         
// ranges, however, the immediates should be trivial to rematerialize by         
// the RA in the event of high register pressure.                                
// TODO : This is currently enabled for stores and binary ops. There are more    
// cases for which this can be enabled, though this catches the bulk of the      
// issues.                                                                       
// TODO2 : This should really also be enabled under O2, but there's currently    
// an issue with RA where we don't pull the constants into their users           
// when we rematerialize them. I'll follow-up on enabling O2 after we fix that   
// issue.                                                                        
// TODO3 : This is currently limited to single basic blocks (DAG creation        
// pulls block immediates to the top and merges them if necessary).              
// Eventually, it would be nice to allow ConstantHoisting to merge constants     
// globally for potentially added savings.                                       
//                                                                               
def imm_su : PatLeaf<(imm), [{                                                   
    return !shouldAvoidImmediateInstFormsForSize(N);                             
}]>;

[Carrot]

llvm/lib/CodeGen/PeepholeOptimizer.cpp

1434

The case I encountered is

%1 = MOV32ri   4
%2 = COPY %1
 // other uses of %1
...

After FoldImmediate it becomes

%1 = MOV32ri 4
%2 = MOV32ri 4

Becuase %1 has multiple uses, so it can't be deleted by FoldImmediate. These two instructions are identical now, so we can replace the uses of %2 by %1, and delete the definition of %2.

llvm/test/CodeGen/X86/popcnt.ll

621

I need to take a look at this.
thanks.

[craig.topper]

llvm/lib/CodeGen/PeepholeOptimizer.cpp

1434

I was questioning whether "happenly" is a real word.

[Carrot]

llvm/lib/CodeGen/PeepholeOptimizer.cpp

1434

Sorry for my poor English. It should be happened.

[Carrot]

Avoid folding immediate if it has more than 1 use and we are optimizing for size.

[Carrot]

ping

[Carrot]

ping

[goldstein.w.n]

Some codesize/Reg slloc stats from llvm test suite would be useful to have for evaluating the benefit of this patch.

llvm/lib/Target/X86/X86InstrInfo.cpp

3882

This is a big patch. I would split.tgis edge case to a follow up and just fail in this case for now.

4837

Does this actually payoff? Do we ever spills and not just rematerialize the constant op in RA?
Because if not, saving code side for the perf benefit may win out for many of these.

4951

Personally would split patch and in the first iteration do only shifts and zero. Those are obviously beneficial cases. The rest are a bit less clear.

4955

Is there really no helper for getting immediate opcide from rr version?

[Carrot]

In D151848#4597644, @goldstein.w.n wrote:

Some codesize/Reg slloc stats from llvm test suite would be useful to have for evaluating the benefit of this patch.

I will collect these stats for spec int 2006.

llvm/lib/Target/X86/X86InstrInfo.cpp

3882

All instructions loading constant to 64bit register have this pattern.

4837

Interesting question, I haven't studied reload vs rematerialization in RA.
But I prefer rematerialization for constant in RA, because if we reload the constant from memory we need a [sp+offset] operand, the offset field is also a 32bit immediate, why not use the bytes to directly rematerialize a register, and save a memory load operation.
So here we keep a constant in register if it is used multiple times to save size. And later if we experience high register pressure, it can be rematerialized on demand.

4951

Actually the motivating case is AND and ADD, as shown in https://reviews.llvm.org/D119916#change-nKHV7D4KKS4t. It is extracted from tcmalloc.

4955

I couldn't find one.

[Carrot]

In D151848#4600416, @Carrot wrote:

In D151848#4597644, @goldstein.w.n wrote:

Some codesize/Reg slloc stats from llvm test suite would be useful to have for evaluating the benefit of this patch.

I will collect these stats for spec int 2006.

Here is the related stats for building spec int 2006

Without this patch

1959884 Number of machine instrs printed

124286 Number of instructions rematerialized
96962 Number of instructions re-materialized
82246 Number of instructions deleted by DCE 
97965 Number of shrinkToUses called

650467 Number of registers assigned
91796 Number of copies inserted for splitting
22727 Number of interferences evicted
20178 Number of splits finished
9640 Number of folded stack accesses
829 Number of folded loads
316 Number of live ranges fractured by DCE 
13285 Number of split global live ranges
227433 Number of identity moves eliminated after rewriting
7536 Number of dead lane conflicts tested
4074 Number of dead lane conflicts resolved
1476 Number of split local live ranges
155548 Number of new live ranges queued
44378 Number of reloads inserted
629 Number of reloads removed
25069 Number of rematerialized defs for spilling
2255 Number of rematerialized defs for splitting
9584 Number of splits that were simple
1156 Number of spilled snippets
16277 Number of spill slots allocated
29934 Number of spilled live ranges
21939 Number of spills inserted
430 Number of spills removed
26830 Number of registers unassigned
612 Number of instruction commuting performed
169128 Number of cross class joins performed
13 Number of copies extended
529170 Number of interval joins performed
20 Number of register classes inflated
35 Number of single use loads folded after DCE

With this patch

45008 Number of move immediate folded

1959709 Number of machine instrs printed

79327 Number of instructions rematerialized
52043 Number of instructions re-materialized
46082 Number of instructions deleted by DCE 
53046 Number of shrinkToUses called

650311 Number of registers assigned
91762 Number of copies inserted for splitting
22698 Number of interferences evicted
20167 Number of splits finished
9653 Number of folded stack accesses
829 Number of folded loads
315 Number of live ranges fractured by DCE 
13282 Number of split global live ranges
227415 Number of identity moves eliminated after rewriting
7536 Number of dead lane conflicts tested
4074 Number of dead lane conflicts resolved
1466 Number of split local live ranges
155411 Number of new live ranges queued
44319 Number of reloads inserted
697 Number of reloads removed
25035 Number of rematerialized defs for spilling
2249 Number of rematerialized defs for splitting
9572 Number of splits that were simple
1166 Number of spilled snippets
16272 Number of spill slots allocated
29925 Number of spilled live ranges
21900 Number of spills inserted
446 Number of spills removed
26800 Number of registers unassigned
613 Number of instruction commuting performed
169130 Number of cross class joins performed
13 Number of copies extended
529157 Number of interval joins performed
20 Number of register classes inflated
35 Number of single use loads folded after DCE

Conclusion:

1. There was no FoldImmediate occurred without this patch. Now there are 45008 occurrences of FoldImmediate with this patch.

2. There are 175 less instructions generated with this patch.

3. There is significantly less rematerialization occurred, because related immediate are already folded into instructions.

There is no big differences for other stats.

[Carrot]

ping

[Carrot]

ping

[davidxl]

Looking at the test case changes, it looks like the change produce fewer instructions and reduces register pressure as well (smaller prologue).

llvm/lib/CodeGen/PeepholeOptimizer.cpp

223

add documentation on the new parameter LocalMIs

llvm/lib/Target/X86/X86InstrInfo.cpp

3867

Add documentation that it follows use-def for subreg2reg instruction to find the real movImm instruction.

4836

Add a test case for optsize case?

4843

Are all the opcode below covered by test cases?

4902

Add brief description for each case (on the transformation)?

[qcolombet]

The peephole optimizer changes look good to me.
+1 on every comments touching on test cases and comments.

For the folding of opc, it may be interesting to use a similar approach as the MemoryFoldTableX. This uses an X86 specific tablegen backend. (See llvm/utils/TableGen/X86FoldTablesEmitter.cpp.)

I only had a cursory look at the X86 changes and they looked fine. The fold table code is quite big though. I leave that part to x86 folks to approve.

llvm/lib/CodeGen/PeepholeOptimizer.cpp

223

+1 without the comment it is impossible to understand why we would skip these instructions.

llvm/lib/Target/X86/X86InstrInfo.h

557

Don't repeat the function name in the comment.

[Carrot]

Thanks for the suggestion for more test cases. It did reveals two missed opportunities in Peephole pass.

1. The function PeepholeOptimizer::isMoveImmediate checks only MI.isMoveImmediate() for immediate, but on x86-64 we use the following code sequence to setup 64b immediate. We can use TII->getConstValDefinedInReg to find more immediate setup instructions.

%8:gr32 = MOV32r0 implicit-def dead $eflags
%6:gr64 = SUBREG_TO_REG 0, killed %8:gr32, %subreg.sub_32bit

2. Peephole only tracks virtual register contains immediate value. On x86 shift instructions use physical register $cl to specify shift amount. In order to fold immediate values into shift instructions we need to track physical register. In SSA form, tracking virtual register is very simple, but tracking physical register is more complex, it also consumes more compile time. So I didn't enhance it here. But FoldImmediate for shift instructions are still useful for D119916, and will be tested in https://reviews.llvm.org/D119916#change-nKHV7D4KKS4t.

llvm/lib/Target/X86/X86InstrInfo.cpp

4902

I refactored the opcode conversion code into a separate function, so it is more clearer.

[davidxl]

llvm/lib/CodeGen/PeepholeOptimizer.cpp

1397–1398

The code looks cleaner with early return like before:

if (!MI.isMoveImmediate() && !TII->getConstValDefinedInReg(MI,Reg,ImmVal))

return false;

1398

Should this be

if (MI.isMoveImmediate() && MCID.getNumDefs() != 1

|| !MI.isMoveImmediate() && !MI.getOperand(0).isReg())
return false;

llvm/lib/Target/X86/X86InstrInfo.cpp

4897

document parameters and return.

4969

brief commend here.

[MatzeB]

llvm/lib/Target/X86/X86InstrInfo.cpp

4802–4818

Does MachineBasicBlock::computeRegisterLiveness work instead?

[MatzeB]

llvm/lib/Target/X86/X86InstrInfo.cpp

4907

Is this correct? I haven't gone over the patch or read up on the details... But I'm worried this mixes up signed/unsigned representations. For example int64_t{-1} passes an isInt<32>() check but does not pass an isUInt<32>() check. Wouldn't it depend on the operation on whether we should interpret the immediate as signed or unsigned?

[Carrot]

llvm/lib/CodeGen/PeepholeOptimizer.cpp

1397–1398

getConstValDefinedInReg needs the Reg parameter, so it can't be moved to the very beginning of the function. But I changed it to return false on the opposite condition.

1398

My code should also be correct. The MI.isMoveImmediate() test is moved to different statement. The !MI.getOperand(0).isReg() is used to make sure the following getReg() works correctly.

llvm/lib/Target/X86/X86InstrInfo.cpp

4907

You are right. The actual legal type is instruction dependent.
All 32 bit immediate in a 32 bit register can be directly used in a ALU32ri instruction, so we can safely skip the checking.
For 64 bit operations only 32 bit signed integers can be used in ALU64ri instructions.

[Carrot]

ping

[MatzeB]

Looks good and I think should be good to go with the getVRegDef thing fixed.

llvm/lib/Target/X86/X86InstrInfo.cpp

3887

getVRegDef must only be used when the code is still in (Machine-)SSA form and we have a guaranteed that there is only one or zero definitions. This code however runs later during regalloc I believe? So probably need to use getUniqueVRegDef here.

The code also has to be prepared for potential nullptrs in case of multiple definitions or no definition (could be live-in to the function or an undef operand for example).

4875

When inspecting something like grep "32ri\s*=" build/lib/Target/X86/X86GenInstrInfo.inc or grep "64rr\s*=" build/lib/Target/X86/X86GenInstrInfo.inc I noticed a couple more that probably work?

• ADC32rr / ADC32ri (also ADD64rr)
• ROL32rCL / ROL32ri
• ROR32rCL / ROR32ri
• RCL32rCL / RCL32ri
• RCR32rCL / RCR32ri
• SBB32rr / SBB32ri (also SBB64rr)
• TEST32rr / TEST32ri (also TEST64rr)

4953–4956

Use the /*xxx=*/ style which is shown in LLVM coding standards. It also has the advantage that clang-tidy can check this style whether the names match the function declarations (though nobody has actually enabled that check in LLVM, but I have seen it work nicely in other projects...)

[MatzeB]

• Did you consider CMP64rr, CMP32rr ?

LGTM with nitpicks resolved.

llvm/lib/Target/X86/X86InstrInfo.cpp

4875

• CMP32rr / CMP32ri

llvm/test/CodeGen/X86/vector-shuffle-combining-avx512bwvl.ll

176–182

Odd to see new xor %edx, %edx added here. Is that the result of poison or undef values? (in that case "meh" we can ignore it)

[MatzeB]

llvm/lib/Target/X86/X86InstrInfo.cpp

4875

heh, just realized I didn't push "submit" on this so you didn't see it on my last review...

[Carrot]

llvm/test/CodeGen/X86/vector-shuffle-combining-avx512bwvl.ll

176–182

This 'xor %edx, %edx' should be the movl instruction in LBB10_2. The first xorl instruction in the function is new. Compare to the source code, %edx and %eax should be variable %i275 and %i277, both are generated from undef value.

[mstorsjo]

This change triggers failed asserts for me:

$ cat repro.c
short a, b;
void c() {
  __asm__("" : : "q"(256 * sizeof a));
  __asm__("" : : "q"(256 * sizeof b));
}
$ clang -target i686-w64-mingw32 -c repro.c -O2
clang: ../include/llvm/CodeGen/MachineOperand.h:375: unsigned int llvm::MachineOperand::getSubReg() const: Assertion `isReg() && "Wrong MachineOperand accessor"' failed. 

Can you look into it, or revert if fixing it takes a while?

[uabelho]

We also see crashes with this patch

clang -O2 bbi-87732.c -S

results in

clang: ../include/llvm/CodeGen/MachineOperand.h:375: unsigned int llvm::MachineOperand::getSubReg() const: Assertion `isReg() && "Wrong MachineOperand accessor"' failed.

bbi-87732.c297 BDownload

[Carrot]

@mstorsjo, @uabelho, thank you for the report and reduced tests. Both failures are caused by the same problem.

Function PeepholeOptimizer::foldRedundantCopy records COPY instructions in CopyMIs, and uses it later. But the COPY instructions is modified by foldImmediate in the tests, so foldRedundantCopy gets an unexpected non COPY instruction, and causes the assertion failed. In theory the COPY instruction can also be changed by other optimizations. We should check if the instructions extracted from CopyMIs are still COPY before using them.

I will send a patch for it.

[Carrot]

Try again.

As suggested by @qcolombet, this time I use delegate to track COPY instructions accurately. It can fix all assertion failures in function foldRedundantCopy.

llvm/include/llvm/CodeGen/MachineFunction.h

class Delegate {
     virtual void anchor();
 
   public:
     virtual ~Delegate() = default;
     /// Callback after an insertion. This should not modify the MI directly.
     virtual void MF_HandleInsertion(MachineInstr &MI) = 0;
     /// Callback before a removal. This should not modify the MI directly.
     virtual void MF_HandleRemoval(MachineInstr &MI) = 0;
+    /// Callback before changing MCInstrDesc. This should not modify the MI
+    /// directly.
+    virtual void MF_HandleChangeDesc(MachineInstr &MI, const MCInstrDesc &TID) {
+      return;
+    }
   };
 
   /// Structure used to represent pair of argument number after call lowering
   /// and register used to transfer that argument.
   /// For now we support only cases when argument is transferred through one
   /// register.
   struct ArgRegPair {
     Register Reg;
private:
   CallSiteInfoMap::iterator getCallSiteInfo(const MachineInstr *MI);
 
   // Callbacks for insertion and removal.
   void handleInsertion(MachineInstr &MI);
   void handleRemoval(MachineInstr &MI);
   friend struct ilist_traits<MachineInstr>;
 
 public:
+  // Need to be accessed from MachineInstr::setDesc.
+  void handleChangeDesc(MachineInstr &MI, const MCInstrDesc &TID);
+
   using VariableDbgInfoMapTy = SmallVector<VariableDbgInfo, 4>;
   VariableDbgInfoMapTy VariableDbgInfos;
 
   /// A count of how many instructions in the function have had numbers
   /// assigned to them. Used for debug value tracking, to determine the
   /// next instruction number.
   unsigned DebugInstrNumberingCount = 0;
 

llvm/include/llvm/CodeGen/MachineInstr.h

   /// instructions that are inserted in a basic block.
   ///
   /// MachineInstrBuilder and the two-argument addOperand(MF, MO) should be
   /// preferred.
   void addOperand(const MachineOperand &Op);
 
   /// Replace the instruction descriptor (thus opcode) of
   /// the current instruction with a new one.
-  void setDesc(const MCInstrDesc &TID) { MCID = &TID; }
+  void setDesc(const MCInstrDesc &TID);
 
   /// Replace current source information with new such.
   /// Avoid using this, the constructor argument is preferable.
   void setDebugLoc(DebugLoc DL) {
     DbgLoc = std::move(DL);
     assert(DbgLoc.hasTrivialDestructor() && "Expected trivial destructor");
   }
 

llvm/lib/CodeGen/MachineFunction.cpp

if (TheDelegate)
     TheDelegate->MF_HandleInsertion(MI);
 }
 
 void MachineFunction::handleRemoval(MachineInstr &MI) {
   if (TheDelegate)
     TheDelegate->MF_HandleRemoval(MI);
 }
 
+void MachineFunction::handleChangeDesc(MachineInstr &MI,
+                                       const MCInstrDesc &TID) {
+  if (TheDelegate)
+    TheDelegate->MF_HandleChangeDesc(MI, TID);
+}
+
 void MachineFunction::init() {
   // Assume the function starts in SSA form with correct liveness.
   Properties.set(MachineFunctionProperties::Property::IsSSA);
   Properties.set(MachineFunctionProperties::Property::TracksLiveness);
   if (STI->getRegisterInfo())
     RegInfo = new (Allocator) MachineRegisterInfo(this);
   else
     RegInfo = nullptr;

llvm/lib/CodeGen/MachineInstr.cpp

for (unsigned i = 0, e = getNumOperands(); i < e; ++i) {
     const MachineOperand &OrigMO = MI.getOperand(i);
     NewMO.TiedTo = OrigMO.TiedTo;
   }
 
   // Copy all the sensible flags.
   setFlags(MI.Flags);
 }
 
+void MachineInstr::setDesc(const MCInstrDesc &TID) {
+  if (getParent())
+    getMF()->handleChangeDesc(*this, TID);
+  MCID = &TID;
+}
+
 void MachineInstr::moveBefore(MachineInstr *MovePos) {
   MovePos->getParent()->splice(MovePos, getParent(), getIterator());
 }
 
 /// getRegInfo - If this instruction is embedded into a MachineFunction,
 /// return the MachineRegisterInfo object for the current function, otherwise
 /// return null.
 MachineRegisterInfo *MachineInstr::getRegInfo() {

llvm/lib/CodeGen/PeepholeOptimizer.cpp

 STATISTIC(NumRewrittenCopies, "Number of copies rewritten");
 STATISTIC(NumNAPhysCopies, "Number of non-allocatable physical copies removed");
 
 namespace {
 
   class ValueTrackerResult;
   class RecurrenceInstr;
 
-  class PeepholeOptimizer : public MachineFunctionPass {
+  class PeepholeOptimizer : public MachineFunctionPass,
+                            private MachineFunction::Delegate {
     const TargetInstrInfo *TII = nullptr;
     const TargetRegisterInfo *TRI = nullptr;
     MachineRegisterInfo *MRI = nullptr;
     MachineDominatorTree *DT = nullptr; // Machine dominator tree
     MachineLoopInfo *MLI = nullptr;
 
   public:
     static char ID; // Pass identification
private:
     bool optimizeCoalescableCopy(MachineInstr &MI);
     bool optimizeUncoalescableCopy(MachineInstr &MI,
                                    SmallPtrSetImpl<MachineInstr *> &LocalMIs);
     bool optimizeRecurrence(MachineInstr &PHI);
     bool findNextSource(RegSubRegPair RegSubReg, RewriteMapTy &RewriteMap);
     bool isMoveImmediate(MachineInstr &MI, SmallSet<Register, 4> &ImmDefRegs,
                          DenseMap<Register, MachineInstr *> &ImmDefMIs);
     bool foldImmediate(MachineInstr &MI, SmallSet<Register, 4> &ImmDefRegs,
-                       DenseMap<Register, MachineInstr *> &ImmDefMIs);
+                       DenseMap<Register, MachineInstr *> &ImmDefMIs,
+                       bool &Deleted);
 
     /// Finds recurrence cycles, but only ones that formulated around
     /// a def operand and a use operand that are tied. If there is a use
     /// operand commutable with the tied use operand, find recurrence cycle
     /// along that operand as well.
     bool findTargetRecurrence(Register Reg,
                               const SmallSet<Register, 2> &TargetReg,
                               RecurrenceCycle &RC);
 
     /// If copy instruction \p MI is a virtual register copy or a copy of a
     /// constant physical register to a virtual register, track it in the
-    /// set \p CopyMIs. If this virtual register was previously seen as a
+    /// set CopySrcMIs. If this virtual register was previously seen as a
     /// copy, replace the uses of this copy with the previously seen copy's
     /// destination register.
-    bool foldRedundantCopy(MachineInstr &MI,
-                           DenseMap<RegSubRegPair, MachineInstr *> &CopyMIs);
+    bool foldRedundantCopy(MachineInstr &MI);
 
     /// Is the register \p Reg a non-allocatable physical register?
     bool isNAPhysCopy(Register Reg);
 
     /// If copy instruction \p MI is a non-allocatable virtual<->physical
     /// register copy, track it in the \p NAPhysToVirtMIs map. If this
     /// non-allocatable physical register was previously copied to a virtual
     /// registered and hasn't been clobbered, the virt->phys copy can be
     /// deleted.
bool isUncoalescableCopy(const MachineInstr &MI) {
       return MI.isBitcast() ||
              (!DisableAdvCopyOpt &&
               (MI.isRegSequenceLike() || MI.isInsertSubregLike() ||
                MI.isExtractSubregLike()));
     }
 
     MachineInstr &rewriteSource(MachineInstr &CopyLike,
                                 RegSubRegPair Def, RewriteMapTy &RewriteMap);
+
+    // Set of copies to virtual registers keyed by source register.  Never
+    // holds any physreg which requires def tracking.
+    DenseMap<RegSubRegPair, MachineInstr *> CopySrcMIs;
+
+    // MachineFunction::Delegate implementation. Used to maintain CopySrcMIs.
+    void MF_HandleInsertion(MachineInstr &MI) override {
+      return;
+    }
+
+    bool getCopySrc(MachineInstr &MI, RegSubRegPair &SrcPair) {
+      if (!MI.isCopy())
+        return false;
+
+      Register SrcReg = MI.getOperand(1).getReg();
+      unsigned SrcSubReg = MI.getOperand(1).getSubReg();
+      if (!SrcReg.isVirtual() && !MRI->isConstantPhysReg(SrcReg))
+        return false;
+
+      SrcPair = RegSubRegPair(SrcReg, SrcSubReg);
+      return true;
+    }
+
+    // If a COPY instruction is to be deleted or changed, we should also remove
+    // it from CopySrcMIs.
+    void deleteChangedCopy(MachineInstr &MI) {
+      RegSubRegPair SrcPair;
+      if (!getCopySrc(MI, SrcPair))
+        return;
+
+      auto It = CopySrcMIs.find(SrcPair);
+      if (It != CopySrcMIs.end() && It->second == &MI)
+        CopySrcMIs.erase(It);
+    }
+
+    void MF_HandleRemoval(MachineInstr &MI) override {
+      deleteChangedCopy(MI);
+    }
+
+    void MF_HandleChangeDesc(MachineInstr &MI, const MCInstrDesc &TID) override
+    {
+      deleteChangedCopy(MI);
+    }
   };
 
   /// Helper class to hold instructions that are inside recurrence cycles.
   /// The recurrence cycle is formulated around 1) a def operand and its
   /// tied use operand, or 2) a def operand and a use operand that is commutable
   /// with another use operand which is tied to the def operand. In the latter
   /// case, index of the tied use operand and the commutable use operand are
   /// maintained with CommutePair.
if (Reg.isVirtual() && !MI.getOperand(0).getSubReg() &&
     return true;
   }
   return false;
 }
 
 bool PeepholeOptimizer::isMoveImmediate(
     MachineInstr &MI, SmallSet<Register, 4> &ImmDefRegs,
     DenseMap<Register, MachineInstr *> &ImmDefMIs) {
   const MCInstrDesc &MCID = MI.getDesc();
-  if (!MI.isMoveImmediate())
-    return false;
-  if (MCID.getNumDefs() != 1)
+  if (MCID.getNumDefs() != 1 || !MI.getOperand(0).isReg())
     return false;
   Register Reg = MI.getOperand(0).getReg();
-  if (Reg.isVirtual()) {
-    ImmDefMIs.insert(std::make_pair(Reg, &MI));
-    ImmDefRegs.insert(Reg);
-    return true;
-  }
+  if (!Reg.isVirtual())
+    return false;
+
+  int64_t ImmVal;
+  if (!MI.isMoveImmediate() && !TII->getConstValDefinedInReg(MI, Reg, ImmVal))
+    return false;
+
 
-  return false;
-}
-
 /// Try folding register operands that are defined by move immediate
 /// instructions, i.e. a trivial constant folding optimization, if
 /// and only if the def and use are in the same BB.
 bool PeepholeOptimizer::foldImmediate(
     MachineInstr &MI, SmallSet<Register, 4> &ImmDefRegs,
-    DenseMap<Register, MachineInstr *> &ImmDefMIs) {
+    DenseMap<Register, MachineInstr *> &ImmDefMIs, bool &Deleted) {
+  Deleted = false;
   for (unsigned i = 0, e = MI.getDesc().getNumOperands(); i != e; ++i) {
     MachineOperand &MO = MI.getOperand(i);
     if (!MO.isReg() || MO.isDef())
       continue;
     Register Reg = MO.getReg();
     if (!Reg.isVirtual())
       continue;
     if (ImmDefRegs.count(Reg) == 0)
       continue;
     DenseMap<Register, MachineInstr *>::iterator II = ImmDefMIs.find(Reg);
     assert(II != ImmDefMIs.end() && "couldn't find immediate definition");
     if (TII->FoldImmediate(MI, *II->second, Reg, MRI)) {
       ++NumImmFold;
+      // FoldImmediate can delete ImmDefMI if MI was its only user. If ImmDefMI
+      // is not deleted, and we happened to get a same MI, we can delete MI and
+      // replace its users.
+      if (MRI->getVRegDef(Reg) &&
+          MI.isIdenticalTo(*II->second, MachineInstr::IgnoreVRegDefs)) {
+        Register DstReg = MI.getOperand(0).getReg();
+        if (DstReg.isVirtual() &&
+            MRI->getRegClass(DstReg) == MRI->getRegClass(Reg)) {
+          MRI->replaceRegWith(DstReg, Reg);
+          MI.eraseFromParent();
+          Deleted = true;
+        }
+      }
       return true;
     }
   }
   return false;
 }
 
 // FIXME: This is very simple and misses some cases which should be handled when
 // motivating examples are found.
 //
 // The copy rewriting logic should look at uses as well as defs and be able to
 // eliminate copies across blocks.
 //
 // Later copies that are subregister extracts will also not be eliminated since
 // only the first copy is considered.
 //
 // e.g.
 // %1 = COPY %0
 // %2 = COPY %0:sub1
 //
 // Should replace %2 uses with %1:sub1
-bool PeepholeOptimizer::foldRedundantCopy(
-    MachineInstr &MI, DenseMap<RegSubRegPair, MachineInstr *> &CopyMIs) {
+bool PeepholeOptimizer::foldRedundantCopy(MachineInstr &MI) {
   assert(MI.isCopy() && "expected a COPY machine instruction");
 
-  Register SrcReg = MI.getOperand(1).getReg();
-  unsigned SrcSubReg = MI.getOperand(1).getSubReg();
-  if (!SrcReg.isVirtual() && !MRI->isConstantPhysReg(SrcReg))
+  RegSubRegPair SrcPair;
+  if (!getCopySrc(MI, SrcPair))
     return false;
 
   Register DstReg = MI.getOperand(0).getReg();
   if (!DstReg.isVirtual())
     return false;
 
-  RegSubRegPair SrcPair(SrcReg, SrcSubReg);
-
-  if (CopyMIs.insert(std::make_pair(SrcPair, &MI)).second) {
+  if (CopySrcMIs.insert(std::make_pair(SrcPair, &MI)).second) {
     // First copy of this reg seen.
     return false;
   }
 
-  MachineInstr *PrevCopy = CopyMIs.find(SrcPair)->second;
+  MachineInstr *PrevCopy = CopySrcMIs.find(SrcPair)->second;
 
-  assert(SrcSubReg == PrevCopy->getOperand(1).getSubReg() &&
+  assert(SrcPair.SubReg == PrevCopy->getOperand(1).getSubReg() &&
          "Unexpected mismatching subreg!");
 
   Register PrevDstReg = PrevCopy->getOperand(0).getReg();
 
   // Only replace if the copy register class is the same.
   //
   // TODO: If we have multiple copies to different register classes, we may want
   // to track multiple copies of the same source register.
bool PeepholeOptimizer::runOnMachineFunction(MachineFunction &MF) {
   if (DisablePeephole)
     return false;
 
   TII = MF.getSubtarget().getInstrInfo();
   TRI = MF.getSubtarget().getRegisterInfo();
   MRI = &MF.getRegInfo();
   DT  = Aggressive ? &getAnalysis<MachineDominatorTree>() : nullptr;
   MLI = &getAnalysis<MachineLoopInfo>();
+  MF.setDelegate(this);
 
   bool Changed = false;
 
   for (MachineBasicBlock &MBB : MF) {
     bool SeenMoveImm = false;
 
     // During this forward scan, at some point it needs to answer the question
     // "given a pointer to an MI in the current BB, is it located before or
     // after the current instruction".
     // To perform this, the following set keeps track of the MIs already seen
     // during the scan, if a MI is not in the set, it is assumed to be located
     // after. Newly created MIs have to be inserted in the set as well.
     SmallPtrSet<MachineInstr*, 16> LocalMIs;
     SmallSet<Register, 4> ImmDefRegs;
     DenseMap<Register, MachineInstr *> ImmDefMIs;
     SmallSet<Register, 16> FoldAsLoadDefCandidates;
 
     // Track when a non-allocatable physical register is copied to a virtual
     // register so that useless moves can be removed.
     //
     // $physreg is the map index; MI is the last valid `%vreg = COPY $physreg`
     // without any intervening re-definition of $physreg.
     DenseMap<Register, MachineInstr *> NAPhysToVirtMIs;
 
-    // Set of copies to virtual registers keyed by source register.  Never
-    // holds any physreg which requires def tracking.
-    DenseMap<RegSubRegPair, MachineInstr *> CopySrcMIs;
+    CopySrcMIs.clear();
 
     bool IsLoopHeader = MLI->isLoopHeader(&MBB);
 
     for (MachineBasicBlock::iterator MII = MBB.begin(), MIE = MBB.end();
          MII != MIE; ) {
       MachineInstr *MI = &*MII;
       // We may be erasing MI below, increment MII now.
       ++MII;
for (MachineBasicBlock::iterator MII = MBB.begin(), MIE = MBB.end();
       }
 
       if (isCoalescableCopy(*MI) && optimizeCoalescableCopy(*MI)) {
         // MI is just rewritten.
         Changed = true;
         continue;
       }
 
-      if (MI->isCopy() && (foldRedundantCopy(*MI, CopySrcMIs) ||
+      if (MI->isCopy() && (foldRedundantCopy(*MI) ||
                            foldRedundantNAPhysCopy(*MI, NAPhysToVirtMIs))) {
         LocalMIs.erase(MI);
         LLVM_DEBUG(dbgs() << "Deleting redundant copy: " << *MI << "\n");
         MI->eraseFromParent();
         Changed = true;
         continue;
       }
 
       if (isMoveImmediate(*MI, ImmDefRegs, ImmDefMIs)) {
         SeenMoveImm = true;
       } else {
         Changed |= optimizeExtInstr(*MI, MBB, LocalMIs);
         // optimizeExtInstr might have created new instructions after MI
         // and before the already incremented MII. Adjust MII so that the
         // next iteration sees the new instructions.
         MII = MI;
         ++MII;
-        if (SeenMoveImm)
-          Changed |= foldImmediate(*MI, ImmDefRegs, ImmDefMIs);
+        if (SeenMoveImm) {
+          bool Deleted;
+          Changed |= foldImmediate(*MI, ImmDefRegs, ImmDefMIs, Deleted);
+          if (Deleted) {
+            LocalMIs.erase(MI);
+            continue;
+          }
+        }
       }
 
       // Check whether MI is a load candidate for folding into a later
       // instruction. If MI is not a candidate, check whether we can fold an
       // earlier load into MI.
       if (!isLoadFoldable(*MI, FoldAsLoadDefCandidates) &&
           !FoldAsLoadDefCandidates.empty()) {
 
for (MachineBasicBlock::iterator MII = MBB.begin(), MIE = MBB.end();
       // instruction, so this needs to be after the folding logic.
       if (MI->isLoadFoldBarrier()) {
         LLVM_DEBUG(dbgs() << "Encountered load fold barrier on " << *MI);
         FoldAsLoadDefCandidates.clear();
       }
     }
   }
 
+  MF.resetDelegate(this);
   return Changed;
 }
 
 ValueTrackerResult ValueTracker::getNextSourceFromCopy() {
   assert(Def->isCopy() && "Invalid definition");
   // Copy instruction are supposed to be: Def = Src.
   // If someone breaks this assumption, bad things will happen everywhere.
   // There may be implicit uses preventing the copy to be moved across

llvm/lib/Target/X86/X86InstrInfo.h

public:
   /// defined by the load we are trying to fold. DefMI returns the machine
   /// instruction that defines FoldAsLoadDefReg, and the function returns
   /// the machine instruction generated due to folding.
   MachineInstr *optimizeLoadInstr(MachineInstr &MI,
                                   const MachineRegisterInfo *MRI,
                                   Register &FoldAsLoadDefReg,
                                   MachineInstr *&DefMI) const override;
 
+  bool FoldImmediateImpl(MachineInstr &UseMI, MachineInstr *DefMI, Register Reg,
+                         int64_t ImmVal, MachineRegisterInfo *MRI,
+                         bool MakeChange) const;
+
+  /// Reg is known to be defined by a move immediate instruction, try to fold
+  /// the immediate into the use instruction.
+  bool FoldImmediate(MachineInstr &UseMI, MachineInstr &DefMI, Register Reg,
+                     MachineRegisterInfo *MRI) const override;
+
   std::pair<unsigned, unsigned>
   decomposeMachineOperandsTargetFlags(unsigned TF) const override;
 
   ArrayRef<std::pair<unsigned, const char *>>
   getSerializableDirectMachineOperandTargetFlags() const override;
 
   std::optional<outliner::OutlinedFunction> getOutliningCandidateInfo(
       std::vector<outliner::Candidate> &RepeatedSequenceLocs) const override;

llvm/lib/Target/X86/X86InstrInfo.cpp

if (!isInt<32>(AM.Displacement)) {
     ErrInfo = "Displacement in address must fit into 32-bit signed "
               "integer";
     return false;
   }
 
   return true;
 }
 
 bool X86InstrInfo::getConstValDefinedInReg(const MachineInstr &MI,
                                            const Register Reg,
                                            int64_t &ImmVal) const {
-  if (MI.getOpcode() != X86::MOV32ri && MI.getOpcode() != X86::MOV64ri)
+  Register MovReg = Reg;
+  const MachineInstr *MovMI = &MI;
+
+  // Follow use-def for SUBREG_TO_REG to find the real move immediate
+  // instruction. It is quite common for x86-64.
+  if (MI.isSubregToReg()) {
+    // We use following pattern to setup 64b immediate.
+    //      %8:gr32 = MOV32r0 implicit-def dead $eflags
+    //      %6:gr64 = SUBREG_TO_REG 0, killed %8:gr32, %subreg.sub_32bit
+    if (!MI.getOperand(1).isImm())
+      return false;
+    unsigned FillBits = MI.getOperand(1).getImm();
+    unsigned SubIdx = MI.getOperand(3).getImm();
+    MovReg = MI.getOperand(2).getReg();
+    if (SubIdx != X86::sub_32bit || FillBits != 0)
+      return false;
+    const MachineRegisterInfo &MRI = MI.getParent()->getParent()->getRegInfo();
+    MovMI = MRI.getUniqueVRegDef(MovReg);
+    if (!MovMI)
+      return false;
+  }
+
+  if (MovMI->getOpcode() == X86::MOV32r0 &&
+      MovMI->getOperand(0).getReg() == MovReg) {
+    ImmVal = 0;
+    return true;
+  }
+
+  if (MovMI->getOpcode() != X86::MOV32ri &&
+      MovMI->getOpcode() != X86::MOV64ri &&
+      MovMI->getOpcode() != X86::MOV32ri64 && MovMI->getOpcode() != X86::MOV8ri)
     return false;
   // Mov Src can be a global address.
-  if (!MI.getOperand(1).isImm() || MI.getOperand(0).getReg() != Reg)
+  if (!MovMI->getOperand(1).isImm() || MovMI->getOperand(0).getReg() != MovReg)
     return false;
-  ImmVal = MI.getOperand(1).getImm();
+  ImmVal = MovMI->getOperand(1).getImm();
   return true;
 }
 
 bool X86InstrInfo::preservesZeroValueInReg(
     const MachineInstr *MI, const Register NullValueReg,
     const TargetRegisterInfo *TRI) const {
   if (!MI->modifiesRegister(NullValueReg, TRI))
     return true;
MachineInstr *X86InstrInfo::optimizeLoadInstr(MachineInstr &MI,
   if (MachineInstr *FoldMI = foldMemoryOperand(MI, SrcOperandIds, *DefMI)) {
     FoldAsLoadDefReg = 0;
     return FoldMI;
   }
 
   return nullptr;
 }
 
+/// Convert an ALUrr opcode to corresponding ALUri opcode. Such as
+///     ADD32rr  ==>  ADD32ri
+/// ShiftRotate will be set to true if the Opcode is shift or rotate.
+/// If the ALUri can be further changed to COPY when the immediate is 0, set
+/// CanConvert2Copy to true.
+static unsigned ConvertALUrr2ALUri(unsigned Opcode, bool &CanConvert2Copy,
+                                   bool &ShiftRotate) {
+  CanConvert2Copy = false;
+  ShiftRotate = false;
+  unsigned NewOpcode = 0;
+  switch (Opcode) {
+    case X86::ADD64rr:
+      NewOpcode = X86::ADD64ri32;
+      CanConvert2Copy = true;
+      break;
+    case X86::ADC64rr:
+      NewOpcode = X86::ADC64ri32;
+      break;
+    case X86::SUB64rr:
+      NewOpcode = X86::SUB64ri32;
+      CanConvert2Copy = true;
+      break;
+    case X86::SBB64rr:
+      NewOpcode = X86::SBB64ri32;
+      break;
+    case X86::AND64rr:
+      NewOpcode = X86::AND64ri32;
+      break;
+    case X86::OR64rr:
+      NewOpcode = X86::OR64ri32;
+      CanConvert2Copy = true;
+      break;
+    case X86::XOR64rr:
+      NewOpcode = X86::XOR64ri32;
+      CanConvert2Copy = true;
+      break;
+    case X86::TEST64rr:
+      NewOpcode = X86::TEST64ri32;
+      break;
+    case X86::CMP64rr:
+      NewOpcode = X86::CMP64ri32;
+      break;
+    case X86::SHR64rCL:
+      NewOpcode = X86::SHR64ri;
+      ShiftRotate = true;
+      break;
+    case X86::SHL64rCL:
+      NewOpcode = X86::SHL64ri;
+      ShiftRotate = true;
+      break;
+    case X86::SAR64rCL:
+      NewOpcode = X86::SAR64ri;
+      ShiftRotate = true;
+      break;
+    case X86::ROL64rCL:
+      NewOpcode = X86::ROL64ri;
+      ShiftRotate = true;
+      break;
+    case X86::ROR64rCL:
+      NewOpcode = X86::ROR64ri;
+      ShiftRotate = true;
+      break;
+    case X86::RCL64rCL:
+      NewOpcode = X86::RCL64ri;
+      ShiftRotate = true;
+      break;
+    case X86::RCR64rCL:
+      NewOpcode = X86::RCR64ri;
+      ShiftRotate = true;
+      break;
+    case X86::ADD32rr:
+      NewOpcode = X86::ADD32ri;
+      CanConvert2Copy = true;
+      break;
+    case X86::ADC32rr:
+      NewOpcode = X86::ADC32ri;
+      break;
+    case X86::SUB32rr:
+      NewOpcode = X86::SUB32ri;
+      CanConvert2Copy = true;
+      break;
+    case X86::SBB32rr:
+      NewOpcode = X86::SBB32ri;
+      break;
+    case X86::AND32rr:
+      NewOpcode = X86::AND32ri;
+      break;
+    case X86::OR32rr:
+      NewOpcode = X86::OR32ri;
+      CanConvert2Copy = true;
+      break;
+    case X86::XOR32rr:
+      NewOpcode = X86::XOR32ri;
+      CanConvert2Copy = true;
+      break;
+    case X86::TEST32rr:
+      NewOpcode = X86::TEST32ri;
+      break;
+    case X86::CMP32rr:
+      NewOpcode = X86::CMP32ri;
+      break;
+    case X86::SHR32rCL:
+      NewOpcode = X86::SHR32ri;
+      ShiftRotate = true;
+      break;
+    case X86::SHL32rCL:
+      NewOpcode = X86::SHL32ri;
+      ShiftRotate = true;
+      break;
+    case X86::SAR32rCL:
+      NewOpcode = X86::SAR32ri;
+      ShiftRotate = true;
+      break;
+    case X86::ROL32rCL:
+      NewOpcode = X86::ROL32ri;
+      ShiftRotate = true;
+      break;
+    case X86::ROR32rCL:
+      NewOpcode = X86::ROR32ri;
+      ShiftRotate = true;
+      break;
+    case X86::RCL32rCL:
+      NewOpcode = X86::RCL32ri;
+      ShiftRotate = true;
+      break;
+    case X86::RCR32rCL:
+      NewOpcode = X86::RCR32ri;
+      ShiftRotate = true;
+      break;
+  }
+  return NewOpcode;
+}
+
+/// Real implementation of FoldImmediate.
+/// Reg is assigned ImmVal in DefMI, and is used in UseMI.
+/// If MakeChange is true, this function tries to replace Reg by ImmVal in
+/// UseMI. If MakeChange is false, just check if folding is possible.
+/// Return true if folding is successful or possible.
+bool X86InstrInfo::FoldImmediateImpl(MachineInstr &UseMI, MachineInstr *DefMI,
+                                     Register Reg, int64_t ImmVal,
+                                     MachineRegisterInfo *MRI,
+                                     bool MakeChange) const {
+  bool Modified = false;
+  bool ShiftRotate = false;
+  // When ImmVal is 0, some instructions can be changed to COPY.
+  bool CanChangeToCopy = false;
+  unsigned Opc = UseMI.getOpcode();
+
+  // 64 bit operations accept sign extended 32 bit immediates.
+  // 32 bit operations accept all 32 bit immediates, so we don't need to check
+  // them.
+  const TargetRegisterClass *RC = nullptr;
+  if (Reg.isVirtual())
+    RC = MRI->getRegClass(Reg);
+  if ((Reg.isPhysical() && X86::GR64RegClass.contains(Reg)) ||
+      (Reg.isVirtual() && X86::GR64RegClass.hasSubClassEq(RC))) {
+    if (!isInt<32>(ImmVal))
+      return false;
+  }
+
+  if (UseMI.findRegisterUseOperand(Reg)->getSubReg())
+    return false;
+  // Immediate has larger code size than register. So avoid folding the
+  // immediate if it has more than 1 use and we are optimizing for size.
+  if (UseMI.getMF()->getFunction().hasOptSize() && Reg.isVirtual() &&
+      !MRI->hasOneNonDBGUse(Reg))
+    return false;
+
+  unsigned NewOpc;
+  if (Opc == TargetOpcode::COPY) {
+    Register ToReg = UseMI.getOperand(0).getReg();
+    const TargetRegisterClass *RC = nullptr;
+    if (ToReg.isVirtual())
+      RC = MRI->getRegClass(ToReg);
+    bool GR32Reg = (ToReg.isVirtual() && X86::GR32RegClass.hasSubClassEq(RC)) ||
+                   (ToReg.isPhysical() && X86::GR32RegClass.contains(ToReg));
+    bool GR64Reg = (ToReg.isVirtual() && X86::GR64RegClass.hasSubClassEq(RC)) ||
+                   (ToReg.isPhysical() && X86::GR64RegClass.contains(ToReg));
+    bool GR8Reg = (ToReg.isVirtual() && X86::GR8RegClass.hasSubClassEq(RC)) ||
+                  (ToReg.isPhysical() && X86::GR8RegClass.contains(ToReg));
+
+    if (ImmVal == 0) {
+      // We have MOV32r0 only.
+      if (!GR32Reg)
+        return false;
+    }
+
+    if (GR64Reg) {
+      if (isUInt<32>(ImmVal))
+        NewOpc = X86::MOV32ri64;
+      else
+        NewOpc = X86::MOV64ri;
+    } else if (GR32Reg) {
+      NewOpc = X86::MOV32ri;
+      if (ImmVal == 0) {
+        // MOV32r0 clobbers EFLAGS.
+        const TargetRegisterInfo *TRI = &getRegisterInfo();
+        if (UseMI.getParent()->computeRegisterLiveness(TRI, X86::EFLAGS, UseMI)
+            != MachineBasicBlock::LQR_Dead)
+          return false;
+
+        // MOV32r0 is different than other cases because it doesn't encode the
+        // immediate in the instruction. So we directly modify it here.
+        if (!MakeChange)
+          return true;
+        UseMI.setDesc(get(X86::MOV32r0));
+        UseMI.removeOperand(UseMI.findRegisterUseOperandIdx(Reg));
+        UseMI.addOperand(MachineOperand::CreateReg(X86::EFLAGS, /*isDef=*/ true,
+                                                   /*isImp=*/ true,
+                                                   /*isKill=*/ false,
+                                                   /*isDead=*/ true));
+        Modified = true;
+      }
+    } else if (GR8Reg)
+      NewOpc = X86::MOV8ri;
+    else
+      return false;
+  } else
+    NewOpc = ConvertALUrr2ALUri(Opc, CanChangeToCopy, ShiftRotate);
+
+  if (!NewOpc)
+    return false;
+
+  // For SUB instructions the immediate can only be the second source operand.
+  if ((NewOpc == X86::SUB64ri32 || NewOpc == X86::SUB32ri ||
+       NewOpc == X86::SBB64ri32 || NewOpc == X86::SBB32ri) &&
+      UseMI.findRegisterUseOperandIdx(Reg) != 2)
+    return false;
+  // For CMP instructions the immediate can only be at index 1.
+  if ((NewOpc == X86::CMP64ri32 || NewOpc == X86::CMP32ri) &&
+      UseMI.findRegisterUseOperandIdx(Reg) != 1)
+    return false;
+
+  if (ShiftRotate) {
+    unsigned RegIdx = UseMI.findRegisterUseOperandIdx(Reg);
+    if (RegIdx < 2)
+      return false;
+    if (!isInt<8>(ImmVal))
+      return false;
+    assert(Reg == X86::CL);
+
+    if (!MakeChange)
+      return true;
+    UseMI.setDesc(get(NewOpc));
+    UseMI.removeOperand(RegIdx);
+    UseMI.addOperand(MachineOperand::CreateImm(ImmVal));
+    // Reg is physical register $cl, so we don't know if DefMI is dead through
+    // MRI. Let the caller handle it, or pass dead-mi-elimination can delete
+    // the dead physical register define instruction.
+    return true;
+  }
+
+  if (!MakeChange)
+    return true;
+
+  if (!Modified) {
+    // Modify the instruction.
+    if (ImmVal == 0 && CanChangeToCopy &&
+        UseMI.registerDefIsDead(X86::EFLAGS)) {
+      //          %100 = add %101, 0
+      //    ==>
+      //          %100 = COPY %101
+      UseMI.setDesc(get(TargetOpcode::COPY));
+      UseMI.removeOperand(UseMI.findRegisterUseOperandIdx(Reg));
+      UseMI.removeOperand(UseMI.findRegisterDefOperandIdx(X86::EFLAGS));
+      UseMI.untieRegOperand(0);
+      UseMI.clearFlag(MachineInstr::MIFlag::NoSWrap);
+      UseMI.clearFlag(MachineInstr::MIFlag::NoUWrap);
+    } else {
+      unsigned Op1 = 1, Op2 = CommuteAnyOperandIndex;
+      unsigned ImmOpNum = 2;
+      if (!UseMI.getOperand(0).isDef()) {
+        Op1 = 0;                                      // TEST, CMP
+        ImmOpNum = 1;
+      }
+      if (Opc == TargetOpcode::COPY)
+        ImmOpNum = 1;
+      if (findCommutedOpIndices(UseMI, Op1, Op2) &&
+          UseMI.getOperand(Op1).getReg() == Reg)
+        commuteInstruction(UseMI);
+
+      assert(UseMI.getOperand(ImmOpNum).getReg() == Reg);
+      UseMI.setDesc(get(NewOpc));
+      UseMI.getOperand(ImmOpNum).ChangeToImmediate(ImmVal);
+    }
+  }
+
+  if (Reg.isVirtual() && MRI->use_nodbg_empty(Reg))
+    DefMI->eraseFromBundle();
+
+  return true;
+}
+
+/// FoldImmediate - 'Reg' is known to be defined by a move immediate
+/// instruction, try to fold the immediate into the use instruction.
+bool X86InstrInfo::FoldImmediate(MachineInstr &UseMI, MachineInstr &DefMI,
+                                 Register Reg, MachineRegisterInfo *MRI) const {
+  int64_t ImmVal;
+  if (!getConstValDefinedInReg(DefMI, Reg, ImmVal))
+    return false;
+
+  return FoldImmediateImpl(UseMI, &DefMI, Reg, ImmVal, MRI, true);
+}
+
 /// Expand a single-def pseudo instruction to a two-addr
 /// instruction with two undef reads of the register being defined.
 /// This is used for mapping:
 ///   %xmm4 = V_SET0
 /// to:
 ///   %xmm4 = PXORrr undef %xmm4, undef %xmm4
 ///
 static bool Expand2AddrUndef(MachineInstrBuilder &MIB,

llvm/test/CodeGen/AMDGPU/peephole-fold-imm.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 # RUN: llc -mtriple=amdgcn--amdhsa -mcpu=gfx940 -verify-machineinstrs -run-pass peephole-opt -o - %s | FileCheck -check-prefix=GCN %s
 
 ---
 name:            fold_simm_virtual
 body:             |
   bb.0:
 
     ; GCN-LABEL: name: fold_simm_virtual
     ; GCN: [[S_MOV_B32_:%[0-9]+]]:sreg_32 = S_MOV_B32 0
-    ; GCN-NEXT: [[S_MOV_B32_1:%[0-9]+]]:sreg_32 = S_MOV_B32 0
     ; GCN-NEXT: SI_RETURN_TO_EPILOG
     %0:sreg_32 = S_MOV_B32 0
     %1:sreg_32 = COPY killed %0
     SI_RETURN_TO_EPILOG
 
 ...
 
 ---

llvm/test/CodeGen/X86/GlobalISel/phi.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-linux-gnu -global-isel -verify-machineinstrs < %s -o - | FileCheck %s --check-prefix=ALL
 
 define i8 @test_i8(i32 %a, i8 %f, i8 %t) {
 ; ALL-LABEL: test_i8:
 ; ALL:       # %bb.0: # %entry
-; ALL-NEXT:    xorl %ecx, %ecx
-; ALL-NEXT:    cmpl %ecx, %edi
+; ALL-NEXT:    cmpl $0, %edi
 ; ALL-NEXT:    setg %cl
 ; ALL-NEXT:    testb $1, %cl
 ; ALL-NEXT:    je .LBB0_2
 ; ALL-NEXT:  # %bb.1:
 ; ALL-NEXT:    movl %esi, %eax
 ; ALL-NEXT:    # kill: def $al killed $al killed $eax
 ; ALL-NEXT:    retq
 ; ALL-NEXT:  .LBB0_2: # %cond.false

 cond.end:                                         ; preds = %cond.false, %cond.true
   %cond = phi i8 [ %f, %cond.true ], [ %t, %cond.false ]
   ret i8 %cond
 }
 
 define i16 @test_i16(i32 %a, i16 %f, i16 %t) {
 ; ALL-LABEL: test_i16:
 ; ALL:       # %bb.0: # %entry
-; ALL-NEXT:    xorl %ecx, %ecx
-; ALL-NEXT:    cmpl %ecx, %edi
+; ALL-NEXT:    cmpl $0, %edi
 ; ALL-NEXT:    setg %cl
 ; ALL-NEXT:    testb $1, %cl
 ; ALL-NEXT:    je .LBB1_2
 ; ALL-NEXT:  # %bb.1:
 ; ALL-NEXT:    movl %esi, %eax
 ; ALL-NEXT:    # kill: def $ax killed $ax killed $eax
 ; ALL-NEXT:    retq
 ; ALL-NEXT:  .LBB1_2: # %cond.false
cond.end:                                         ; preds = %cond.false, %cond.true
   %cond = phi i16 [ %f, %cond.true ], [ %t, %cond.false ]
   ret i16 %cond
 }
 
 define i32 @test_i32(i32 %a, i32 %f, i32 %t) {
 ; ALL-LABEL: test_i32:
 ; ALL:       # %bb.0: # %entry
 ; ALL-NEXT:    movl %esi, %eax
-; ALL-NEXT:    xorl %ecx, %ecx
-; ALL-NEXT:    cmpl %ecx, %edi
+; ALL-NEXT:    cmpl $0, %edi
 ; ALL-NEXT:    setg %cl
 ; ALL-NEXT:    testb $1, %cl
 ; ALL-NEXT:    je .LBB2_1
 ; ALL-NEXT:  # %bb.2: # %cond.end
 ; ALL-NEXT:    retq
 ; ALL-NEXT:  .LBB2_1: # %cond.false
 ; ALL-NEXT:    movl %edx, %eax
 ; ALL-NEXT:    retq
cond.end:                                         ; preds = %cond.false, %cond.true
   %cond = phi i32 [ %f, %cond.true ], [ %t, %cond.false ]
   ret i32 %cond
 }
 
 define i64 @test_i64(i32 %a, i64 %f, i64 %t) {
 ; ALL-LABEL: test_i64:
 ; ALL:       # %bb.0: # %entry
 ; ALL-NEXT:    movq %rsi, %rax
-; ALL-NEXT:    xorl %ecx, %ecx
-; ALL-NEXT:    cmpl %ecx, %edi
+; ALL-NEXT:    cmpl $0, %edi
 ; ALL-NEXT:    setg %cl
 ; ALL-NEXT:    testb $1, %cl
 ; ALL-NEXT:    je .LBB3_1
 ; ALL-NEXT:  # %bb.2: # %cond.end
 ; ALL-NEXT:    retq
 ; ALL-NEXT:  .LBB3_1: # %cond.false
 ; ALL-NEXT:    movq %rdx, %rax
 ; ALL-NEXT:    retq

 cond.end:                                         ; preds = %cond.false, %cond.true
   %cond = phi i64 [ %f, %cond.true ], [ %t, %cond.false ]
   ret i64 %cond
 }
 
 define float @test_float(i32 %a, float %f, float %t) {
 ; ALL-LABEL: test_float:
 ; ALL:       # %bb.0: # %entry
-; ALL-NEXT:    xorl %eax, %eax
-; ALL-NEXT:    cmpl %eax, %edi
+; ALL-NEXT:    cmpl $0, %edi
 ; ALL-NEXT:    setg %al
 ; ALL-NEXT:    testb $1, %al
 ; ALL-NEXT:    je .LBB4_1
 ; ALL-NEXT:  # %bb.2: # %cond.end
 ; ALL-NEXT:    retq
 ; ALL-NEXT:  .LBB4_1: # %cond.false
 ; ALL-NEXT:    movaps %xmm1, %xmm0
 ; ALL-NEXT:    retq

 cond.end:                                         ; preds = %cond.false, %cond.true
   %cond = phi float [ %f, %cond.true ], [ %t, %cond.false ]
   ret float %cond
 }
 
 define double @test_double(i32 %a, double %f, double %t) {
 ; ALL-LABEL: test_double:
 ; ALL:       # %bb.0: # %entry
-; ALL-NEXT:    xorl %eax, %eax
-; ALL-NEXT:    cmpl %eax, %edi
+; ALL-NEXT:    cmpl $0, %edi
 ; ALL-NEXT:    setg %al
 ; ALL-NEXT:    testb $1, %al
 ; ALL-NEXT:    je .LBB5_1
 ; ALL-NEXT:  # %bb.2: # %cond.end
 ; ALL-NEXT:    retq
 ; ALL-NEXT:  .LBB5_1: # %cond.false
 ; ALL-NEXT:    movaps %xmm1, %xmm0
 ; ALL-NEXT:    retq

llvm/test/CodeGen/X86/div-rem-pair-recomposition-signed.ll

 define i128 @scalar_i128(i128 %x, i128 %y, ptr %divdst) nounwind {
 ; X86-LABEL: scalar_i128:
 ; X86:       # %bb.0: # %_udiv-special-cases
 ; X86-NEXT:    pushl %ebp
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %edi
 ; X86-NEXT:    pushl %esi
 ; X86-NEXT:    subl $152, %esp
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    sarl $31, %eax
-; X86-NEXT:    movl %ebp, %edx
-; X86-NEXT:    sarl $31, %edx
+; X86-NEXT:    movl %edx, %edi
+; X86-NEXT:    sarl $31, %edi
 ; X86-NEXT:    movl %eax, %esi
 ; X86-NEXT:    xorl %ecx, %esi
-; X86-NEXT:    movl %esi, %edi
+; X86-NEXT:    movl %esi, %ebp
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    xorl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl %ecx, %ebx
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    xorl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl %eax, %esi
 ; X86-NEXT:    xorl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    subl %eax, %esi
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %esi, (%esp) # 4-byte Spill
 ; X86-NEXT:    sbbl %eax, %ecx
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %eax, %ebx
-; X86-NEXT:    movl %ebx, (%esp) # 4-byte Spill
-; X86-NEXT:    sbbl %eax, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edx, %edi
-; X86-NEXT:    xorl %ebp, %edi
-; X86-NEXT:    movl %edx, %ebx
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    sbbl %eax, %ebp
+; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %edi, %esi
+; X86-NEXT:    xorl %edx, %esi
+; X86-NEXT:    movl %edi, %edx
+; X86-NEXT:    xorl {{[0-9]+}}(%esp), %edx
+; X86-NEXT:    movl %edi, %ebx
 ; X86-NEXT:    xorl {{[0-9]+}}(%esp), %ebx
-; X86-NEXT:    movl %edx, %esi
-; X86-NEXT:    xorl {{[0-9]+}}(%esp), %esi
-; X86-NEXT:    movl %edx, %ebp
+; X86-NEXT:    movl %edi, %ebp
 ; X86-NEXT:    xorl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    subl %edx, %ebp
-; X86-NEXT:    sbbl %edx, %esi
-; X86-NEXT:    sbbl %edx, %ebx
-; X86-NEXT:    sbbl %edx, %edi
-; X86-NEXT:    xorl %eax, %edx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %esi, %eax
-; X86-NEXT:    orl %edi, %eax
+; X86-NEXT:    subl %edi, %ebp
+; X86-NEXT:    sbbl %edi, %ebx
+; X86-NEXT:    sbbl %edi, %edx
+; X86-NEXT:    sbbl %edi, %esi
+; X86-NEXT:    xorl %eax, %edi
+; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebx, %eax
+; X86-NEXT:    orl %esi, %eax
 ; X86-NEXT:    movl %ebp, %ecx
-; X86-NEXT:    orl %ebx, %ecx
+; X86-NEXT:    orl %edx, %ecx
+; X86-NEXT:    movl %edx, %edi
 ; X86-NEXT:    orl %eax, %ecx
 ; X86-NEXT:    sete %cl
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    orl (%esp), %edx # 4-byte Folded Reload
+; X86-NEXT:    movl (%esp), %edx # 4-byte Reload
+; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
 ; X86-NEXT:    orl %eax, %edx
 ; X86-NEXT:    sete %al
 ; X86-NEXT:    orb %cl, %al
 ; X86-NEXT:    movb %al, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
-; X86-NEXT:    bsrl %edi, %edx
+; X86-NEXT:    bsrl %esi, %edx
 ; X86-NEXT:    xorl $31, %edx
-; X86-NEXT:    bsrl %ebx, %ecx
+; X86-NEXT:    bsrl %edi, %ecx
 ; X86-NEXT:    xorl $31, %ecx
 ; X86-NEXT:    addl $32, %ecx
-; X86-NEXT:    testl %edi, %edi
+; X86-NEXT:    testl %esi, %esi
 ; X86-NEXT:    cmovnel %edx, %ecx
-; X86-NEXT:    bsrl %esi, %edx
+; X86-NEXT:    bsrl %ebx, %edx
 ; X86-NEXT:    xorl $31, %edx
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    bsrl %ebp, %ebp
 ; X86-NEXT:    xorl $31, %ebp
 ; X86-NEXT:    addl $32, %ebp
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    testl %esi, %esi
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    testl %ebx, %ebx
 ; X86-NEXT:    cmovnel %edx, %ebp
 ; X86-NEXT:    addl $64, %ebp
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %edi, %ebx
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl %esi, %edi
 ; X86-NEXT:    cmovnel %ecx, %ebp
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
 ; X86-NEXT:    bsrl %edi, %edx
 ; X86-NEXT:    xorl $31, %edx
-; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    bsrl %eax, %ecx
 ; X86-NEXT:    xorl $31, %ecx
 ; X86-NEXT:    addl $32, %ecx
 ; X86-NEXT:    testl %edi, %edi
 ; X86-NEXT:    cmovnel %edx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:    bsrl %ebx, %esi
 ; X86-NEXT:    xorl $31, %esi
-; X86-NEXT:    bsrl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
+; X86-NEXT:    bsrl (%esp), %edx # 4-byte Folded Reload
 ; X86-NEXT:    xorl $31, %edx
 ; X86-NEXT:    addl $32, %edx
 ; X86-NEXT:    testl %ebx, %ebx
 ; X86-NEXT:    cmovnel %esi, %edx
 ; X86-NEXT:    addl $64, %edx
 ; X86-NEXT:    movl %eax, %esi
 ; X86-NEXT:    orl %edi, %esi
-; X86-NEXT:    movl %edi, %ebx
 ; X86-NEXT:    cmovnel %ecx, %edx
 ; X86-NEXT:    xorl %esi, %esi
 ; X86-NEXT:    subl %edx, %ebp
+; X86-NEXT:    movl $0, %ebx
+; X86-NEXT:    sbbl %ebx, %ebx
 ; X86-NEXT:    movl $0, %edx
 ; X86-NEXT:    sbbl %edx, %edx
 ; X86-NEXT:    movl $0, %eax
 ; X86-NEXT:    sbbl %eax, %eax
-; X86-NEXT:    movl $0, %edi
-; X86-NEXT:    sbbl %edi, %edi
 ; X86-NEXT:    movl $127, %ecx
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    cmpl %ebp, %ecx
 ; X86-NEXT:    movl $0, %ecx
+; X86-NEXT:    sbbl %ebx, %ecx
+; X86-NEXT:    movl $0, %ecx
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %edx, %ecx
 ; X86-NEXT:    movl $0, %ecx
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    sbbl %eax, %ecx
-; X86-NEXT:    movl $0, %ecx
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %edi, %ecx
 ; X86-NEXT:    setb %cl
 ; X86-NEXT:    orb {{[-0-9]+}}(%e{{[sb]}}p), %cl # 1-byte Folded Reload
-; X86-NEXT:    movl %ebx, %edi
 ; X86-NEXT:    cmovnel %esi, %edi
-; X86-NEXT:    movl (%esp), %edx # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NEXT:    cmovnel %esi, %edx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    cmovnel %esi, %eax
-; X86-NEXT:    cmovel {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    jne .LBB4_1
-; X86-NEXT:  # %bb.8: # %_udiv-special-cases
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    xorl $127, %ebp
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; X86-NEXT:    orl %ebp, %ecx
+; X86-NEXT:    cmovel (%esp), %esi # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
+; X86-NEXT:    jne .LBB4_8
+; X86-NEXT:  # %bb.1: # %_udiv-special-cases
+; X86-NEXT:    movl %ebx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    je .LBB4_9
-; X86-NEXT:  # %bb.5: # %udiv-bb1
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    xorl $127, %ebx
+; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Folded Reload
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
+; X86-NEXT:    orl %ebx, %ecx
+; X86-NEXT:    je .LBB4_8
+; X86-NEXT:  # %bb.2: # %udiv-bb1
+; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
 ; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
-; X86-NEXT:    movl (%esp), %eax # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    # kill: def $al killed $al killed $eax
 ; X86-NEXT:    xorb $127, %al
 ; X86-NEXT:    movb %al, %ch
 ; X86-NEXT:    andb $7, %ch
 ; X86-NEXT:    shrb $3, %al
 ; X86-NEXT:    andb $15, %al
 ; X86-NEXT:    negb %al
-; X86-NEXT:    movsbl %al, %edi
-; X86-NEXT:    movl 144(%esp,%edi), %edx
-; X86-NEXT:    movl 148(%esp,%edi), %esi
+; X86-NEXT:    movsbl %al, %ebx
+; X86-NEXT:    movl 144(%esp,%ebx), %edx
+; X86-NEXT:    movl 148(%esp,%ebx), %edi
 ; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    shldl %cl, %edx, %esi
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    shldl %cl, %edx, %edi
 ; X86-NEXT:    shll %cl, %edx
 ; X86-NEXT:    notb %cl
-; X86-NEXT:    movl 140(%esp,%edi), %eax
+; X86-NEXT:    movl 140(%esp,%ebx), %eax
 ; X86-NEXT:    movl %eax, %esi
 ; X86-NEXT:    shrl %esi
 ; X86-NEXT:    shrl %cl, %esi
 ; X86-NEXT:    orl %edx, %esi
 ; X86-NEXT:    movl %esi, %edx
-; X86-NEXT:    movl 136(%esp,%edi), %esi
+; X86-NEXT:    movl 136(%esp,%ebx), %esi
 ; X86-NEXT:    movb %ch, %cl
 ; X86-NEXT:    shldl %cl, %esi, %eax
 ; X86-NEXT:    shll %cl, %esi
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    addl $1, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    adcl $0, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    adcl $0, %edi
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT:    adcl $0, %ebx
 ; X86-NEXT:    adcl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    jae .LBB4_2
+; X86-NEXT:    jae .LBB4_3
 ; X86-NEXT:  # %bb.6:
-; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    xorl %ecx, %ecx
+; X86-NEXT:    xorl %ebx, %ebx
+; X86-NEXT:    xorl %esi, %esi
 ; X86-NEXT:    jmp .LBB4_7
-; X86-NEXT:  .LBB4_1:
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    jmp .LBB4_9
-; X86-NEXT:  .LBB4_2: # %udiv-preheader
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
-; X86-NEXT:    movl (%esp), %ebx # 4-byte Reload
-; X86-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    movl %ebx, {{[0-9]+}}(%esp)
+; X86-NEXT:  .LBB4_3: # %udiv-preheader
+; X86-NEXT:    movl (%esp), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:    movb %bl, %ch
 ; X86-NEXT:    andb $7, %ch
 ; X86-NEXT:    movb %bl, %cl
 ; X86-NEXT:    shrb $3, %cl
 ; X86-NEXT:    andb $15, %cl
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movzbl %cl, %ebx
-; X86-NEXT:    movl 100(%esp,%ebx), %ebp
-; X86-NEXT:    movl %ebp, (%esp) # 4-byte Spill
-; X86-NEXT:    movl 96(%esp,%ebx), %edi
+; X86-NEXT:    movzbl %cl, %ebp
+; X86-NEXT:    movl 100(%esp,%ebp), %esi
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl 96(%esp,%ebp), %ebx
+; X86-NEXT:    movl %ebp, %eax
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edi, %edx
+; X86-NEXT:    movl %ebx, %edx
 ; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    shrdl %cl, %ebp, %edx
-; X86-NEXT:    movl %edx, %ebp
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl 88(%esp,%ebx), %edx
-; X86-NEXT:    movl 92(%esp,%ebx), %ebx
-; X86-NEXT:    movl %ebx, %eax
+; X86-NEXT:    shrdl %cl, %esi, %edx
+; X86-NEXT:    movl 88(%esp,%ebp), %ebp
+; X86-NEXT:    movl 92(%esp,%eax), %esi
+; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    shrl %cl, %eax
 ; X86-NEXT:    notb %cl
-; X86-NEXT:    addl %edi, %edi
-; X86-NEXT:    shll %cl, %edi
-; X86-NEXT:    orl %eax, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    addl %ebx, %ebx
+; X86-NEXT:    shll %cl, %ebx
+; X86-NEXT:    orl %eax, %ebx
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    shrl %cl, (%esp) # 4-byte Folded Spill
-; X86-NEXT:    shrdl %cl, %ebx, %edx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    shrl %cl, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT:    shrdl %cl, %esi, %ebp
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    addl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    xorl %esi, %esi
 ; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:    .p2align 4, 0x90
-; X86-NEXT:  .LBB4_3: # %udiv-do-while
+; X86-NEXT:  .LBB4_4: # %udiv-do-while
 ; X86-NEXT:    # =>This Inner Loop Header: Depth=1
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %ebp, %edx
-; X86-NEXT:    shldl $1, %ebp, (%esp) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebp, %edx
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebx, %ebp
-; X86-NEXT:    shldl $1, %edi, %ebx
+; X86-NEXT:    shldl $1, %edx, %ebx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
+; X86-NEXT:    shldl $1, %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
+; X86-NEXT:    shldl $1, %ebp, %edx
+; X86-NEXT:    shldl $1, %edi, %ebp
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    shldl $1, %eax, %edi
+; X86-NEXT:    orl %esi, %edi
+; X86-NEXT:    movl %edi, (%esp) # 4-byte Spill
+; X86-NEXT:    movl %ecx, %edi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ecx, %edi
+; X86-NEXT:    shldl $1, %ecx, %eax
+; X86-NEXT:    orl %esi, %eax
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    orl %eax, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    shldl $1, %edi, %ecx
-; X86-NEXT:    orl %eax, %ecx
+; X86-NEXT:    shldl $1, %eax, %ecx
+; X86-NEXT:    orl %esi, %ecx
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    shldl $1, %esi, %edi
-; X86-NEXT:    orl %eax, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    addl %esi, %esi
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    cmpl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    sbbl %ebp, %ecx
+; X86-NEXT:    addl %eax, %eax
+; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    cmpl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    sbbl %edx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    sbbl (%esp), %ecx # 4-byte Folded Reload
+; X86-NEXT:    sbbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    sbbl %ebx, %ecx
 ; X86-NEXT:    sarl $31, %ecx
+; X86-NEXT:    movl %ecx, %eax
+; X86-NEXT:    andl $1, %eax
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ecx, %esi
-; X86-NEXT:    andl $1, %esi
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %ecx, %esi
-; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
+; X86-NEXT:    andl %edi, %esi
 ; X86-NEXT:    movl %ecx, %edi
 ; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
 ; X86-NEXT:    andl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
-; X86-NEXT:    subl %ecx, %ebx
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %eax, %ebp
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %edi, %edx
-; X86-NEXT:    movl %edx, %ebp
+; X86-NEXT:    subl %ecx, %ebp
+; X86-NEXT:    sbbl %eax, %edx
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    sbbl %esi, (%esp) # 4-byte Folded Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    sbbl %edi, %edx
+; X86-NEXT:    movl (%esp), %edi # 4-byte Reload
+; X86-NEXT:    sbbl %esi, %ebx
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    addl $-1, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
-; X86-NEXT:    adcl $-1, %edi
-; X86-NEXT:    adcl $-1, %edx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
+; X86-NEXT:    adcl $-1, %esi
+; X86-NEXT:    adcl $-1, %ebx
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %edx, %eax
+; X86-NEXT:    orl %ebx, %eax
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %edi, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl %esi, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; X86-NEXT:    orl %eax, %ecx
-; X86-NEXT:    jne .LBB4_3
-; X86-NEXT:  # %bb.4:
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    jne .LBB4_4
+; X86-NEXT:  # %bb.5:
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:  .LBB4_7: # %udiv-loop-exit
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
 ; X86-NEXT:    shldl $1, %edx, %edi
-; X86-NEXT:    orl %ecx, %edi
+; X86-NEXT:    orl %esi, %edi
 ; X86-NEXT:    shldl $1, %eax, %edx
-; X86-NEXT:    orl %ecx, %edx
+; X86-NEXT:    orl %esi, %edx
+; X86-NEXT:    movl %esi, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; X86-NEXT:    shldl $1, %esi, %eax
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    addl %esi, %esi
-; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
-; X86-NEXT:  .LBB4_9: # %udiv-end
-; X86-NEXT:    xorl %ebx, %edi
-; X86-NEXT:    xorl %ebx, %edx
-; X86-NEXT:    xorl %ebx, %eax
-; X86-NEXT:    xorl %ebx, %esi
-; X86-NEXT:    subl %ebx, %esi
+; X86-NEXT:    orl %ebx, %esi
+; X86-NEXT:  .LBB4_8: # %udiv-end
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    xorl %ecx, %edi
+; X86-NEXT:    xorl %ecx, %edx
+; X86-NEXT:    xorl %ecx, %eax
+; X86-NEXT:    xorl %ecx, %esi
+; X86-NEXT:    subl %ecx, %esi
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %ebx, %eax
+; X86-NEXT:    sbbl %ecx, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %ebx, %edx
-; X86-NEXT:    sbbl %ebx, %edi
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    sbbl %ecx, %edx
+; X86-NEXT:    sbbl %ecx, %edi
+; X86-NEXT:    movl %edi, (%esp) # 4-byte Spill
 ; X86-NEXT:    movl %esi, (%ebp)
 ; X86-NEXT:    movl %eax, 4(%ebp)
 ; X86-NEXT:    movl %edx, 8(%ebp)
 ; X86-NEXT:    movl %edi, 12(%ebp)
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; X86-NEXT:    movl %edx, %ebx
 ; X86-NEXT:    mull %edi
 ; X86-NEXT:    movl %edx, %ecx
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    mull %edi
-; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
+; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %edx, %edi
 ; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
 ; X86-NEXT:    adcl $0, %ecx
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    mull %esi
 ; X86-NEXT:    addl %edi, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    adcl %ecx, %edx
 ; X86-NEXT:    movl %edx, %edi
 ; X86-NEXT:    setb %cl
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Reload
 ; X86-NEXT:    movl %esi, %eax
 ; X86-NEXT:    mull {{[0-9]+}}(%esp)
 ; X86-NEXT:    addl %edi, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movzbl %cl, %eax
 ; X86-NEXT:    adcl %eax, %edx
 ; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    movl (%esp), %ecx # 4-byte Reload
 ; X86-NEXT:    imull %eax, %ecx
 ; X86-NEXT:    mull %ebx
-; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X86-NEXT:    imull {{[0-9]+}}(%esp), %ebx
 ; X86-NEXT:    addl %edx, %ebx
 ; X86-NEXT:    addl %ecx, %ebx
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    imull %esi, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NEXT:    imull %edx, %ebp
 ; X86-NEXT:    mull %edx
 ; X86-NEXT:    addl %edx, %ebp
 ; X86-NEXT:    addl %ecx, %ebp
-; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
+; X86-NEXT:    addl (%esp), %eax # 4-byte Folded Reload
 ; X86-NEXT:    adcl %ebx, %ebp
 ; X86-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Folded Reload
 ; X86-NEXT:    adcl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edx
-; X86-NEXT:    subl (%esp), %edx # 4-byte Folded Reload
+; X86-NEXT:    subl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    sbbl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    sbbl %eax, %esi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    sbbl %ebp, %edi
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %edx, (%eax)

llvm/test/CodeGen/X86/div-rem-pair-recomposition-unsigned.ll

 ; X86-NEXT:    orl %edx, %esi
 ; X86-NEXT:    movl %ebp, %edx
 ; X86-NEXT:    movl 120(%esp,%eax), %ebp
 ; X86-NEXT:    movb %ch, %cl
 ; X86-NEXT:    shldl %cl, %ebp, %edx
 ; X86-NEXT:    shll %cl, %ebp
 ; X86-NEXT:    addl $1, %edi
 ; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
-; X86-NEXT:    adcl $0, %eax
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    adcl $0, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
+; X86-NEXT:    adcl $0, %eax
 ; X86-NEXT:    adcl $0, %ebx
 ; X86-NEXT:    jae .LBB4_3
 ; X86-NEXT:  # %bb.6:
 ; X86-NEXT:    xorl %eax, %eax
 ; X86-NEXT:    xorl %ecx, %ecx
 ; X86-NEXT:    movl %edx, %ebx
 ; X86-NEXT:    jmp .LBB4_7
 ; X86-NEXT:  .LBB4_3: # %udiv-preheader
-; X86-NEXT:    movl %ecx, %edi
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X86-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NEXT:    movl %edi, {{[0-9]+}}(%esp)
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    movl %ecx, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    movb %al, %ch
 ; X86-NEXT:    andb $7, %ch
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    # kill: def $al killed $al killed $eax
 ; X86-NEXT:    shrb $3, %al
 ; X86-NEXT:    andb $15, %al
 ; X86-NEXT:    movzbl %al, %eax
-; X86-NEXT:    movl 84(%esp,%eax), %ebx
+; X86-NEXT:    movl 84(%esp,%eax), %ebp
 ; X86-NEXT:    movl %esi, %edi
-; X86-NEXT:    movl %edx, %esi
-; X86-NEXT:    movl 80(%esp,%eax), %edx
-; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edx, %esi
+; X86-NEXT:    movl 80(%esp,%eax), %ebx
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %ebx, %esi
 ; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    shrdl %cl, %ebx, %esi
+; X86-NEXT:    shrdl %cl, %ebp, %esi
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl 72(%esp,%eax), %ebp
+; X86-NEXT:    movl 72(%esp,%eax), %esi
 ; X86-NEXT:    movl 76(%esp,%eax), %eax
-; X86-NEXT:    movl %eax, %esi
-; X86-NEXT:    shrl %cl, %esi
+; X86-NEXT:    movl %eax, %edx
+; X86-NEXT:    shrl %cl, %edx
 ; X86-NEXT:    notb %cl
-; X86-NEXT:    addl %edx, %edx
-; X86-NEXT:    shll %cl, %edx
-; X86-NEXT:    orl %esi, %edx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    addl %ebx, %ebx
+; X86-NEXT:    shll %cl, %ebx
+; X86-NEXT:    orl %edx, %ebx
 ; X86-NEXT:    movb %ch, %cl
-; X86-NEXT:    shrl %cl, %ebx
-; X86-NEXT:    movl %ebx, %edx
-; X86-NEXT:    shrdl %cl, %eax, %ebp
-; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    shrl %cl, %ebp
+; X86-NEXT:    shrdl %cl, %eax, %esi
+; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    addl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    adcl $-1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    adcl $-1, %ecx
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X86-NEXT:    adcl $-1, %esi
 ; X86-NEXT:    movl %esi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    xorl %ecx, %ecx
 ; X86-NEXT:    movl $0, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Spill
 ; X86-NEXT:    movl (%esp), %esi # 4-byte Reload
 ; X86-NEXT:    .p2align 4, 0x90
 ; X86-NEXT:  .LBB4_4: # %udiv-do-while
 ; X86-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebx, %edx
-; X86-NEXT:    movl %edx, (%esp) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
-; X86-NEXT:    shldl $1, %edx, %ebx
+; X86-NEXT:    shldl $1, %edx, %ebp
+; X86-NEXT:    movl %ebp, (%esp) # 4-byte Spill
+; X86-NEXT:    shldl $1, %ebx, %edx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload
-; X86-NEXT:    shldl $1, %ebp, %edx
+; X86-NEXT:    shldl $1, %ebp, %ebx
 ; X86-NEXT:    shldl $1, %esi, %ebp
 ; X86-NEXT:    shldl $1, %edi, %esi
 ; X86-NEXT:    orl %ecx, %esi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    shldl $1, %eax, %edi
 ; X86-NEXT:    orl %ecx, %edi
 ; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
 ; X86-NEXT:    shldl $1, %edi, %eax
 ; X86-NEXT:    orl %ecx, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    addl %edi, %edi
 ; X86-NEXT:    orl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
 ; X86-NEXT:    movl %edi, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    cmpl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Folded Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
-; X86-NEXT:    sbbl %edx, %ecx
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    sbbl %ebx, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
+; X86-NEXT:    sbbl %edx, %ecx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    sbbl (%esp), %ecx # 4-byte Folded Reload
 ; X86-NEXT:    sarl $31, %ecx
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    andl $1, %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ecx, %edi
 ; X86-NEXT:    andl {{[0-9]+}}(%esp), %edi
 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    andl {{[0-9]+}}(%esp), %ecx
 ; X86-NEXT:    subl %ecx, %ebp
 ; X86-NEXT:    movl %ebp, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %eax, %edx
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    sbbl %edi, %ebx
+; X86-NEXT:    sbbl %eax, %ebx
 ; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    sbbl %edi, %edx
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    sbbl %eax, (%esp) # 4-byte Folded Spill
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    addl $-1, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %eax # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %eax
-; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
-; X86-NEXT:    adcl $-1, %ebx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %edx # 4-byte Reload
 ; X86-NEXT:    adcl $-1, %edx
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT:    adcl $-1, %ebx
 ; X86-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %edx, %eax
-; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
 ; X86-NEXT:    movl %ebx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; X86-NEXT:    orl %ebx, %ecx
-; X86-NEXT:    movl (%esp), %edx # 4-byte Reload
+; X86-NEXT:    orl %ebx, %eax
+; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
+; X86-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    movl %edx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; X86-NEXT:    orl %edx, %ecx
+; X86-NEXT:    movl (%esp), %ebp # 4-byte Reload
 ; X86-NEXT:    orl %eax, %ecx
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; X86-NEXT:    jne .LBB4_4
 ; X86-NEXT:  # %bb.5:
 ; X86-NEXT:    movl %esi, (%esp) # 4-byte Spill
 ; X86-NEXT:    movl %edi, %esi
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebx # 4-byte Reload
 ; X86-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ebp # 4-byte Reload

llvm/test/CodeGen/X86/fast-isel-freeze.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s                               -mtriple=x86_64-unknown-linux | FileCheck %s --check-prefix=SDAG
 ; RUN: llc < %s -fast-isel -fast-isel-abort=1 -mtriple=x86_64-unknown-linux | FileCheck %s --check-prefix=FAST
 
 define i32 @freeze(i32 %t) {
 ; SDAG-LABEL: freeze:
 ; SDAG:       # %bb.0:
 ; SDAG-NEXT:    movl %edi, %eax
 ; SDAG-NEXT:    xorl $10, %eax
 ; SDAG-NEXT:    retq
 ;
 ; FAST-LABEL: freeze:
 ; FAST:       # %bb.0:
-; FAST-NEXT:    movl $10, %eax
-; FAST-NEXT:    xorl %edi, %eax
+; FAST-NEXT:    movl %edi, %eax
+; FAST-NEXT:    xorl $10, %eax
 ; FAST-NEXT:    retq
   %1 = freeze i32 %t
   %2 = freeze i32 10
   %3 = xor i32 %1, %2
   ret i32 %3
 }

llvm/test/CodeGen/X86/foldimmediate-size.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc < %s -mtriple=x86_64-unknown | FileCheck %s
+
+; When optimize for size, the constant $858993459 is moved into a register,
+; and use that register in following two andl instructions.
+
+define i32 @cnt32_optsize(i32 %x) nounwind readnone optsize {
+; CHECK-LABEL: cnt32_optsize:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    shrl %eax
+; CHECK-NEXT:    andl $1431655765, %eax # imm = 0x55555555
+; CHECK-NEXT:    subl %eax, %edi
+; CHECK-NEXT:    movl $858993459, %eax # imm = 0x33333333
+; CHECK-NEXT:    movl %edi, %ecx
+; CHECK-NEXT:    andl %eax, %ecx
+; CHECK-NEXT:    shrl $2, %edi
+; CHECK-NEXT:    andl %eax, %edi
+; CHECK-NEXT:    addl %ecx, %edi
+; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    shrl $4, %eax
+; CHECK-NEXT:    addl %edi, %eax
+; CHECK-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
+; CHECK-NEXT:    imull $16843009, %eax, %eax # imm = 0x1010101
+; CHECK-NEXT:    shrl $24, %eax
+; CHECK-NEXT:    retq
+  %cnt = tail call i32 @llvm.ctpop.i32(i32 %x)
+  ret i32 %cnt
+}
+
+; When optimize for speed, the constant $858993459 can be directly folded into
+; two andl instructions.
+
+define i32 @cnt32_optspeed(i32 %x) nounwind readnone {
+; CHECK-LABEL: cnt32_optspeed:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    shrl %eax
+; CHECK-NEXT:    andl $1431655765, %eax # imm = 0x55555555
+; CHECK-NEXT:    subl %eax, %edi
+; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    andl $858993459, %eax # imm = 0x33333333
+; CHECK-NEXT:    shrl $2, %edi
+; CHECK-NEXT:    andl $858993459, %edi # imm = 0x33333333
+; CHECK-NEXT:    addl %eax, %edi
+; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    shrl $4, %eax
+; CHECK-NEXT:    addl %edi, %eax
+; CHECK-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
+; CHECK-NEXT:    imull $16843009, %eax, %eax # imm = 0x1010101
+; CHECK-NEXT:    shrl $24, %eax
+; CHECK-NEXT:    retq
+  %cnt = tail call i32 @llvm.ctpop.i32(i32 %x)
+  ret i32 %cnt
+}
+
+declare i32 @llvm.ctpop.i32(i32) nounwind readnone

llvm/test/CodeGen/X86/foldimmediate.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 3
+# RUN: llc -mtriple=x86_64-- -run-pass=peephole-opt %s -o - | FileCheck %s
+--- |
+  define void @foldImmediate() { ret void }
+...
+---
+# Check that immediates can be folded into ALU instructions.
+name: foldImmediate
+registers:
+  - { id: 0, class: gr32 }
+  - { id: 1, class: gr32 }
+  - { id: 2, class: gr32 }
+  - { id: 3, class: gr32 }
+  - { id: 4, class: gr32 }
+  - { id: 5, class: gr32 }
+  - { id: 6, class: gr32 }
+  - { id: 7, class: gr64 }
+  - { id: 8, class: gr64 }
+  - { id: 9, class: gr64 }
+  - { id: 10, class: gr64 }
+  - { id: 11, class: gr64 }
+  - { id: 12, class: gr64 }
+  - { id: 13, class: gr64 }
+  - { id: 14, class: gr64 }
+  - { id: 15, class: gr64 }
+  - { id: 16, class: gr32 }
+  - { id: 17, class: gr64 }
+  - { id: 18, class: gr32 }
+
+body: |
+  bb.0:
+    liveins: $rdi, $rsi
+
+    ; CHECK-LABEL: name: foldImmediate
+    ; CHECK: liveins: $rdi, $rsi
+    ; CHECK-NEXT: {{  $}}
+    ; CHECK-NEXT: [[MOV32ri:%[0-9]+]]:gr32 = MOV32ri 81
+    ; CHECK-NEXT: [[COPY:%[0-9]+]]:gr32 = COPY $edi
+    ; CHECK-NEXT: [[ADD32ri:%[0-9]+]]:gr32 = ADD32ri [[COPY]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[ADD32ri]]
+    ; CHECK-NEXT: [[SUB32ri:%[0-9]+]]:gr32 = SUB32ri [[COPY]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[SUB32ri]]
+    ; CHECK-NEXT: [[AND32ri:%[0-9]+]]:gr32 = AND32ri [[COPY]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[AND32ri]]
+    ; CHECK-NEXT: [[OR32ri:%[0-9]+]]:gr32 = OR32ri [[COPY]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[OR32ri]]
+    ; CHECK-NEXT: [[XOR32ri:%[0-9]+]]:gr32 = XOR32ri [[COPY]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[XOR32ri]]
+    ; CHECK-NEXT: TEST32ri [[COPY]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit $eflags
+    ; CHECK-NEXT: CMP32ri [[COPY]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit $eflags
+    ; CHECK-NEXT: [[ADC32ri:%[0-9]+]]:gr32 = ADC32ri [[COPY]], 81, implicit-def $eflags, implicit $eflags
+    ; CHECK-NEXT: NOOP implicit [[ADC32ri]]
+    ; CHECK-NEXT: [[SBB32ri:%[0-9]+]]:gr32 = SBB32ri [[COPY]], 81, implicit-def $eflags, implicit $eflags
+    ; CHECK-NEXT: NOOP implicit [[SBB32ri]]
+    ; CHECK-NEXT: [[SUBREG_TO_REG:%[0-9]+]]:gr64 = SUBREG_TO_REG 0, killed [[MOV32ri]], %subreg.sub_32bit
+    ; CHECK-NEXT: [[COPY1:%[0-9]+]]:gr64 = COPY $rsi
+    ; CHECK-NEXT: [[ADD64ri32_:%[0-9]+]]:gr64 = ADD64ri32 [[COPY1]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[ADD64ri32_]]
+    ; CHECK-NEXT: [[SUB64ri32_:%[0-9]+]]:gr64 = SUB64ri32 [[COPY1]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[SUB64ri32_]]
+    ; CHECK-NEXT: [[AND64ri32_:%[0-9]+]]:gr64 = AND64ri32 [[COPY1]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[AND64ri32_]]
+    ; CHECK-NEXT: [[OR64ri32_:%[0-9]+]]:gr64 = OR64ri32 [[COPY1]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[OR64ri32_]]
+    ; CHECK-NEXT: [[XOR64ri32_:%[0-9]+]]:gr64 = XOR64ri32 [[COPY1]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit [[XOR64ri32_]]
+    ; CHECK-NEXT: [[MOV32ri64_:%[0-9]+]]:gr64 = MOV32ri64 81
+    ; CHECK-NEXT: NOOP implicit [[MOV32ri64_]]
+    ; CHECK-NEXT: TEST64ri32 [[COPY1]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit $eflags
+    ; CHECK-NEXT: [[ADC64ri32_:%[0-9]+]]:gr64 = ADC64ri32 [[COPY1]], 81, implicit-def $eflags, implicit $eflags
+    ; CHECK-NEXT: NOOP implicit [[ADC64ri32_]]
+    ; CHECK-NEXT: [[SBB64ri32_:%[0-9]+]]:gr64 = SBB64ri32 [[COPY1]], 81, implicit-def $eflags, implicit $eflags
+    ; CHECK-NEXT: NOOP implicit [[SBB64ri32_]]
+    ; CHECK-NEXT: CMP64ri32 [[COPY1]], 81, implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit $eflags
+    ; CHECK-NEXT: CMP64rr [[SUBREG_TO_REG]], [[COPY1]], implicit-def $eflags
+    ; CHECK-NEXT: NOOP implicit $eflags
+    %0 = MOV32ri 81
+    %1 = COPY $edi
+    %2 = ADD32rr %0, %1, implicit-def $eflags
+    NOOP implicit %2
+
+    %3 = SUB32rr %1, %0, implicit-def $eflags
+    NOOP implicit %3
+
+    %4 = AND32rr %0, %1, implicit-def $eflags
+    NOOP implicit %4
+
+    %5 = OR32rr %0, %1, implicit-def $eflags
+    NOOP implicit %5
+
+    %6 = XOR32rr %0, %1, implicit-def $eflags
+    NOOP implicit %6
+
+    TEST32rr %0, %1, implicit-def $eflags
+    NOOP implicit $eflags
+
+    CMP32rr %1, %0, implicit-def $eflags
+    NOOP implicit $eflags
+
+    %16 = ADC32rr %0, %1, implicit-def $eflags, implicit $eflags
+    NOOP implicit %16
+
+    %18 = SBB32rr %1, %0, implicit-def $eflags, implicit $eflags
+    NOOP implicit %18
+
+    %7 = SUBREG_TO_REG 0, killed %0:gr32, %subreg.sub_32bit
+    %8 = COPY $rsi
+    %9 = ADD64rr %7, %8, implicit-def $eflags
+    NOOP implicit %9
+
+    %10 = SUB64rr %8, %7, implicit-def $eflags
+    NOOP implicit %10
+
+    %11 = AND64rr %8, %7, implicit-def $eflags
+    NOOP implicit %11
+
+    %12 = OR64rr %8, %7, implicit-def $eflags
+    NOOP implicit %12
+
+    %13 = XOR64rr %8, %7, implicit-def $eflags
+    NOOP implicit %13
+
+    %14 = COPY %7
+    NOOP implicit %14
+
+    TEST64rr %8, %7, implicit-def $eflags
+    NOOP implicit $eflags
+
+    %15 = ADC64rr %8, %7, implicit-def $eflags, implicit $eflags
+    NOOP implicit %15
+
+    %17 = SBB64rr %8, %7, implicit-def $eflags, implicit $eflags
+    NOOP implicit %17
+
+    CMP64rr %8, %7, implicit-def $eflags
+    NOOP implicit $eflags
+    CMP64rr %7, %8, implicit-def $eflags
+    NOOP implicit $eflags
+...

llvm/test/CodeGen/X86/isel-icmp.ll

 ; FAST-X86-NEXT:    andb $1, %al
 ; FAST-X86-NEXT:    movzbl %al, %eax
 ; FAST-X86-NEXT:    retl
 ;
 ; GISEL-X86-LABEL: test_icmp_eq_i64:
 ; GISEL-X86:       ## %bb.0:
 ; GISEL-X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; GISEL-X86-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; GISEL-X86-NEXT:    xorl %edx, %edx
 ; GISEL-X86-NEXT:    xorl {{[0-9]+}}(%esp), %eax
 ; GISEL-X86-NEXT:    xorl {{[0-9]+}}(%esp), %ecx
 ; GISEL-X86-NEXT:    orl %eax, %ecx
 ; GISEL-X86-NEXT:    xorl %eax, %eax
-; GISEL-X86-NEXT:    cmpl %edx, %ecx
+; GISEL-X86-NEXT:    cmpl $0, %ecx
 ; GISEL-X86-NEXT:    sete %al
 ; GISEL-X86-NEXT:    andl $1, %eax
 ; GISEL-X86-NEXT:    retl
   %r = icmp eq i64 %a, %b
   %res =  zext i1 %r to i32
   ret i32 %res
 }
 

llvm/test/CodeGen/X86/pcsections-atomics.ll

 ; O0-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic8_cas_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movb $1, %cl
-; O1-NEXT:    movb $42, %al
 ; O1-NEXT:  .Lpcsection65:
-; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O1-NEXT:    movb $42, %al
 ; O1-NEXT:  .Lpcsection66:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
-; O1-NEXT:    movb $42, %al
 ; O1-NEXT:  .Lpcsection67:
+; O1-NEXT:    movb $42, %al
+; O1-NEXT:  .Lpcsection68:
+; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection69:
+; O1-NEXT:    movb $42, %al
+; O1-NEXT:  .Lpcsection70:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic8_cas_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movb $1, %cl
-; O2-NEXT:    movb $42, %al
 ; O2-NEXT:  .Lpcsection65:
-; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O2-NEXT:    movb $42, %al
 ; O2-NEXT:  .Lpcsection66:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
-; O2-NEXT:    movb $42, %al
 ; O2-NEXT:  .Lpcsection67:
+; O2-NEXT:    movb $42, %al
+; O2-NEXT:  .Lpcsection68:
+; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection69:
+; O2-NEXT:    movb $42, %al
+; O2-NEXT:  .Lpcsection70:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic8_cas_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movb $1, %cl
-; O3-NEXT:    movb $42, %al
 ; O3-NEXT:  .Lpcsection65:
-; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O3-NEXT:    movb $42, %al
 ; O3-NEXT:  .Lpcsection66:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
-; O3-NEXT:    movb $42, %al
 ; O3-NEXT:  .Lpcsection67:
+; O3-NEXT:    movb $42, %al
+; O3-NEXT:  .Lpcsection68:
+; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection69:
+; O3-NEXT:    movb $42, %al
+; O3-NEXT:  .Lpcsection70:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i8 42, i8 1 monotonic monotonic, align 1, !pcsections !0
   %y = cmpxchg ptr %a, i8 42, i8 1 monotonic acquire, align 1, !pcsections !0
   %z = cmpxchg ptr %a, i8 42, i8 1 monotonic seq_cst, align 1, !pcsections !0

 ; O0-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic8_cas_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movb $1, %cl
+; O1-NEXT:  .Lpcsection71:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection68:
+; O1-NEXT:  .Lpcsection72:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection73:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection69:
+; O1-NEXT:  .Lpcsection74:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection75:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection70:
+; O1-NEXT:  .Lpcsection76:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic8_cas_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movb $1, %cl
+; O2-NEXT:  .Lpcsection71:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection68:
+; O2-NEXT:  .Lpcsection72:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection73:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection69:
+; O2-NEXT:  .Lpcsection74:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection75:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection70:
+; O2-NEXT:  .Lpcsection76:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic8_cas_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movb $1, %cl
+; O3-NEXT:  .Lpcsection71:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection68:
+; O3-NEXT:  .Lpcsection72:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection73:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection69:
+; O3-NEXT:  .Lpcsection74:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection75:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection70:
+; O3-NEXT:  .Lpcsection76:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i8 42, i8 1 acquire monotonic, align 1, !pcsections !0
   %y = cmpxchg ptr %a, i8 42, i8 1 acquire acquire, align 1, !pcsections !0
   %z = cmpxchg ptr %a, i8 42, i8 1 acquire seq_cst, align 1, !pcsections !0

 ; O0-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic8_cas_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movb $1, %cl
+; O1-NEXT:  .Lpcsection77:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection71:
+; O1-NEXT:  .Lpcsection78:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection79:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection72:
+; O1-NEXT:  .Lpcsection80:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection81:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection73:
+; O1-NEXT:  .Lpcsection82:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic8_cas_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movb $1, %cl
+; O2-NEXT:  .Lpcsection77:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection71:
+; O2-NEXT:  .Lpcsection78:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection79:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection72:
+; O2-NEXT:  .Lpcsection80:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection81:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection73:
+; O2-NEXT:  .Lpcsection82:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic8_cas_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movb $1, %cl
+; O3-NEXT:  .Lpcsection77:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection71:
+; O3-NEXT:  .Lpcsection78:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection79:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection72:
+; O3-NEXT:  .Lpcsection80:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection81:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection73:
+; O3-NEXT:  .Lpcsection82:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i8 42, i8 1 release monotonic, align 1, !pcsections !0
   %y = cmpxchg ptr %a, i8 42, i8 1 release acquire, align 1, !pcsections !0
   %z = cmpxchg ptr %a, i8 42, i8 1 release seq_cst, align 1, !pcsections !0

 ; O0-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic8_cas_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movb $1, %cl
+; O1-NEXT:  .Lpcsection83:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection74:
+; O1-NEXT:  .Lpcsection84:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection85:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection75:
+; O1-NEXT:  .Lpcsection86:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection87:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection76:
+; O1-NEXT:  .Lpcsection88:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic8_cas_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movb $1, %cl
+; O2-NEXT:  .Lpcsection83:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection74:
+; O2-NEXT:  .Lpcsection84:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection85:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection75:
+; O2-NEXT:  .Lpcsection86:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection87:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection76:
+; O2-NEXT:  .Lpcsection88:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic8_cas_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movb $1, %cl
+; O3-NEXT:  .Lpcsection83:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection74:
+; O3-NEXT:  .Lpcsection84:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection85:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection75:
+; O3-NEXT:  .Lpcsection86:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection87:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection76:
+; O3-NEXT:  .Lpcsection88:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i8 42, i8 1 acq_rel monotonic, align 1, !pcsections !0
   %y = cmpxchg ptr %a, i8 42, i8 1 acq_rel acquire, align 1, !pcsections !0
   %z = cmpxchg ptr %a, i8 42, i8 1 acq_rel seq_cst, align 1, !pcsections !0

 ; O0-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic8_cas_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movb $1, %cl
+; O1-NEXT:  .Lpcsection89:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection77:
+; O1-NEXT:  .Lpcsection90:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection91:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection78:
+; O1-NEXT:  .Lpcsection92:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O1-NEXT:  .Lpcsection93:
 ; O1-NEXT:    movb $42, %al
-; O1-NEXT:  .Lpcsection79:
+; O1-NEXT:  .Lpcsection94:
 ; O1-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic8_cas_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movb $1, %cl
+; O2-NEXT:  .Lpcsection89:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection77:
+; O2-NEXT:  .Lpcsection90:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection91:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection78:
+; O2-NEXT:  .Lpcsection92:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O2-NEXT:  .Lpcsection93:
 ; O2-NEXT:    movb $42, %al
-; O2-NEXT:  .Lpcsection79:
+; O2-NEXT:  .Lpcsection94:
 ; O2-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic8_cas_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movb $1, %cl
+; O3-NEXT:  .Lpcsection89:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection77:
+; O3-NEXT:  .Lpcsection90:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection91:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection78:
+; O3-NEXT:  .Lpcsection92:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
+; O3-NEXT:  .Lpcsection93:
 ; O3-NEXT:    movb $42, %al
-; O3-NEXT:  .Lpcsection79:
+; O3-NEXT:  .Lpcsection94:
 ; O3-NEXT:    lock cmpxchgb %cl, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i8 42, i8 1 seq_cst monotonic, align 1, !pcsections !0
   %y = cmpxchg ptr %a, i8 42, i8 1 seq_cst acquire, align 1, !pcsections !0
   %z = cmpxchg ptr %a, i8 42, i8 1 seq_cst seq_cst, align 1, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define i16 @atomic16_load_unordered(ptr %a) {
 ; O0-LABEL: atomic16_load_unordered:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection100:
 ; O0-NEXT:    movw (%rdi), %ax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_load_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection80:
+; O1-NEXT:  .Lpcsection95:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_load_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection80:
+; O2-NEXT:  .Lpcsection95:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_load_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection80:
+; O3-NEXT:  .Lpcsection95:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i16, ptr %a unordered, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i16 %x
 }
 
 define i16 @atomic16_load_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_load_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection101:
 ; O0-NEXT:    movw (%rdi), %ax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_load_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection81:
+; O1-NEXT:  .Lpcsection96:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_load_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection81:
+; O2-NEXT:  .Lpcsection96:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_load_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection81:
+; O3-NEXT:  .Lpcsection96:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i16, ptr %a monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i16 %x
 }
 
 define i16 @atomic16_load_acquire(ptr %a) {
 ; O0-LABEL: atomic16_load_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection102:
 ; O0-NEXT:    movw (%rdi), %ax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_load_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection82:
+; O1-NEXT:  .Lpcsection97:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_load_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection82:
+; O2-NEXT:  .Lpcsection97:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_load_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection82:
+; O3-NEXT:  .Lpcsection97:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i16, ptr %a acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i16 %x
 }
 
 define i16 @atomic16_load_seq_cst(ptr %a) {
 ; O0-LABEL: atomic16_load_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection103:
 ; O0-NEXT:    movw (%rdi), %ax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_load_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection83:
+; O1-NEXT:  .Lpcsection98:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_load_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection83:
+; O2-NEXT:  .Lpcsection98:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_load_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection83:
+; O3-NEXT:  .Lpcsection98:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i16, ptr %a seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i16 %x
 }
 
 define void @atomic16_store_unordered(ptr %a) {
 ; O0-LABEL: atomic16_store_unordered:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection104:
 ; O0-NEXT:    movw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_store_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection84:
+; O1-NEXT:  .Lpcsection99:
 ; O1-NEXT:    movw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_store_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection84:
+; O2-NEXT:  .Lpcsection99:
 ; O2-NEXT:    movw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_store_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection84:
+; O3-NEXT:  .Lpcsection99:
 ; O3-NEXT:    movw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i16 42, ptr %a unordered, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_store_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_store_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection105:
 ; O0-NEXT:    movw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_store_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection85:
+; O1-NEXT:  .Lpcsection100:
 ; O1-NEXT:    movw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_store_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection85:
+; O2-NEXT:  .Lpcsection100:
 ; O2-NEXT:    movw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_store_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection85:
+; O3-NEXT:  .Lpcsection100:
 ; O3-NEXT:    movw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i16 42, ptr %a monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_store_release(ptr %a) {
 ; O0-LABEL: atomic16_store_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection106:
 ; O0-NEXT:    movw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_store_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection86:
+; O1-NEXT:  .Lpcsection101:
 ; O1-NEXT:    movw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_store_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection86:
+; O2-NEXT:  .Lpcsection101:
 ; O2-NEXT:    movw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_store_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection86:
+; O3-NEXT:  .Lpcsection101:
 ; O3-NEXT:    movw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i16 42, ptr %a release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_store_seq_cst(ptr %a) {
 ; O0-LABEL: atomic16_store_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:    movw $42, %ax
 ; O0-NEXT:  .Lpcsection107:
 ; O0-NEXT:    xchgw %ax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_store_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection87:
+; O1-NEXT:  .Lpcsection102:
 ; O1-NEXT:    xchgw %ax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_store_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection87:
+; O2-NEXT:  .Lpcsection102:
 ; O2-NEXT:    xchgw %ax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_store_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection87:
+; O3-NEXT:  .Lpcsection102:
 ; O3-NEXT:    xchgw %ax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i16 42, ptr %a seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_xchg_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_xchg_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:    movw $42, %ax
 ; O0-NEXT:  .Lpcsection108:
 ; O0-NEXT:    xchgw %ax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xchg_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection88:
+; O1-NEXT:  .Lpcsection103:
 ; O1-NEXT:    xchgw %ax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xchg_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection88:
+; O2-NEXT:  .Lpcsection103:
 ; O2-NEXT:    xchgw %ax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xchg_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection88:
+; O3-NEXT:  .Lpcsection103:
 ; O3-NEXT:    xchgw %ax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i16 42 monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_add_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_add_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection109:
 ; O0-NEXT:    lock addw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_add_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection89:
+; O1-NEXT:  .Lpcsection104:
 ; O1-NEXT:    lock addw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_add_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection89:
+; O2-NEXT:  .Lpcsection104:
 ; O2-NEXT:    lock addw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_add_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection89:
+; O3-NEXT:  .Lpcsection104:
 ; O3-NEXT:    lock addw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i16 42 monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_sub_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_sub_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection110:
 ; O0-NEXT:    lock subw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_sub_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection90:
+; O1-NEXT:  .Lpcsection105:
 ; O1-NEXT:    lock subw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_sub_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection90:
+; O2-NEXT:  .Lpcsection105:
 ; O2-NEXT:    lock subw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_sub_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection90:
+; O3-NEXT:  .Lpcsection105:
 ; O3-NEXT:    lock subw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i16 42 monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_and_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_and_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection111:
 ; O0-NEXT:    lock andw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_and_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection91:
+; O1-NEXT:  .Lpcsection106:
 ; O1-NEXT:    lock andw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_and_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection91:
+; O2-NEXT:  .Lpcsection106:
 ; O2-NEXT:    lock andw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_and_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection91:
+; O3-NEXT:  .Lpcsection106:
 ; O3-NEXT:    lock andw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i16 42 monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_or_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_or_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection112:
 ; O0-NEXT:    lock orw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_or_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection92:
+; O1-NEXT:  .Lpcsection107:
 ; O1-NEXT:    lock orw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_or_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection92:
+; O2-NEXT:  .Lpcsection107:
 ; O2-NEXT:    lock orw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_or_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection92:
+; O3-NEXT:  .Lpcsection107:
 ; O3-NEXT:    lock orw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i16 42 monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_xor_monotonic(ptr %a) {
 ; O0-LABEL: atomic16_xor_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection113:
 ; O0-NEXT:    lock xorw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xor_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection93:
+; O1-NEXT:  .Lpcsection108:
 ; O1-NEXT:    lock xorw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xor_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection93:
+; O2-NEXT:  .Lpcsection108:
 ; O2-NEXT:    lock xorw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xor_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection93:
+; O3-NEXT:  .Lpcsection108:
 ; O3-NEXT:    lock xorw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i16 42 monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB64_1
 ; O0-NEXT:  .LBB64_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_nand_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection94:
+; O1-NEXT:  .Lpcsection109:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB64_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection95:
+; O1-NEXT:  .Lpcsection110:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection96:
+; O1-NEXT:  .Lpcsection111:
 ; O1-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O1-NEXT:  .Lpcsection97:
+; O1-NEXT:  .Lpcsection112:
 ; O1-NEXT:    # kill: def $ax killed $ax killed $eax
-; O1-NEXT:  .Lpcsection98:
+; O1-NEXT:  .Lpcsection113:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O1-NEXT:  .Lpcsection99:
+; O1-NEXT:  .Lpcsection114:
 ; O1-NEXT:    # kill: def $ax killed $ax def $eax
-; O1-NEXT:  .Lpcsection100:
+; O1-NEXT:  .Lpcsection115:
 ; O1-NEXT:    jne .LBB64_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_nand_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection94:
+; O2-NEXT:  .Lpcsection109:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB64_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection95:
+; O2-NEXT:  .Lpcsection110:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection96:
+; O2-NEXT:  .Lpcsection111:
 ; O2-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O2-NEXT:  .Lpcsection97:
+; O2-NEXT:  .Lpcsection112:
 ; O2-NEXT:    # kill: def $ax killed $ax killed $eax
-; O2-NEXT:  .Lpcsection98:
+; O2-NEXT:  .Lpcsection113:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O2-NEXT:  .Lpcsection99:
+; O2-NEXT:  .Lpcsection114:
 ; O2-NEXT:    # kill: def $ax killed $ax def $eax
-; O2-NEXT:  .Lpcsection100:
+; O2-NEXT:  .Lpcsection115:
 ; O2-NEXT:    jne .LBB64_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_nand_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection94:
+; O3-NEXT:  .Lpcsection109:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB64_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection95:
+; O3-NEXT:  .Lpcsection110:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection96:
+; O3-NEXT:  .Lpcsection111:
 ; O3-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O3-NEXT:  .Lpcsection97:
+; O3-NEXT:  .Lpcsection112:
 ; O3-NEXT:    # kill: def $ax killed $ax killed $eax
-; O3-NEXT:  .Lpcsection98:
+; O3-NEXT:  .Lpcsection113:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O3-NEXT:  .Lpcsection99:
+; O3-NEXT:  .Lpcsection114:
 ; O3-NEXT:    # kill: def $ax killed $ax def $eax
-; O3-NEXT:  .Lpcsection100:
+; O3-NEXT:  .Lpcsection115:
 ; O3-NEXT:    jne .LBB64_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i16 42 monotonic, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgw %ax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xchg_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection101:
+; O1-NEXT:  .Lpcsection116:
 ; O1-NEXT:    xchgw %ax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xchg_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection101:
+; O2-NEXT:  .Lpcsection116:
 ; O2-NEXT:    xchgw %ax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xchg_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection101:
+; O3-NEXT:  .Lpcsection116:
 ; O3-NEXT:    xchgw %ax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i16 42 acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_add_acquire(ptr %a) {
 ; O0-LABEL: atomic16_add_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection124:
 ; O0-NEXT:    lock addw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_add_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection102:
+; O1-NEXT:  .Lpcsection117:
 ; O1-NEXT:    lock addw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_add_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection102:
+; O2-NEXT:  .Lpcsection117:
 ; O2-NEXT:    lock addw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_add_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection102:
+; O3-NEXT:  .Lpcsection117:
 ; O3-NEXT:    lock addw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i16 42 acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_sub_acquire(ptr %a) {
 ; O0-LABEL: atomic16_sub_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection125:
 ; O0-NEXT:    lock subw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_sub_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection103:
+; O1-NEXT:  .Lpcsection118:
 ; O1-NEXT:    lock subw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_sub_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection103:
+; O2-NEXT:  .Lpcsection118:
 ; O2-NEXT:    lock subw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_sub_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection103:
+; O3-NEXT:  .Lpcsection118:
 ; O3-NEXT:    lock subw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i16 42 acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_and_acquire(ptr %a) {
 ; O0-LABEL: atomic16_and_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection126:
 ; O0-NEXT:    lock andw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_and_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection104:
+; O1-NEXT:  .Lpcsection119:
 ; O1-NEXT:    lock andw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_and_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection104:
+; O2-NEXT:  .Lpcsection119:
 ; O2-NEXT:    lock andw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_and_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection104:
+; O3-NEXT:  .Lpcsection119:
 ; O3-NEXT:    lock andw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i16 42 acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_or_acquire(ptr %a) {
 ; O0-LABEL: atomic16_or_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection127:
 ; O0-NEXT:    lock orw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_or_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection105:
+; O1-NEXT:  .Lpcsection120:
 ; O1-NEXT:    lock orw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_or_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection105:
+; O2-NEXT:  .Lpcsection120:
 ; O2-NEXT:    lock orw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_or_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection105:
+; O3-NEXT:  .Lpcsection120:
 ; O3-NEXT:    lock orw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i16 42 acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_xor_acquire(ptr %a) {
 ; O0-LABEL: atomic16_xor_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection128:
 ; O0-NEXT:    lock xorw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xor_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection106:
+; O1-NEXT:  .Lpcsection121:
 ; O1-NEXT:    lock xorw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xor_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection106:
+; O2-NEXT:  .Lpcsection121:
 ; O2-NEXT:    lock xorw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xor_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection106:
+; O3-NEXT:  .Lpcsection121:
 ; O3-NEXT:    lock xorw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i16 42 acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB71_1
 ; O0-NEXT:  .LBB71_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_nand_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection107:
+; O1-NEXT:  .Lpcsection122:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB71_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection108:
+; O1-NEXT:  .Lpcsection123:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection109:
+; O1-NEXT:  .Lpcsection124:
 ; O1-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O1-NEXT:  .Lpcsection110:
+; O1-NEXT:  .Lpcsection125:
 ; O1-NEXT:    # kill: def $ax killed $ax killed $eax
-; O1-NEXT:  .Lpcsection111:
+; O1-NEXT:  .Lpcsection126:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O1-NEXT:  .Lpcsection112:
+; O1-NEXT:  .Lpcsection127:
 ; O1-NEXT:    # kill: def $ax killed $ax def $eax
-; O1-NEXT:  .Lpcsection113:
+; O1-NEXT:  .Lpcsection128:
 ; O1-NEXT:    jne .LBB71_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_nand_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection107:
+; O2-NEXT:  .Lpcsection122:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB71_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection108:
+; O2-NEXT:  .Lpcsection123:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection109:
+; O2-NEXT:  .Lpcsection124:
 ; O2-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O2-NEXT:  .Lpcsection110:
+; O2-NEXT:  .Lpcsection125:
 ; O2-NEXT:    # kill: def $ax killed $ax killed $eax
-; O2-NEXT:  .Lpcsection111:
+; O2-NEXT:  .Lpcsection126:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O2-NEXT:  .Lpcsection112:
+; O2-NEXT:  .Lpcsection127:
 ; O2-NEXT:    # kill: def $ax killed $ax def $eax
-; O2-NEXT:  .Lpcsection113:
+; O2-NEXT:  .Lpcsection128:
 ; O2-NEXT:    jne .LBB71_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_nand_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection107:
+; O3-NEXT:  .Lpcsection122:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB71_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection108:
+; O3-NEXT:  .Lpcsection123:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection109:
+; O3-NEXT:  .Lpcsection124:
 ; O3-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O3-NEXT:  .Lpcsection110:
+; O3-NEXT:  .Lpcsection125:
 ; O3-NEXT:    # kill: def $ax killed $ax killed $eax
-; O3-NEXT:  .Lpcsection111:
+; O3-NEXT:  .Lpcsection126:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O3-NEXT:  .Lpcsection112:
+; O3-NEXT:  .Lpcsection127:
 ; O3-NEXT:    # kill: def $ax killed $ax def $eax
-; O3-NEXT:  .Lpcsection113:
+; O3-NEXT:  .Lpcsection128:
 ; O3-NEXT:    jne .LBB71_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i16 42 acquire, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgw %ax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xchg_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection114:
+; O1-NEXT:  .Lpcsection129:
 ; O1-NEXT:    xchgw %ax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xchg_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection114:
+; O2-NEXT:  .Lpcsection129:
 ; O2-NEXT:    xchgw %ax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xchg_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection114:
+; O3-NEXT:  .Lpcsection129:
 ; O3-NEXT:    xchgw %ax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i16 42 release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_add_release(ptr %a) {
 ; O0-LABEL: atomic16_add_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection139:
 ; O0-NEXT:    lock addw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_add_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection115:
+; O1-NEXT:  .Lpcsection130:
 ; O1-NEXT:    lock addw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_add_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection115:
+; O2-NEXT:  .Lpcsection130:
 ; O2-NEXT:    lock addw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_add_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection115:
+; O3-NEXT:  .Lpcsection130:
 ; O3-NEXT:    lock addw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i16 42 release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_sub_release(ptr %a) {
 ; O0-LABEL: atomic16_sub_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection140:
 ; O0-NEXT:    lock subw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_sub_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection116:
+; O1-NEXT:  .Lpcsection131:
 ; O1-NEXT:    lock subw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_sub_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection116:
+; O2-NEXT:  .Lpcsection131:
 ; O2-NEXT:    lock subw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_sub_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection116:
+; O3-NEXT:  .Lpcsection131:
 ; O3-NEXT:    lock subw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i16 42 release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_and_release(ptr %a) {
 ; O0-LABEL: atomic16_and_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection141:
 ; O0-NEXT:    lock andw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_and_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection117:
+; O1-NEXT:  .Lpcsection132:
 ; O1-NEXT:    lock andw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_and_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection117:
+; O2-NEXT:  .Lpcsection132:
 ; O2-NEXT:    lock andw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_and_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection117:
+; O3-NEXT:  .Lpcsection132:
 ; O3-NEXT:    lock andw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i16 42 release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_or_release(ptr %a) {
 ; O0-LABEL: atomic16_or_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection142:
 ; O0-NEXT:    lock orw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_or_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection118:
+; O1-NEXT:  .Lpcsection133:
 ; O1-NEXT:    lock orw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_or_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection118:
+; O2-NEXT:  .Lpcsection133:
 ; O2-NEXT:    lock orw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_or_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection118:
+; O3-NEXT:  .Lpcsection133:
 ; O3-NEXT:    lock orw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i16 42 release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_xor_release(ptr %a) {
 ; O0-LABEL: atomic16_xor_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection143:
 ; O0-NEXT:    lock xorw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xor_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection119:
+; O1-NEXT:  .Lpcsection134:
 ; O1-NEXT:    lock xorw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xor_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection119:
+; O2-NEXT:  .Lpcsection134:
 ; O2-NEXT:    lock xorw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xor_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection119:
+; O3-NEXT:  .Lpcsection134:
 ; O3-NEXT:    lock xorw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i16 42 release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB78_1
 ; O0-NEXT:  .LBB78_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_nand_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection120:
+; O1-NEXT:  .Lpcsection135:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB78_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection121:
+; O1-NEXT:  .Lpcsection136:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection122:
+; O1-NEXT:  .Lpcsection137:
 ; O1-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O1-NEXT:  .Lpcsection123:
+; O1-NEXT:  .Lpcsection138:
 ; O1-NEXT:    # kill: def $ax killed $ax killed $eax
-; O1-NEXT:  .Lpcsection124:
+; O1-NEXT:  .Lpcsection139:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O1-NEXT:  .Lpcsection125:
+; O1-NEXT:  .Lpcsection140:
 ; O1-NEXT:    # kill: def $ax killed $ax def $eax
-; O1-NEXT:  .Lpcsection126:
+; O1-NEXT:  .Lpcsection141:
 ; O1-NEXT:    jne .LBB78_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_nand_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection120:
+; O2-NEXT:  .Lpcsection135:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB78_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection121:
+; O2-NEXT:  .Lpcsection136:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection122:
+; O2-NEXT:  .Lpcsection137:
 ; O2-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O2-NEXT:  .Lpcsection123:
+; O2-NEXT:  .Lpcsection138:
 ; O2-NEXT:    # kill: def $ax killed $ax killed $eax
-; O2-NEXT:  .Lpcsection124:
+; O2-NEXT:  .Lpcsection139:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O2-NEXT:  .Lpcsection125:
+; O2-NEXT:  .Lpcsection140:
 ; O2-NEXT:    # kill: def $ax killed $ax def $eax
-; O2-NEXT:  .Lpcsection126:
+; O2-NEXT:  .Lpcsection141:
 ; O2-NEXT:    jne .LBB78_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_nand_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection120:
+; O3-NEXT:  .Lpcsection135:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB78_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection121:
+; O3-NEXT:  .Lpcsection136:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection122:
+; O3-NEXT:  .Lpcsection137:
 ; O3-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O3-NEXT:  .Lpcsection123:
+; O3-NEXT:  .Lpcsection138:
 ; O3-NEXT:    # kill: def $ax killed $ax killed $eax
-; O3-NEXT:  .Lpcsection124:
+; O3-NEXT:  .Lpcsection139:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O3-NEXT:  .Lpcsection125:
+; O3-NEXT:  .Lpcsection140:
 ; O3-NEXT:    # kill: def $ax killed $ax def $eax
-; O3-NEXT:  .Lpcsection126:
+; O3-NEXT:  .Lpcsection141:
 ; O3-NEXT:    jne .LBB78_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i16 42 release, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgw %ax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xchg_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection127:
+; O1-NEXT:  .Lpcsection142:
 ; O1-NEXT:    xchgw %ax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xchg_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection127:
+; O2-NEXT:  .Lpcsection142:
 ; O2-NEXT:    xchgw %ax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xchg_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection127:
+; O3-NEXT:  .Lpcsection142:
 ; O3-NEXT:    xchgw %ax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i16 42 acq_rel, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_add_acq_rel(ptr %a) {
 ; O0-LABEL: atomic16_add_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection154:
 ; O0-NEXT:    lock addw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_add_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection128:
+; O1-NEXT:  .Lpcsection143:
 ; O1-NEXT:    lock addw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_add_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection128:
+; O2-NEXT:  .Lpcsection143:
 ; O2-NEXT:    lock addw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_add_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection128:
+; O3-NEXT:  .Lpcsection143:
 ; O3-NEXT:    lock addw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i16 42 acq_rel, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_sub_acq_rel(ptr %a) {
 ; O0-LABEL: atomic16_sub_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection155:
 ; O0-NEXT:    lock subw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_sub_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection129:
+; O1-NEXT:  .Lpcsection144:
 ; O1-NEXT:    lock subw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_sub_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection129:
+; O2-NEXT:  .Lpcsection144:
 ; O2-NEXT:    lock subw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_sub_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection129:
+; O3-NEXT:  .Lpcsection144:
 ; O3-NEXT:    lock subw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i16 42 acq_rel, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_and_acq_rel(ptr %a) {
 ; O0-LABEL: atomic16_and_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection156:
 ; O0-NEXT:    lock andw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_and_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection130:
+; O1-NEXT:  .Lpcsection145:
 ; O1-NEXT:    lock andw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_and_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection130:
+; O2-NEXT:  .Lpcsection145:
 ; O2-NEXT:    lock andw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_and_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection130:
+; O3-NEXT:  .Lpcsection145:
 ; O3-NEXT:    lock andw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i16 42 acq_rel, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_or_acq_rel(ptr %a) {
 ; O0-LABEL: atomic16_or_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection157:
 ; O0-NEXT:    lock orw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_or_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection131:
+; O1-NEXT:  .Lpcsection146:
 ; O1-NEXT:    lock orw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_or_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection131:
+; O2-NEXT:  .Lpcsection146:
 ; O2-NEXT:    lock orw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_or_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection131:
+; O3-NEXT:  .Lpcsection146:
 ; O3-NEXT:    lock orw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i16 42 acq_rel, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_xor_acq_rel(ptr %a) {
 ; O0-LABEL: atomic16_xor_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection158:
 ; O0-NEXT:    lock xorw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xor_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection132:
+; O1-NEXT:  .Lpcsection147:
 ; O1-NEXT:    lock xorw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xor_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection132:
+; O2-NEXT:  .Lpcsection147:
 ; O2-NEXT:    lock xorw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xor_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection132:
+; O3-NEXT:  .Lpcsection147:
 ; O3-NEXT:    lock xorw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i16 42 acq_rel, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB85_1
 ; O0-NEXT:  .LBB85_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_nand_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection133:
+; O1-NEXT:  .Lpcsection148:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB85_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection134:
+; O1-NEXT:  .Lpcsection149:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection135:
+; O1-NEXT:  .Lpcsection150:
 ; O1-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O1-NEXT:  .Lpcsection136:
+; O1-NEXT:  .Lpcsection151:
 ; O1-NEXT:    # kill: def $ax killed $ax killed $eax
-; O1-NEXT:  .Lpcsection137:
+; O1-NEXT:  .Lpcsection152:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O1-NEXT:  .Lpcsection138:
+; O1-NEXT:  .Lpcsection153:
 ; O1-NEXT:    # kill: def $ax killed $ax def $eax
-; O1-NEXT:  .Lpcsection139:
+; O1-NEXT:  .Lpcsection154:
 ; O1-NEXT:    jne .LBB85_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_nand_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection133:
+; O2-NEXT:  .Lpcsection148:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB85_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection134:
+; O2-NEXT:  .Lpcsection149:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection135:
+; O2-NEXT:  .Lpcsection150:
 ; O2-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O2-NEXT:  .Lpcsection136:
+; O2-NEXT:  .Lpcsection151:
 ; O2-NEXT:    # kill: def $ax killed $ax killed $eax
-; O2-NEXT:  .Lpcsection137:
+; O2-NEXT:  .Lpcsection152:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O2-NEXT:  .Lpcsection138:
+; O2-NEXT:  .Lpcsection153:
 ; O2-NEXT:    # kill: def $ax killed $ax def $eax
-; O2-NEXT:  .Lpcsection139:
+; O2-NEXT:  .Lpcsection154:
 ; O2-NEXT:    jne .LBB85_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_nand_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection133:
+; O3-NEXT:  .Lpcsection148:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB85_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection134:
+; O3-NEXT:  .Lpcsection149:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection135:
+; O3-NEXT:  .Lpcsection150:
 ; O3-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O3-NEXT:  .Lpcsection136:
+; O3-NEXT:  .Lpcsection151:
 ; O3-NEXT:    # kill: def $ax killed $ax killed $eax
-; O3-NEXT:  .Lpcsection137:
+; O3-NEXT:  .Lpcsection152:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O3-NEXT:  .Lpcsection138:
+; O3-NEXT:  .Lpcsection153:
 ; O3-NEXT:    # kill: def $ax killed $ax def $eax
-; O3-NEXT:  .Lpcsection139:
+; O3-NEXT:  .Lpcsection154:
 ; O3-NEXT:    jne .LBB85_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i16 42 acq_rel, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgw %ax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xchg_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection140:
+; O1-NEXT:  .Lpcsection155:
 ; O1-NEXT:    xchgw %ax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xchg_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection140:
+; O2-NEXT:  .Lpcsection155:
 ; O2-NEXT:    xchgw %ax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xchg_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection140:
+; O3-NEXT:  .Lpcsection155:
 ; O3-NEXT:    xchgw %ax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i16 42 seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_add_seq_cst(ptr %a) {
 ; O0-LABEL: atomic16_add_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection169:
 ; O0-NEXT:    lock addw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_add_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection141:
+; O1-NEXT:  .Lpcsection156:
 ; O1-NEXT:    lock addw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_add_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection141:
+; O2-NEXT:  .Lpcsection156:
 ; O2-NEXT:    lock addw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_add_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection141:
+; O3-NEXT:  .Lpcsection156:
 ; O3-NEXT:    lock addw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i16 42 seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_sub_seq_cst(ptr %a) {
 ; O0-LABEL: atomic16_sub_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection170:
 ; O0-NEXT:    lock subw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_sub_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection142:
+; O1-NEXT:  .Lpcsection157:
 ; O1-NEXT:    lock subw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_sub_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection142:
+; O2-NEXT:  .Lpcsection157:
 ; O2-NEXT:    lock subw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_sub_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection142:
+; O3-NEXT:  .Lpcsection157:
 ; O3-NEXT:    lock subw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i16 42 seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_and_seq_cst(ptr %a) {
 ; O0-LABEL: atomic16_and_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection171:
 ; O0-NEXT:    lock andw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_and_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection143:
+; O1-NEXT:  .Lpcsection158:
 ; O1-NEXT:    lock andw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_and_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection143:
+; O2-NEXT:  .Lpcsection158:
 ; O2-NEXT:    lock andw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_and_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection143:
+; O3-NEXT:  .Lpcsection158:
 ; O3-NEXT:    lock andw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i16 42 seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_or_seq_cst(ptr %a) {
 ; O0-LABEL: atomic16_or_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection172:
 ; O0-NEXT:    lock orw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_or_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection144:
+; O1-NEXT:  .Lpcsection159:
 ; O1-NEXT:    lock orw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_or_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection144:
+; O2-NEXT:  .Lpcsection159:
 ; O2-NEXT:    lock orw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_or_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection144:
+; O3-NEXT:  .Lpcsection159:
 ; O3-NEXT:    lock orw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i16 42 seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic16_xor_seq_cst(ptr %a) {
 ; O0-LABEL: atomic16_xor_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection173:
 ; O0-NEXT:    lock xorw $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_xor_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection145:
+; O1-NEXT:  .Lpcsection160:
 ; O1-NEXT:    lock xorw $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_xor_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection145:
+; O2-NEXT:  .Lpcsection160:
 ; O2-NEXT:    lock xorw $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_xor_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection145:
+; O3-NEXT:  .Lpcsection160:
 ; O3-NEXT:    lock xorw $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i16 42 seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB92_1
 ; O0-NEXT:  .LBB92_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_nand_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection146:
+; O1-NEXT:  .Lpcsection161:
 ; O1-NEXT:    movzwl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB92_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection147:
+; O1-NEXT:  .Lpcsection162:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection148:
+; O1-NEXT:  .Lpcsection163:
 ; O1-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O1-NEXT:  .Lpcsection149:
+; O1-NEXT:  .Lpcsection164:
 ; O1-NEXT:    # kill: def $ax killed $ax killed $eax
-; O1-NEXT:  .Lpcsection150:
+; O1-NEXT:  .Lpcsection165:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O1-NEXT:  .Lpcsection151:
+; O1-NEXT:  .Lpcsection166:
 ; O1-NEXT:    # kill: def $ax killed $ax def $eax
-; O1-NEXT:  .Lpcsection152:
+; O1-NEXT:  .Lpcsection167:
 ; O1-NEXT:    jne .LBB92_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_nand_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection146:
+; O2-NEXT:  .Lpcsection161:
 ; O2-NEXT:    movzwl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB92_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection147:
+; O2-NEXT:  .Lpcsection162:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection148:
+; O2-NEXT:  .Lpcsection163:
 ; O2-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O2-NEXT:  .Lpcsection149:
+; O2-NEXT:  .Lpcsection164:
 ; O2-NEXT:    # kill: def $ax killed $ax killed $eax
-; O2-NEXT:  .Lpcsection150:
+; O2-NEXT:  .Lpcsection165:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O2-NEXT:  .Lpcsection151:
+; O2-NEXT:  .Lpcsection166:
 ; O2-NEXT:    # kill: def $ax killed $ax def $eax
-; O2-NEXT:  .Lpcsection152:
+; O2-NEXT:  .Lpcsection167:
 ; O2-NEXT:    jne .LBB92_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_nand_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection146:
+; O3-NEXT:  .Lpcsection161:
 ; O3-NEXT:    movzwl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB92_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection147:
+; O3-NEXT:  .Lpcsection162:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection148:
+; O3-NEXT:  .Lpcsection163:
 ; O3-NEXT:    orl $65493, %ecx # imm = 0xFFD5
-; O3-NEXT:  .Lpcsection149:
+; O3-NEXT:  .Lpcsection164:
 ; O3-NEXT:    # kill: def $ax killed $ax killed $eax
-; O3-NEXT:  .Lpcsection150:
+; O3-NEXT:  .Lpcsection165:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
-; O3-NEXT:  .Lpcsection151:
+; O3-NEXT:  .Lpcsection166:
 ; O3-NEXT:    # kill: def $ax killed $ax def $eax
-; O3-NEXT:  .Lpcsection152:
+; O3-NEXT:  .Lpcsection167:
 ; O3-NEXT:    jne .LBB92_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i16 42 seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_cas_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $1, %cx
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection153:
+; O1-NEXT:  .Lpcsection168:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection154:
+; O1-NEXT:  .Lpcsection169:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection155:
+; O1-NEXT:  .Lpcsection170:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_cas_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $1, %cx
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection153:
+; O2-NEXT:  .Lpcsection168:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection154:
+; O2-NEXT:  .Lpcsection169:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection155:
+; O2-NEXT:  .Lpcsection170:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_cas_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $1, %cx
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection153:
+; O3-NEXT:  .Lpcsection168:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection154:
+; O3-NEXT:  .Lpcsection169:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection155:
+; O3-NEXT:  .Lpcsection170:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i16 42, i16 1 monotonic monotonic, align 2, !pcsections !0
   %y = cmpxchg ptr %a, i16 42, i16 1 monotonic acquire, align 2, !pcsections !0
   %z = cmpxchg ptr %a, i16 42, i16 1 monotonic seq_cst, align 2, !pcsections !0

 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_cas_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $1, %cx
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection156:
+; O1-NEXT:  .Lpcsection171:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection157:
+; O1-NEXT:  .Lpcsection172:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection158:
+; O1-NEXT:  .Lpcsection173:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_cas_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $1, %cx
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection156:
+; O2-NEXT:  .Lpcsection171:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection157:
+; O2-NEXT:  .Lpcsection172:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection158:
+; O2-NEXT:  .Lpcsection173:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_cas_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $1, %cx
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection156:
+; O3-NEXT:  .Lpcsection171:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection157:
+; O3-NEXT:  .Lpcsection172:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection158:
+; O3-NEXT:  .Lpcsection173:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i16 42, i16 1 acquire monotonic, align 2, !pcsections !0
   %y = cmpxchg ptr %a, i16 42, i16 1 acquire acquire, align 2, !pcsections !0
   %z = cmpxchg ptr %a, i16 42, i16 1 acquire seq_cst, align 2, !pcsections !0

 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_cas_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $1, %cx
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection159:
+; O1-NEXT:  .Lpcsection174:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection160:
+; O1-NEXT:  .Lpcsection175:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection161:
+; O1-NEXT:  .Lpcsection176:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_cas_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $1, %cx
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection159:
+; O2-NEXT:  .Lpcsection174:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection160:
+; O2-NEXT:  .Lpcsection175:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection161:
+; O2-NEXT:  .Lpcsection176:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_cas_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $1, %cx
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection159:
+; O3-NEXT:  .Lpcsection174:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection160:
+; O3-NEXT:  .Lpcsection175:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection161:
+; O3-NEXT:  .Lpcsection176:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i16 42, i16 1 release monotonic, align 2, !pcsections !0
   %y = cmpxchg ptr %a, i16 42, i16 1 release acquire, align 2, !pcsections !0
   %z = cmpxchg ptr %a, i16 42, i16 1 release seq_cst, align 2, !pcsections !0

 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_cas_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $1, %cx
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection162:
+; O1-NEXT:  .Lpcsection177:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection163:
+; O1-NEXT:  .Lpcsection178:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection164:
+; O1-NEXT:  .Lpcsection179:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_cas_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $1, %cx
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection162:
+; O2-NEXT:  .Lpcsection177:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection163:
+; O2-NEXT:  .Lpcsection178:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection164:
+; O2-NEXT:  .Lpcsection179:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_cas_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $1, %cx
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection162:
+; O3-NEXT:  .Lpcsection177:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection163:
+; O3-NEXT:  .Lpcsection178:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection164:
+; O3-NEXT:  .Lpcsection179:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i16 42, i16 1 acq_rel monotonic, align 2, !pcsections !0
   %y = cmpxchg ptr %a, i16 42, i16 1 acq_rel acquire, align 2, !pcsections !0
   %z = cmpxchg ptr %a, i16 42, i16 1 acq_rel seq_cst, align 2, !pcsections !0

 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic16_cas_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movw $1, %cx
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection165:
+; O1-NEXT:  .Lpcsection180:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection166:
+; O1-NEXT:  .Lpcsection181:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movw $42, %ax
-; O1-NEXT:  .Lpcsection167:
+; O1-NEXT:  .Lpcsection182:
 ; O1-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic16_cas_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movw $1, %cx
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection165:
+; O2-NEXT:  .Lpcsection180:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection166:
+; O2-NEXT:  .Lpcsection181:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movw $42, %ax
-; O2-NEXT:  .Lpcsection167:
+; O2-NEXT:  .Lpcsection182:
 ; O2-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic16_cas_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movw $1, %cx
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection165:
+; O3-NEXT:  .Lpcsection180:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection166:
+; O3-NEXT:  .Lpcsection181:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movw $42, %ax
-; O3-NEXT:  .Lpcsection167:
+; O3-NEXT:  .Lpcsection182:
 ; O3-NEXT:    lock cmpxchgw %cx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i16 42, i16 1 seq_cst monotonic, align 2, !pcsections !0
   %y = cmpxchg ptr %a, i16 42, i16 1 seq_cst acquire, align 2, !pcsections !0
   %z = cmpxchg ptr %a, i16 42, i16 1 seq_cst seq_cst, align 2, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define i32 @atomic32_load_unordered(ptr %a) {
 ; O0-LABEL: atomic32_load_unordered:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection208:
 ; O0-NEXT:    movl (%rdi), %eax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_load_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection168:
+; O1-NEXT:  .Lpcsection183:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_load_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection168:
+; O2-NEXT:  .Lpcsection183:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_load_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection168:
+; O3-NEXT:  .Lpcsection183:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i32, ptr %a unordered, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i32 %x
 }
 
 define i32 @atomic32_load_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_load_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection209:
 ; O0-NEXT:    movl (%rdi), %eax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_load_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection169:
+; O1-NEXT:  .Lpcsection184:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_load_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection169:
+; O2-NEXT:  .Lpcsection184:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_load_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection169:
+; O3-NEXT:  .Lpcsection184:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i32, ptr %a monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i32 %x
 }
 
 define i32 @atomic32_load_acquire(ptr %a) {
 ; O0-LABEL: atomic32_load_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection210:
 ; O0-NEXT:    movl (%rdi), %eax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_load_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection170:
+; O1-NEXT:  .Lpcsection185:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_load_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection170:
+; O2-NEXT:  .Lpcsection185:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_load_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection170:
+; O3-NEXT:  .Lpcsection185:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i32, ptr %a acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i32 %x
 }
 
 define i32 @atomic32_load_seq_cst(ptr %a) {
 ; O0-LABEL: atomic32_load_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection211:
 ; O0-NEXT:    movl (%rdi), %eax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_load_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection171:
+; O1-NEXT:  .Lpcsection186:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_load_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection171:
+; O2-NEXT:  .Lpcsection186:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_load_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection171:
+; O3-NEXT:  .Lpcsection186:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i32, ptr %a seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i32 %x
 }
 
 define void @atomic32_store_unordered(ptr %a) {
 ; O0-LABEL: atomic32_store_unordered:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection212:
 ; O0-NEXT:    movl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_store_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection172:
+; O1-NEXT:  .Lpcsection187:
 ; O1-NEXT:    movl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_store_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection172:
+; O2-NEXT:  .Lpcsection187:
 ; O2-NEXT:    movl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_store_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection172:
+; O3-NEXT:  .Lpcsection187:
 ; O3-NEXT:    movl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i32 42, ptr %a unordered, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_store_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_store_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection213:
 ; O0-NEXT:    movl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_store_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection173:
+; O1-NEXT:  .Lpcsection188:
 ; O1-NEXT:    movl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_store_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection173:
+; O2-NEXT:  .Lpcsection188:
 ; O2-NEXT:    movl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_store_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection173:
+; O3-NEXT:  .Lpcsection188:
 ; O3-NEXT:    movl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i32 42, ptr %a monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_store_release(ptr %a) {
 ; O0-LABEL: atomic32_store_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection214:
 ; O0-NEXT:    movl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_store_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection174:
+; O1-NEXT:  .Lpcsection189:
 ; O1-NEXT:    movl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_store_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection174:
+; O2-NEXT:  .Lpcsection189:
 ; O2-NEXT:    movl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_store_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection174:
+; O3-NEXT:  .Lpcsection189:
 ; O3-NEXT:    movl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i32 42, ptr %a release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_store_seq_cst(ptr %a) {
 ; O0-LABEL: atomic32_store_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:    movl $42, %eax
 ; O0-NEXT:  .Lpcsection215:
 ; O0-NEXT:    xchgl %eax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_store_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection175:
+; O1-NEXT:  .Lpcsection190:
 ; O1-NEXT:    xchgl %eax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_store_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection175:
+; O2-NEXT:  .Lpcsection190:
 ; O2-NEXT:    xchgl %eax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_store_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection175:
+; O3-NEXT:  .Lpcsection190:
 ; O3-NEXT:    xchgl %eax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i32 42, ptr %a seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_xchg_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_xchg_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:    movl $42, %eax
 ; O0-NEXT:  .Lpcsection216:
 ; O0-NEXT:    xchgl %eax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xchg_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection176:
+; O1-NEXT:  .Lpcsection191:
 ; O1-NEXT:    xchgl %eax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xchg_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection176:
+; O2-NEXT:  .Lpcsection191:
 ; O2-NEXT:    xchgl %eax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xchg_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection176:
+; O3-NEXT:  .Lpcsection191:
 ; O3-NEXT:    xchgl %eax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i32 42 monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_add_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_add_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection217:
 ; O0-NEXT:    lock addl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_add_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection177:
+; O1-NEXT:  .Lpcsection192:
 ; O1-NEXT:    lock addl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_add_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection177:
+; O2-NEXT:  .Lpcsection192:
 ; O2-NEXT:    lock addl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_add_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection177:
+; O3-NEXT:  .Lpcsection192:
 ; O3-NEXT:    lock addl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i32 42 monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_sub_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_sub_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection218:
 ; O0-NEXT:    lock subl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_sub_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection178:
+; O1-NEXT:  .Lpcsection193:
 ; O1-NEXT:    lock subl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_sub_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection178:
+; O2-NEXT:  .Lpcsection193:
 ; O2-NEXT:    lock subl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_sub_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection178:
+; O3-NEXT:  .Lpcsection193:
 ; O3-NEXT:    lock subl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i32 42 monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_and_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_and_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection219:
 ; O0-NEXT:    lock andl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_and_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection179:
+; O1-NEXT:  .Lpcsection194:
 ; O1-NEXT:    lock andl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_and_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection179:
+; O2-NEXT:  .Lpcsection194:
 ; O2-NEXT:    lock andl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_and_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection179:
+; O3-NEXT:  .Lpcsection194:
 ; O3-NEXT:    lock andl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i32 42 monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_or_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_or_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection220:
 ; O0-NEXT:    lock orl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_or_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection180:
+; O1-NEXT:  .Lpcsection195:
 ; O1-NEXT:    lock orl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_or_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection180:
+; O2-NEXT:  .Lpcsection195:
 ; O2-NEXT:    lock orl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_or_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection180:
+; O3-NEXT:  .Lpcsection195:
 ; O3-NEXT:    lock orl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i32 42 monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_xor_monotonic(ptr %a) {
 ; O0-LABEL: atomic32_xor_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection221:
 ; O0-NEXT:    lock xorl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xor_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection181:
+; O1-NEXT:  .Lpcsection196:
 ; O1-NEXT:    lock xorl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xor_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection181:
+; O2-NEXT:  .Lpcsection196:
 ; O2-NEXT:    lock xorl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xor_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection181:
+; O3-NEXT:  .Lpcsection196:
 ; O3-NEXT:    lock xorl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i32 42 monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB112_1
 ; O0-NEXT:  .LBB112_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_nand_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection182:
+; O1-NEXT:  .Lpcsection197:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB112_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection183:
+; O1-NEXT:  .Lpcsection198:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection184:
+; O1-NEXT:  .Lpcsection199:
 ; O1-NEXT:    orl $-43, %ecx
-; O1-NEXT:  .Lpcsection185:
+; O1-NEXT:  .Lpcsection200:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O1-NEXT:  .Lpcsection186:
+; O1-NEXT:  .Lpcsection201:
 ; O1-NEXT:    jne .LBB112_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_nand_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection182:
+; O2-NEXT:  .Lpcsection197:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB112_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection183:
+; O2-NEXT:  .Lpcsection198:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection184:
+; O2-NEXT:  .Lpcsection199:
 ; O2-NEXT:    orl $-43, %ecx
-; O2-NEXT:  .Lpcsection185:
+; O2-NEXT:  .Lpcsection200:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O2-NEXT:  .Lpcsection186:
+; O2-NEXT:  .Lpcsection201:
 ; O2-NEXT:    jne .LBB112_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_nand_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection182:
+; O3-NEXT:  .Lpcsection197:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB112_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection183:
+; O3-NEXT:  .Lpcsection198:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection184:
+; O3-NEXT:  .Lpcsection199:
 ; O3-NEXT:    orl $-43, %ecx
-; O3-NEXT:  .Lpcsection185:
+; O3-NEXT:  .Lpcsection200:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O3-NEXT:  .Lpcsection186:
+; O3-NEXT:  .Lpcsection201:
 ; O3-NEXT:    jne .LBB112_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i32 42 monotonic, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgl %eax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xchg_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection187:
+; O1-NEXT:  .Lpcsection202:
 ; O1-NEXT:    xchgl %eax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xchg_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection187:
+; O2-NEXT:  .Lpcsection202:
 ; O2-NEXT:    xchgl %eax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xchg_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection187:
+; O3-NEXT:  .Lpcsection202:
 ; O3-NEXT:    xchgl %eax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i32 42 acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_add_acquire(ptr %a) {
 ; O0-LABEL: atomic32_add_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection230:
 ; O0-NEXT:    lock addl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_add_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection188:
+; O1-NEXT:  .Lpcsection203:
 ; O1-NEXT:    lock addl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_add_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection188:
+; O2-NEXT:  .Lpcsection203:
 ; O2-NEXT:    lock addl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_add_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection188:
+; O3-NEXT:  .Lpcsection203:
 ; O3-NEXT:    lock addl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i32 42 acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_sub_acquire(ptr %a) {
 ; O0-LABEL: atomic32_sub_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection231:
 ; O0-NEXT:    lock subl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_sub_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection189:
+; O1-NEXT:  .Lpcsection204:
 ; O1-NEXT:    lock subl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_sub_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection189:
+; O2-NEXT:  .Lpcsection204:
 ; O2-NEXT:    lock subl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_sub_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection189:
+; O3-NEXT:  .Lpcsection204:
 ; O3-NEXT:    lock subl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i32 42 acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_and_acquire(ptr %a) {
 ; O0-LABEL: atomic32_and_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection232:
 ; O0-NEXT:    lock andl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_and_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection190:
+; O1-NEXT:  .Lpcsection205:
 ; O1-NEXT:    lock andl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_and_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection190:
+; O2-NEXT:  .Lpcsection205:
 ; O2-NEXT:    lock andl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_and_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection190:
+; O3-NEXT:  .Lpcsection205:
 ; O3-NEXT:    lock andl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i32 42 acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_or_acquire(ptr %a) {
 ; O0-LABEL: atomic32_or_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection233:
 ; O0-NEXT:    lock orl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_or_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection191:
+; O1-NEXT:  .Lpcsection206:
 ; O1-NEXT:    lock orl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_or_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection191:
+; O2-NEXT:  .Lpcsection206:
 ; O2-NEXT:    lock orl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_or_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection191:
+; O3-NEXT:  .Lpcsection206:
 ; O3-NEXT:    lock orl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i32 42 acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_xor_acquire(ptr %a) {
 ; O0-LABEL: atomic32_xor_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection234:
 ; O0-NEXT:    lock xorl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xor_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection192:
+; O1-NEXT:  .Lpcsection207:
 ; O1-NEXT:    lock xorl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xor_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection192:
+; O2-NEXT:  .Lpcsection207:
 ; O2-NEXT:    lock xorl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xor_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection192:
+; O3-NEXT:  .Lpcsection207:
 ; O3-NEXT:    lock xorl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i32 42 acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB119_1
 ; O0-NEXT:  .LBB119_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_nand_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection193:
+; O1-NEXT:  .Lpcsection208:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB119_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection194:
+; O1-NEXT:  .Lpcsection209:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection195:
+; O1-NEXT:  .Lpcsection210:
 ; O1-NEXT:    orl $-43, %ecx
-; O1-NEXT:  .Lpcsection196:
+; O1-NEXT:  .Lpcsection211:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O1-NEXT:  .Lpcsection197:
+; O1-NEXT:  .Lpcsection212:
 ; O1-NEXT:    jne .LBB119_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_nand_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection193:
+; O2-NEXT:  .Lpcsection208:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB119_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection194:
+; O2-NEXT:  .Lpcsection209:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection195:
+; O2-NEXT:  .Lpcsection210:
 ; O2-NEXT:    orl $-43, %ecx
-; O2-NEXT:  .Lpcsection196:
+; O2-NEXT:  .Lpcsection211:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O2-NEXT:  .Lpcsection197:
+; O2-NEXT:  .Lpcsection212:
 ; O2-NEXT:    jne .LBB119_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_nand_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection193:
+; O3-NEXT:  .Lpcsection208:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB119_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection194:
+; O3-NEXT:  .Lpcsection209:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection195:
+; O3-NEXT:  .Lpcsection210:
 ; O3-NEXT:    orl $-43, %ecx
-; O3-NEXT:  .Lpcsection196:
+; O3-NEXT:  .Lpcsection211:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O3-NEXT:  .Lpcsection197:
+; O3-NEXT:  .Lpcsection212:
 ; O3-NEXT:    jne .LBB119_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i32 42 acquire, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgl %eax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xchg_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection198:
+; O1-NEXT:  .Lpcsection213:
 ; O1-NEXT:    xchgl %eax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xchg_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection198:
+; O2-NEXT:  .Lpcsection213:
 ; O2-NEXT:    xchgl %eax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xchg_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection198:
+; O3-NEXT:  .Lpcsection213:
 ; O3-NEXT:    xchgl %eax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i32 42 release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_add_release(ptr %a) {
 ; O0-LABEL: atomic32_add_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection243:
 ; O0-NEXT:    lock addl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_add_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection199:
+; O1-NEXT:  .Lpcsection214:
 ; O1-NEXT:    lock addl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_add_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection199:
+; O2-NEXT:  .Lpcsection214:
 ; O2-NEXT:    lock addl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_add_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection199:
+; O3-NEXT:  .Lpcsection214:
 ; O3-NEXT:    lock addl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i32 42 release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_sub_release(ptr %a) {
 ; O0-LABEL: atomic32_sub_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection244:
 ; O0-NEXT:    lock subl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_sub_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection200:
+; O1-NEXT:  .Lpcsection215:
 ; O1-NEXT:    lock subl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_sub_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection200:
+; O2-NEXT:  .Lpcsection215:
 ; O2-NEXT:    lock subl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_sub_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection200:
+; O3-NEXT:  .Lpcsection215:
 ; O3-NEXT:    lock subl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i32 42 release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_and_release(ptr %a) {
 ; O0-LABEL: atomic32_and_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection245:
 ; O0-NEXT:    lock andl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_and_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection201:
+; O1-NEXT:  .Lpcsection216:
 ; O1-NEXT:    lock andl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_and_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection201:
+; O2-NEXT:  .Lpcsection216:
 ; O2-NEXT:    lock andl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_and_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection201:
+; O3-NEXT:  .Lpcsection216:
 ; O3-NEXT:    lock andl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i32 42 release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_or_release(ptr %a) {
 ; O0-LABEL: atomic32_or_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection246:
 ; O0-NEXT:    lock orl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_or_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection202:
+; O1-NEXT:  .Lpcsection217:
 ; O1-NEXT:    lock orl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_or_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection202:
+; O2-NEXT:  .Lpcsection217:
 ; O2-NEXT:    lock orl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_or_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection202:
+; O3-NEXT:  .Lpcsection217:
 ; O3-NEXT:    lock orl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i32 42 release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_xor_release(ptr %a) {
 ; O0-LABEL: atomic32_xor_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection247:
 ; O0-NEXT:    lock xorl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xor_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection203:
+; O1-NEXT:  .Lpcsection218:
 ; O1-NEXT:    lock xorl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xor_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection203:
+; O2-NEXT:  .Lpcsection218:
 ; O2-NEXT:    lock xorl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xor_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection203:
+; O3-NEXT:  .Lpcsection218:
 ; O3-NEXT:    lock xorl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i32 42 release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB126_1
 ; O0-NEXT:  .LBB126_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_nand_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection204:
+; O1-NEXT:  .Lpcsection219:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB126_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection205:
+; O1-NEXT:  .Lpcsection220:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection206:
+; O1-NEXT:  .Lpcsection221:
 ; O1-NEXT:    orl $-43, %ecx
-; O1-NEXT:  .Lpcsection207:
+; O1-NEXT:  .Lpcsection222:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O1-NEXT:  .Lpcsection208:
+; O1-NEXT:  .Lpcsection223:
 ; O1-NEXT:    jne .LBB126_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_nand_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection204:
+; O2-NEXT:  .Lpcsection219:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB126_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection205:
+; O2-NEXT:  .Lpcsection220:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection206:
+; O2-NEXT:  .Lpcsection221:
 ; O2-NEXT:    orl $-43, %ecx
-; O2-NEXT:  .Lpcsection207:
+; O2-NEXT:  .Lpcsection222:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O2-NEXT:  .Lpcsection208:
+; O2-NEXT:  .Lpcsection223:
 ; O2-NEXT:    jne .LBB126_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_nand_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection204:
+; O3-NEXT:  .Lpcsection219:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB126_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection205:
+; O3-NEXT:  .Lpcsection220:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection206:
+; O3-NEXT:  .Lpcsection221:
 ; O3-NEXT:    orl $-43, %ecx
-; O3-NEXT:  .Lpcsection207:
+; O3-NEXT:  .Lpcsection222:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O3-NEXT:  .Lpcsection208:
+; O3-NEXT:  .Lpcsection223:
 ; O3-NEXT:    jne .LBB126_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i32 42 release, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgl %eax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xchg_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection209:
+; O1-NEXT:  .Lpcsection224:
 ; O1-NEXT:    xchgl %eax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xchg_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection209:
+; O2-NEXT:  .Lpcsection224:
 ; O2-NEXT:    xchgl %eax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xchg_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection209:
+; O3-NEXT:  .Lpcsection224:
 ; O3-NEXT:    xchgl %eax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i32 42 acq_rel, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_add_acq_rel(ptr %a) {
 ; O0-LABEL: atomic32_add_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection256:
 ; O0-NEXT:    lock addl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_add_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection210:
+; O1-NEXT:  .Lpcsection225:
 ; O1-NEXT:    lock addl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_add_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection210:
+; O2-NEXT:  .Lpcsection225:
 ; O2-NEXT:    lock addl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_add_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection210:
+; O3-NEXT:  .Lpcsection225:
 ; O3-NEXT:    lock addl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i32 42 acq_rel, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_sub_acq_rel(ptr %a) {
 ; O0-LABEL: atomic32_sub_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection257:
 ; O0-NEXT:    lock subl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_sub_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection211:
+; O1-NEXT:  .Lpcsection226:
 ; O1-NEXT:    lock subl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_sub_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection211:
+; O2-NEXT:  .Lpcsection226:
 ; O2-NEXT:    lock subl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_sub_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection211:
+; O3-NEXT:  .Lpcsection226:
 ; O3-NEXT:    lock subl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i32 42 acq_rel, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_and_acq_rel(ptr %a) {
 ; O0-LABEL: atomic32_and_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection258:
 ; O0-NEXT:    lock andl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_and_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection212:
+; O1-NEXT:  .Lpcsection227:
 ; O1-NEXT:    lock andl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_and_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection212:
+; O2-NEXT:  .Lpcsection227:
 ; O2-NEXT:    lock andl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_and_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection212:
+; O3-NEXT:  .Lpcsection227:
 ; O3-NEXT:    lock andl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i32 42 acq_rel, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_or_acq_rel(ptr %a) {
 ; O0-LABEL: atomic32_or_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection259:
 ; O0-NEXT:    lock orl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_or_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection213:
+; O1-NEXT:  .Lpcsection228:
 ; O1-NEXT:    lock orl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_or_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection213:
+; O2-NEXT:  .Lpcsection228:
 ; O2-NEXT:    lock orl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_or_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection213:
+; O3-NEXT:  .Lpcsection228:
 ; O3-NEXT:    lock orl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i32 42 acq_rel, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_xor_acq_rel(ptr %a) {
 ; O0-LABEL: atomic32_xor_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection260:
 ; O0-NEXT:    lock xorl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xor_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection214:
+; O1-NEXT:  .Lpcsection229:
 ; O1-NEXT:    lock xorl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xor_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection214:
+; O2-NEXT:  .Lpcsection229:
 ; O2-NEXT:    lock xorl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xor_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection214:
+; O3-NEXT:  .Lpcsection229:
 ; O3-NEXT:    lock xorl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i32 42 acq_rel, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB133_1
 ; O0-NEXT:  .LBB133_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_nand_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection215:
+; O1-NEXT:  .Lpcsection230:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB133_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection216:
+; O1-NEXT:  .Lpcsection231:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection217:
+; O1-NEXT:  .Lpcsection232:
 ; O1-NEXT:    orl $-43, %ecx
-; O1-NEXT:  .Lpcsection218:
+; O1-NEXT:  .Lpcsection233:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O1-NEXT:  .Lpcsection219:
+; O1-NEXT:  .Lpcsection234:
 ; O1-NEXT:    jne .LBB133_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_nand_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection215:
+; O2-NEXT:  .Lpcsection230:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB133_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection216:
+; O2-NEXT:  .Lpcsection231:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection217:
+; O2-NEXT:  .Lpcsection232:
 ; O2-NEXT:    orl $-43, %ecx
-; O2-NEXT:  .Lpcsection218:
+; O2-NEXT:  .Lpcsection233:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O2-NEXT:  .Lpcsection219:
+; O2-NEXT:  .Lpcsection234:
 ; O2-NEXT:    jne .LBB133_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_nand_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection215:
+; O3-NEXT:  .Lpcsection230:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB133_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection216:
+; O3-NEXT:  .Lpcsection231:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection217:
+; O3-NEXT:  .Lpcsection232:
 ; O3-NEXT:    orl $-43, %ecx
-; O3-NEXT:  .Lpcsection218:
+; O3-NEXT:  .Lpcsection233:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O3-NEXT:  .Lpcsection219:
+; O3-NEXT:  .Lpcsection234:
 ; O3-NEXT:    jne .LBB133_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i32 42 acq_rel, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgl %eax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xchg_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection220:
+; O1-NEXT:  .Lpcsection235:
 ; O1-NEXT:    xchgl %eax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xchg_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection220:
+; O2-NEXT:  .Lpcsection235:
 ; O2-NEXT:    xchgl %eax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xchg_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection220:
+; O3-NEXT:  .Lpcsection235:
 ; O3-NEXT:    xchgl %eax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i32 42 seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_add_seq_cst(ptr %a) {
 ; O0-LABEL: atomic32_add_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection269:
 ; O0-NEXT:    lock addl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_add_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection221:
+; O1-NEXT:  .Lpcsection236:
 ; O1-NEXT:    lock addl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_add_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection221:
+; O2-NEXT:  .Lpcsection236:
 ; O2-NEXT:    lock addl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_add_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection221:
+; O3-NEXT:  .Lpcsection236:
 ; O3-NEXT:    lock addl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i32 42 seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_sub_seq_cst(ptr %a) {
 ; O0-LABEL: atomic32_sub_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection270:
 ; O0-NEXT:    lock subl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_sub_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection222:
+; O1-NEXT:  .Lpcsection237:
 ; O1-NEXT:    lock subl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_sub_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection222:
+; O2-NEXT:  .Lpcsection237:
 ; O2-NEXT:    lock subl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_sub_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection222:
+; O3-NEXT:  .Lpcsection237:
 ; O3-NEXT:    lock subl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i32 42 seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_and_seq_cst(ptr %a) {
 ; O0-LABEL: atomic32_and_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection271:
 ; O0-NEXT:    lock andl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_and_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection223:
+; O1-NEXT:  .Lpcsection238:
 ; O1-NEXT:    lock andl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_and_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection223:
+; O2-NEXT:  .Lpcsection238:
 ; O2-NEXT:    lock andl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_and_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection223:
+; O3-NEXT:  .Lpcsection238:
 ; O3-NEXT:    lock andl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i32 42 seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_or_seq_cst(ptr %a) {
 ; O0-LABEL: atomic32_or_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection272:
 ; O0-NEXT:    lock orl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_or_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection224:
+; O1-NEXT:  .Lpcsection239:
 ; O1-NEXT:    lock orl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_or_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection224:
+; O2-NEXT:  .Lpcsection239:
 ; O2-NEXT:    lock orl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_or_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection224:
+; O3-NEXT:  .Lpcsection239:
 ; O3-NEXT:    lock orl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i32 42 seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic32_xor_seq_cst(ptr %a) {
 ; O0-LABEL: atomic32_xor_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection273:
 ; O0-NEXT:    lock xorl $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_xor_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection225:
+; O1-NEXT:  .Lpcsection240:
 ; O1-NEXT:    lock xorl $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_xor_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection225:
+; O2-NEXT:  .Lpcsection240:
 ; O2-NEXT:    lock xorl $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_xor_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection225:
+; O3-NEXT:  .Lpcsection240:
 ; O3-NEXT:    lock xorl $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i32 42 seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB140_1
 ; O0-NEXT:  .LBB140_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_nand_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection226:
+; O1-NEXT:  .Lpcsection241:
 ; O1-NEXT:    movl (%rdi), %eax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB140_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection227:
+; O1-NEXT:  .Lpcsection242:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection228:
+; O1-NEXT:  .Lpcsection243:
 ; O1-NEXT:    orl $-43, %ecx
-; O1-NEXT:  .Lpcsection229:
+; O1-NEXT:  .Lpcsection244:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O1-NEXT:  .Lpcsection230:
+; O1-NEXT:  .Lpcsection245:
 ; O1-NEXT:    jne .LBB140_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_nand_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection226:
+; O2-NEXT:  .Lpcsection241:
 ; O2-NEXT:    movl (%rdi), %eax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB140_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection227:
+; O2-NEXT:  .Lpcsection242:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection228:
+; O2-NEXT:  .Lpcsection243:
 ; O2-NEXT:    orl $-43, %ecx
-; O2-NEXT:  .Lpcsection229:
+; O2-NEXT:  .Lpcsection244:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O2-NEXT:  .Lpcsection230:
+; O2-NEXT:  .Lpcsection245:
 ; O2-NEXT:    jne .LBB140_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_nand_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection226:
+; O3-NEXT:  .Lpcsection241:
 ; O3-NEXT:    movl (%rdi), %eax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB140_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection227:
+; O3-NEXT:  .Lpcsection242:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection228:
+; O3-NEXT:  .Lpcsection243:
 ; O3-NEXT:    orl $-43, %ecx
-; O3-NEXT:  .Lpcsection229:
+; O3-NEXT:  .Lpcsection244:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
-; O3-NEXT:  .Lpcsection230:
+; O3-NEXT:  .Lpcsection245:
 ; O3-NEXT:    jne .LBB140_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i32 42 seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_cas_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection246:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection231:
+; O1-NEXT:  .Lpcsection247:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection248:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection232:
+; O1-NEXT:  .Lpcsection249:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection250:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection233:
+; O1-NEXT:  .Lpcsection251:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_cas_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection246:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection231:
+; O2-NEXT:  .Lpcsection247:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection248:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection232:
+; O2-NEXT:  .Lpcsection249:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection250:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection233:
+; O2-NEXT:  .Lpcsection251:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_cas_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection246:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection231:
+; O3-NEXT:  .Lpcsection247:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection248:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection232:
+; O3-NEXT:  .Lpcsection249:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection250:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection233:
+; O3-NEXT:  .Lpcsection251:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i32 42, i32 1 monotonic monotonic, align 4, !pcsections !0
   %y = cmpxchg ptr %a, i32 42, i32 1 monotonic acquire, align 4, !pcsections !0
   %z = cmpxchg ptr %a, i32 42, i32 1 monotonic seq_cst, align 4, !pcsections !0

 ; O0-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_cas_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection252:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection234:
+; O1-NEXT:  .Lpcsection253:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection254:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection235:
+; O1-NEXT:  .Lpcsection255:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection256:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection236:
+; O1-NEXT:  .Lpcsection257:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_cas_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection252:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection234:
+; O2-NEXT:  .Lpcsection253:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection254:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection235:
+; O2-NEXT:  .Lpcsection255:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection256:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection236:
+; O2-NEXT:  .Lpcsection257:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_cas_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection252:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection234:
+; O3-NEXT:  .Lpcsection253:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection254:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection235:
+; O3-NEXT:  .Lpcsection255:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection256:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection236:
+; O3-NEXT:  .Lpcsection257:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i32 42, i32 1 acquire monotonic, align 4, !pcsections !0
   %y = cmpxchg ptr %a, i32 42, i32 1 acquire acquire, align 4, !pcsections !0
   %z = cmpxchg ptr %a, i32 42, i32 1 acquire seq_cst, align 4, !pcsections !0

 ; O0-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_cas_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection258:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection237:
+; O1-NEXT:  .Lpcsection259:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection260:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection238:
+; O1-NEXT:  .Lpcsection261:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection262:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection239:
+; O1-NEXT:  .Lpcsection263:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_cas_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection258:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection237:
+; O2-NEXT:  .Lpcsection259:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection260:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection238:
+; O2-NEXT:  .Lpcsection261:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection262:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection239:
+; O2-NEXT:  .Lpcsection263:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_cas_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection258:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection237:
+; O3-NEXT:  .Lpcsection259:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection260:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection238:
+; O3-NEXT:  .Lpcsection261:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection262:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection239:
+; O3-NEXT:  .Lpcsection263:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i32 42, i32 1 release monotonic, align 4, !pcsections !0
   %y = cmpxchg ptr %a, i32 42, i32 1 release acquire, align 4, !pcsections !0
   %z = cmpxchg ptr %a, i32 42, i32 1 release seq_cst, align 4, !pcsections !0

 ; O0-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_cas_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection264:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection240:
+; O1-NEXT:  .Lpcsection265:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection266:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection241:
+; O1-NEXT:  .Lpcsection267:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection268:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection242:
+; O1-NEXT:  .Lpcsection269:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_cas_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection264:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection240:
+; O2-NEXT:  .Lpcsection265:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection266:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection241:
+; O2-NEXT:  .Lpcsection267:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection268:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection242:
+; O2-NEXT:  .Lpcsection269:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_cas_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection264:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection240:
+; O3-NEXT:  .Lpcsection265:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection266:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection241:
+; O3-NEXT:  .Lpcsection267:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection268:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection242:
+; O3-NEXT:  .Lpcsection269:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i32 42, i32 1 acq_rel monotonic, align 4, !pcsections !0
   %y = cmpxchg ptr %a, i32 42, i32 1 acq_rel acquire, align 4, !pcsections !0
   %z = cmpxchg ptr %a, i32 42, i32 1 acq_rel seq_cst, align 4, !pcsections !0

 ; O0-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic32_cas_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection270:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection243:
+; O1-NEXT:  .Lpcsection271:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection272:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection244:
+; O1-NEXT:  .Lpcsection273:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O1-NEXT:  .Lpcsection274:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection245:
+; O1-NEXT:  .Lpcsection275:
 ; O1-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic32_cas_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection270:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection243:
+; O2-NEXT:  .Lpcsection271:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection272:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection244:
+; O2-NEXT:  .Lpcsection273:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O2-NEXT:  .Lpcsection274:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection245:
+; O2-NEXT:  .Lpcsection275:
 ; O2-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic32_cas_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection270:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection243:
+; O3-NEXT:  .Lpcsection271:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection272:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection244:
+; O3-NEXT:  .Lpcsection273:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
+; O3-NEXT:  .Lpcsection274:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection245:
+; O3-NEXT:  .Lpcsection275:
 ; O3-NEXT:    lock cmpxchgl %ecx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i32 42, i32 1 seq_cst monotonic, align 4, !pcsections !0
   %y = cmpxchg ptr %a, i32 42, i32 1 seq_cst acquire, align 4, !pcsections !0
   %z = cmpxchg ptr %a, i32 42, i32 1 seq_cst seq_cst, align 4, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define i64 @atomic64_load_unordered(ptr %a) {
 ; O0-LABEL: atomic64_load_unordered:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection306:
 ; O0-NEXT:    movq (%rdi), %rax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_load_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection246:
+; O1-NEXT:  .Lpcsection276:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_load_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection246:
+; O2-NEXT:  .Lpcsection276:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_load_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection246:
+; O3-NEXT:  .Lpcsection276:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i64, ptr %a unordered, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i64 %x
 }
 
 define i64 @atomic64_load_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_load_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection307:
 ; O0-NEXT:    movq (%rdi), %rax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_load_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection247:
+; O1-NEXT:  .Lpcsection277:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_load_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection247:
+; O2-NEXT:  .Lpcsection277:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_load_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection247:
+; O3-NEXT:  .Lpcsection277:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i64, ptr %a monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i64 %x
 }
 
 define i64 @atomic64_load_acquire(ptr %a) {
 ; O0-LABEL: atomic64_load_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection308:
 ; O0-NEXT:    movq (%rdi), %rax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_load_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection248:
+; O1-NEXT:  .Lpcsection278:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_load_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection248:
+; O2-NEXT:  .Lpcsection278:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_load_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection248:
+; O3-NEXT:  .Lpcsection278:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i64, ptr %a acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i64 %x
 }
 
 define i64 @atomic64_load_seq_cst(ptr %a) {
 ; O0-LABEL: atomic64_load_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection309:
 ; O0-NEXT:    movq (%rdi), %rax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_load_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection249:
+; O1-NEXT:  .Lpcsection279:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_load_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection249:
+; O2-NEXT:  .Lpcsection279:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_load_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection249:
+; O3-NEXT:  .Lpcsection279:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i64, ptr %a seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i64 %x
 }
 
 define ptr @atomic64_load_seq_cst_ptr_ty(ptr %a) {
 ; O0-LABEL: atomic64_load_seq_cst_ptr_ty:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection310:
 ; O0-NEXT:    movq (%rdi), %rax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_load_seq_cst_ptr_ty:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection250:
+; O1-NEXT:  .Lpcsection280:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_load_seq_cst_ptr_ty:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection250:
+; O2-NEXT:  .Lpcsection280:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_load_seq_cst_ptr_ty:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection250:
+; O3-NEXT:  .Lpcsection280:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic ptr, ptr %a seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret ptr %x
 }
 
 define void @atomic64_store_unordered(ptr %a) {
 ; O0-LABEL: atomic64_store_unordered:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection311:
 ; O0-NEXT:    movq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_store_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection251:
+; O1-NEXT:  .Lpcsection281:
 ; O1-NEXT:    movq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_store_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection251:
+; O2-NEXT:  .Lpcsection281:
 ; O2-NEXT:    movq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_store_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection251:
+; O3-NEXT:  .Lpcsection281:
 ; O3-NEXT:    movq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i64 42, ptr %a unordered, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_store_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_store_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection312:
 ; O0-NEXT:    movq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_store_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection252:
+; O1-NEXT:  .Lpcsection282:
 ; O1-NEXT:    movq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_store_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection252:
+; O2-NEXT:  .Lpcsection282:
 ; O2-NEXT:    movq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_store_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection252:
+; O3-NEXT:  .Lpcsection282:
 ; O3-NEXT:    movq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i64 42, ptr %a monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_store_release(ptr %a) {
 ; O0-LABEL: atomic64_store_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection313:
 ; O0-NEXT:    movq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_store_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection253:
+; O1-NEXT:  .Lpcsection283:
 ; O1-NEXT:    movq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_store_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection253:
+; O2-NEXT:  .Lpcsection283:
 ; O2-NEXT:    movq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_store_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection253:
+; O3-NEXT:  .Lpcsection283:
 ; O3-NEXT:    movq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i64 42, ptr %a release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_store_seq_cst(ptr %a) {
 ; O0-LABEL: atomic64_store_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:    movl $42, %eax
 ; O0-NEXT:  .Lpcsection314:
 ; O0-NEXT:    xchgq %rax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_store_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection254:
+; O1-NEXT:  .Lpcsection284:
 ; O1-NEXT:    xchgq %rax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_store_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection254:
+; O2-NEXT:  .Lpcsection284:
 ; O2-NEXT:    xchgq %rax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_store_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection254:
+; O3-NEXT:  .Lpcsection284:
 ; O3-NEXT:    xchgq %rax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic i64 42, ptr %a seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_store_seq_cst_ptr_ty(ptr %a, ptr %v) {
 ; O0-LABEL: atomic64_store_seq_cst_ptr_ty:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection315:
 ; O0-NEXT:    xchgq %rsi, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_store_seq_cst_ptr_ty:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection255:
+; O1-NEXT:  .Lpcsection285:
 ; O1-NEXT:    xchgq %rsi, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_store_seq_cst_ptr_ty:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection255:
+; O2-NEXT:  .Lpcsection285:
 ; O2-NEXT:    xchgq %rsi, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_store_seq_cst_ptr_ty:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection255:
+; O3-NEXT:  .Lpcsection285:
 ; O3-NEXT:    xchgq %rsi, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic ptr %v, ptr %a seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_xchg_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_xchg_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:    movl $42, %eax
 ; O0-NEXT:  .Lpcsection316:
 ; O0-NEXT:    xchgq %rax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xchg_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection256:
+; O1-NEXT:  .Lpcsection286:
 ; O1-NEXT:    xchgq %rax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xchg_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection256:
+; O2-NEXT:  .Lpcsection286:
 ; O2-NEXT:    xchgq %rax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xchg_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection256:
+; O3-NEXT:  .Lpcsection286:
 ; O3-NEXT:    xchgq %rax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i64 42 monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_add_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_add_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection317:
 ; O0-NEXT:    lock addq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_add_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection257:
+; O1-NEXT:  .Lpcsection287:
 ; O1-NEXT:    lock addq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_add_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection257:
+; O2-NEXT:  .Lpcsection287:
 ; O2-NEXT:    lock addq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_add_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection257:
+; O3-NEXT:  .Lpcsection287:
 ; O3-NEXT:    lock addq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i64 42 monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_sub_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_sub_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection318:
 ; O0-NEXT:    lock subq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_sub_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection258:
+; O1-NEXT:  .Lpcsection288:
 ; O1-NEXT:    lock subq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_sub_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection258:
+; O2-NEXT:  .Lpcsection288:
 ; O2-NEXT:    lock subq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_sub_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection258:
+; O3-NEXT:  .Lpcsection288:
 ; O3-NEXT:    lock subq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i64 42 monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_and_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_and_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection319:
 ; O0-NEXT:    lock andq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_and_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection259:
+; O1-NEXT:  .Lpcsection289:
 ; O1-NEXT:    lock andq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_and_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection259:
+; O2-NEXT:  .Lpcsection289:
 ; O2-NEXT:    lock andq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_and_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection259:
+; O3-NEXT:  .Lpcsection289:
 ; O3-NEXT:    lock andq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i64 42 monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_or_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_or_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection320:
 ; O0-NEXT:    lock orq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_or_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection260:
+; O1-NEXT:  .Lpcsection290:
 ; O1-NEXT:    lock orq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_or_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection260:
+; O2-NEXT:  .Lpcsection290:
 ; O2-NEXT:    lock orq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_or_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection260:
+; O3-NEXT:  .Lpcsection290:
 ; O3-NEXT:    lock orq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i64 42 monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_xor_monotonic(ptr %a) {
 ; O0-LABEL: atomic64_xor_monotonic:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection321:
 ; O0-NEXT:    lock xorq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xor_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection261:
+; O1-NEXT:  .Lpcsection291:
 ; O1-NEXT:    lock xorq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xor_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection261:
+; O2-NEXT:  .Lpcsection291:
 ; O2-NEXT:    lock xorq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xor_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection261:
+; O3-NEXT:  .Lpcsection291:
 ; O3-NEXT:    lock xorq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i64 42 monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB162_1
 ; O0-NEXT:  .LBB162_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_nand_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection262:
+; O1-NEXT:  .Lpcsection292:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB162_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection263:
+; O1-NEXT:  .Lpcsection293:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection264:
+; O1-NEXT:  .Lpcsection294:
 ; O1-NEXT:    orq $-43, %rcx
-; O1-NEXT:  .Lpcsection265:
+; O1-NEXT:  .Lpcsection295:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O1-NEXT:  .Lpcsection266:
+; O1-NEXT:  .Lpcsection296:
 ; O1-NEXT:    jne .LBB162_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_nand_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection262:
+; O2-NEXT:  .Lpcsection292:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB162_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection263:
+; O2-NEXT:  .Lpcsection293:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection264:
+; O2-NEXT:  .Lpcsection294:
 ; O2-NEXT:    orq $-43, %rcx
-; O2-NEXT:  .Lpcsection265:
+; O2-NEXT:  .Lpcsection295:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O2-NEXT:  .Lpcsection266:
+; O2-NEXT:  .Lpcsection296:
 ; O2-NEXT:    jne .LBB162_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_nand_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection262:
+; O3-NEXT:  .Lpcsection292:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB162_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection263:
+; O3-NEXT:  .Lpcsection293:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection264:
+; O3-NEXT:  .Lpcsection294:
 ; O3-NEXT:    orq $-43, %rcx
-; O3-NEXT:  .Lpcsection265:
+; O3-NEXT:  .Lpcsection295:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O3-NEXT:  .Lpcsection266:
+; O3-NEXT:  .Lpcsection296:
 ; O3-NEXT:    jne .LBB162_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i64 42 monotonic, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgq %rax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xchg_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection267:
+; O1-NEXT:  .Lpcsection297:
 ; O1-NEXT:    xchgq %rax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xchg_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection267:
+; O2-NEXT:  .Lpcsection297:
 ; O2-NEXT:    xchgq %rax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xchg_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection267:
+; O3-NEXT:  .Lpcsection297:
 ; O3-NEXT:    xchgq %rax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i64 42 acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_add_acquire(ptr %a) {
 ; O0-LABEL: atomic64_add_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection331:
 ; O0-NEXT:    lock addq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_add_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection268:
+; O1-NEXT:  .Lpcsection298:
 ; O1-NEXT:    lock addq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_add_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection268:
+; O2-NEXT:  .Lpcsection298:
 ; O2-NEXT:    lock addq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_add_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection268:
+; O3-NEXT:  .Lpcsection298:
 ; O3-NEXT:    lock addq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i64 42 acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_sub_acquire(ptr %a) {
 ; O0-LABEL: atomic64_sub_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection332:
 ; O0-NEXT:    lock subq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_sub_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection269:
+; O1-NEXT:  .Lpcsection299:
 ; O1-NEXT:    lock subq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_sub_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection269:
+; O2-NEXT:  .Lpcsection299:
 ; O2-NEXT:    lock subq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_sub_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection269:
+; O3-NEXT:  .Lpcsection299:
 ; O3-NEXT:    lock subq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i64 42 acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_and_acquire(ptr %a) {
 ; O0-LABEL: atomic64_and_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection333:
 ; O0-NEXT:    lock andq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_and_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection270:
+; O1-NEXT:  .Lpcsection300:
 ; O1-NEXT:    lock andq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_and_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection270:
+; O2-NEXT:  .Lpcsection300:
 ; O2-NEXT:    lock andq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_and_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection270:
+; O3-NEXT:  .Lpcsection300:
 ; O3-NEXT:    lock andq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i64 42 acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_or_acquire(ptr %a) {
 ; O0-LABEL: atomic64_or_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection334:
 ; O0-NEXT:    lock orq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_or_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection271:
+; O1-NEXT:  .Lpcsection301:
 ; O1-NEXT:    lock orq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_or_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection271:
+; O2-NEXT:  .Lpcsection301:
 ; O2-NEXT:    lock orq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_or_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection271:
+; O3-NEXT:  .Lpcsection301:
 ; O3-NEXT:    lock orq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i64 42 acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_xor_acquire(ptr %a) {
 ; O0-LABEL: atomic64_xor_acquire:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection335:
 ; O0-NEXT:    lock xorq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xor_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection272:
+; O1-NEXT:  .Lpcsection302:
 ; O1-NEXT:    lock xorq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xor_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection272:
+; O2-NEXT:  .Lpcsection302:
 ; O2-NEXT:    lock xorq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xor_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection272:
+; O3-NEXT:  .Lpcsection302:
 ; O3-NEXT:    lock xorq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i64 42 acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB169_1
 ; O0-NEXT:  .LBB169_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_nand_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection273:
+; O1-NEXT:  .Lpcsection303:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB169_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection274:
+; O1-NEXT:  .Lpcsection304:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection275:
+; O1-NEXT:  .Lpcsection305:
 ; O1-NEXT:    orq $-43, %rcx
-; O1-NEXT:  .Lpcsection276:
+; O1-NEXT:  .Lpcsection306:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O1-NEXT:  .Lpcsection277:
+; O1-NEXT:  .Lpcsection307:
 ; O1-NEXT:    jne .LBB169_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_nand_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection273:
+; O2-NEXT:  .Lpcsection303:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB169_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection274:
+; O2-NEXT:  .Lpcsection304:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection275:
+; O2-NEXT:  .Lpcsection305:
 ; O2-NEXT:    orq $-43, %rcx
-; O2-NEXT:  .Lpcsection276:
+; O2-NEXT:  .Lpcsection306:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O2-NEXT:  .Lpcsection277:
+; O2-NEXT:  .Lpcsection307:
 ; O2-NEXT:    jne .LBB169_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_nand_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection273:
+; O3-NEXT:  .Lpcsection303:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB169_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection274:
+; O3-NEXT:  .Lpcsection304:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection275:
+; O3-NEXT:  .Lpcsection305:
 ; O3-NEXT:    orq $-43, %rcx
-; O3-NEXT:  .Lpcsection276:
+; O3-NEXT:  .Lpcsection306:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O3-NEXT:  .Lpcsection277:
+; O3-NEXT:  .Lpcsection307:
 ; O3-NEXT:    jne .LBB169_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i64 42 acquire, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgq %rax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xchg_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection278:
+; O1-NEXT:  .Lpcsection308:
 ; O1-NEXT:    xchgq %rax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xchg_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection278:
+; O2-NEXT:  .Lpcsection308:
 ; O2-NEXT:    xchgq %rax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xchg_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection278:
+; O3-NEXT:  .Lpcsection308:
 ; O3-NEXT:    xchgq %rax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i64 42 release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_add_release(ptr %a) {
 ; O0-LABEL: atomic64_add_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection345:
 ; O0-NEXT:    lock addq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_add_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection279:
+; O1-NEXT:  .Lpcsection309:
 ; O1-NEXT:    lock addq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_add_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection279:
+; O2-NEXT:  .Lpcsection309:
 ; O2-NEXT:    lock addq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_add_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection279:
+; O3-NEXT:  .Lpcsection309:
 ; O3-NEXT:    lock addq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i64 42 release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_sub_release(ptr %a) {
 ; O0-LABEL: atomic64_sub_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection346:
 ; O0-NEXT:    lock subq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_sub_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection280:
+; O1-NEXT:  .Lpcsection310:
 ; O1-NEXT:    lock subq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_sub_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection280:
+; O2-NEXT:  .Lpcsection310:
 ; O2-NEXT:    lock subq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_sub_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection280:
+; O3-NEXT:  .Lpcsection310:
 ; O3-NEXT:    lock subq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i64 42 release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_and_release(ptr %a) {
 ; O0-LABEL: atomic64_and_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection347:
 ; O0-NEXT:    lock andq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_and_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection281:
+; O1-NEXT:  .Lpcsection311:
 ; O1-NEXT:    lock andq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_and_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection281:
+; O2-NEXT:  .Lpcsection311:
 ; O2-NEXT:    lock andq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_and_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection281:
+; O3-NEXT:  .Lpcsection311:
 ; O3-NEXT:    lock andq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i64 42 release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_or_release(ptr %a) {
 ; O0-LABEL: atomic64_or_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection348:
 ; O0-NEXT:    lock orq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_or_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection282:
+; O1-NEXT:  .Lpcsection312:
 ; O1-NEXT:    lock orq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_or_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection282:
+; O2-NEXT:  .Lpcsection312:
 ; O2-NEXT:    lock orq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_or_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection282:
+; O3-NEXT:  .Lpcsection312:
 ; O3-NEXT:    lock orq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i64 42 release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_xor_release(ptr %a) {
 ; O0-LABEL: atomic64_xor_release:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection349:
 ; O0-NEXT:    lock xorq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xor_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection283:
+; O1-NEXT:  .Lpcsection313:
 ; O1-NEXT:    lock xorq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xor_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection283:
+; O2-NEXT:  .Lpcsection313:
 ; O2-NEXT:    lock xorq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xor_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection283:
+; O3-NEXT:  .Lpcsection313:
 ; O3-NEXT:    lock xorq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i64 42 release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB176_1
 ; O0-NEXT:  .LBB176_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_nand_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection284:
+; O1-NEXT:  .Lpcsection314:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB176_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection285:
+; O1-NEXT:  .Lpcsection315:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection286:
+; O1-NEXT:  .Lpcsection316:
 ; O1-NEXT:    orq $-43, %rcx
-; O1-NEXT:  .Lpcsection287:
+; O1-NEXT:  .Lpcsection317:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O1-NEXT:  .Lpcsection288:
+; O1-NEXT:  .Lpcsection318:
 ; O1-NEXT:    jne .LBB176_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_nand_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection284:
+; O2-NEXT:  .Lpcsection314:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB176_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection285:
+; O2-NEXT:  .Lpcsection315:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection286:
+; O2-NEXT:  .Lpcsection316:
 ; O2-NEXT:    orq $-43, %rcx
-; O2-NEXT:  .Lpcsection287:
+; O2-NEXT:  .Lpcsection317:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O2-NEXT:  .Lpcsection288:
+; O2-NEXT:  .Lpcsection318:
 ; O2-NEXT:    jne .LBB176_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_nand_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection284:
+; O3-NEXT:  .Lpcsection314:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB176_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection285:
+; O3-NEXT:  .Lpcsection315:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection286:
+; O3-NEXT:  .Lpcsection316:
 ; O3-NEXT:    orq $-43, %rcx
-; O3-NEXT:  .Lpcsection287:
+; O3-NEXT:  .Lpcsection317:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O3-NEXT:  .Lpcsection288:
+; O3-NEXT:  .Lpcsection318:
 ; O3-NEXT:    jne .LBB176_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i64 42 release, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgq %rax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xchg_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection289:
+; O1-NEXT:  .Lpcsection319:
 ; O1-NEXT:    xchgq %rax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xchg_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection289:
+; O2-NEXT:  .Lpcsection319:
 ; O2-NEXT:    xchgq %rax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xchg_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection289:
+; O3-NEXT:  .Lpcsection319:
 ; O3-NEXT:    xchgq %rax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i64 42 acq_rel, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_add_acq_rel(ptr %a) {
 ; O0-LABEL: atomic64_add_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection359:
 ; O0-NEXT:    lock addq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_add_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection290:
+; O1-NEXT:  .Lpcsection320:
 ; O1-NEXT:    lock addq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_add_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection290:
+; O2-NEXT:  .Lpcsection320:
 ; O2-NEXT:    lock addq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_add_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection290:
+; O3-NEXT:  .Lpcsection320:
 ; O3-NEXT:    lock addq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i64 42 acq_rel, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_sub_acq_rel(ptr %a) {
 ; O0-LABEL: atomic64_sub_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection360:
 ; O0-NEXT:    lock subq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_sub_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection291:
+; O1-NEXT:  .Lpcsection321:
 ; O1-NEXT:    lock subq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_sub_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection291:
+; O2-NEXT:  .Lpcsection321:
 ; O2-NEXT:    lock subq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_sub_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection291:
+; O3-NEXT:  .Lpcsection321:
 ; O3-NEXT:    lock subq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i64 42 acq_rel, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_and_acq_rel(ptr %a) {
 ; O0-LABEL: atomic64_and_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection361:
 ; O0-NEXT:    lock andq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_and_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection292:
+; O1-NEXT:  .Lpcsection322:
 ; O1-NEXT:    lock andq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_and_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection292:
+; O2-NEXT:  .Lpcsection322:
 ; O2-NEXT:    lock andq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_and_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection292:
+; O3-NEXT:  .Lpcsection322:
 ; O3-NEXT:    lock andq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i64 42 acq_rel, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_or_acq_rel(ptr %a) {
 ; O0-LABEL: atomic64_or_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection362:
 ; O0-NEXT:    lock orq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_or_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection293:
+; O1-NEXT:  .Lpcsection323:
 ; O1-NEXT:    lock orq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_or_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection293:
+; O2-NEXT:  .Lpcsection323:
 ; O2-NEXT:    lock orq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_or_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection293:
+; O3-NEXT:  .Lpcsection323:
 ; O3-NEXT:    lock orq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i64 42 acq_rel, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_xor_acq_rel(ptr %a) {
 ; O0-LABEL: atomic64_xor_acq_rel:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection363:
 ; O0-NEXT:    lock xorq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xor_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection294:
+; O1-NEXT:  .Lpcsection324:
 ; O1-NEXT:    lock xorq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xor_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection294:
+; O2-NEXT:  .Lpcsection324:
 ; O2-NEXT:    lock xorq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xor_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection294:
+; O3-NEXT:  .Lpcsection324:
 ; O3-NEXT:    lock xorq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i64 42 acq_rel, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB183_1
 ; O0-NEXT:  .LBB183_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_nand_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection295:
+; O1-NEXT:  .Lpcsection325:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB183_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection296:
+; O1-NEXT:  .Lpcsection326:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection297:
+; O1-NEXT:  .Lpcsection327:
 ; O1-NEXT:    orq $-43, %rcx
-; O1-NEXT:  .Lpcsection298:
+; O1-NEXT:  .Lpcsection328:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O1-NEXT:  .Lpcsection299:
+; O1-NEXT:  .Lpcsection329:
 ; O1-NEXT:    jne .LBB183_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_nand_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection295:
+; O2-NEXT:  .Lpcsection325:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB183_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection296:
+; O2-NEXT:  .Lpcsection326:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection297:
+; O2-NEXT:  .Lpcsection327:
 ; O2-NEXT:    orq $-43, %rcx
-; O2-NEXT:  .Lpcsection298:
+; O2-NEXT:  .Lpcsection328:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O2-NEXT:  .Lpcsection299:
+; O2-NEXT:  .Lpcsection329:
 ; O2-NEXT:    jne .LBB183_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_nand_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection295:
+; O3-NEXT:  .Lpcsection325:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB183_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection296:
+; O3-NEXT:  .Lpcsection326:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection297:
+; O3-NEXT:  .Lpcsection327:
 ; O3-NEXT:    orq $-43, %rcx
-; O3-NEXT:  .Lpcsection298:
+; O3-NEXT:  .Lpcsection328:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O3-NEXT:  .Lpcsection299:
+; O3-NEXT:  .Lpcsection329:
 ; O3-NEXT:    jne .LBB183_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i64 42 acq_rel, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    xchgq %rax, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xchg_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection300:
+; O1-NEXT:  .Lpcsection330:
 ; O1-NEXT:    xchgq %rax, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xchg_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection300:
+; O2-NEXT:  .Lpcsection330:
 ; O2-NEXT:    xchgq %rax, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xchg_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection300:
+; O3-NEXT:  .Lpcsection330:
 ; O3-NEXT:    xchgq %rax, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i64 42 seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_add_seq_cst(ptr %a) {
 ; O0-LABEL: atomic64_add_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection373:
 ; O0-NEXT:    lock addq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_add_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection301:
+; O1-NEXT:  .Lpcsection331:
 ; O1-NEXT:    lock addq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_add_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection301:
+; O2-NEXT:  .Lpcsection331:
 ; O2-NEXT:    lock addq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_add_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection301:
+; O3-NEXT:  .Lpcsection331:
 ; O3-NEXT:    lock addq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw add ptr %a, i64 42 seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_sub_seq_cst(ptr %a) {
 ; O0-LABEL: atomic64_sub_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection374:
 ; O0-NEXT:    lock subq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_sub_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection302:
+; O1-NEXT:  .Lpcsection332:
 ; O1-NEXT:    lock subq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_sub_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection302:
+; O2-NEXT:  .Lpcsection332:
 ; O2-NEXT:    lock subq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_sub_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection302:
+; O3-NEXT:  .Lpcsection332:
 ; O3-NEXT:    lock subq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw sub ptr %a, i64 42 seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_and_seq_cst(ptr %a) {
 ; O0-LABEL: atomic64_and_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection375:
 ; O0-NEXT:    lock andq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_and_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection303:
+; O1-NEXT:  .Lpcsection333:
 ; O1-NEXT:    lock andq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_and_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection303:
+; O2-NEXT:  .Lpcsection333:
 ; O2-NEXT:    lock andq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_and_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection303:
+; O3-NEXT:  .Lpcsection333:
 ; O3-NEXT:    lock andq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw and ptr %a, i64 42 seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_or_seq_cst(ptr %a) {
 ; O0-LABEL: atomic64_or_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection376:
 ; O0-NEXT:    lock orq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_or_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection304:
+; O1-NEXT:  .Lpcsection334:
 ; O1-NEXT:    lock orq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_or_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection304:
+; O2-NEXT:  .Lpcsection334:
 ; O2-NEXT:    lock orq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_or_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection304:
+; O3-NEXT:  .Lpcsection334:
 ; O3-NEXT:    lock orq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw or ptr %a, i64 42 seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void
 }
 
 define void @atomic64_xor_seq_cst(ptr %a) {
 ; O0-LABEL: atomic64_xor_seq_cst:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection377:
 ; O0-NEXT:    lock xorq $42, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_xor_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection305:
+; O1-NEXT:  .Lpcsection335:
 ; O1-NEXT:    lock xorq $42, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_xor_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection305:
+; O2-NEXT:  .Lpcsection335:
 ; O2-NEXT:    lock xorq $42, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_xor_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection305:
+; O3-NEXT:  .Lpcsection335:
 ; O3-NEXT:    lock xorq $42, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xor ptr %a, i64 42 seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    jmp .LBB190_1
 ; O0-NEXT:  .LBB190_2: # %atomicrmw.end
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_nand_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection306:
+; O1-NEXT:  .Lpcsection336:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB190_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ecx
-; O1-NEXT:  .Lpcsection307:
+; O1-NEXT:  .Lpcsection337:
 ; O1-NEXT:    notl %ecx
-; O1-NEXT:  .Lpcsection308:
+; O1-NEXT:  .Lpcsection338:
 ; O1-NEXT:    orq $-43, %rcx
-; O1-NEXT:  .Lpcsection309:
+; O1-NEXT:  .Lpcsection339:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O1-NEXT:  .Lpcsection310:
+; O1-NEXT:  .Lpcsection340:
 ; O1-NEXT:    jne .LBB190_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_nand_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection306:
+; O2-NEXT:  .Lpcsection336:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB190_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ecx
-; O2-NEXT:  .Lpcsection307:
+; O2-NEXT:  .Lpcsection337:
 ; O2-NEXT:    notl %ecx
-; O2-NEXT:  .Lpcsection308:
+; O2-NEXT:  .Lpcsection338:
 ; O2-NEXT:    orq $-43, %rcx
-; O2-NEXT:  .Lpcsection309:
+; O2-NEXT:  .Lpcsection339:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O2-NEXT:  .Lpcsection310:
+; O2-NEXT:  .Lpcsection340:
 ; O2-NEXT:    jne .LBB190_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_nand_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection306:
+; O3-NEXT:  .Lpcsection336:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB190_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ecx
-; O3-NEXT:  .Lpcsection307:
+; O3-NEXT:  .Lpcsection337:
 ; O3-NEXT:    notl %ecx
-; O3-NEXT:  .Lpcsection308:
+; O3-NEXT:  .Lpcsection338:
 ; O3-NEXT:    orq $-43, %rcx
-; O3-NEXT:  .Lpcsection309:
+; O3-NEXT:  .Lpcsection339:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
-; O3-NEXT:  .Lpcsection310:
+; O3-NEXT:  .Lpcsection340:
 ; O3-NEXT:    jne .LBB190_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw nand ptr %a, i64 42 seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8

 ; O0-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_cas_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection341:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection311:
+; O1-NEXT:  .Lpcsection342:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection343:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection312:
+; O1-NEXT:  .Lpcsection344:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection345:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection313:
+; O1-NEXT:  .Lpcsection346:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_cas_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection341:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection311:
+; O2-NEXT:  .Lpcsection342:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection343:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection312:
+; O2-NEXT:  .Lpcsection344:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection345:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection313:
+; O2-NEXT:  .Lpcsection346:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_cas_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection341:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection311:
+; O3-NEXT:  .Lpcsection342:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection343:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection312:
+; O3-NEXT:  .Lpcsection344:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection345:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection313:
+; O3-NEXT:  .Lpcsection346:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i64 42, i64 1 monotonic monotonic, align 8, !pcsections !0
   %y = cmpxchg ptr %a, i64 42, i64 1 monotonic acquire, align 8, !pcsections !0
   %z = cmpxchg ptr %a, i64 42, i64 1 monotonic seq_cst, align 8, !pcsections !0

 ; O0-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_cas_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection347:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection314:
+; O1-NEXT:  .Lpcsection348:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection349:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection315:
+; O1-NEXT:  .Lpcsection350:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection351:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection316:
+; O1-NEXT:  .Lpcsection352:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_cas_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection347:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection314:
+; O2-NEXT:  .Lpcsection348:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection349:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection315:
+; O2-NEXT:  .Lpcsection350:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection351:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection316:
+; O2-NEXT:  .Lpcsection352:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_cas_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection347:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection314:
+; O3-NEXT:  .Lpcsection348:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection349:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection315:
+; O3-NEXT:  .Lpcsection350:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection351:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection316:
+; O3-NEXT:  .Lpcsection352:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i64 42, i64 1 acquire monotonic, align 8, !pcsections !0
   %y = cmpxchg ptr %a, i64 42, i64 1 acquire acquire, align 8, !pcsections !0
   %z = cmpxchg ptr %a, i64 42, i64 1 acquire seq_cst, align 8, !pcsections !0

 ; O0-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_cas_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection353:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection317:
+; O1-NEXT:  .Lpcsection354:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection355:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection318:
+; O1-NEXT:  .Lpcsection356:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection357:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection319:
+; O1-NEXT:  .Lpcsection358:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_cas_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection353:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection317:
+; O2-NEXT:  .Lpcsection354:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection355:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection318:
+; O2-NEXT:  .Lpcsection356:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection357:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection319:
+; O2-NEXT:  .Lpcsection358:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_cas_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection353:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection317:
+; O3-NEXT:  .Lpcsection354:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection355:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection318:
+; O3-NEXT:  .Lpcsection356:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection357:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection319:
+; O3-NEXT:  .Lpcsection358:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i64 42, i64 1 release monotonic, align 8, !pcsections !0
   %y = cmpxchg ptr %a, i64 42, i64 1 release acquire, align 8, !pcsections !0
   %z = cmpxchg ptr %a, i64 42, i64 1 release seq_cst, align 8, !pcsections !0

 ; O0-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_cas_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection359:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection320:
+; O1-NEXT:  .Lpcsection360:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection361:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection321:
+; O1-NEXT:  .Lpcsection362:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection363:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection322:
+; O1-NEXT:  .Lpcsection364:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_cas_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection359:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection320:
+; O2-NEXT:  .Lpcsection360:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection361:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection321:
+; O2-NEXT:  .Lpcsection362:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection363:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection322:
+; O2-NEXT:  .Lpcsection364:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_cas_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection359:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection320:
+; O3-NEXT:  .Lpcsection360:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection361:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection321:
+; O3-NEXT:  .Lpcsection362:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection363:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection322:
+; O3-NEXT:  .Lpcsection364:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i64 42, i64 1 acq_rel monotonic, align 8, !pcsections !0
   %y = cmpxchg ptr %a, i64 42, i64 1 acq_rel acquire, align 8, !pcsections !0
   %z = cmpxchg ptr %a, i64 42, i64 1 acq_rel seq_cst, align 8, !pcsections !0

 ; O0-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O0-NEXT:    movq $3, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_cas_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
 ; O1-NEXT:    movl $1, %ecx
+; O1-NEXT:  .Lpcsection365:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection323:
+; O1-NEXT:  .Lpcsection366:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection367:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection324:
+; O1-NEXT:  .Lpcsection368:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O1-NEXT:  .Lpcsection369:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection325:
+; O1-NEXT:  .Lpcsection370:
 ; O1-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O1-NEXT:    movq $3, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_cas_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
 ; O2-NEXT:    movl $1, %ecx
+; O2-NEXT:  .Lpcsection365:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection323:
+; O2-NEXT:  .Lpcsection366:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection367:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection324:
+; O2-NEXT:  .Lpcsection368:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O2-NEXT:  .Lpcsection369:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection325:
+; O2-NEXT:  .Lpcsection370:
 ; O2-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O2-NEXT:    movq $3, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_cas_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
 ; O3-NEXT:    movl $1, %ecx
+; O3-NEXT:  .Lpcsection365:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection323:
+; O3-NEXT:  .Lpcsection366:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection367:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection324:
+; O3-NEXT:  .Lpcsection368:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
+; O3-NEXT:  .Lpcsection369:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection325:
+; O3-NEXT:  .Lpcsection370:
 ; O3-NEXT:    lock cmpxchgq %rcx, (%rdi)
 ; O3-NEXT:    movq $3, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i64 42, i64 1 seq_cst monotonic, align 8, !pcsections !0
   %y = cmpxchg ptr %a, i64 42, i64 1 seq_cst acquire, align 8, !pcsections !0
   %z = cmpxchg ptr %a, i64 42, i64 1 seq_cst seq_cst, align 8, !pcsections !0

 ; O0-NEXT:    lock cmpxchgq %rdx, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic64_cas_seq_cst_ptr_ty:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq %rsi, %rax
 ; O1-NEXT:    movq foo(%rip), %rcx
-; O1-NEXT:  .Lpcsection326:
+; O1-NEXT:  .Lpcsection371:
 ; O1-NEXT:    lock cmpxchgq %rdx, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic64_cas_seq_cst_ptr_ty:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq %rsi, %rax
 ; O2-NEXT:    movq foo(%rip), %rcx
-; O2-NEXT:  .Lpcsection326:
+; O2-NEXT:  .Lpcsection371:
 ; O2-NEXT:    lock cmpxchgq %rdx, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic64_cas_seq_cst_ptr_ty:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq %rsi, %rax
 ; O3-NEXT:    movq foo(%rip), %rcx
-; O3-NEXT:  .Lpcsection326:
+; O3-NEXT:  .Lpcsection371:
 ; O3-NEXT:    lock cmpxchgq %rdx, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, ptr %v1, ptr %v2 seq_cst seq_cst, align 8, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    movl $1, %eax
 ; O0-NEXT:    retq
 ; O0-NEXT:  .LBB197_2: # %else
 ; O0-NEXT:    movl $2, %eax
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic_use_cond:
 ; O1:       # %bb.0: # %entry
-; O1-NEXT:  .Lpcsection327:
+; O1-NEXT:  .Lpcsection372:
 ; O1-NEXT:    lock decq (%rdi)
 ; O1-NEXT:    jne .LBB197_2
 ; O1-NEXT:  # %bb.1: # %then
 ; O1-NEXT:    movl $1, %eax
 ; O1-NEXT:    retq
 ; O1-NEXT:  .LBB197_2: # %else
 ; O1-NEXT:    movl $2, %eax
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic_use_cond:
 ; O2:       # %bb.0: # %entry
-; O2-NEXT:  .Lpcsection327:
+; O2-NEXT:  .Lpcsection372:
 ; O2-NEXT:    lock decq (%rdi)
 ; O2-NEXT:    jne .LBB197_2
 ; O2-NEXT:  # %bb.1: # %then
 ; O2-NEXT:    movl $1, %eax
 ; O2-NEXT:    retq
 ; O2-NEXT:  .LBB197_2: # %else
 ; O2-NEXT:    movl $2, %eax
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic_use_cond:
 ; O3:       # %bb.0: # %entry
-; O3-NEXT:  .Lpcsection327:
+; O3-NEXT:  .Lpcsection372:
 ; O3-NEXT:    lock decq (%rdi)
 ; O3-NEXT:    jne .LBB197_2
 ; O3-NEXT:  # %bb.1: # %then
 ; O3-NEXT:    movl $1, %eax
 ; O3-NEXT:    retq
 ; O3-NEXT:  .LBB197_2: # %else
 ; O3-NEXT:    movl $2, %eax
 ; O3-NEXT:    retq

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_load_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection328:
+; O1-NEXT:  .Lpcsection373:
 ; O1-NEXT:    xorl %eax, %eax
-; O1-NEXT:  .Lpcsection329:
+; O1-NEXT:  .Lpcsection374:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection330:
+; O1-NEXT:  .Lpcsection375:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection331:
+; O1-NEXT:  .Lpcsection376:
 ; O1-NEXT:    xorl %ebx, %ebx
-; O1-NEXT:  .Lpcsection332:
+; O1-NEXT:  .Lpcsection377:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_load_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection328:
+; O2-NEXT:  .Lpcsection373:
 ; O2-NEXT:    xorl %eax, %eax
-; O2-NEXT:  .Lpcsection329:
+; O2-NEXT:  .Lpcsection374:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection330:
+; O2-NEXT:  .Lpcsection375:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection331:
+; O2-NEXT:  .Lpcsection376:
 ; O2-NEXT:    xorl %ebx, %ebx
-; O2-NEXT:  .Lpcsection332:
+; O2-NEXT:  .Lpcsection377:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_load_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection328:
+; O3-NEXT:  .Lpcsection373:
 ; O3-NEXT:    xorl %eax, %eax
-; O3-NEXT:  .Lpcsection329:
+; O3-NEXT:  .Lpcsection374:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection330:
+; O3-NEXT:  .Lpcsection375:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection331:
+; O3-NEXT:  .Lpcsection376:
 ; O3-NEXT:    xorl %ebx, %ebx
-; O3-NEXT:  .Lpcsection332:
+; O3-NEXT:  .Lpcsection377:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i128, ptr %a unordered, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_load_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection333:
+; O1-NEXT:  .Lpcsection378:
 ; O1-NEXT:    xorl %eax, %eax
-; O1-NEXT:  .Lpcsection334:
+; O1-NEXT:  .Lpcsection379:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection335:
+; O1-NEXT:  .Lpcsection380:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection336:
+; O1-NEXT:  .Lpcsection381:
 ; O1-NEXT:    xorl %ebx, %ebx
-; O1-NEXT:  .Lpcsection337:
+; O1-NEXT:  .Lpcsection382:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_load_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection333:
+; O2-NEXT:  .Lpcsection378:
 ; O2-NEXT:    xorl %eax, %eax
-; O2-NEXT:  .Lpcsection334:
+; O2-NEXT:  .Lpcsection379:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection335:
+; O2-NEXT:  .Lpcsection380:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection336:
+; O2-NEXT:  .Lpcsection381:
 ; O2-NEXT:    xorl %ebx, %ebx
-; O2-NEXT:  .Lpcsection337:
+; O2-NEXT:  .Lpcsection382:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_load_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection333:
+; O3-NEXT:  .Lpcsection378:
 ; O3-NEXT:    xorl %eax, %eax
-; O3-NEXT:  .Lpcsection334:
+; O3-NEXT:  .Lpcsection379:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection335:
+; O3-NEXT:  .Lpcsection380:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection336:
+; O3-NEXT:  .Lpcsection381:
 ; O3-NEXT:    xorl %ebx, %ebx
-; O3-NEXT:  .Lpcsection337:
+; O3-NEXT:  .Lpcsection382:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i128, ptr %a monotonic, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_load_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection338:
+; O1-NEXT:  .Lpcsection383:
 ; O1-NEXT:    xorl %eax, %eax
-; O1-NEXT:  .Lpcsection339:
+; O1-NEXT:  .Lpcsection384:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection340:
+; O1-NEXT:  .Lpcsection385:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection341:
+; O1-NEXT:  .Lpcsection386:
 ; O1-NEXT:    xorl %ebx, %ebx
-; O1-NEXT:  .Lpcsection342:
+; O1-NEXT:  .Lpcsection387:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_load_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection338:
+; O2-NEXT:  .Lpcsection383:
 ; O2-NEXT:    xorl %eax, %eax
-; O2-NEXT:  .Lpcsection339:
+; O2-NEXT:  .Lpcsection384:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection340:
+; O2-NEXT:  .Lpcsection385:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection341:
+; O2-NEXT:  .Lpcsection386:
 ; O2-NEXT:    xorl %ebx, %ebx
-; O2-NEXT:  .Lpcsection342:
+; O2-NEXT:  .Lpcsection387:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_load_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection338:
+; O3-NEXT:  .Lpcsection383:
 ; O3-NEXT:    xorl %eax, %eax
-; O3-NEXT:  .Lpcsection339:
+; O3-NEXT:  .Lpcsection384:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection340:
+; O3-NEXT:  .Lpcsection385:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection341:
+; O3-NEXT:  .Lpcsection386:
 ; O3-NEXT:    xorl %ebx, %ebx
-; O3-NEXT:  .Lpcsection342:
+; O3-NEXT:  .Lpcsection387:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i128, ptr %a acquire, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_load_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection343:
+; O1-NEXT:  .Lpcsection388:
 ; O1-NEXT:    xorl %eax, %eax
-; O1-NEXT:  .Lpcsection344:
+; O1-NEXT:  .Lpcsection389:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection345:
+; O1-NEXT:  .Lpcsection390:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection346:
+; O1-NEXT:  .Lpcsection391:
 ; O1-NEXT:    xorl %ebx, %ebx
-; O1-NEXT:  .Lpcsection347:
+; O1-NEXT:  .Lpcsection392:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_load_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection343:
+; O2-NEXT:  .Lpcsection388:
 ; O2-NEXT:    xorl %eax, %eax
-; O2-NEXT:  .Lpcsection344:
+; O2-NEXT:  .Lpcsection389:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection345:
+; O2-NEXT:  .Lpcsection390:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection346:
+; O2-NEXT:  .Lpcsection391:
 ; O2-NEXT:    xorl %ebx, %ebx
-; O2-NEXT:  .Lpcsection347:
+; O2-NEXT:  .Lpcsection392:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_load_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection343:
+; O3-NEXT:  .Lpcsection388:
 ; O3-NEXT:    xorl %eax, %eax
-; O3-NEXT:  .Lpcsection344:
+; O3-NEXT:  .Lpcsection389:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection345:
+; O3-NEXT:  .Lpcsection390:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection346:
+; O3-NEXT:  .Lpcsection391:
 ; O3-NEXT:    xorl %ebx, %ebx
-; O3-NEXT:  .Lpcsection347:
+; O3-NEXT:  .Lpcsection392:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic i128, ptr %a seq_cst, align 16, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret i128 %x
 }
 
 define ptr @atomic128_load_seq_cst_ptr_ty(ptr %a) {
 ; O0-LABEL: atomic128_load_seq_cst_ptr_ty:
 ; O0:       # %bb.0: # %entry
 ; O0-NEXT:    movq foo(%rip), %rax
 ; O0-NEXT:  .Lpcsection422:
 ; O0-NEXT:    movq (%rdi), %rax
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_load_seq_cst_ptr_ty:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection348:
+; O1-NEXT:  .Lpcsection393:
 ; O1-NEXT:    movq (%rdi), %rax
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_load_seq_cst_ptr_ty:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection348:
+; O2-NEXT:  .Lpcsection393:
 ; O2-NEXT:    movq (%rdi), %rax
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_load_seq_cst_ptr_ty:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection348:
+; O3-NEXT:  .Lpcsection393:
 ; O3-NEXT:    movq (%rdi), %rax
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = load atomic ptr, ptr %a seq_cst, align 16, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret ptr %x

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_store_unordered:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection349:
+; O1-NEXT:  .Lpcsection394:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection350:
+; O1-NEXT:  .Lpcsection395:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection351:
+; O1-NEXT:  .Lpcsection396:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB203_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection352:
+; O1-NEXT:  .Lpcsection397:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection353:
+; O1-NEXT:  .Lpcsection398:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection354:
+; O1-NEXT:  .Lpcsection399:
 ; O1-NEXT:    jne .LBB203_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_store_unordered:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection349:
+; O2-NEXT:  .Lpcsection394:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection350:
+; O2-NEXT:  .Lpcsection395:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection351:
+; O2-NEXT:  .Lpcsection396:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB203_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection352:
+; O2-NEXT:  .Lpcsection397:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection353:
+; O2-NEXT:  .Lpcsection398:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection354:
+; O2-NEXT:  .Lpcsection399:
 ; O2-NEXT:    jne .LBB203_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_store_unordered:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection349:
+; O3-NEXT:  .Lpcsection394:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection350:
+; O3-NEXT:  .Lpcsection395:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection351:
+; O3-NEXT:  .Lpcsection396:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB203_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection352:
+; O3-NEXT:  .Lpcsection397:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection353:
+; O3-NEXT:  .Lpcsection398:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection354:
+; O3-NEXT:  .Lpcsection399:
 ; O3-NEXT:    jne .LBB203_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_store_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection355:
+; O1-NEXT:  .Lpcsection400:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection356:
+; O1-NEXT:  .Lpcsection401:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection357:
+; O1-NEXT:  .Lpcsection402:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB204_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection358:
+; O1-NEXT:  .Lpcsection403:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection359:
+; O1-NEXT:  .Lpcsection404:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection360:
+; O1-NEXT:  .Lpcsection405:
 ; O1-NEXT:    jne .LBB204_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_store_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection355:
+; O2-NEXT:  .Lpcsection400:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection356:
+; O2-NEXT:  .Lpcsection401:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection357:
+; O2-NEXT:  .Lpcsection402:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB204_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection358:
+; O2-NEXT:  .Lpcsection403:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection359:
+; O2-NEXT:  .Lpcsection404:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection360:
+; O2-NEXT:  .Lpcsection405:
 ; O2-NEXT:    jne .LBB204_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_store_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection355:
+; O3-NEXT:  .Lpcsection400:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection356:
+; O3-NEXT:  .Lpcsection401:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection357:
+; O3-NEXT:  .Lpcsection402:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB204_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection358:
+; O3-NEXT:  .Lpcsection403:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection359:
+; O3-NEXT:  .Lpcsection404:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection360:
+; O3-NEXT:  .Lpcsection405:
 ; O3-NEXT:    jne .LBB204_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_store_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection361:
+; O1-NEXT:  .Lpcsection406:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection362:
+; O1-NEXT:  .Lpcsection407:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection363:
+; O1-NEXT:  .Lpcsection408:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB205_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection364:
+; O1-NEXT:  .Lpcsection409:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection365:
+; O1-NEXT:  .Lpcsection410:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection366:
+; O1-NEXT:  .Lpcsection411:
 ; O1-NEXT:    jne .LBB205_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_store_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection361:
+; O2-NEXT:  .Lpcsection406:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection362:
+; O2-NEXT:  .Lpcsection407:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection363:
+; O2-NEXT:  .Lpcsection408:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB205_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection364:
+; O2-NEXT:  .Lpcsection409:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection365:
+; O2-NEXT:  .Lpcsection410:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection366:
+; O2-NEXT:  .Lpcsection411:
 ; O2-NEXT:    jne .LBB205_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_store_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection361:
+; O3-NEXT:  .Lpcsection406:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection362:
+; O3-NEXT:  .Lpcsection407:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection363:
+; O3-NEXT:  .Lpcsection408:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB205_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection364:
+; O3-NEXT:  .Lpcsection409:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection365:
+; O3-NEXT:  .Lpcsection410:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection366:
+; O3-NEXT:  .Lpcsection411:
 ; O3-NEXT:    jne .LBB205_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_store_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection367:
+; O1-NEXT:  .Lpcsection412:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection368:
+; O1-NEXT:  .Lpcsection413:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection369:
+; O1-NEXT:  .Lpcsection414:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB206_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection370:
+; O1-NEXT:  .Lpcsection415:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection371:
+; O1-NEXT:  .Lpcsection416:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection372:
+; O1-NEXT:  .Lpcsection417:
 ; O1-NEXT:    jne .LBB206_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_store_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection367:
+; O2-NEXT:  .Lpcsection412:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection368:
+; O2-NEXT:  .Lpcsection413:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection369:
+; O2-NEXT:  .Lpcsection414:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB206_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection370:
+; O2-NEXT:  .Lpcsection415:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection371:
+; O2-NEXT:  .Lpcsection416:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection372:
+; O2-NEXT:  .Lpcsection417:
 ; O2-NEXT:    jne .LBB206_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_store_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection367:
+; O3-NEXT:  .Lpcsection412:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection368:
+; O3-NEXT:  .Lpcsection413:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection369:
+; O3-NEXT:  .Lpcsection414:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB206_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection370:
+; O3-NEXT:  .Lpcsection415:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection371:
+; O3-NEXT:  .Lpcsection416:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection372:
+; O3-NEXT:  .Lpcsection417:
 ; O3-NEXT:    jne .LBB206_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:  .Lpcsection455:
 ; O0-NEXT:    xchgq %rsi, (%rdi)
 ; O0-NEXT:    movq $1, foo
 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_store_seq_cst_ptr_ty:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection373:
+; O1-NEXT:  .Lpcsection418:
 ; O1-NEXT:    xchgq %rsi, (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_store_seq_cst_ptr_ty:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection373:
+; O2-NEXT:  .Lpcsection418:
 ; O2-NEXT:    xchgq %rsi, (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_store_seq_cst_ptr_ty:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection373:
+; O3-NEXT:  .Lpcsection418:
 ; O3-NEXT:    xchgq %rsi, (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   store atomic ptr %v, ptr %a seq_cst, align 16, !pcsections !0
   store volatile i64 1, ptr @foo, align 8
   ret void

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xchg_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection374:
+; O1-NEXT:  .Lpcsection419:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection375:
+; O1-NEXT:  .Lpcsection420:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection376:
+; O1-NEXT:  .Lpcsection421:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB208_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection377:
+; O1-NEXT:  .Lpcsection422:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection378:
+; O1-NEXT:  .Lpcsection423:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection379:
+; O1-NEXT:  .Lpcsection424:
 ; O1-NEXT:    jne .LBB208_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xchg_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection374:
+; O2-NEXT:  .Lpcsection419:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection375:
+; O2-NEXT:  .Lpcsection420:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection376:
+; O2-NEXT:  .Lpcsection421:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB208_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection377:
+; O2-NEXT:  .Lpcsection422:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection378:
+; O2-NEXT:  .Lpcsection423:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection379:
+; O2-NEXT:  .Lpcsection424:
 ; O2-NEXT:    jne .LBB208_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xchg_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection374:
+; O3-NEXT:  .Lpcsection419:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection375:
+; O3-NEXT:  .Lpcsection420:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection376:
+; O3-NEXT:  .Lpcsection421:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB208_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection377:
+; O3-NEXT:  .Lpcsection422:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection378:
+; O3-NEXT:  .Lpcsection423:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection379:
+; O3-NEXT:  .Lpcsection424:
 ; O3-NEXT:    jne .LBB208_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_add_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection380:
+; O1-NEXT:  .Lpcsection425:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection381:
+; O1-NEXT:  .Lpcsection426:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB209_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection382:
+; O1-NEXT:  .Lpcsection427:
 ; O1-NEXT:    addq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection383:
+; O1-NEXT:  .Lpcsection428:
 ; O1-NEXT:    adcq $0, %rcx
-; O1-NEXT:  .Lpcsection384:
+; O1-NEXT:  .Lpcsection429:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection385:
+; O1-NEXT:  .Lpcsection430:
 ; O1-NEXT:    jne .LBB209_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_add_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection380:
+; O2-NEXT:  .Lpcsection425:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection381:
+; O2-NEXT:  .Lpcsection426:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB209_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection382:
+; O2-NEXT:  .Lpcsection427:
 ; O2-NEXT:    addq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection383:
+; O2-NEXT:  .Lpcsection428:
 ; O2-NEXT:    adcq $0, %rcx
-; O2-NEXT:  .Lpcsection384:
+; O2-NEXT:  .Lpcsection429:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection385:
+; O2-NEXT:  .Lpcsection430:
 ; O2-NEXT:    jne .LBB209_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_add_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection380:
+; O3-NEXT:  .Lpcsection425:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection381:
+; O3-NEXT:  .Lpcsection426:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB209_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection382:
+; O3-NEXT:  .Lpcsection427:
 ; O3-NEXT:    addq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection383:
+; O3-NEXT:  .Lpcsection428:
 ; O3-NEXT:    adcq $0, %rcx
-; O3-NEXT:  .Lpcsection384:
+; O3-NEXT:  .Lpcsection429:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection385:
+; O3-NEXT:  .Lpcsection430:
 ; O3-NEXT:    jne .LBB209_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_sub_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection386:
+; O1-NEXT:  .Lpcsection431:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection387:
+; O1-NEXT:  .Lpcsection432:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB210_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection388:
+; O1-NEXT:  .Lpcsection433:
 ; O1-NEXT:    addq $-42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection389:
+; O1-NEXT:  .Lpcsection434:
 ; O1-NEXT:    adcq $-1, %rcx
-; O1-NEXT:  .Lpcsection390:
+; O1-NEXT:  .Lpcsection435:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection391:
+; O1-NEXT:  .Lpcsection436:
 ; O1-NEXT:    jne .LBB210_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_sub_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection386:
+; O2-NEXT:  .Lpcsection431:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection387:
+; O2-NEXT:  .Lpcsection432:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB210_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection388:
+; O2-NEXT:  .Lpcsection433:
 ; O2-NEXT:    addq $-42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection389:
+; O2-NEXT:  .Lpcsection434:
 ; O2-NEXT:    adcq $-1, %rcx
-; O2-NEXT:  .Lpcsection390:
+; O2-NEXT:  .Lpcsection435:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection391:
+; O2-NEXT:  .Lpcsection436:
 ; O2-NEXT:    jne .LBB210_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_sub_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection386:
+; O3-NEXT:  .Lpcsection431:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection387:
+; O3-NEXT:  .Lpcsection432:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB210_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection388:
+; O3-NEXT:  .Lpcsection433:
 ; O3-NEXT:    addq $-42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection389:
+; O3-NEXT:  .Lpcsection434:
 ; O3-NEXT:    adcq $-1, %rcx
-; O3-NEXT:  .Lpcsection390:
+; O3-NEXT:  .Lpcsection435:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection391:
+; O3-NEXT:  .Lpcsection436:
 ; O3-NEXT:    jne .LBB210_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_and_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection392:
+; O1-NEXT:  .Lpcsection437:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection393:
+; O1-NEXT:  .Lpcsection438:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB211_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection394:
+; O1-NEXT:  .Lpcsection439:
 ; O1-NEXT:    andl $42, %ebx
-; O1-NEXT:  .Lpcsection395:
+; O1-NEXT:  .Lpcsection440:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection396:
+; O1-NEXT:  .Lpcsection441:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection397:
+; O1-NEXT:  .Lpcsection442:
 ; O1-NEXT:    jne .LBB211_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_and_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection392:
+; O2-NEXT:  .Lpcsection437:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection393:
+; O2-NEXT:  .Lpcsection438:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB211_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection394:
+; O2-NEXT:  .Lpcsection439:
 ; O2-NEXT:    andl $42, %ebx
-; O2-NEXT:  .Lpcsection395:
+; O2-NEXT:  .Lpcsection440:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection396:
+; O2-NEXT:  .Lpcsection441:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection397:
+; O2-NEXT:  .Lpcsection442:
 ; O2-NEXT:    jne .LBB211_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_and_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection392:
+; O3-NEXT:  .Lpcsection437:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection393:
+; O3-NEXT:  .Lpcsection438:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB211_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection394:
+; O3-NEXT:  .Lpcsection439:
 ; O3-NEXT:    andl $42, %ebx
-; O3-NEXT:  .Lpcsection395:
+; O3-NEXT:  .Lpcsection440:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection396:
+; O3-NEXT:  .Lpcsection441:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection397:
+; O3-NEXT:  .Lpcsection442:
 ; O3-NEXT:    jne .LBB211_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_or_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection398:
+; O1-NEXT:  .Lpcsection443:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection399:
+; O1-NEXT:  .Lpcsection444:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB212_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection400:
+; O1-NEXT:  .Lpcsection445:
 ; O1-NEXT:    orq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection401:
+; O1-NEXT:  .Lpcsection446:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection402:
+; O1-NEXT:  .Lpcsection447:
 ; O1-NEXT:    jne .LBB212_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_or_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection398:
+; O2-NEXT:  .Lpcsection443:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection399:
+; O2-NEXT:  .Lpcsection444:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB212_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection400:
+; O2-NEXT:  .Lpcsection445:
 ; O2-NEXT:    orq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection401:
+; O2-NEXT:  .Lpcsection446:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection402:
+; O2-NEXT:  .Lpcsection447:
 ; O2-NEXT:    jne .LBB212_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_or_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection398:
+; O3-NEXT:  .Lpcsection443:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection399:
+; O3-NEXT:  .Lpcsection444:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB212_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection400:
+; O3-NEXT:  .Lpcsection445:
 ; O3-NEXT:    orq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection401:
+; O3-NEXT:  .Lpcsection446:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection402:
+; O3-NEXT:  .Lpcsection447:
 ; O3-NEXT:    jne .LBB212_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xor_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection403:
+; O1-NEXT:  .Lpcsection448:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection404:
+; O1-NEXT:  .Lpcsection449:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB213_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection405:
+; O1-NEXT:  .Lpcsection450:
 ; O1-NEXT:    xorq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection406:
+; O1-NEXT:  .Lpcsection451:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection407:
+; O1-NEXT:  .Lpcsection452:
 ; O1-NEXT:    jne .LBB213_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xor_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection403:
+; O2-NEXT:  .Lpcsection448:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection404:
+; O2-NEXT:  .Lpcsection449:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB213_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection405:
+; O2-NEXT:  .Lpcsection450:
 ; O2-NEXT:    xorq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection406:
+; O2-NEXT:  .Lpcsection451:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection407:
+; O2-NEXT:  .Lpcsection452:
 ; O2-NEXT:    jne .LBB213_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xor_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection403:
+; O3-NEXT:  .Lpcsection448:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection404:
+; O3-NEXT:  .Lpcsection449:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB213_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection405:
+; O3-NEXT:  .Lpcsection450:
 ; O3-NEXT:    xorq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection406:
+; O3-NEXT:  .Lpcsection451:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection407:
+; O3-NEXT:  .Lpcsection452:
 ; O3-NEXT:    jne .LBB213_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_nand_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection408:
+; O1-NEXT:  .Lpcsection453:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection409:
+; O1-NEXT:  .Lpcsection454:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection410:
+; O1-NEXT:  .Lpcsection455:
 ; O1-NEXT:    movq $-1, %rcx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB214_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection411:
+; O1-NEXT:  .Lpcsection456:
 ; O1-NEXT:    notl %ebx
-; O1-NEXT:  .Lpcsection412:
+; O1-NEXT:  .Lpcsection457:
 ; O1-NEXT:    orq $-43, %rbx
-; O1-NEXT:  .Lpcsection413:
+; O1-NEXT:  .Lpcsection458:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection414:
+; O1-NEXT:  .Lpcsection459:
 ; O1-NEXT:    jne .LBB214_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_nand_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection408:
+; O2-NEXT:  .Lpcsection453:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection409:
+; O2-NEXT:  .Lpcsection454:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection410:
+; O2-NEXT:  .Lpcsection455:
 ; O2-NEXT:    movq $-1, %rcx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB214_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection411:
+; O2-NEXT:  .Lpcsection456:
 ; O2-NEXT:    notl %ebx
-; O2-NEXT:  .Lpcsection412:
+; O2-NEXT:  .Lpcsection457:
 ; O2-NEXT:    orq $-43, %rbx
-; O2-NEXT:  .Lpcsection413:
+; O2-NEXT:  .Lpcsection458:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection414:
+; O2-NEXT:  .Lpcsection459:
 ; O2-NEXT:    jne .LBB214_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_nand_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection408:
+; O3-NEXT:  .Lpcsection453:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection409:
+; O3-NEXT:  .Lpcsection454:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection410:
+; O3-NEXT:  .Lpcsection455:
 ; O3-NEXT:    movq $-1, %rcx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB214_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection411:
+; O3-NEXT:  .Lpcsection456:
 ; O3-NEXT:    notl %ebx
-; O3-NEXT:  .Lpcsection412:
+; O3-NEXT:  .Lpcsection457:
 ; O3-NEXT:    orq $-43, %rbx
-; O3-NEXT:  .Lpcsection413:
+; O3-NEXT:  .Lpcsection458:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection414:
+; O3-NEXT:  .Lpcsection459:
 ; O3-NEXT:    jne .LBB214_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xchg_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection415:
+; O1-NEXT:  .Lpcsection460:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection416:
+; O1-NEXT:  .Lpcsection461:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection417:
+; O1-NEXT:  .Lpcsection462:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB215_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection418:
+; O1-NEXT:  .Lpcsection463:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection419:
+; O1-NEXT:  .Lpcsection464:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection420:
+; O1-NEXT:  .Lpcsection465:
 ; O1-NEXT:    jne .LBB215_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xchg_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection415:
+; O2-NEXT:  .Lpcsection460:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection416:
+; O2-NEXT:  .Lpcsection461:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection417:
+; O2-NEXT:  .Lpcsection462:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB215_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection418:
+; O2-NEXT:  .Lpcsection463:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection419:
+; O2-NEXT:  .Lpcsection464:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection420:
+; O2-NEXT:  .Lpcsection465:
 ; O2-NEXT:    jne .LBB215_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xchg_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection415:
+; O3-NEXT:  .Lpcsection460:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection416:
+; O3-NEXT:  .Lpcsection461:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection417:
+; O3-NEXT:  .Lpcsection462:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB215_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection418:
+; O3-NEXT:  .Lpcsection463:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection419:
+; O3-NEXT:  .Lpcsection464:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection420:
+; O3-NEXT:  .Lpcsection465:
 ; O3-NEXT:    jne .LBB215_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_add_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection421:
+; O1-NEXT:  .Lpcsection466:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection422:
+; O1-NEXT:  .Lpcsection467:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB216_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection423:
+; O1-NEXT:  .Lpcsection468:
 ; O1-NEXT:    addq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection424:
+; O1-NEXT:  .Lpcsection469:
 ; O1-NEXT:    adcq $0, %rcx
-; O1-NEXT:  .Lpcsection425:
+; O1-NEXT:  .Lpcsection470:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection426:
+; O1-NEXT:  .Lpcsection471:
 ; O1-NEXT:    jne .LBB216_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_add_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection421:
+; O2-NEXT:  .Lpcsection466:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection422:
+; O2-NEXT:  .Lpcsection467:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB216_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection423:
+; O2-NEXT:  .Lpcsection468:
 ; O2-NEXT:    addq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection424:
+; O2-NEXT:  .Lpcsection469:
 ; O2-NEXT:    adcq $0, %rcx
-; O2-NEXT:  .Lpcsection425:
+; O2-NEXT:  .Lpcsection470:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection426:
+; O2-NEXT:  .Lpcsection471:
 ; O2-NEXT:    jne .LBB216_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_add_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection421:
+; O3-NEXT:  .Lpcsection466:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection422:
+; O3-NEXT:  .Lpcsection467:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB216_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection423:
+; O3-NEXT:  .Lpcsection468:
 ; O3-NEXT:    addq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection424:
+; O3-NEXT:  .Lpcsection469:
 ; O3-NEXT:    adcq $0, %rcx
-; O3-NEXT:  .Lpcsection425:
+; O3-NEXT:  .Lpcsection470:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection426:
+; O3-NEXT:  .Lpcsection471:
 ; O3-NEXT:    jne .LBB216_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_sub_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection427:
+; O1-NEXT:  .Lpcsection472:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection428:
+; O1-NEXT:  .Lpcsection473:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB217_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection429:
+; O1-NEXT:  .Lpcsection474:
 ; O1-NEXT:    addq $-42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection430:
+; O1-NEXT:  .Lpcsection475:
 ; O1-NEXT:    adcq $-1, %rcx
-; O1-NEXT:  .Lpcsection431:
+; O1-NEXT:  .Lpcsection476:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection432:
+; O1-NEXT:  .Lpcsection477:
 ; O1-NEXT:    jne .LBB217_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_sub_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection427:
+; O2-NEXT:  .Lpcsection472:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection428:
+; O2-NEXT:  .Lpcsection473:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB217_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection429:
+; O2-NEXT:  .Lpcsection474:
 ; O2-NEXT:    addq $-42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection430:
+; O2-NEXT:  .Lpcsection475:
 ; O2-NEXT:    adcq $-1, %rcx
-; O2-NEXT:  .Lpcsection431:
+; O2-NEXT:  .Lpcsection476:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection432:
+; O2-NEXT:  .Lpcsection477:
 ; O2-NEXT:    jne .LBB217_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_sub_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection427:
+; O3-NEXT:  .Lpcsection472:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection428:
+; O3-NEXT:  .Lpcsection473:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB217_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection429:
+; O3-NEXT:  .Lpcsection474:
 ; O3-NEXT:    addq $-42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection430:
+; O3-NEXT:  .Lpcsection475:
 ; O3-NEXT:    adcq $-1, %rcx
-; O3-NEXT:  .Lpcsection431:
+; O3-NEXT:  .Lpcsection476:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection432:
+; O3-NEXT:  .Lpcsection477:
 ; O3-NEXT:    jne .LBB217_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_and_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection433:
+; O1-NEXT:  .Lpcsection478:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection434:
+; O1-NEXT:  .Lpcsection479:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB218_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection435:
+; O1-NEXT:  .Lpcsection480:
 ; O1-NEXT:    andl $42, %ebx
-; O1-NEXT:  .Lpcsection436:
+; O1-NEXT:  .Lpcsection481:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection437:
+; O1-NEXT:  .Lpcsection482:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection438:
+; O1-NEXT:  .Lpcsection483:
 ; O1-NEXT:    jne .LBB218_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_and_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection433:
+; O2-NEXT:  .Lpcsection478:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection434:
+; O2-NEXT:  .Lpcsection479:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB218_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection435:
+; O2-NEXT:  .Lpcsection480:
 ; O2-NEXT:    andl $42, %ebx
-; O2-NEXT:  .Lpcsection436:
+; O2-NEXT:  .Lpcsection481:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection437:
+; O2-NEXT:  .Lpcsection482:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection438:
+; O2-NEXT:  .Lpcsection483:
 ; O2-NEXT:    jne .LBB218_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_and_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection433:
+; O3-NEXT:  .Lpcsection478:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection434:
+; O3-NEXT:  .Lpcsection479:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB218_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection435:
+; O3-NEXT:  .Lpcsection480:
 ; O3-NEXT:    andl $42, %ebx
-; O3-NEXT:  .Lpcsection436:
+; O3-NEXT:  .Lpcsection481:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection437:
+; O3-NEXT:  .Lpcsection482:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection438:
+; O3-NEXT:  .Lpcsection483:
 ; O3-NEXT:    jne .LBB218_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_or_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection439:
+; O1-NEXT:  .Lpcsection484:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection440:
+; O1-NEXT:  .Lpcsection485:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB219_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection441:
+; O1-NEXT:  .Lpcsection486:
 ; O1-NEXT:    orq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection442:
+; O1-NEXT:  .Lpcsection487:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection443:
+; O1-NEXT:  .Lpcsection488:
 ; O1-NEXT:    jne .LBB219_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_or_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection439:
+; O2-NEXT:  .Lpcsection484:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection440:
+; O2-NEXT:  .Lpcsection485:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB219_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection441:
+; O2-NEXT:  .Lpcsection486:
 ; O2-NEXT:    orq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection442:
+; O2-NEXT:  .Lpcsection487:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection443:
+; O2-NEXT:  .Lpcsection488:
 ; O2-NEXT:    jne .LBB219_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_or_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection439:
+; O3-NEXT:  .Lpcsection484:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection440:
+; O3-NEXT:  .Lpcsection485:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB219_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection441:
+; O3-NEXT:  .Lpcsection486:
 ; O3-NEXT:    orq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection442:
+; O3-NEXT:  .Lpcsection487:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection443:
+; O3-NEXT:  .Lpcsection488:
 ; O3-NEXT:    jne .LBB219_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xor_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection444:
+; O1-NEXT:  .Lpcsection489:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection445:
+; O1-NEXT:  .Lpcsection490:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB220_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection446:
+; O1-NEXT:  .Lpcsection491:
 ; O1-NEXT:    xorq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection447:
+; O1-NEXT:  .Lpcsection492:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection448:
+; O1-NEXT:  .Lpcsection493:
 ; O1-NEXT:    jne .LBB220_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xor_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection444:
+; O2-NEXT:  .Lpcsection489:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection445:
+; O2-NEXT:  .Lpcsection490:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB220_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection446:
+; O2-NEXT:  .Lpcsection491:
 ; O2-NEXT:    xorq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection447:
+; O2-NEXT:  .Lpcsection492:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection448:
+; O2-NEXT:  .Lpcsection493:
 ; O2-NEXT:    jne .LBB220_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xor_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection444:
+; O3-NEXT:  .Lpcsection489:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection445:
+; O3-NEXT:  .Lpcsection490:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB220_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection446:
+; O3-NEXT:  .Lpcsection491:
 ; O3-NEXT:    xorq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection447:
+; O3-NEXT:  .Lpcsection492:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection448:
+; O3-NEXT:  .Lpcsection493:
 ; O3-NEXT:    jne .LBB220_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_nand_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection449:
+; O1-NEXT:  .Lpcsection494:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection450:
+; O1-NEXT:  .Lpcsection495:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection451:
+; O1-NEXT:  .Lpcsection496:
 ; O1-NEXT:    movq $-1, %rcx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB221_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection452:
+; O1-NEXT:  .Lpcsection497:
 ; O1-NEXT:    notl %ebx
-; O1-NEXT:  .Lpcsection453:
+; O1-NEXT:  .Lpcsection498:
 ; O1-NEXT:    orq $-43, %rbx
-; O1-NEXT:  .Lpcsection454:
+; O1-NEXT:  .Lpcsection499:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection455:
+; O1-NEXT:  .Lpcsection500:
 ; O1-NEXT:    jne .LBB221_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_nand_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection449:
+; O2-NEXT:  .Lpcsection494:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection450:
+; O2-NEXT:  .Lpcsection495:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection451:
+; O2-NEXT:  .Lpcsection496:
 ; O2-NEXT:    movq $-1, %rcx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB221_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection452:
+; O2-NEXT:  .Lpcsection497:
 ; O2-NEXT:    notl %ebx
-; O2-NEXT:  .Lpcsection453:
+; O2-NEXT:  .Lpcsection498:
 ; O2-NEXT:    orq $-43, %rbx
-; O2-NEXT:  .Lpcsection454:
+; O2-NEXT:  .Lpcsection499:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection455:
+; O2-NEXT:  .Lpcsection500:
 ; O2-NEXT:    jne .LBB221_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_nand_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection449:
+; O3-NEXT:  .Lpcsection494:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection450:
+; O3-NEXT:  .Lpcsection495:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection451:
+; O3-NEXT:  .Lpcsection496:
 ; O3-NEXT:    movq $-1, %rcx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB221_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection452:
+; O3-NEXT:  .Lpcsection497:
 ; O3-NEXT:    notl %ebx
-; O3-NEXT:  .Lpcsection453:
+; O3-NEXT:  .Lpcsection498:
 ; O3-NEXT:    orq $-43, %rbx
-; O3-NEXT:  .Lpcsection454:
+; O3-NEXT:  .Lpcsection499:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection455:
+; O3-NEXT:  .Lpcsection500:
 ; O3-NEXT:    jne .LBB221_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xchg_release:
 ; O1:       # %bb.0:
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection456:
+; O1-NEXT:  .Lpcsection501:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection457:
+; O1-NEXT:  .Lpcsection502:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection458:
+; O1-NEXT:  .Lpcsection503:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB222_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection459:
+; O1-NEXT:  .Lpcsection504:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection460:
+; O1-NEXT:  .Lpcsection505:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection461:
+; O1-NEXT:  .Lpcsection506:
 ; O1-NEXT:    jne .LBB222_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xchg_release:
 ; O2:       # %bb.0:
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection456:
+; O2-NEXT:  .Lpcsection501:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection457:
+; O2-NEXT:  .Lpcsection502:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection458:
+; O2-NEXT:  .Lpcsection503:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB222_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection459:
+; O2-NEXT:  .Lpcsection504:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection460:
+; O2-NEXT:  .Lpcsection505:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection461:
+; O2-NEXT:  .Lpcsection506:
 ; O2-NEXT:    jne .LBB222_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xchg_release:
 ; O3:       # %bb.0:
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection456:
+; O3-NEXT:  .Lpcsection501:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection457:
+; O3-NEXT:  .Lpcsection502:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection458:
+; O3-NEXT:  .Lpcsection503:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB222_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection459:
+; O3-NEXT:  .Lpcsection504:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection460:
+; O3-NEXT:  .Lpcsection505:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection461:
+; O3-NEXT:  .Lpcsection506:
 ; O3-NEXT:    jne .LBB222_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
   load volatile i64, ptr @foo, align 8
   %x = atomicrmw xchg ptr %a, i128 42 release, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_add_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection462:
+; O1-NEXT:  .Lpcsection507:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection463:
+; O1-NEXT:  .Lpcsection508:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB223_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection464:
+; O1-NEXT:  .Lpcsection509:
 ; O1-NEXT:    addq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection465:
+; O1-NEXT:  .Lpcsection510:
 ; O1-NEXT:    adcq $0, %rcx
-; O1-NEXT:  .Lpcsection466:
+; O1-NEXT:  .Lpcsection511:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection467:
+; O1-NEXT:  .Lpcsection512:
 ; O1-NEXT:    jne .LBB223_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_add_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection462:
+; O2-NEXT:  .Lpcsection507:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection463:
+; O2-NEXT:  .Lpcsection508:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB223_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection464:
+; O2-NEXT:  .Lpcsection509:
 ; O2-NEXT:    addq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection465:
+; O2-NEXT:  .Lpcsection510:
 ; O2-NEXT:    adcq $0, %rcx
-; O2-NEXT:  .Lpcsection466:
+; O2-NEXT:  .Lpcsection511:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection467:
+; O2-NEXT:  .Lpcsection512:
 ; O2-NEXT:    jne .LBB223_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_add_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection462:
+; O3-NEXT:  .Lpcsection507:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection463:
+; O3-NEXT:  .Lpcsection508:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB223_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection464:
+; O3-NEXT:  .Lpcsection509:
 ; O3-NEXT:    addq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection465:
+; O3-NEXT:  .Lpcsection510:
 ; O3-NEXT:    adcq $0, %rcx
-; O3-NEXT:  .Lpcsection466:
+; O3-NEXT:  .Lpcsection511:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection467:
+; O3-NEXT:  .Lpcsection512:
 ; O3-NEXT:    jne .LBB223_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_sub_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection468:
+; O1-NEXT:  .Lpcsection513:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection469:
+; O1-NEXT:  .Lpcsection514:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB224_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection470:
+; O1-NEXT:  .Lpcsection515:
 ; O1-NEXT:    addq $-42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection471:
+; O1-NEXT:  .Lpcsection516:
 ; O1-NEXT:    adcq $-1, %rcx
-; O1-NEXT:  .Lpcsection472:
+; O1-NEXT:  .Lpcsection517:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection473:
+; O1-NEXT:  .Lpcsection518:
 ; O1-NEXT:    jne .LBB224_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_sub_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection468:
+; O2-NEXT:  .Lpcsection513:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection469:
+; O2-NEXT:  .Lpcsection514:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB224_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection470:
+; O2-NEXT:  .Lpcsection515:
 ; O2-NEXT:    addq $-42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection471:
+; O2-NEXT:  .Lpcsection516:
 ; O2-NEXT:    adcq $-1, %rcx
-; O2-NEXT:  .Lpcsection472:
+; O2-NEXT:  .Lpcsection517:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection473:
+; O2-NEXT:  .Lpcsection518:
 ; O2-NEXT:    jne .LBB224_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_sub_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection468:
+; O3-NEXT:  .Lpcsection513:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection469:
+; O3-NEXT:  .Lpcsection514:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB224_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection470:
+; O3-NEXT:  .Lpcsection515:
 ; O3-NEXT:    addq $-42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection471:
+; O3-NEXT:  .Lpcsection516:
 ; O3-NEXT:    adcq $-1, %rcx
-; O3-NEXT:  .Lpcsection472:
+; O3-NEXT:  .Lpcsection517:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection473:
+; O3-NEXT:  .Lpcsection518:
 ; O3-NEXT:    jne .LBB224_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_and_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection474:
+; O1-NEXT:  .Lpcsection519:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection475:
+; O1-NEXT:  .Lpcsection520:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB225_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection476:
+; O1-NEXT:  .Lpcsection521:
 ; O1-NEXT:    andl $42, %ebx
-; O1-NEXT:  .Lpcsection477:
+; O1-NEXT:  .Lpcsection522:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection478:
+; O1-NEXT:  .Lpcsection523:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection479:
+; O1-NEXT:  .Lpcsection524:
 ; O1-NEXT:    jne .LBB225_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_and_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection474:
+; O2-NEXT:  .Lpcsection519:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection475:
+; O2-NEXT:  .Lpcsection520:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB225_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection476:
+; O2-NEXT:  .Lpcsection521:
 ; O2-NEXT:    andl $42, %ebx
-; O2-NEXT:  .Lpcsection477:
+; O2-NEXT:  .Lpcsection522:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection478:
+; O2-NEXT:  .Lpcsection523:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection479:
+; O2-NEXT:  .Lpcsection524:
 ; O2-NEXT:    jne .LBB225_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_and_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection474:
+; O3-NEXT:  .Lpcsection519:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection475:
+; O3-NEXT:  .Lpcsection520:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB225_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection476:
+; O3-NEXT:  .Lpcsection521:
 ; O3-NEXT:    andl $42, %ebx
-; O3-NEXT:  .Lpcsection477:
+; O3-NEXT:  .Lpcsection522:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection478:
+; O3-NEXT:  .Lpcsection523:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection479:
+; O3-NEXT:  .Lpcsection524:
 ; O3-NEXT:    jne .LBB225_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_or_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection480:
+; O1-NEXT:  .Lpcsection525:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection481:
+; O1-NEXT:  .Lpcsection526:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB226_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection482:
+; O1-NEXT:  .Lpcsection527:
 ; O1-NEXT:    orq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection483:
+; O1-NEXT:  .Lpcsection528:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection484:
+; O1-NEXT:  .Lpcsection529:
 ; O1-NEXT:    jne .LBB226_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_or_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection480:
+; O2-NEXT:  .Lpcsection525:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection481:
+; O2-NEXT:  .Lpcsection526:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB226_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection482:
+; O2-NEXT:  .Lpcsection527:
 ; O2-NEXT:    orq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection483:
+; O2-NEXT:  .Lpcsection528:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection484:
+; O2-NEXT:  .Lpcsection529:
 ; O2-NEXT:    jne .LBB226_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_or_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection480:
+; O3-NEXT:  .Lpcsection525:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection481:
+; O3-NEXT:  .Lpcsection526:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB226_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection482:
+; O3-NEXT:  .Lpcsection527:
 ; O3-NEXT:    orq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection483:
+; O3-NEXT:  .Lpcsection528:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection484:
+; O3-NEXT:  .Lpcsection529:
 ; O3-NEXT:    jne .LBB226_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xor_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection485:
+; O1-NEXT:  .Lpcsection530:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection486:
+; O1-NEXT:  .Lpcsection531:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB227_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection487:
+; O1-NEXT:  .Lpcsection532:
 ; O1-NEXT:    xorq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection488:
+; O1-NEXT:  .Lpcsection533:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection489:
+; O1-NEXT:  .Lpcsection534:
 ; O1-NEXT:    jne .LBB227_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xor_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection485:
+; O2-NEXT:  .Lpcsection530:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection486:
+; O2-NEXT:  .Lpcsection531:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB227_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection487:
+; O2-NEXT:  .Lpcsection532:
 ; O2-NEXT:    xorq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection488:
+; O2-NEXT:  .Lpcsection533:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection489:
+; O2-NEXT:  .Lpcsection534:
 ; O2-NEXT:    jne .LBB227_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xor_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection485:
+; O3-NEXT:  .Lpcsection530:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection486:
+; O3-NEXT:  .Lpcsection531:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB227_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection487:
+; O3-NEXT:  .Lpcsection532:
 ; O3-NEXT:    xorq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection488:
+; O3-NEXT:  .Lpcsection533:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection489:
+; O3-NEXT:  .Lpcsection534:
 ; O3-NEXT:    jne .LBB227_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_nand_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection490:
+; O1-NEXT:  .Lpcsection535:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection491:
+; O1-NEXT:  .Lpcsection536:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection492:
+; O1-NEXT:  .Lpcsection537:
 ; O1-NEXT:    movq $-1, %rcx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB228_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection493:
+; O1-NEXT:  .Lpcsection538:
 ; O1-NEXT:    notl %ebx
-; O1-NEXT:  .Lpcsection494:
+; O1-NEXT:  .Lpcsection539:
 ; O1-NEXT:    orq $-43, %rbx
-; O1-NEXT:  .Lpcsection495:
+; O1-NEXT:  .Lpcsection540:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection496:
+; O1-NEXT:  .Lpcsection541:
 ; O1-NEXT:    jne .LBB228_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_nand_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection490:
+; O2-NEXT:  .Lpcsection535:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection491:
+; O2-NEXT:  .Lpcsection536:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection492:
+; O2-NEXT:  .Lpcsection537:
 ; O2-NEXT:    movq $-1, %rcx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB228_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection493:
+; O2-NEXT:  .Lpcsection538:
 ; O2-NEXT:    notl %ebx
-; O2-NEXT:  .Lpcsection494:
+; O2-NEXT:  .Lpcsection539:
 ; O2-NEXT:    orq $-43, %rbx
-; O2-NEXT:  .Lpcsection495:
+; O2-NEXT:  .Lpcsection540:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection496:
+; O2-NEXT:  .Lpcsection541:
 ; O2-NEXT:    jne .LBB228_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_nand_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection490:
+; O3-NEXT:  .Lpcsection535:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection491:
+; O3-NEXT:  .Lpcsection536:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection492:
+; O3-NEXT:  .Lpcsection537:
 ; O3-NEXT:    movq $-1, %rcx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB228_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection493:
+; O3-NEXT:  .Lpcsection538:
 ; O3-NEXT:    notl %ebx
-; O3-NEXT:  .Lpcsection494:
+; O3-NEXT:  .Lpcsection539:
 ; O3-NEXT:    orq $-43, %rbx
-; O3-NEXT:  .Lpcsection495:
+; O3-NEXT:  .Lpcsection540:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection496:
+; O3-NEXT:  .Lpcsection541:
 ; O3-NEXT:    jne .LBB228_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xchg_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection497:
+; O1-NEXT:  .Lpcsection542:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection498:
+; O1-NEXT:  .Lpcsection543:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection499:
+; O1-NEXT:  .Lpcsection544:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB229_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection500:
+; O1-NEXT:  .Lpcsection545:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection501:
+; O1-NEXT:  .Lpcsection546:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection502:
+; O1-NEXT:  .Lpcsection547:
 ; O1-NEXT:    jne .LBB229_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xchg_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection497:
+; O2-NEXT:  .Lpcsection542:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection498:
+; O2-NEXT:  .Lpcsection543:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection499:
+; O2-NEXT:  .Lpcsection544:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB229_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection500:
+; O2-NEXT:  .Lpcsection545:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection501:
+; O2-NEXT:  .Lpcsection546:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection502:
+; O2-NEXT:  .Lpcsection547:
 ; O2-NEXT:    jne .LBB229_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xchg_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection497:
+; O3-NEXT:  .Lpcsection542:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection498:
+; O3-NEXT:  .Lpcsection543:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection499:
+; O3-NEXT:  .Lpcsection544:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB229_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection500:
+; O3-NEXT:  .Lpcsection545:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection501:
+; O3-NEXT:  .Lpcsection546:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection502:
+; O3-NEXT:  .Lpcsection547:
 ; O3-NEXT:    jne .LBB229_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_add_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection503:
+; O1-NEXT:  .Lpcsection548:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection504:
+; O1-NEXT:  .Lpcsection549:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB230_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection505:
+; O1-NEXT:  .Lpcsection550:
 ; O1-NEXT:    addq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection506:
+; O1-NEXT:  .Lpcsection551:
 ; O1-NEXT:    adcq $0, %rcx
-; O1-NEXT:  .Lpcsection507:
+; O1-NEXT:  .Lpcsection552:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection508:
+; O1-NEXT:  .Lpcsection553:
 ; O1-NEXT:    jne .LBB230_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_add_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection503:
+; O2-NEXT:  .Lpcsection548:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection504:
+; O2-NEXT:  .Lpcsection549:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB230_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection505:
+; O2-NEXT:  .Lpcsection550:
 ; O2-NEXT:    addq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection506:
+; O2-NEXT:  .Lpcsection551:
 ; O2-NEXT:    adcq $0, %rcx
-; O2-NEXT:  .Lpcsection507:
+; O2-NEXT:  .Lpcsection552:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection508:
+; O2-NEXT:  .Lpcsection553:
 ; O2-NEXT:    jne .LBB230_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_add_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection503:
+; O3-NEXT:  .Lpcsection548:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection504:
+; O3-NEXT:  .Lpcsection549:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB230_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection505:
+; O3-NEXT:  .Lpcsection550:
 ; O3-NEXT:    addq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection506:
+; O3-NEXT:  .Lpcsection551:
 ; O3-NEXT:    adcq $0, %rcx
-; O3-NEXT:  .Lpcsection507:
+; O3-NEXT:  .Lpcsection552:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection508:
+; O3-NEXT:  .Lpcsection553:
 ; O3-NEXT:    jne .LBB230_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_sub_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection509:
+; O1-NEXT:  .Lpcsection554:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection510:
+; O1-NEXT:  .Lpcsection555:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB231_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection511:
+; O1-NEXT:  .Lpcsection556:
 ; O1-NEXT:    addq $-42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection512:
+; O1-NEXT:  .Lpcsection557:
 ; O1-NEXT:    adcq $-1, %rcx
-; O1-NEXT:  .Lpcsection513:
+; O1-NEXT:  .Lpcsection558:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection514:
+; O1-NEXT:  .Lpcsection559:
 ; O1-NEXT:    jne .LBB231_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_sub_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection509:
+; O2-NEXT:  .Lpcsection554:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection510:
+; O2-NEXT:  .Lpcsection555:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB231_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection511:
+; O2-NEXT:  .Lpcsection556:
 ; O2-NEXT:    addq $-42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection512:
+; O2-NEXT:  .Lpcsection557:
 ; O2-NEXT:    adcq $-1, %rcx
-; O2-NEXT:  .Lpcsection513:
+; O2-NEXT:  .Lpcsection558:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection514:
+; O2-NEXT:  .Lpcsection559:
 ; O2-NEXT:    jne .LBB231_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_sub_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection509:
+; O3-NEXT:  .Lpcsection554:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection510:
+; O3-NEXT:  .Lpcsection555:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB231_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection511:
+; O3-NEXT:  .Lpcsection556:
 ; O3-NEXT:    addq $-42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection512:
+; O3-NEXT:  .Lpcsection557:
 ; O3-NEXT:    adcq $-1, %rcx
-; O3-NEXT:  .Lpcsection513:
+; O3-NEXT:  .Lpcsection558:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection514:
+; O3-NEXT:  .Lpcsection559:
 ; O3-NEXT:    jne .LBB231_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_and_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection515:
+; O1-NEXT:  .Lpcsection560:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection516:
+; O1-NEXT:  .Lpcsection561:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB232_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection517:
+; O1-NEXT:  .Lpcsection562:
 ; O1-NEXT:    andl $42, %ebx
-; O1-NEXT:  .Lpcsection518:
+; O1-NEXT:  .Lpcsection563:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection519:
+; O1-NEXT:  .Lpcsection564:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection520:
+; O1-NEXT:  .Lpcsection565:
 ; O1-NEXT:    jne .LBB232_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_and_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection515:
+; O2-NEXT:  .Lpcsection560:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection516:
+; O2-NEXT:  .Lpcsection561:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB232_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection517:
+; O2-NEXT:  .Lpcsection562:
 ; O2-NEXT:    andl $42, %ebx
-; O2-NEXT:  .Lpcsection518:
+; O2-NEXT:  .Lpcsection563:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection519:
+; O2-NEXT:  .Lpcsection564:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection520:
+; O2-NEXT:  .Lpcsection565:
 ; O2-NEXT:    jne .LBB232_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_and_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection515:
+; O3-NEXT:  .Lpcsection560:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection516:
+; O3-NEXT:  .Lpcsection561:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB232_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection517:
+; O3-NEXT:  .Lpcsection562:
 ; O3-NEXT:    andl $42, %ebx
-; O3-NEXT:  .Lpcsection518:
+; O3-NEXT:  .Lpcsection563:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection519:
+; O3-NEXT:  .Lpcsection564:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection520:
+; O3-NEXT:  .Lpcsection565:
 ; O3-NEXT:    jne .LBB232_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_or_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection521:
+; O1-NEXT:  .Lpcsection566:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection522:
+; O1-NEXT:  .Lpcsection567:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB233_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection523:
+; O1-NEXT:  .Lpcsection568:
 ; O1-NEXT:    orq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection524:
+; O1-NEXT:  .Lpcsection569:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection525:
+; O1-NEXT:  .Lpcsection570:
 ; O1-NEXT:    jne .LBB233_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_or_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection521:
+; O2-NEXT:  .Lpcsection566:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection522:
+; O2-NEXT:  .Lpcsection567:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB233_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection523:
+; O2-NEXT:  .Lpcsection568:
 ; O2-NEXT:    orq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection524:
+; O2-NEXT:  .Lpcsection569:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection525:
+; O2-NEXT:  .Lpcsection570:
 ; O2-NEXT:    jne .LBB233_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_or_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection521:
+; O3-NEXT:  .Lpcsection566:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection522:
+; O3-NEXT:  .Lpcsection567:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB233_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection523:
+; O3-NEXT:  .Lpcsection568:
 ; O3-NEXT:    orq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection524:
+; O3-NEXT:  .Lpcsection569:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection525:
+; O3-NEXT:  .Lpcsection570:
 ; O3-NEXT:    jne .LBB233_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xor_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection526:
+; O1-NEXT:  .Lpcsection571:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection527:
+; O1-NEXT:  .Lpcsection572:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB234_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection528:
+; O1-NEXT:  .Lpcsection573:
 ; O1-NEXT:    xorq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection529:
+; O1-NEXT:  .Lpcsection574:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection530:
+; O1-NEXT:  .Lpcsection575:
 ; O1-NEXT:    jne .LBB234_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xor_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection526:
+; O2-NEXT:  .Lpcsection571:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection527:
+; O2-NEXT:  .Lpcsection572:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB234_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection528:
+; O2-NEXT:  .Lpcsection573:
 ; O2-NEXT:    xorq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection529:
+; O2-NEXT:  .Lpcsection574:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection530:
+; O2-NEXT:  .Lpcsection575:
 ; O2-NEXT:    jne .LBB234_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xor_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection526:
+; O3-NEXT:  .Lpcsection571:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection527:
+; O3-NEXT:  .Lpcsection572:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB234_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection528:
+; O3-NEXT:  .Lpcsection573:
 ; O3-NEXT:    xorq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection529:
+; O3-NEXT:  .Lpcsection574:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection530:
+; O3-NEXT:  .Lpcsection575:
 ; O3-NEXT:    jne .LBB234_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_nand_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection531:
+; O1-NEXT:  .Lpcsection576:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection532:
+; O1-NEXT:  .Lpcsection577:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection533:
+; O1-NEXT:  .Lpcsection578:
 ; O1-NEXT:    movq $-1, %rcx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB235_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection534:
+; O1-NEXT:  .Lpcsection579:
 ; O1-NEXT:    notl %ebx
-; O1-NEXT:  .Lpcsection535:
+; O1-NEXT:  .Lpcsection580:
 ; O1-NEXT:    orq $-43, %rbx
-; O1-NEXT:  .Lpcsection536:
+; O1-NEXT:  .Lpcsection581:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection537:
+; O1-NEXT:  .Lpcsection582:
 ; O1-NEXT:    jne .LBB235_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_nand_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection531:
+; O2-NEXT:  .Lpcsection576:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection532:
+; O2-NEXT:  .Lpcsection577:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection533:
+; O2-NEXT:  .Lpcsection578:
 ; O2-NEXT:    movq $-1, %rcx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB235_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection534:
+; O2-NEXT:  .Lpcsection579:
 ; O2-NEXT:    notl %ebx
-; O2-NEXT:  .Lpcsection535:
+; O2-NEXT:  .Lpcsection580:
 ; O2-NEXT:    orq $-43, %rbx
-; O2-NEXT:  .Lpcsection536:
+; O2-NEXT:  .Lpcsection581:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection537:
+; O2-NEXT:  .Lpcsection582:
 ; O2-NEXT:    jne .LBB235_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_nand_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection531:
+; O3-NEXT:  .Lpcsection576:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection532:
+; O3-NEXT:  .Lpcsection577:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection533:
+; O3-NEXT:  .Lpcsection578:
 ; O3-NEXT:    movq $-1, %rcx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB235_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection534:
+; O3-NEXT:  .Lpcsection579:
 ; O3-NEXT:    notl %ebx
-; O3-NEXT:  .Lpcsection535:
+; O3-NEXT:  .Lpcsection580:
 ; O3-NEXT:    orq $-43, %rbx
-; O3-NEXT:  .Lpcsection536:
+; O3-NEXT:  .Lpcsection581:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection537:
+; O3-NEXT:  .Lpcsection582:
 ; O3-NEXT:    jne .LBB235_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xchg_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection538:
+; O1-NEXT:  .Lpcsection583:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection539:
+; O1-NEXT:  .Lpcsection584:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection540:
+; O1-NEXT:  .Lpcsection585:
 ; O1-NEXT:    movl $42, %ebx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB236_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
-; O1-NEXT:  .Lpcsection541:
+; O1-NEXT:  .Lpcsection586:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection542:
+; O1-NEXT:  .Lpcsection587:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection543:
+; O1-NEXT:  .Lpcsection588:
 ; O1-NEXT:    jne .LBB236_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xchg_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection538:
+; O2-NEXT:  .Lpcsection583:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection539:
+; O2-NEXT:  .Lpcsection584:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection540:
+; O2-NEXT:  .Lpcsection585:
 ; O2-NEXT:    movl $42, %ebx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB236_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
-; O2-NEXT:  .Lpcsection541:
+; O2-NEXT:  .Lpcsection586:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection542:
+; O2-NEXT:  .Lpcsection587:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection543:
+; O2-NEXT:  .Lpcsection588:
 ; O2-NEXT:    jne .LBB236_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xchg_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection538:
+; O3-NEXT:  .Lpcsection583:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection539:
+; O3-NEXT:  .Lpcsection584:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection540:
+; O3-NEXT:  .Lpcsection585:
 ; O3-NEXT:    movl $42, %ebx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB236_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
-; O3-NEXT:  .Lpcsection541:
+; O3-NEXT:  .Lpcsection586:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection542:
+; O3-NEXT:  .Lpcsection587:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection543:
+; O3-NEXT:  .Lpcsection588:
 ; O3-NEXT:    jne .LBB236_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_add_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection544:
+; O1-NEXT:  .Lpcsection589:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection545:
+; O1-NEXT:  .Lpcsection590:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB237_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection546:
+; O1-NEXT:  .Lpcsection591:
 ; O1-NEXT:    addq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection547:
+; O1-NEXT:  .Lpcsection592:
 ; O1-NEXT:    adcq $0, %rcx
-; O1-NEXT:  .Lpcsection548:
+; O1-NEXT:  .Lpcsection593:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection549:
+; O1-NEXT:  .Lpcsection594:
 ; O1-NEXT:    jne .LBB237_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_add_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection544:
+; O2-NEXT:  .Lpcsection589:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection545:
+; O2-NEXT:  .Lpcsection590:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB237_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection546:
+; O2-NEXT:  .Lpcsection591:
 ; O2-NEXT:    addq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection547:
+; O2-NEXT:  .Lpcsection592:
 ; O2-NEXT:    adcq $0, %rcx
-; O2-NEXT:  .Lpcsection548:
+; O2-NEXT:  .Lpcsection593:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection549:
+; O2-NEXT:  .Lpcsection594:
 ; O2-NEXT:    jne .LBB237_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_add_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection544:
+; O3-NEXT:  .Lpcsection589:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection545:
+; O3-NEXT:  .Lpcsection590:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB237_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection546:
+; O3-NEXT:  .Lpcsection591:
 ; O3-NEXT:    addq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection547:
+; O3-NEXT:  .Lpcsection592:
 ; O3-NEXT:    adcq $0, %rcx
-; O3-NEXT:  .Lpcsection548:
+; O3-NEXT:  .Lpcsection593:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection549:
+; O3-NEXT:  .Lpcsection594:
 ; O3-NEXT:    jne .LBB237_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_sub_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection550:
+; O1-NEXT:  .Lpcsection595:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection551:
+; O1-NEXT:  .Lpcsection596:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB238_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection552:
+; O1-NEXT:  .Lpcsection597:
 ; O1-NEXT:    addq $-42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection553:
+; O1-NEXT:  .Lpcsection598:
 ; O1-NEXT:    adcq $-1, %rcx
-; O1-NEXT:  .Lpcsection554:
+; O1-NEXT:  .Lpcsection599:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection555:
+; O1-NEXT:  .Lpcsection600:
 ; O1-NEXT:    jne .LBB238_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_sub_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection550:
+; O2-NEXT:  .Lpcsection595:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection551:
+; O2-NEXT:  .Lpcsection596:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB238_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection552:
+; O2-NEXT:  .Lpcsection597:
 ; O2-NEXT:    addq $-42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection553:
+; O2-NEXT:  .Lpcsection598:
 ; O2-NEXT:    adcq $-1, %rcx
-; O2-NEXT:  .Lpcsection554:
+; O2-NEXT:  .Lpcsection599:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection555:
+; O2-NEXT:  .Lpcsection600:
 ; O2-NEXT:    jne .LBB238_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_sub_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection550:
+; O3-NEXT:  .Lpcsection595:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection551:
+; O3-NEXT:  .Lpcsection596:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB238_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection552:
+; O3-NEXT:  .Lpcsection597:
 ; O3-NEXT:    addq $-42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection553:
+; O3-NEXT:  .Lpcsection598:
 ; O3-NEXT:    adcq $-1, %rcx
-; O3-NEXT:  .Lpcsection554:
+; O3-NEXT:  .Lpcsection599:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection555:
+; O3-NEXT:  .Lpcsection600:
 ; O3-NEXT:    jne .LBB238_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_and_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection556:
+; O1-NEXT:  .Lpcsection601:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection557:
+; O1-NEXT:  .Lpcsection602:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB239_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection558:
+; O1-NEXT:  .Lpcsection603:
 ; O1-NEXT:    andl $42, %ebx
-; O1-NEXT:  .Lpcsection559:
+; O1-NEXT:  .Lpcsection604:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection560:
+; O1-NEXT:  .Lpcsection605:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection561:
+; O1-NEXT:  .Lpcsection606:
 ; O1-NEXT:    jne .LBB239_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_and_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection556:
+; O2-NEXT:  .Lpcsection601:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection557:
+; O2-NEXT:  .Lpcsection602:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB239_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection558:
+; O2-NEXT:  .Lpcsection603:
 ; O2-NEXT:    andl $42, %ebx
-; O2-NEXT:  .Lpcsection559:
+; O2-NEXT:  .Lpcsection604:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection560:
+; O2-NEXT:  .Lpcsection605:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection561:
+; O2-NEXT:  .Lpcsection606:
 ; O2-NEXT:    jne .LBB239_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_and_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection556:
+; O3-NEXT:  .Lpcsection601:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection557:
+; O3-NEXT:  .Lpcsection602:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB239_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection558:
+; O3-NEXT:  .Lpcsection603:
 ; O3-NEXT:    andl $42, %ebx
-; O3-NEXT:  .Lpcsection559:
+; O3-NEXT:  .Lpcsection604:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection560:
+; O3-NEXT:  .Lpcsection605:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection561:
+; O3-NEXT:  .Lpcsection606:
 ; O3-NEXT:    jne .LBB239_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_or_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection562:
+; O1-NEXT:  .Lpcsection607:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection563:
+; O1-NEXT:  .Lpcsection608:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB240_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection564:
+; O1-NEXT:  .Lpcsection609:
 ; O1-NEXT:    orq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection565:
+; O1-NEXT:  .Lpcsection610:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection566:
+; O1-NEXT:  .Lpcsection611:
 ; O1-NEXT:    jne .LBB240_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_or_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection562:
+; O2-NEXT:  .Lpcsection607:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection563:
+; O2-NEXT:  .Lpcsection608:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB240_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection564:
+; O2-NEXT:  .Lpcsection609:
 ; O2-NEXT:    orq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection565:
+; O2-NEXT:  .Lpcsection610:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection566:
+; O2-NEXT:  .Lpcsection611:
 ; O2-NEXT:    jne .LBB240_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_or_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection562:
+; O3-NEXT:  .Lpcsection607:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection563:
+; O3-NEXT:  .Lpcsection608:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB240_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection564:
+; O3-NEXT:  .Lpcsection609:
 ; O3-NEXT:    orq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection565:
+; O3-NEXT:  .Lpcsection610:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection566:
+; O3-NEXT:  .Lpcsection611:
 ; O3-NEXT:    jne .LBB240_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_xor_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection567:
+; O1-NEXT:  .Lpcsection612:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection568:
+; O1-NEXT:  .Lpcsection613:
 ; O1-NEXT:    movq 8(%rdi), %rdx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB241_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movq %rax, %rbx
-; O1-NEXT:  .Lpcsection569:
+; O1-NEXT:  .Lpcsection614:
 ; O1-NEXT:    xorq $42, %rbx
 ; O1-NEXT:    movq %rdx, %rcx
-; O1-NEXT:  .Lpcsection570:
+; O1-NEXT:  .Lpcsection615:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection571:
+; O1-NEXT:  .Lpcsection616:
 ; O1-NEXT:    jne .LBB241_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_xor_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection567:
+; O2-NEXT:  .Lpcsection612:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection568:
+; O2-NEXT:  .Lpcsection613:
 ; O2-NEXT:    movq 8(%rdi), %rdx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB241_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movq %rax, %rbx
-; O2-NEXT:  .Lpcsection569:
+; O2-NEXT:  .Lpcsection614:
 ; O2-NEXT:    xorq $42, %rbx
 ; O2-NEXT:    movq %rdx, %rcx
-; O2-NEXT:  .Lpcsection570:
+; O2-NEXT:  .Lpcsection615:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection571:
+; O2-NEXT:  .Lpcsection616:
 ; O2-NEXT:    jne .LBB241_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_xor_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection567:
+; O3-NEXT:  .Lpcsection612:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection568:
+; O3-NEXT:  .Lpcsection613:
 ; O3-NEXT:    movq 8(%rdi), %rdx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB241_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movq %rax, %rbx
-; O3-NEXT:  .Lpcsection569:
+; O3-NEXT:  .Lpcsection614:
 ; O3-NEXT:    xorq $42, %rbx
 ; O3-NEXT:    movq %rdx, %rcx
-; O3-NEXT:  .Lpcsection570:
+; O3-NEXT:  .Lpcsection615:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection571:
+; O3-NEXT:  .Lpcsection616:
 ; O3-NEXT:    jne .LBB241_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_nand_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection572:
+; O1-NEXT:  .Lpcsection617:
 ; O1-NEXT:    movq (%rdi), %rax
-; O1-NEXT:  .Lpcsection573:
+; O1-NEXT:  .Lpcsection618:
 ; O1-NEXT:    movq 8(%rdi), %rdx
-; O1-NEXT:  .Lpcsection574:
+; O1-NEXT:  .Lpcsection619:
 ; O1-NEXT:    movq $-1, %rcx
 ; O1-NEXT:    .p2align 4, 0x90
 ; O1-NEXT:  .LBB242_1: # %atomicrmw.start
 ; O1-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O1-NEXT:    movl %eax, %ebx
-; O1-NEXT:  .Lpcsection575:
+; O1-NEXT:  .Lpcsection620:
 ; O1-NEXT:    notl %ebx
-; O1-NEXT:  .Lpcsection576:
+; O1-NEXT:  .Lpcsection621:
 ; O1-NEXT:    orq $-43, %rbx
-; O1-NEXT:  .Lpcsection577:
+; O1-NEXT:  .Lpcsection622:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection578:
+; O1-NEXT:  .Lpcsection623:
 ; O1-NEXT:    jne .LBB242_1
 ; O1-NEXT:  # %bb.2: # %atomicrmw.end
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_nand_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection572:
+; O2-NEXT:  .Lpcsection617:
 ; O2-NEXT:    movq (%rdi), %rax
-; O2-NEXT:  .Lpcsection573:
+; O2-NEXT:  .Lpcsection618:
 ; O2-NEXT:    movq 8(%rdi), %rdx
-; O2-NEXT:  .Lpcsection574:
+; O2-NEXT:  .Lpcsection619:
 ; O2-NEXT:    movq $-1, %rcx
 ; O2-NEXT:    .p2align 4, 0x90
 ; O2-NEXT:  .LBB242_1: # %atomicrmw.start
 ; O2-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O2-NEXT:    movl %eax, %ebx
-; O2-NEXT:  .Lpcsection575:
+; O2-NEXT:  .Lpcsection620:
 ; O2-NEXT:    notl %ebx
-; O2-NEXT:  .Lpcsection576:
+; O2-NEXT:  .Lpcsection621:
 ; O2-NEXT:    orq $-43, %rbx
-; O2-NEXT:  .Lpcsection577:
+; O2-NEXT:  .Lpcsection622:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection578:
+; O2-NEXT:  .Lpcsection623:
 ; O2-NEXT:    jne .LBB242_1
 ; O2-NEXT:  # %bb.2: # %atomicrmw.end
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_nand_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection572:
+; O3-NEXT:  .Lpcsection617:
 ; O3-NEXT:    movq (%rdi), %rax
-; O3-NEXT:  .Lpcsection573:
+; O3-NEXT:  .Lpcsection618:
 ; O3-NEXT:    movq 8(%rdi), %rdx
-; O3-NEXT:  .Lpcsection574:
+; O3-NEXT:  .Lpcsection619:
 ; O3-NEXT:    movq $-1, %rcx
 ; O3-NEXT:    .p2align 4, 0x90
 ; O3-NEXT:  .LBB242_1: # %atomicrmw.start
 ; O3-NEXT:    # =>This Inner Loop Header: Depth=1
 ; O3-NEXT:    movl %eax, %ebx
-; O3-NEXT:  .Lpcsection575:
+; O3-NEXT:  .Lpcsection620:
 ; O3-NEXT:    notl %ebx
-; O3-NEXT:  .Lpcsection576:
+; O3-NEXT:  .Lpcsection621:
 ; O3-NEXT:    orq $-43, %rbx
-; O3-NEXT:  .Lpcsection577:
+; O3-NEXT:  .Lpcsection622:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection578:
+; O3-NEXT:  .Lpcsection623:
 ; O3-NEXT:    jne .LBB242_1
 ; O3-NEXT:  # %bb.2: # %atomicrmw.end
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_cas_monotonic:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection579:
+; O1-NEXT:  .Lpcsection624:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection580:
+; O1-NEXT:  .Lpcsection625:
 ; O1-NEXT:    movl $1, %ebx
-; O1-NEXT:  .Lpcsection581:
+; O1-NEXT:  .Lpcsection626:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection582:
+; O1-NEXT:  .Lpcsection627:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection583:
+; O1-NEXT:  .Lpcsection628:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection584:
+; O1-NEXT:  .Lpcsection629:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection585:
+; O1-NEXT:  .Lpcsection630:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection586:
+; O1-NEXT:  .Lpcsection631:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection587:
+; O1-NEXT:  .Lpcsection632:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection588:
+; O1-NEXT:  .Lpcsection633:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection589:
+; O1-NEXT:  .Lpcsection634:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection590:
+; O1-NEXT:  .Lpcsection635:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection591:
+; O1-NEXT:  .Lpcsection636:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_cas_monotonic:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection579:
+; O2-NEXT:  .Lpcsection624:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection580:
+; O2-NEXT:  .Lpcsection625:
 ; O2-NEXT:    movl $1, %ebx
-; O2-NEXT:  .Lpcsection581:
+; O2-NEXT:  .Lpcsection626:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection582:
+; O2-NEXT:  .Lpcsection627:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection583:
+; O2-NEXT:  .Lpcsection628:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection584:
+; O2-NEXT:  .Lpcsection629:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection585:
+; O2-NEXT:  .Lpcsection630:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection586:
+; O2-NEXT:  .Lpcsection631:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection587:
+; O2-NEXT:  .Lpcsection632:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection588:
+; O2-NEXT:  .Lpcsection633:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection589:
+; O2-NEXT:  .Lpcsection634:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection590:
+; O2-NEXT:  .Lpcsection635:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection591:
+; O2-NEXT:  .Lpcsection636:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_cas_monotonic:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection579:
+; O3-NEXT:  .Lpcsection624:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection580:
+; O3-NEXT:  .Lpcsection625:
 ; O3-NEXT:    movl $1, %ebx
-; O3-NEXT:  .Lpcsection581:
+; O3-NEXT:  .Lpcsection626:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection582:
+; O3-NEXT:  .Lpcsection627:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection583:
+; O3-NEXT:  .Lpcsection628:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection584:
+; O3-NEXT:  .Lpcsection629:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection585:
+; O3-NEXT:  .Lpcsection630:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection586:
+; O3-NEXT:  .Lpcsection631:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection587:
+; O3-NEXT:  .Lpcsection632:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection588:
+; O3-NEXT:  .Lpcsection633:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection589:
+; O3-NEXT:  .Lpcsection634:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection590:
+; O3-NEXT:  .Lpcsection635:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection591:
+; O3-NEXT:  .Lpcsection636:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i128 42, i128 1 monotonic monotonic, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_cas_acquire:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection592:
+; O1-NEXT:  .Lpcsection637:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection593:
+; O1-NEXT:  .Lpcsection638:
 ; O1-NEXT:    movl $1, %ebx
-; O1-NEXT:  .Lpcsection594:
+; O1-NEXT:  .Lpcsection639:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection595:
+; O1-NEXT:  .Lpcsection640:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection596:
+; O1-NEXT:  .Lpcsection641:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection597:
+; O1-NEXT:  .Lpcsection642:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection598:
+; O1-NEXT:  .Lpcsection643:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection599:
+; O1-NEXT:  .Lpcsection644:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection600:
+; O1-NEXT:  .Lpcsection645:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection601:
+; O1-NEXT:  .Lpcsection646:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection602:
+; O1-NEXT:  .Lpcsection647:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection603:
+; O1-NEXT:  .Lpcsection648:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection604:
+; O1-NEXT:  .Lpcsection649:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_cas_acquire:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection592:
+; O2-NEXT:  .Lpcsection637:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection593:
+; O2-NEXT:  .Lpcsection638:
 ; O2-NEXT:    movl $1, %ebx
-; O2-NEXT:  .Lpcsection594:
+; O2-NEXT:  .Lpcsection639:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection595:
+; O2-NEXT:  .Lpcsection640:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection596:
+; O2-NEXT:  .Lpcsection641:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection597:
+; O2-NEXT:  .Lpcsection642:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection598:
+; O2-NEXT:  .Lpcsection643:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection599:
+; O2-NEXT:  .Lpcsection644:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection600:
+; O2-NEXT:  .Lpcsection645:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection601:
+; O2-NEXT:  .Lpcsection646:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection602:
+; O2-NEXT:  .Lpcsection647:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection603:
+; O2-NEXT:  .Lpcsection648:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection604:
+; O2-NEXT:  .Lpcsection649:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_cas_acquire:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection592:
+; O3-NEXT:  .Lpcsection637:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection593:
+; O3-NEXT:  .Lpcsection638:
 ; O3-NEXT:    movl $1, %ebx
-; O3-NEXT:  .Lpcsection594:
+; O3-NEXT:  .Lpcsection639:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection595:
+; O3-NEXT:  .Lpcsection640:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection596:
+; O3-NEXT:  .Lpcsection641:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection597:
+; O3-NEXT:  .Lpcsection642:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection598:
+; O3-NEXT:  .Lpcsection643:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection599:
+; O3-NEXT:  .Lpcsection644:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection600:
+; O3-NEXT:  .Lpcsection645:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection601:
+; O3-NEXT:  .Lpcsection646:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection602:
+; O3-NEXT:  .Lpcsection647:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection603:
+; O3-NEXT:  .Lpcsection648:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection604:
+; O3-NEXT:  .Lpcsection649:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i128 42, i128 1 acquire monotonic, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_cas_release:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection605:
+; O1-NEXT:  .Lpcsection650:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection606:
+; O1-NEXT:  .Lpcsection651:
 ; O1-NEXT:    movl $1, %ebx
-; O1-NEXT:  .Lpcsection607:
+; O1-NEXT:  .Lpcsection652:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection608:
+; O1-NEXT:  .Lpcsection653:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection609:
+; O1-NEXT:  .Lpcsection654:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection610:
+; O1-NEXT:  .Lpcsection655:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection611:
+; O1-NEXT:  .Lpcsection656:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection612:
+; O1-NEXT:  .Lpcsection657:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection613:
+; O1-NEXT:  .Lpcsection658:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection614:
+; O1-NEXT:  .Lpcsection659:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection615:
+; O1-NEXT:  .Lpcsection660:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection616:
+; O1-NEXT:  .Lpcsection661:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection617:
+; O1-NEXT:  .Lpcsection662:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_cas_release:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection605:
+; O2-NEXT:  .Lpcsection650:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection606:
+; O2-NEXT:  .Lpcsection651:
 ; O2-NEXT:    movl $1, %ebx
-; O2-NEXT:  .Lpcsection607:
+; O2-NEXT:  .Lpcsection652:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection608:
+; O2-NEXT:  .Lpcsection653:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection609:
+; O2-NEXT:  .Lpcsection654:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection610:
+; O2-NEXT:  .Lpcsection655:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection611:
+; O2-NEXT:  .Lpcsection656:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection612:
+; O2-NEXT:  .Lpcsection657:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection613:
+; O2-NEXT:  .Lpcsection658:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection614:
+; O2-NEXT:  .Lpcsection659:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection615:
+; O2-NEXT:  .Lpcsection660:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection616:
+; O2-NEXT:  .Lpcsection661:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection617:
+; O2-NEXT:  .Lpcsection662:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_cas_release:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection605:
+; O3-NEXT:  .Lpcsection650:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection606:
+; O3-NEXT:  .Lpcsection651:
 ; O3-NEXT:    movl $1, %ebx
-; O3-NEXT:  .Lpcsection607:
+; O3-NEXT:  .Lpcsection652:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection608:
+; O3-NEXT:  .Lpcsection653:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection609:
+; O3-NEXT:  .Lpcsection654:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection610:
+; O3-NEXT:  .Lpcsection655:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection611:
+; O3-NEXT:  .Lpcsection656:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection612:
+; O3-NEXT:  .Lpcsection657:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection613:
+; O3-NEXT:  .Lpcsection658:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection614:
+; O3-NEXT:  .Lpcsection659:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection615:
+; O3-NEXT:  .Lpcsection660:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection616:
+; O3-NEXT:  .Lpcsection661:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection617:
+; O3-NEXT:  .Lpcsection662:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i128 42, i128 1 release monotonic, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_cas_acq_rel:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection618:
+; O1-NEXT:  .Lpcsection663:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection619:
+; O1-NEXT:  .Lpcsection664:
 ; O1-NEXT:    movl $1, %ebx
-; O1-NEXT:  .Lpcsection620:
+; O1-NEXT:  .Lpcsection665:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection621:
+; O1-NEXT:  .Lpcsection666:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection622:
+; O1-NEXT:  .Lpcsection667:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection623:
+; O1-NEXT:  .Lpcsection668:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection624:
+; O1-NEXT:  .Lpcsection669:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection625:
+; O1-NEXT:  .Lpcsection670:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection626:
+; O1-NEXT:  .Lpcsection671:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection627:
+; O1-NEXT:  .Lpcsection672:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection628:
+; O1-NEXT:  .Lpcsection673:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection629:
+; O1-NEXT:  .Lpcsection674:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection630:
+; O1-NEXT:  .Lpcsection675:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $1, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_cas_acq_rel:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection618:
+; O2-NEXT:  .Lpcsection663:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection619:
+; O2-NEXT:  .Lpcsection664:
 ; O2-NEXT:    movl $1, %ebx
-; O2-NEXT:  .Lpcsection620:
+; O2-NEXT:  .Lpcsection665:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection621:
+; O2-NEXT:  .Lpcsection666:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection622:
+; O2-NEXT:  .Lpcsection667:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection623:
+; O2-NEXT:  .Lpcsection668:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection624:
+; O2-NEXT:  .Lpcsection669:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection625:
+; O2-NEXT:  .Lpcsection670:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection626:
+; O2-NEXT:  .Lpcsection671:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection627:
+; O2-NEXT:  .Lpcsection672:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection628:
+; O2-NEXT:  .Lpcsection673:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection629:
+; O2-NEXT:  .Lpcsection674:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection630:
+; O2-NEXT:  .Lpcsection675:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $1, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_cas_acq_rel:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection618:
+; O3-NEXT:  .Lpcsection663:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection619:
+; O3-NEXT:  .Lpcsection664:
 ; O3-NEXT:    movl $1, %ebx
-; O3-NEXT:  .Lpcsection620:
+; O3-NEXT:  .Lpcsection665:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection621:
+; O3-NEXT:  .Lpcsection666:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection622:
+; O3-NEXT:  .Lpcsection667:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection623:
+; O3-NEXT:  .Lpcsection668:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection624:
+; O3-NEXT:  .Lpcsection669:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection625:
+; O3-NEXT:  .Lpcsection670:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection626:
+; O3-NEXT:  .Lpcsection671:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection627:
+; O3-NEXT:  .Lpcsection672:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection628:
+; O3-NEXT:  .Lpcsection673:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection629:
+; O3-NEXT:  .Lpcsection674:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection630:
+; O3-NEXT:  .Lpcsection675:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $1, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i128 42, i128 1 acq_rel monotonic, align 16, !pcsections !0

 ; O0-NEXT:    retq
 ;
 ; O1-LABEL: atomic128_cas_seq_cst:
 ; O1:       # %bb.0: # %entry
 ; O1-NEXT:    pushq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 16
 ; O1-NEXT:    .cfi_offset %rbx, -16
 ; O1-NEXT:    movq foo(%rip), %rax
-; O1-NEXT:  .Lpcsection631:
+; O1-NEXT:  .Lpcsection676:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection632:
+; O1-NEXT:  .Lpcsection677:
 ; O1-NEXT:    movl $1, %ebx
-; O1-NEXT:  .Lpcsection633:
+; O1-NEXT:  .Lpcsection678:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection634:
+; O1-NEXT:  .Lpcsection679:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection635:
+; O1-NEXT:  .Lpcsection680:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection636:
+; O1-NEXT:  .Lpcsection681:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection637:
+; O1-NEXT:  .Lpcsection682:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection638:
+; O1-NEXT:  .Lpcsection683:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection639:
+; O1-NEXT:  .Lpcsection684:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
-; O1-NEXT:  .Lpcsection640:
+; O1-NEXT:  .Lpcsection685:
 ; O1-NEXT:    movl $42, %eax
-; O1-NEXT:  .Lpcsection641:
+; O1-NEXT:  .Lpcsection686:
 ; O1-NEXT:    xorl %edx, %edx
-; O1-NEXT:  .Lpcsection642:
+; O1-NEXT:  .Lpcsection687:
 ; O1-NEXT:    xorl %ecx, %ecx
-; O1-NEXT:  .Lpcsection643:
+; O1-NEXT:  .Lpcsection688:
 ; O1-NEXT:    lock cmpxchg16b (%rdi)
 ; O1-NEXT:    movq $3, foo(%rip)
 ; O1-NEXT:    popq %rbx
 ; O1-NEXT:    .cfi_def_cfa_offset 8
 ; O1-NEXT:    retq
 ;
 ; O2-LABEL: atomic128_cas_seq_cst:
 ; O2:       # %bb.0: # %entry
 ; O2-NEXT:    pushq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 16
 ; O2-NEXT:    .cfi_offset %rbx, -16
 ; O2-NEXT:    movq foo(%rip), %rax
-; O2-NEXT:  .Lpcsection631:
+; O2-NEXT:  .Lpcsection676:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection632:
+; O2-NEXT:  .Lpcsection677:
 ; O2-NEXT:    movl $1, %ebx
-; O2-NEXT:  .Lpcsection633:
+; O2-NEXT:  .Lpcsection678:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection634:
+; O2-NEXT:  .Lpcsection679:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection635:
+; O2-NEXT:  .Lpcsection680:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection636:
+; O2-NEXT:  .Lpcsection681:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection637:
+; O2-NEXT:  .Lpcsection682:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection638:
+; O2-NEXT:  .Lpcsection683:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection639:
+; O2-NEXT:  .Lpcsection684:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
-; O2-NEXT:  .Lpcsection640:
+; O2-NEXT:  .Lpcsection685:
 ; O2-NEXT:    movl $42, %eax
-; O2-NEXT:  .Lpcsection641:
+; O2-NEXT:  .Lpcsection686:
 ; O2-NEXT:    xorl %edx, %edx
-; O2-NEXT:  .Lpcsection642:
+; O2-NEXT:  .Lpcsection687:
 ; O2-NEXT:    xorl %ecx, %ecx
-; O2-NEXT:  .Lpcsection643:
+; O2-NEXT:  .Lpcsection688:
 ; O2-NEXT:    lock cmpxchg16b (%rdi)
 ; O2-NEXT:    movq $3, foo(%rip)
 ; O2-NEXT:    popq %rbx
 ; O2-NEXT:    .cfi_def_cfa_offset 8
 ; O2-NEXT:    retq
 ;
 ; O3-LABEL: atomic128_cas_seq_cst:
 ; O3:       # %bb.0: # %entry
 ; O3-NEXT:    pushq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 16
 ; O3-NEXT:    .cfi_offset %rbx, -16
 ; O3-NEXT:    movq foo(%rip), %rax
-; O3-NEXT:  .Lpcsection631:
+; O3-NEXT:  .Lpcsection676:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection632:
+; O3-NEXT:  .Lpcsection677:
 ; O3-NEXT:    movl $1, %ebx
-; O3-NEXT:  .Lpcsection633:
+; O3-NEXT:  .Lpcsection678:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection634:
+; O3-NEXT:  .Lpcsection679:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection635:
+; O3-NEXT:  .Lpcsection680:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection636:
+; O3-NEXT:  .Lpcsection681:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection637:
+; O3-NEXT:  .Lpcsection682:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection638:
+; O3-NEXT:  .Lpcsection683:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection639:
+; O3-NEXT:  .Lpcsection684:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
-; O3-NEXT:  .Lpcsection640:
+; O3-NEXT:  .Lpcsection685:
 ; O3-NEXT:    movl $42, %eax
-; O3-NEXT:  .Lpcsection641:
+; O3-NEXT:  .Lpcsection686:
 ; O3-NEXT:    xorl %edx, %edx
-; O3-NEXT:  .Lpcsection642:
+; O3-NEXT:  .Lpcsection687:
 ; O3-NEXT:    xorl %ecx, %ecx
-; O3-NEXT:  .Lpcsection643:
+; O3-NEXT:  .Lpcsection688:
 ; O3-NEXT:    lock cmpxchg16b (%rdi)
 ; O3-NEXT:    movq $3, foo(%rip)
 ; O3-NEXT:    popq %rbx
 ; O3-NEXT:    .cfi_def_cfa_offset 8
 ; O3-NEXT:    retq
 entry:
   load volatile i64, ptr @foo, align 8
   %x = cmpxchg ptr %a, i128 42, i128 1 seq_cst monotonic, align 16, !pcsections !0
   %y = cmpxchg ptr %a, i128 42, i128 1 seq_cst acquire, align 16, !pcsections !0
   %z = cmpxchg ptr %a, i128 42, i128 1 seq_cst seq_cst, align 16, !pcsections !0
   store volatile i64 3, ptr @foo, align 8
   ret void
 }
 
 !0 = !{!"somesection"}

llvm/test/CodeGen/X86/peephole-copy.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
+; RUN: llc < %s -mtriple=x86_64-pc-linux-gnu | FileCheck %s
+
+; Correctly tracking COPY instructions in peephole should not crash compiler.
+
+declare void @llvm.lifetime.start.p0(i64 immarg, ptr nocapture)
+
+define void @foo(ptr %p1, ptr %p2, ptr %p3, ptr %p4) {
+; CHECK-LABEL: foo:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    movq %rcx, %rax
+; CHECK-NEXT:    movl $5, -{{[0-9]+}}(%rsp)
+; CHECK-NEXT:    movl $5, %r8d
+; CHECK-NEXT:    xorl %ecx, %ecx
+; CHECK-NEXT:    cmpl $0, (%rdi)
+; CHECK-NEXT:    setle %cl
+; CHECK-NEXT:    movl (%rax), %eax
+; CHECK-NEXT:    shll %cl, %eax
+; CHECK-NEXT:    movl %r8d, (%rdx)
+; CHECK-NEXT:    .p2align 4, 0x90
+; CHECK-NEXT:  .LBB0_1: # %loop2.header
+; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
+; CHECK-NEXT:    movl $5, %edi
+; CHECK-NEXT:    orl %eax, %edi
+; CHECK-NEXT:    je .LBB0_1
+; CHECK-NEXT:  # %bb.2: # %exit
+; CHECK-NEXT:    movl %r8d, (%rdx)
+; CHECK-NEXT:    movl %ecx, (%rsi)
+; CHECK-NEXT:    retq
+entry:
+  %q = alloca i32, align 4
+  %r = alloca i32, align 4
+  store i32 5, ptr %q, align 4
+  call void @llvm.lifetime.start.p0(i64 4, ptr %r)
+  %vq1 = load i32, ptr %q, align 4
+  %vb = load i32, ptr %p1, align 4
+  %cmp3 = icmp slt i32 %vb, 1
+  %conv4 = zext i1 %cmp3 to i32
+  %va = load i32, ptr %p4, align 4
+  %shl5 = shl i32 %va, %conv4
+  %vq2 = load i32, ptr %q, align 4
+  %cmp = icmp slt i32 %vb, 1
+  %conv = zext i1 %cmp to i32
+  %shl = shl i32 %va, %conv
+  %vq3 = load i32, ptr %q, align 4
+  %cmp.us = icmp slt i32 %vb, 1
+  %conv.us = zext i1 %cmp.us to i32
+  %shl.us = shl i32 %va, %conv.us
+  br label %loop2.preheader
+
+loop2.preheader:
+  store i32 %vq1, ptr %p3, align 4
+  br label %loop2.header
+
+loop2.header:
+  %cond1 = icmp eq i32 %vq3, 0
+  br i1 %cond1, label %loop2.body, label %exit
+
+loop2.body:
+  %cond2 = icmp eq i32 %shl.us, 0
+  br i1 %cond2, label %loop2.header, label %exit
+
+exit:
+  store i32 %vq2, ptr %p3, align 4
+  store i32 %conv.us, ptr %p2, align 4
+  ret void
+}

llvm/test/CodeGen/X86/peephole-copy.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 3
+# RUN: llc -mtriple=i686-- -run-pass=peephole-opt %s -o - | FileCheck %s
+--- |
+  define void @c() {
+  entry:
+    tail call void asm sideeffect "", "q,~{dirflag},~{fpsr},~{flags}"(i32 512)
+    tail call void asm sideeffect "", "q,~{dirflag},~{fpsr},~{flags}"(i32 512)
+    ret void
+  }
+...
+---
+# In peephole optimization the modified COPY instruction should not cause
+# compiler failure.
+name: c
+tracksRegLiveness: true
+registers:
+  - { id: 0, class: gr32_abcd }
+  - { id: 1, class: gr32_abcd }
+  - { id: 2, class: gr32 }
+
+body: |
+  bb.0:
+    ; CHECK-LABEL: name: c
+    ; CHECK: [[MOV32ri:%[0-9]+]]:gr32_abcd = MOV32ri 512
+    ; CHECK-NEXT: INLINEASM &"", 1 /* sideeffect attdialect */, 2359305 /* reguse:GR32 */, [[MOV32ri]], 1 /* reguse */, implicit-def early-clobber $df
+    ; CHECK-NEXT: [[MOV32ri1:%[0-9]+]]:gr32_abcd = MOV32ri 512
+    ; CHECK-NEXT: INLINEASM &"", 1 /* sideeffect attdialect */, 2359305 /* reguse:GR32 */, [[MOV32ri1]], 1 /* reguse */, implicit-def early-clobber $df
+    ; CHECK-NEXT: RET 0
+    %2 = MOV32ri 512
+    %0 = COPY %2
+    INLINEASM &"", 1 /* sideeffect attdialect */, 2359305 /* reguse:GR32_ABCD */, %0:gr32_abcd, 1 /* clobber */, implicit-def early-clobber $df
+    %1 = COPY %2
+    INLINEASM &"", 1 /* sideeffect attdialect */, 2359305 /* reguse:GR32_ABCD */, %1:gr32_abcd, 1 /* clobber */, implicit-def early-clobber $df
+    RET 0
+...

llvm/test/CodeGen/X86/physreg-pairs.ll

entry:
   %conv1 = sext i32 %add to i64
   ret i64 %conv1
 }
 
 define dso_local i64 @test_ebp(i64 %in) local_unnamed_addr nounwind {
 ; CHECK-LABEL: test_ebp:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pushl %ebp
-; CHECK-NEXT:    movl $19088743, %esp # imm = 0x1234567
 ; CHECK-NEXT:    movl $-1985229329, %ebp # imm = 0x89ABCDEF
+; CHECK-NEXT:    movl $19088743, %esp # imm = 0x1234567
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    movl %ebp, %eax
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    addl $3, %eax
 ; CHECK-NEXT:    movl %eax, %edx
 ; CHECK-NEXT:    sarl $31, %edx
 ; CHECK-NEXT:    popl %ebp
 ; CHECK-NEXT:    retl
 entry:
   %0 = tail call i64 asm sideeffect "mov $1, $0", "=r,{ebp},~{dirflag},~{fpsr},~{flags}"(i64 81985529216486895)
   %conv = trunc i64 %0 to i32
   %add = add nsw i32 %conv, 3
   %conv1 = sext i32 %add to i64
   ret i64 %conv1
 }
 

llvm/test/CodeGen/X86/popcnt.ll

 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrl %ecx
 ; X86-NEXT:    andl $1431655765, %ecx # imm = 0x55555555
 ; X86-NEXT:    subl %ecx, %eax
 ; X86-NEXT:    movl $858993459, %ecx # imm = 0x33333333
 ; X86-NEXT:    movl %eax, %edx
 ; X86-NEXT:    andl %ecx, %edx
 ; X86-NEXT:    shrl $2, %eax
 ; X86-NEXT:    andl %ecx, %eax
 ; X86-NEXT:    addl %edx, %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrl $4, %ecx
 ; X86-NEXT:    addl %eax, %ecx
 ; X86-NEXT:    andl $252645135, %ecx # imm = 0xF0F0F0F
 ; X86-NEXT:    imull $16843009, %ecx, %eax # imm = 0x1010101
 ; X86-NEXT:    shrl $24, %eax

 define i32 @cnt32_pgso(i32 %x) nounwind readnone !prof !14 {
 ; X86-LABEL: cnt32_pgso:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrl %ecx
 ; X86-NEXT:    andl $1431655765, %ecx # imm = 0x55555555
 ; X86-NEXT:    subl %ecx, %eax
-; X86-NEXT:    movl $858993459, %ecx # imm = 0x33333333
-; X86-NEXT:    movl %eax, %edx
-; X86-NEXT:    andl %ecx, %edx
+; X86-NEXT:    movl %eax, %ecx
+; X86-NEXT:    andl $858993459, %ecx # imm = 0x33333333
 ; X86-NEXT:    shrl $2, %eax
-; X86-NEXT:    andl %ecx, %eax
-; X86-NEXT:    addl %edx, %eax
+; X86-NEXT:    andl $858993459, %eax # imm = 0x33333333
+; X86-NEXT:    addl %ecx, %eax
 ; X86-NEXT:    movl %eax, %ecx
 ; X86-NEXT:    shrl $4, %ecx
 ; X86-NEXT:    addl %eax, %ecx
 ; X86-NEXT:    andl $252645135, %ecx # imm = 0xF0F0F0F
 ; X86-NEXT:    imull $16843009, %ecx, %eax # imm = 0x1010101
 ; X86-NEXT:    shrl $24, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: cnt32_pgso:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl %eax
 ; X64-NEXT:    andl $1431655765, %eax # imm = 0x55555555
 ; X64-NEXT:    subl %eax, %edi
-; X64-NEXT:    movl $858993459, %eax # imm = 0x33333333
-; X64-NEXT:    movl %edi, %ecx
-; X64-NEXT:    andl %eax, %ecx
+; X64-NEXT:    movl %edi, %eax
+; X64-NEXT:    andl $858993459, %eax # imm = 0x33333333
 ; X64-NEXT:    shrl $2, %edi
-; X64-NEXT:    andl %eax, %edi
-; X64-NEXT:    addl %ecx, %edi
+; X64-NEXT:    andl $858993459, %edi # imm = 0x33333333
+; X64-NEXT:    addl %eax, %edi
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl $4, %eax
 ; X64-NEXT:    addl %edi, %eax
 ; X64-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
 ; X64-NEXT:    imull $16843009, %eax, %eax # imm = 0x1010101
 ; X64-NEXT:    shrl $24, %eax
 ; X64-NEXT:    retq
 ;
 ; X86-POPCNT-LABEL: cnt32_pgso:
 ; X86-POPCNT:       # %bb.0:
 ; X86-POPCNT-NEXT:    popcntl {{[0-9]+}}(%esp), %eax
 ; X86-POPCNT-NEXT:    retl
 ;
 ; X64-POPCNT-LABEL: cnt32_pgso:
 ; X64-POPCNT:       # %bb.0:
 ; X64-POPCNT-NEXT:    popcntl %edi, %eax
 ; X64-POPCNT-NEXT:    retq
   %cnt = tail call i32 @llvm.ctpop.i32(i32 %x)
   ret i32 %cnt
 }
 
 define i64 @cnt64_pgso(i64 %x) nounwind readnone !prof !14 {
 ; X86-NOSSE-LABEL: cnt64_pgso:
 ; X86-NOSSE:       # %bb.0:
-; X86-NOSSE-NEXT:    pushl %ebx
-; X86-NOSSE-NEXT:    pushl %edi
-; X86-NOSSE-NEXT:    pushl %esi
 ; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NOSSE-NEXT:    movl %esi, %ecx
-; X86-NOSSE-NEXT:    shrl %ecx
-; X86-NOSSE-NEXT:    movl $1431655765, %edx # imm = 0x55555555
-; X86-NOSSE-NEXT:    andl %edx, %ecx
-; X86-NOSSE-NEXT:    subl %ecx, %esi
-; X86-NOSSE-NEXT:    movl $858993459, %ecx # imm = 0x33333333
-; X86-NOSSE-NEXT:    movl %esi, %edi
-; X86-NOSSE-NEXT:    andl %ecx, %edi
-; X86-NOSSE-NEXT:    shrl $2, %esi
-; X86-NOSSE-NEXT:    andl %ecx, %esi
-; X86-NOSSE-NEXT:    addl %edi, %esi
-; X86-NOSSE-NEXT:    movl %esi, %ebx
-; X86-NOSSE-NEXT:    shrl $4, %ebx
-; X86-NOSSE-NEXT:    addl %esi, %ebx
-; X86-NOSSE-NEXT:    movl $252645135, %edi # imm = 0xF0F0F0F
-; X86-NOSSE-NEXT:    andl %edi, %ebx
-; X86-NOSSE-NEXT:    imull $16843009, %ebx, %esi # imm = 0x1010101
-; X86-NOSSE-NEXT:    shrl $24, %esi
-; X86-NOSSE-NEXT:    movl %eax, %ebx
-; X86-NOSSE-NEXT:    shrl %ebx
-; X86-NOSSE-NEXT:    andl %edx, %ebx
-; X86-NOSSE-NEXT:    subl %ebx, %eax
+; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; X86-NOSSE-NEXT:    movl %ecx, %edx
+; X86-NOSSE-NEXT:    shrl %edx
+; X86-NOSSE-NEXT:    andl $1431655765, %edx # imm = 0x55555555
+; X86-NOSSE-NEXT:    subl %edx, %ecx
+; X86-NOSSE-NEXT:    movl %ecx, %edx
+; X86-NOSSE-NEXT:    andl $858993459, %edx # imm = 0x33333333
+; X86-NOSSE-NEXT:    shrl $2, %ecx
+; X86-NOSSE-NEXT:    andl $858993459, %ecx # imm = 0x33333333
+; X86-NOSSE-NEXT:    addl %edx, %ecx
+; X86-NOSSE-NEXT:    movl %ecx, %edx
+; X86-NOSSE-NEXT:    shrl $4, %edx
+; X86-NOSSE-NEXT:    addl %ecx, %edx
+; X86-NOSSE-NEXT:    andl $252645135, %edx # imm = 0xF0F0F0F
+; X86-NOSSE-NEXT:    imull $16843009, %edx, %ecx # imm = 0x1010101
+; X86-NOSSE-NEXT:    shrl $24, %ecx
 ; X86-NOSSE-NEXT:    movl %eax, %edx
-; X86-NOSSE-NEXT:    andl %ecx, %edx
+; X86-NOSSE-NEXT:    shrl %edx
+; X86-NOSSE-NEXT:    andl $1431655765, %edx # imm = 0x55555555
+; X86-NOSSE-NEXT:    subl %edx, %eax
+; X86-NOSSE-NEXT:    movl %eax, %edx
+; X86-NOSSE-NEXT:    andl $858993459, %edx # imm = 0x33333333
 ; X86-NOSSE-NEXT:    shrl $2, %eax
-; X86-NOSSE-NEXT:    andl %ecx, %eax
+; X86-NOSSE-NEXT:    andl $858993459, %eax # imm = 0x33333333
 ; X86-NOSSE-NEXT:    addl %edx, %eax
-; X86-NOSSE-NEXT:    movl %eax, %ecx
-; X86-NOSSE-NEXT:    shrl $4, %ecx
-; X86-NOSSE-NEXT:    addl %eax, %ecx
-; X86-NOSSE-NEXT:    andl %edi, %ecx
-; X86-NOSSE-NEXT:    imull $16843009, %ecx, %eax # imm = 0x1010101
+; X86-NOSSE-NEXT:    movl %eax, %edx
+; X86-NOSSE-NEXT:    shrl $4, %edx
+; X86-NOSSE-NEXT:    addl %eax, %edx
+; X86-NOSSE-NEXT:    andl $252645135, %edx # imm = 0xF0F0F0F
+; X86-NOSSE-NEXT:    imull $16843009, %edx, %eax # imm = 0x1010101
 ; X86-NOSSE-NEXT:    shrl $24, %eax
-; X86-NOSSE-NEXT:    addl %esi, %eax
+; X86-NOSSE-NEXT:    addl %ecx, %eax
 ; X86-NOSSE-NEXT:    xorl %edx, %edx
-; X86-NOSSE-NEXT:    popl %esi
-; X86-NOSSE-NEXT:    popl %edi
-; X86-NOSSE-NEXT:    popl %ebx
 ; X86-NOSSE-NEXT:    retl
 ;
 ; X64-LABEL: cnt64_pgso:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    shrq %rax
 ; X64-NEXT:    movabsq $6148914691236517205, %rcx # imm = 0x5555555555555555
 ; X64-NEXT:    andq %rax, %rcx

 ; X86-SSSE3-NEXT:    retl
   %cnt = tail call i64 @llvm.ctpop.i64(i64 %x)
   ret i64 %cnt
 }
 
 define i128 @cnt128_pgso(i128 %x) nounwind readnone !prof !14 {
 ; X86-NOSSE-LABEL: cnt128_pgso:
 ; X86-NOSSE:       # %bb.0:
-; X86-NOSSE-NEXT:    pushl %ebp
 ; X86-NOSSE-NEXT:    pushl %ebx
 ; X86-NOSSE-NEXT:    pushl %edi
 ; X86-NOSSE-NEXT:    pushl %esi
+; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %esi
-; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %eax
-; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %ebx
-; X86-NOSSE-NEXT:    movl %ebx, %ecx
-; X86-NOSSE-NEXT:    shrl %ecx
-; X86-NOSSE-NEXT:    movl $1431655765, %edi # imm = 0x55555555
-; X86-NOSSE-NEXT:    andl %edi, %ecx
-; X86-NOSSE-NEXT:    subl %ecx, %ebx
-; X86-NOSSE-NEXT:    movl $858993459, %ecx # imm = 0x33333333
-; X86-NOSSE-NEXT:    movl %ebx, %ebp
-; X86-NOSSE-NEXT:    andl %ecx, %ebp
-; X86-NOSSE-NEXT:    shrl $2, %ebx
-; X86-NOSSE-NEXT:    andl %ecx, %ebx
-; X86-NOSSE-NEXT:    addl %ebp, %ebx
-; X86-NOSSE-NEXT:    movl %ebx, %ebp
-; X86-NOSSE-NEXT:    shrl $4, %ebp
-; X86-NOSSE-NEXT:    addl %ebx, %ebp
-; X86-NOSSE-NEXT:    movl %eax, %ebx
+; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X86-NOSSE-NEXT:    movl %edi, %ebx
 ; X86-NOSSE-NEXT:    shrl %ebx
-; X86-NOSSE-NEXT:    andl %edi, %ebx
-; X86-NOSSE-NEXT:    subl %ebx, %eax
-; X86-NOSSE-NEXT:    movl %eax, %ebx
-; X86-NOSSE-NEXT:    andl %ecx, %ebx
-; X86-NOSSE-NEXT:    shrl $2, %eax
-; X86-NOSSE-NEXT:    andl %ecx, %eax
-; X86-NOSSE-NEXT:    addl %ebx, %eax
-; X86-NOSSE-NEXT:    movl %eax, %edi
-; X86-NOSSE-NEXT:    shrl $4, %edi
-; X86-NOSSE-NEXT:    addl %eax, %edi
-; X86-NOSSE-NEXT:    movl $252645135, %ebx # imm = 0xF0F0F0F
-; X86-NOSSE-NEXT:    andl %ebx, %ebp
-; X86-NOSSE-NEXT:    imull $16843009, %ebp, %eax # imm = 0x1010101
-; X86-NOSSE-NEXT:    shrl $24, %eax
-; X86-NOSSE-NEXT:    andl %ebx, %edi
-; X86-NOSSE-NEXT:    imull $16843009, %edi, %edi # imm = 0x1010101
+; X86-NOSSE-NEXT:    andl $1431655765, %ebx # imm = 0x55555555
+; X86-NOSSE-NEXT:    subl %ebx, %edi
+; X86-NOSSE-NEXT:    movl %edi, %ebx
+; X86-NOSSE-NEXT:    andl $858993459, %ebx # imm = 0x33333333
+; X86-NOSSE-NEXT:    shrl $2, %edi
+; X86-NOSSE-NEXT:    andl $858993459, %edi # imm = 0x33333333
+; X86-NOSSE-NEXT:    addl %ebx, %edi
+; X86-NOSSE-NEXT:    movl %edi, %ebx
+; X86-NOSSE-NEXT:    shrl $4, %ebx
+; X86-NOSSE-NEXT:    addl %edi, %ebx
+; X86-NOSSE-NEXT:    andl $252645135, %ebx # imm = 0xF0F0F0F
+; X86-NOSSE-NEXT:    imull $16843009, %ebx, %edi # imm = 0x1010101
 ; X86-NOSSE-NEXT:    shrl $24, %edi
-; X86-NOSSE-NEXT:    addl %eax, %edi
-; X86-NOSSE-NEXT:    movl %esi, %eax
-; X86-NOSSE-NEXT:    shrl %eax
-; X86-NOSSE-NEXT:    movl $1431655765, %ebp # imm = 0x55555555
-; X86-NOSSE-NEXT:    andl %ebp, %eax
-; X86-NOSSE-NEXT:    subl %eax, %esi
-; X86-NOSSE-NEXT:    movl %esi, %eax
-; X86-NOSSE-NEXT:    andl %ecx, %eax
+; X86-NOSSE-NEXT:    movl %esi, %ebx
+; X86-NOSSE-NEXT:    shrl %ebx
+; X86-NOSSE-NEXT:    andl $1431655765, %ebx # imm = 0x55555555
+; X86-NOSSE-NEXT:    subl %ebx, %esi
+; X86-NOSSE-NEXT:    movl %esi, %ebx
+; X86-NOSSE-NEXT:    andl $858993459, %ebx # imm = 0x33333333
 ; X86-NOSSE-NEXT:    shrl $2, %esi
-; X86-NOSSE-NEXT:    andl %ecx, %esi
-; X86-NOSSE-NEXT:    addl %eax, %esi
-; X86-NOSSE-NEXT:    movl %esi, %ebp
-; X86-NOSSE-NEXT:    shrl $4, %ebp
-; X86-NOSSE-NEXT:    addl %esi, %ebp
-; X86-NOSSE-NEXT:    movl %edx, %eax
-; X86-NOSSE-NEXT:    shrl %eax
-; X86-NOSSE-NEXT:    movl $1431655765, %esi # imm = 0x55555555
-; X86-NOSSE-NEXT:    andl %esi, %eax
-; X86-NOSSE-NEXT:    subl %eax, %edx
-; X86-NOSSE-NEXT:    movl %edx, %eax
-; X86-NOSSE-NEXT:    andl %ecx, %eax
+; X86-NOSSE-NEXT:    andl $858993459, %esi # imm = 0x33333333
+; X86-NOSSE-NEXT:    addl %ebx, %esi
+; X86-NOSSE-NEXT:    movl %esi, %ebx
+; X86-NOSSE-NEXT:    shrl $4, %ebx
+; X86-NOSSE-NEXT:    addl %esi, %ebx
+; X86-NOSSE-NEXT:    andl $252645135, %ebx # imm = 0xF0F0F0F
+; X86-NOSSE-NEXT:    imull $16843009, %ebx, %esi # imm = 0x1010101
+; X86-NOSSE-NEXT:    shrl $24, %esi
+; X86-NOSSE-NEXT:    addl %edi, %esi
+; X86-NOSSE-NEXT:    movl %edx, %edi
+; X86-NOSSE-NEXT:    shrl %edi
+; X86-NOSSE-NEXT:    andl $1431655765, %edi # imm = 0x55555555
+; X86-NOSSE-NEXT:    subl %edi, %edx
+; X86-NOSSE-NEXT:    movl %edx, %edi
+; X86-NOSSE-NEXT:    andl $858993459, %edi # imm = 0x33333333
 ; X86-NOSSE-NEXT:    shrl $2, %edx
-; X86-NOSSE-NEXT:    andl %ecx, %edx
-; X86-NOSSE-NEXT:    addl %eax, %edx
-; X86-NOSSE-NEXT:    movl %edx, %eax
-; X86-NOSSE-NEXT:    shrl $4, %eax
-; X86-NOSSE-NEXT:    addl %edx, %eax
-; X86-NOSSE-NEXT:    andl %ebx, %ebp
-; X86-NOSSE-NEXT:    andl %ebx, %eax
-; X86-NOSSE-NEXT:    imull $16843009, %ebp, %ecx # imm = 0x1010101
-; X86-NOSSE-NEXT:    shrl $24, %ecx
-; X86-NOSSE-NEXT:    imull $16843009, %eax, %edx # imm = 0x1010101
-; X86-NOSSE-NEXT:    shrl $24, %edx
-; X86-NOSSE-NEXT:    addl %ecx, %edx
-; X86-NOSSE-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NOSSE-NEXT:    andl $858993459, %edx # imm = 0x33333333
 ; X86-NOSSE-NEXT:    addl %edi, %edx
-; X86-NOSSE-NEXT:    xorl %ecx, %ecx
-; X86-NOSSE-NEXT:    movl %ecx, 12(%eax)
-; X86-NOSSE-NEXT:    movl %ecx, 8(%eax)
-; X86-NOSSE-NEXT:    movl %ecx, 4(%eax)
-; X86-NOSSE-NEXT:    movl %edx, (%eax)
+; X86-NOSSE-NEXT:    movl %edx, %edi
+; X86-NOSSE-NEXT:    shrl $4, %edi
+; X86-NOSSE-NEXT:    addl %edx, %edi
+; X86-NOSSE-NEXT:    andl $252645135, %edi # imm = 0xF0F0F0F
+; X86-NOSSE-NEXT:    imull $16843009, %edi, %edx # imm = 0x1010101
+; X86-NOSSE-NEXT:    shrl $24, %edx
+; X86-NOSSE-NEXT:    movl %ecx, %edi
+; X86-NOSSE-NEXT:    shrl %edi
+; X86-NOSSE-NEXT:    andl $1431655765, %edi # imm = 0x55555555
+; X86-NOSSE-NEXT:    subl %edi, %ecx
+; X86-NOSSE-NEXT:    movl %ecx, %edi
+; X86-NOSSE-NEXT:    andl $858993459, %edi # imm = 0x33333333
+; X86-NOSSE-NEXT:    shrl $2, %ecx
+; X86-NOSSE-NEXT:    andl $858993459, %ecx # imm = 0x33333333
+; X86-NOSSE-NEXT:    addl %edi, %ecx
+; X86-NOSSE-NEXT:    movl %ecx, %edi
+; X86-NOSSE-NEXT:    shrl $4, %edi
+; X86-NOSSE-NEXT:    addl %ecx, %edi
+; X86-NOSSE-NEXT:    andl $252645135, %edi # imm = 0xF0F0F0F
+; X86-NOSSE-NEXT:    imull $16843009, %edi, %ecx # imm = 0x1010101
+; X86-NOSSE-NEXT:    shrl $24, %ecx
+; X86-NOSSE-NEXT:    addl %edx, %ecx
+; X86-NOSSE-NEXT:    addl %esi, %ecx
+; X86-NOSSE-NEXT:    xorl %edx, %edx
+; X86-NOSSE-NEXT:    movl %edx, 12(%eax)
+; X86-NOSSE-NEXT:    movl %edx, 8(%eax)
+; X86-NOSSE-NEXT:    movl %edx, 4(%eax)
+; X86-NOSSE-NEXT:    movl %ecx, (%eax)
 ; X86-NOSSE-NEXT:    popl %esi
 ; X86-NOSSE-NEXT:    popl %edi
 ; X86-NOSSE-NEXT:    popl %ebx
-; X86-NOSSE-NEXT:    popl %ebp
 ; X86-NOSSE-NEXT:    retl $4
 ;
 ; X64-LABEL: cnt128_pgso:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rsi, %rax
 ; X64-NEXT:    shrq %rax
 ; X64-NEXT:    movabsq $6148914691236517205, %r8 # imm = 0x5555555555555555
 ; X64-NEXT:    andq %r8, %rax

llvm/test/CodeGen/X86/ragreedy-hoist-spill.ll

 ; CHECK-NEXT:    movq %rdi, %r14
 ; CHECK-NEXT:    callq _memset
 ; CHECK-NEXT:    movq %r14, %rdi
 ; CHECK-NEXT:    movq %rbx, %rdx
 ; CHECK-NEXT:  LBB0_8: ## %while.body.preheader
 ; CHECK-NEXT:    imulq $1040, %rdx, %rax ## imm = 0x410
 ; CHECK-NEXT:    movq _syBuf@GOTPCREL(%rip), %rcx
 ; CHECK-NEXT:    leaq 8(%rcx,%rax), %rdx
-; CHECK-NEXT:    movl $1, %r13d
 ; CHECK-NEXT:    movq _syCTRO@GOTPCREL(%rip), %rax
+; CHECK-NEXT:    movl $1, %r13d
 ; CHECK-NEXT:    movb $1, %cl
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_9: ## %do.body
 ; CHECK-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    movl $0, (%rax)
 ; CHECK-NEXT:    testb %cl, %cl
 ; CHECK-NEXT:    jne LBB0_9
 ; CHECK-NEXT:  ## %bb.10: ## %do.end

llvm/test/CodeGen/X86/remat-phys-dead.ll

 }
 
 ; On the other hand, if it's already the correct width, we really shouldn't be
 ; marking the definition register as dead.
 
 define i32 @test_remat32() {
   ret i32 0
 ; CHECK: REGISTER COALESCER
-; CHECK: Remat: $eax = MOV32r0 implicit-def dead $eflags
+; CHECK: $eax = MOV32r0 implicit-def dead $eflags
 }
 

llvm/test/CodeGen/X86/select_const_i128.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 3
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mcpu=x86-64-v2 | FileCheck %s
 
 define i128 @select_eq_i128(ptr %a) {
 ; CHECK-LABEL: select_eq_i128:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movdqa (%rdi), %xmm0
 ; CHECK-NEXT:    pxor {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    ptest %xmm0, %xmm0
 ; CHECK-NEXT:    setne %al
-; CHECK-NEXT:    xorl %ecx, %ecx
 ; CHECK-NEXT:    addq $-1, %rax
 ; CHECK-NEXT:    movabsq $9223372036854775807, %rdx # imm = 0x7FFFFFFFFFFFFFFF
-; CHECK-NEXT:    adcq %rcx, %rdx
+; CHECK-NEXT:    adcq $0, %rdx
 ; CHECK-NEXT:    retq
   %1 = load i128, ptr %a, align 16
   %cmp = icmp eq i128 %1, 1
   %cond = select i1 %cmp, i128 170141183460469231731687303715884105727, i128 -170141183460469231731687303715884105728
   ret i128 %cond
 }

llvm/test/CodeGen/X86/shrink_vmul.ll

 ; X86-SSE-NEXT:    pushl %eax
 ; X86-SSE-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X86-SSE-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-SSE-NEXT:    movzwl 16(%eax), %edx
 ; X86-SSE-NEXT:    movl %edx, (%esp) # 4-byte Spill
 ; X86-SSE-NEXT:    movdqa (%eax), %xmm3
 ; X86-SSE-NEXT:    movdqa (%ecx), %xmm0
 ; X86-SSE-NEXT:    movdqa 16(%ecx), %xmm1
-; X86-SSE-NEXT:    pxor %xmm4, %xmm4
+; X86-SSE-NEXT:    pxor %xmm5, %xmm5
 ; X86-SSE-NEXT:    movdqa %xmm3, %xmm2
 ; X86-SSE-NEXT:    pextrw $7, %xmm3, %eax
 ; X86-SSE-NEXT:    pextrw $4, %xmm3, %edi
 ; X86-SSE-NEXT:    pextrw $0, %xmm3, %ebp
 ; X86-SSE-NEXT:    pextrw $1, %xmm3, %esi
 ; X86-SSE-NEXT:    pextrw $3, %xmm3, %ebx
-; X86-SSE-NEXT:    movdqa %xmm3, %xmm5
-; X86-SSE-NEXT:    punpcklwd {{.*#+}} xmm5 = xmm5[0],xmm4[0],xmm5[1],xmm4[1],xmm5[2],xmm4[2],xmm5[3],xmm4[3]
-; X86-SSE-NEXT:    punpckhwd {{.*#+}} xmm2 = xmm2[4],xmm4[4],xmm2[5],xmm4[5],xmm2[6],xmm4[6],xmm2[7],xmm4[7]
+; X86-SSE-NEXT:    movdqa %xmm3, %xmm4
+; X86-SSE-NEXT:    punpcklwd {{.*#+}} xmm4 = xmm4[0],xmm5[0],xmm4[1],xmm5[1],xmm4[2],xmm5[2],xmm4[3],xmm5[3]
+; X86-SSE-NEXT:    punpckhwd {{.*#+}} xmm2 = xmm2[4],xmm5[4],xmm2[5],xmm5[5],xmm2[6],xmm5[6],xmm2[7],xmm5[7]
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[3,3,3,3]
 ; X86-SSE-NEXT:    movd %xmm3, %ecx
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm3
-; X86-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm2[2,3,2,3]
-; X86-SSE-NEXT:    movd %xmm4, %eax
-; X86-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm1[2,3,2,3]
-; X86-SSE-NEXT:    movd %xmm4, %ecx
+; X86-SSE-NEXT:    pshufd {{.*#+}} xmm5 = xmm2[2,3,2,3]
+; X86-SSE-NEXT:    movd %xmm5, %eax
+; X86-SSE-NEXT:    pshufd {{.*#+}} xmm5 = xmm1[2,3,2,3]
+; X86-SSE-NEXT:    movd %xmm5, %ecx
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
-; X86-SSE-NEXT:    movd %edx, %xmm4
-; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm4 = xmm4[0],xmm3[0],xmm4[1],xmm3[1]
+; X86-SSE-NEXT:    movd %edx, %xmm5
+; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm5 = xmm5[0],xmm3[0],xmm5[1],xmm3[1]
 ; X86-SSE-NEXT:    movl %edi, %eax
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X86-SSE-NEXT:    divl 16(%edi)
 ; X86-SSE-NEXT:    movd %edx, %xmm3
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[1,1,1,1]
 ; X86-SSE-NEXT:    movd %xmm2, %eax
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,1,1]
 ; X86-SSE-NEXT:    movd %xmm1, %ecx
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm1
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm1[0],xmm3[1],xmm1[1]
-; X86-SSE-NEXT:    punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm4[0]
+; X86-SSE-NEXT:    punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
 ; X86-SSE-NEXT:    movl %ebp, %eax
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl (%edi)
 ; X86-SSE-NEXT:    movd %edx, %xmm1
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[1,1,1,1]
 ; X86-SSE-NEXT:    movd %xmm2, %ecx
 ; X86-SSE-NEXT:    movl %esi, %eax
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm2
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[3,3,3,3]
 ; X86-SSE-NEXT:    movd %xmm2, %ecx
 ; X86-SSE-NEXT:    movl %ebx, %eax
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm2
-; X86-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm5[2,3,2,3]
+; X86-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm4[2,3,2,3]
 ; X86-SSE-NEXT:    movd %xmm4, %eax
 ; X86-SSE-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
 ; X86-SSE-NEXT:    movd %xmm0, %ecx
 ; X86-SSE-NEXT:    xorl %edx, %edx
 ; X86-SSE-NEXT:    divl %ecx
 ; X86-SSE-NEXT:    movd %edx, %xmm0
 ; X86-SSE-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; X86-SSE-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]

 ; X86-AVX2-NEXT:    retl
 ;
 ; X64-SSE-LABEL: PR34947:
 ; X64-SSE:       # %bb.0:
 ; X64-SSE-NEXT:    movzwl 16(%rdi), %ecx
 ; X64-SSE-NEXT:    movdqa (%rdi), %xmm3
 ; X64-SSE-NEXT:    movdqa (%rsi), %xmm0
 ; X64-SSE-NEXT:    movdqa 16(%rsi), %xmm1
-; X64-SSE-NEXT:    pxor %xmm4, %xmm4
+; X64-SSE-NEXT:    pxor %xmm5, %xmm5
 ; X64-SSE-NEXT:    movdqa %xmm3, %xmm2
 ; X64-SSE-NEXT:    pextrw $7, %xmm3, %eax
 ; X64-SSE-NEXT:    pextrw $4, %xmm3, %r8d
 ; X64-SSE-NEXT:    pextrw $0, %xmm3, %r10d
 ; X64-SSE-NEXT:    pextrw $1, %xmm3, %edi
 ; X64-SSE-NEXT:    pextrw $3, %xmm3, %r9d
-; X64-SSE-NEXT:    movdqa %xmm3, %xmm5
-; X64-SSE-NEXT:    punpcklwd {{.*#+}} xmm5 = xmm5[0],xmm4[0],xmm5[1],xmm4[1],xmm5[2],xmm4[2],xmm5[3],xmm4[3]
-; X64-SSE-NEXT:    punpckhwd {{.*#+}} xmm2 = xmm2[4],xmm4[4],xmm2[5],xmm4[5],xmm2[6],xmm4[6],xmm2[7],xmm4[7]
+; X64-SSE-NEXT:    movdqa %xmm3, %xmm4
+; X64-SSE-NEXT:    punpcklwd {{.*#+}} xmm4 = xmm4[0],xmm5[0],xmm4[1],xmm5[1],xmm4[2],xmm5[2],xmm4[3],xmm5[3]
+; X64-SSE-NEXT:    punpckhwd {{.*#+}} xmm2 = xmm2[4],xmm5[4],xmm2[5],xmm5[5],xmm2[6],xmm5[6],xmm2[7],xmm5[7]
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[3,3,3,3]
 ; X64-SSE-NEXT:    movd %xmm3, %r11d
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %r11d
 ; X64-SSE-NEXT:    movd %edx, %xmm3
-; X64-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm2[2,3,2,3]
-; X64-SSE-NEXT:    movd %xmm4, %eax
-; X64-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm1[2,3,2,3]
-; X64-SSE-NEXT:    movd %xmm4, %r11d
+; X64-SSE-NEXT:    pshufd {{.*#+}} xmm5 = xmm2[2,3,2,3]
+; X64-SSE-NEXT:    movd %xmm5, %eax
+; X64-SSE-NEXT:    pshufd {{.*#+}} xmm5 = xmm1[2,3,2,3]
+; X64-SSE-NEXT:    movd %xmm5, %r11d
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %r11d
-; X64-SSE-NEXT:    movd %edx, %xmm4
-; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm4 = xmm4[0],xmm3[0],xmm4[1],xmm3[1]
+; X64-SSE-NEXT:    movd %edx, %xmm5
+; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm5 = xmm5[0],xmm3[0],xmm5[1],xmm3[1]
 ; X64-SSE-NEXT:    movl %r8d, %eax
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl 16(%rsi)
 ; X64-SSE-NEXT:    movd %edx, %xmm3
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[1,1,1,1]
 ; X64-SSE-NEXT:    movd %xmm2, %eax
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,1,1]
 ; X64-SSE-NEXT:    movd %xmm1, %r8d
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %r8d
 ; X64-SSE-NEXT:    movd %edx, %xmm1
 ; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm1[0],xmm3[1],xmm1[1]
-; X64-SSE-NEXT:    punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm4[0]
+; X64-SSE-NEXT:    punpcklqdq {{.*#+}} xmm3 = xmm3[0],xmm5[0]
 ; X64-SSE-NEXT:    movl %r10d, %eax
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl (%rsi)
 ; X64-SSE-NEXT:    movd %edx, %xmm1
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[1,1,1,1]
 ; X64-SSE-NEXT:    movd %xmm2, %r8d
 ; X64-SSE-NEXT:    movl %edi, %eax
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %r8d
 ; X64-SSE-NEXT:    movd %edx, %xmm2
 ; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm2[0],xmm1[1],xmm2[1]
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[3,3,3,3]
 ; X64-SSE-NEXT:    movd %xmm2, %edi
 ; X64-SSE-NEXT:    movl %r9d, %eax
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %edi
 ; X64-SSE-NEXT:    movd %edx, %xmm2
-; X64-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm5[2,3,2,3]
+; X64-SSE-NEXT:    pshufd {{.*#+}} xmm4 = xmm4[2,3,2,3]
 ; X64-SSE-NEXT:    movd %xmm4, %eax
 ; X64-SSE-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[2,3,2,3]
 ; X64-SSE-NEXT:    movd %xmm0, %edi
 ; X64-SSE-NEXT:    xorl %edx, %edx
 ; X64-SSE-NEXT:    divl %edi
 ; X64-SSE-NEXT:    movd %edx, %xmm0
 ; X64-SSE-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; X64-SSE-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm0[0]

llvm/test/CodeGen/X86/speculative-load-hardening-call-and-ret.ll

 declare i32 @__sigsetjmp(ptr %foo, ptr %bar, i32 %baz) returns_twice
 
 define i32 @test_call_setjmp(ptr%ptr) nounwind {
 ; X64-NOPIC-LABEL: test_call_setjmp:
 ; X64-NOPIC:       # %bb.0: # %entry
 ; X64-NOPIC-NEXT:    pushq %rbp
 ; X64-NOPIC-NEXT:    pushq %r15
 ; X64-NOPIC-NEXT:    pushq %r14
-; X64-NOPIC-NEXT:    pushq %r13
 ; X64-NOPIC-NEXT:    pushq %r12
 ; X64-NOPIC-NEXT:    pushq %rbx
-; X64-NOPIC-NEXT:    subq $24, %rsp
+; X64-NOPIC-NEXT:    subq $16, %rsp
 ; X64-NOPIC-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-NEXT:    movq %rdi, %rbx
 ; X64-NOPIC-NEXT:    movq $-1, %r15
 ; X64-NOPIC-NEXT:    sarq $63, %rax
-; X64-NOPIC-NEXT:    leaq {{[0-9]+}}(%rsp), %r14
+; X64-NOPIC-NEXT:    movq %rsp, %r14
 ; X64-NOPIC-NEXT:    shlq $47, %rax
 ; X64-NOPIC-NEXT:    movq %r14, %rdi
 ; X64-NOPIC-NEXT:    orq %rax, %rsp
 ; X64-NOPIC-NEXT:    movq $.Lslh_ret_addr4, %r12
 ; X64-NOPIC-NEXT:    callq setjmp@PLT
 ; X64-NOPIC-NEXT:  .Lslh_ret_addr4:
 ; X64-NOPIC-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-NEXT:    sarq $63, %rax
 ; X64-NOPIC-NEXT:    cmpq $.Lslh_ret_addr4, %r12
 ; X64-NOPIC-NEXT:    cmovneq %r15, %rax
-; X64-NOPIC-NEXT:    movl (%rbx), %r12d
-; X64-NOPIC-NEXT:    movl $42, %ebp
+; X64-NOPIC-NEXT:    movl (%rbx), %ebp
 ; X64-NOPIC-NEXT:    shlq $47, %rax
 ; X64-NOPIC-NEXT:    movq %r14, %rdi
-; X64-NOPIC-NEXT:    movl %ebp, %esi
+; X64-NOPIC-NEXT:    movl $42, %esi
 ; X64-NOPIC-NEXT:    orq %rax, %rsp
-; X64-NOPIC-NEXT:    movq $.Lslh_ret_addr5, %r13
+; X64-NOPIC-NEXT:    movq $.Lslh_ret_addr5, %r12
 ; X64-NOPIC-NEXT:    callq sigsetjmp@PLT
 ; X64-NOPIC-NEXT:  .Lslh_ret_addr5:
 ; X64-NOPIC-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-NEXT:    sarq $63, %rax
-; X64-NOPIC-NEXT:    cmpq $.Lslh_ret_addr5, %r13
+; X64-NOPIC-NEXT:    cmpq $.Lslh_ret_addr5, %r12
 ; X64-NOPIC-NEXT:    cmovneq %r15, %rax
-; X64-NOPIC-NEXT:    addl (%rbx), %r12d
+; X64-NOPIC-NEXT:    addl (%rbx), %ebp
 ; X64-NOPIC-NEXT:    shlq $47, %rax
 ; X64-NOPIC-NEXT:    movq %r14, %rdi
 ; X64-NOPIC-NEXT:    movq %r14, %rsi
-; X64-NOPIC-NEXT:    movl %ebp, %edx
+; X64-NOPIC-NEXT:    movl $42, %edx
 ; X64-NOPIC-NEXT:    orq %rax, %rsp
 ; X64-NOPIC-NEXT:    movq $.Lslh_ret_addr6, %r14
 ; X64-NOPIC-NEXT:    callq __sigsetjmp@PLT
 ; X64-NOPIC-NEXT:  .Lslh_ret_addr6:
 ; X64-NOPIC-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-NEXT:    sarq $63, %rax
 ; X64-NOPIC-NEXT:    cmpq $.Lslh_ret_addr6, %r14
 ; X64-NOPIC-NEXT:    movq %rax, %rcx
 ; X64-NOPIC-NEXT:    cmovneq %r15, %rcx
-; X64-NOPIC-NEXT:    addl (%rbx), %r12d
-; X64-NOPIC-NEXT:    movl %r12d, %eax
+; X64-NOPIC-NEXT:    addl (%rbx), %ebp
+; X64-NOPIC-NEXT:    movl %ebp, %eax
 ; X64-NOPIC-NEXT:    orl %ecx, %eax
 ; X64-NOPIC-NEXT:    shlq $47, %rcx
 ; X64-NOPIC-NEXT:    orq %rcx, %rsp
-; X64-NOPIC-NEXT:    addq $24, %rsp
+; X64-NOPIC-NEXT:    addq $16, %rsp
 ; X64-NOPIC-NEXT:    popq %rbx
 ; X64-NOPIC-NEXT:    popq %r12
-; X64-NOPIC-NEXT:    popq %r13
 ; X64-NOPIC-NEXT:    popq %r14
 ; X64-NOPIC-NEXT:    popq %r15
 ; X64-NOPIC-NEXT:    popq %rbp
 ; X64-NOPIC-NEXT:    retq
 ;
 ; X64-NOPIC-MCM-LABEL: test_call_setjmp:
 ; X64-NOPIC-MCM:       # %bb.0: # %entry
 ; X64-NOPIC-MCM-NEXT:    pushq %rbp
 ; X64-NOPIC-MCM-NEXT:    pushq %r15
 ; X64-NOPIC-MCM-NEXT:    pushq %r14
-; X64-NOPIC-MCM-NEXT:    pushq %r13
 ; X64-NOPIC-MCM-NEXT:    pushq %r12
 ; X64-NOPIC-MCM-NEXT:    pushq %rbx
-; X64-NOPIC-MCM-NEXT:    subq $24, %rsp
+; X64-NOPIC-MCM-NEXT:    subq $16, %rsp
 ; X64-NOPIC-MCM-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-MCM-NEXT:    movq %rdi, %rbx
 ; X64-NOPIC-MCM-NEXT:    movq $-1, %r15
 ; X64-NOPIC-MCM-NEXT:    sarq $63, %rax
-; X64-NOPIC-MCM-NEXT:    leaq {{[0-9]+}}(%rsp), %r14
+; X64-NOPIC-MCM-NEXT:    movq %rsp, %r14
 ; X64-NOPIC-MCM-NEXT:    shlq $47, %rax
 ; X64-NOPIC-MCM-NEXT:    movq %r14, %rdi
 ; X64-NOPIC-MCM-NEXT:    orq %rax, %rsp
 ; X64-NOPIC-MCM-NEXT:    leaq .Lslh_ret_addr4(%rip), %r12
 ; X64-NOPIC-MCM-NEXT:    callq setjmp@PLT
 ; X64-NOPIC-MCM-NEXT:  .Lslh_ret_addr4:
 ; X64-NOPIC-MCM-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-MCM-NEXT:    sarq $63, %rax
 ; X64-NOPIC-MCM-NEXT:    leaq .Lslh_ret_addr4(%rip), %rcx
 ; X64-NOPIC-MCM-NEXT:    cmpq %rcx, %r12
 ; X64-NOPIC-MCM-NEXT:    cmovneq %r15, %rax
-; X64-NOPIC-MCM-NEXT:    movl (%rbx), %r12d
-; X64-NOPIC-MCM-NEXT:    movl $42, %ebp
+; X64-NOPIC-MCM-NEXT:    movl (%rbx), %ebp
 ; X64-NOPIC-MCM-NEXT:    shlq $47, %rax
 ; X64-NOPIC-MCM-NEXT:    movq %r14, %rdi
-; X64-NOPIC-MCM-NEXT:    movl %ebp, %esi
+; X64-NOPIC-MCM-NEXT:    movl $42, %esi
 ; X64-NOPIC-MCM-NEXT:    orq %rax, %rsp
-; X64-NOPIC-MCM-NEXT:    leaq .Lslh_ret_addr5(%rip), %r13
+; X64-NOPIC-MCM-NEXT:    leaq .Lslh_ret_addr5(%rip), %r12
 ; X64-NOPIC-MCM-NEXT:    callq sigsetjmp@PLT
 ; X64-NOPIC-MCM-NEXT:  .Lslh_ret_addr5:
 ; X64-NOPIC-MCM-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-MCM-NEXT:    sarq $63, %rax
 ; X64-NOPIC-MCM-NEXT:    leaq .Lslh_ret_addr5(%rip), %rcx
-; X64-NOPIC-MCM-NEXT:    cmpq %rcx, %r13
+; X64-NOPIC-MCM-NEXT:    cmpq %rcx, %r12
 ; X64-NOPIC-MCM-NEXT:    cmovneq %r15, %rax
-; X64-NOPIC-MCM-NEXT:    addl (%rbx), %r12d
+; X64-NOPIC-MCM-NEXT:    addl (%rbx), %ebp
 ; X64-NOPIC-MCM-NEXT:    shlq $47, %rax
 ; X64-NOPIC-MCM-NEXT:    movq %r14, %rdi
 ; X64-NOPIC-MCM-NEXT:    movq %r14, %rsi
-; X64-NOPIC-MCM-NEXT:    movl %ebp, %edx
+; X64-NOPIC-MCM-NEXT:    movl $42, %edx
 ; X64-NOPIC-MCM-NEXT:    orq %rax, %rsp
 ; X64-NOPIC-MCM-NEXT:    leaq .Lslh_ret_addr6(%rip), %r14
 ; X64-NOPIC-MCM-NEXT:    callq __sigsetjmp@PLT
 ; X64-NOPIC-MCM-NEXT:  .Lslh_ret_addr6:
 ; X64-NOPIC-MCM-NEXT:    movq %rsp, %rax
 ; X64-NOPIC-MCM-NEXT:    sarq $63, %rax
 ; X64-NOPIC-MCM-NEXT:    leaq .Lslh_ret_addr6(%rip), %rcx
 ; X64-NOPIC-MCM-NEXT:    cmpq %rcx, %r14
 ; X64-NOPIC-MCM-NEXT:    movq %rax, %rcx
 ; X64-NOPIC-MCM-NEXT:    cmovneq %r15, %rcx
-; X64-NOPIC-MCM-NEXT:    addl (%rbx), %r12d
-; X64-NOPIC-MCM-NEXT:    movl %r12d, %eax
+; X64-NOPIC-MCM-NEXT:    addl (%rbx), %ebp
+; X64-NOPIC-MCM-NEXT:    movl %ebp, %eax
 ; X64-NOPIC-MCM-NEXT:    orl %ecx, %eax
 ; X64-NOPIC-MCM-NEXT:    shlq $47, %rcx
 ; X64-NOPIC-MCM-NEXT:    orq %rcx, %rsp
-; X64-NOPIC-MCM-NEXT:    addq $24, %rsp
+; X64-NOPIC-MCM-NEXT:    addq $16, %rsp
 ; X64-NOPIC-MCM-NEXT:    popq %rbx
 ; X64-NOPIC-MCM-NEXT:    popq %r12
-; X64-NOPIC-MCM-NEXT:    popq %r13
 ; X64-NOPIC-MCM-NEXT:    popq %r14
 ; X64-NOPIC-MCM-NEXT:    popq %r15
 ; X64-NOPIC-MCM-NEXT:    popq %rbp
 ; X64-NOPIC-MCM-NEXT:    retq
 ;
 ; X64-PIC-LABEL: test_call_setjmp:
 ; X64-PIC:       # %bb.0: # %entry
 ; X64-PIC-NEXT:    pushq %rbp
 ; X64-PIC-NEXT:    pushq %r15
 ; X64-PIC-NEXT:    pushq %r14
-; X64-PIC-NEXT:    pushq %r13
 ; X64-PIC-NEXT:    pushq %r12
 ; X64-PIC-NEXT:    pushq %rbx
-; X64-PIC-NEXT:    subq $24, %rsp
+; X64-PIC-NEXT:    subq $16, %rsp
 ; X64-PIC-NEXT:    movq %rsp, %rax
 ; X64-PIC-NEXT:    movq %rdi, %rbx
 ; X64-PIC-NEXT:    movq $-1, %r15
 ; X64-PIC-NEXT:    sarq $63, %rax
-; X64-PIC-NEXT:    leaq {{[0-9]+}}(%rsp), %r14
+; X64-PIC-NEXT:    movq %rsp, %r14
 ; X64-PIC-NEXT:    shlq $47, %rax
 ; X64-PIC-NEXT:    movq %r14, %rdi
 ; X64-PIC-NEXT:    orq %rax, %rsp
 ; X64-PIC-NEXT:    leaq .Lslh_ret_addr4(%rip), %r12
 ; X64-PIC-NEXT:    callq setjmp@PLT
 ; X64-PIC-NEXT:  .Lslh_ret_addr4:
 ; X64-PIC-NEXT:    movq %rsp, %rax
 ; X64-PIC-NEXT:    sarq $63, %rax
 ; X64-PIC-NEXT:    leaq .Lslh_ret_addr4(%rip), %rcx
 ; X64-PIC-NEXT:    cmpq %rcx, %r12
 ; X64-PIC-NEXT:    cmovneq %r15, %rax
-; X64-PIC-NEXT:    movl (%rbx), %r12d
-; X64-PIC-NEXT:    movl $42, %ebp
+; X64-PIC-NEXT:    movl (%rbx), %ebp
 ; X64-PIC-NEXT:    shlq $47, %rax
 ; X64-PIC-NEXT:    movq %r14, %rdi
-; X64-PIC-NEXT:    movl %ebp, %esi
+; X64-PIC-NEXT:    movl $42, %esi
 ; X64-PIC-NEXT:    orq %rax, %rsp
-; X64-PIC-NEXT:    leaq .Lslh_ret_addr5(%rip), %r13
+; X64-PIC-NEXT:    leaq .Lslh_ret_addr5(%rip), %r12
 ; X64-PIC-NEXT:    callq sigsetjmp@PLT
 ; X64-PIC-NEXT:  .Lslh_ret_addr5:
 ; X64-PIC-NEXT:    movq %rsp, %rax
 ; X64-PIC-NEXT:    sarq $63, %rax
 ; X64-PIC-NEXT:    leaq .Lslh_ret_addr5(%rip), %rcx
-; X64-PIC-NEXT:    cmpq %rcx, %r13
+; X64-PIC-NEXT:    cmpq %rcx, %r12
 ; X64-PIC-NEXT:    cmovneq %r15, %rax
-; X64-PIC-NEXT:    addl (%rbx), %r12d
+; X64-PIC-NEXT:    addl (%rbx), %ebp
 ; X64-PIC-NEXT:    shlq $47, %rax
 ; X64-PIC-NEXT:    movq %r14, %rdi
 ; X64-PIC-NEXT:    movq %r14, %rsi
-; X64-PIC-NEXT:    movl %ebp, %edx
+; X64-PIC-NEXT:    movl $42, %edx
 ; X64-PIC-NEXT:    orq %rax, %rsp
 ; X64-PIC-NEXT:    leaq .Lslh_ret_addr6(%rip), %r14
 ; X64-PIC-NEXT:    callq __sigsetjmp@PLT
 ; X64-PIC-NEXT:  .Lslh_ret_addr6:
 ; X64-PIC-NEXT:    movq %rsp, %rax
 ; X64-PIC-NEXT:    sarq $63, %rax
 ; X64-PIC-NEXT:    leaq .Lslh_ret_addr6(%rip), %rcx
 ; X64-PIC-NEXT:    cmpq %rcx, %r14
 ; X64-PIC-NEXT:    movq %rax, %rcx
 ; X64-PIC-NEXT:    cmovneq %r15, %rcx
-; X64-PIC-NEXT:    addl (%rbx), %r12d
-; X64-PIC-NEXT:    movl %r12d, %eax
+; X64-PIC-NEXT:    addl (%rbx), %ebp
+; X64-PIC-NEXT:    movl %ebp, %eax
 ; X64-PIC-NEXT:    orl %ecx, %eax
 ; X64-PIC-NEXT:    shlq $47, %rcx
 ; X64-PIC-NEXT:    orq %rcx, %rsp
-; X64-PIC-NEXT:    addq $24, %rsp
+; X64-PIC-NEXT:    addq $16, %rsp
 ; X64-PIC-NEXT:    popq %rbx
 ; X64-PIC-NEXT:    popq %r12
-; X64-PIC-NEXT:    popq %r13
 ; X64-PIC-NEXT:    popq %r14
 ; X64-PIC-NEXT:    popq %r15
 ; X64-PIC-NEXT:    popq %rbp
 ; X64-PIC-NEXT:    retq
 entry:
   %env = alloca i8, i32 16
   ; Call a normal setjmp function.
   call i32 @setjmp(ptr %env)

llvm/test/CodeGen/X86/swifterror.ll

 ; CHECK-APPLE-NEXT:    .cfi_def_cfa_offset 48
 ; CHECK-APPLE-NEXT:    subq $48, %rsp
 ; CHECK-APPLE-NEXT:    .cfi_def_cfa_offset 96
 ; CHECK-APPLE-NEXT:    .cfi_offset %rbx, -48
 ; CHECK-APPLE-NEXT:    .cfi_offset %r13, -40
 ; CHECK-APPLE-NEXT:    .cfi_offset %r14, -32
 ; CHECK-APPLE-NEXT:    .cfi_offset %r15, -24
 ; CHECK-APPLE-NEXT:    .cfi_offset %rbp, -16
-; CHECK-APPLE-NEXT:    movq %r12, %rbx
-; CHECK-APPLE-NEXT:    movq %r13, (%rsp) ## 8-byte Spill
+; CHECK-APPLE-NEXT:    movq %r12, (%rsp) ## 8-byte Spill
+; CHECK-APPLE-NEXT:    movq %r13, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
 ; CHECK-APPLE-NEXT:    movq %r9, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
 ; CHECK-APPLE-NEXT:    movq %r8, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
-; CHECK-APPLE-NEXT:    movq %rcx, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
+; CHECK-APPLE-NEXT:    movq %rcx, %rbx
 ; CHECK-APPLE-NEXT:    movq %rdx, %r14
 ; CHECK-APPLE-NEXT:    movq %rsi, %r15
 ; CHECK-APPLE-NEXT:    movq %rdi, %rbp
 ; CHECK-APPLE-NEXT:    movl $1, %edi
 ; CHECK-APPLE-NEXT:    movl $2, %esi
 ; CHECK-APPLE-NEXT:    movl $3, %edx
 ; CHECK-APPLE-NEXT:    movl $4, %ecx
 ; CHECK-APPLE-NEXT:    movl $5, %r8d
 ; CHECK-APPLE-NEXT:    movl $6, %r9d
 ; CHECK-APPLE-NEXT:    xorl %r13d, %r13d
 ; CHECK-APPLE-NEXT:    xorl %r12d, %r12d
 ; CHECK-APPLE-NEXT:    callq _params_in_reg2
 ; CHECK-APPLE-NEXT:    movq %r12, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
 ; CHECK-APPLE-NEXT:    movq %rbp, %rdi
 ; CHECK-APPLE-NEXT:    movq %r15, %rsi
 ; CHECK-APPLE-NEXT:    movq %r14, %rdx
-; CHECK-APPLE-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %rcx ## 8-byte Reload
+; CHECK-APPLE-NEXT:    movq %rbx, %rcx
 ; CHECK-APPLE-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %r8 ## 8-byte Reload
 ; CHECK-APPLE-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %r9 ## 8-byte Reload
-; CHECK-APPLE-NEXT:    movq (%rsp), %r13 ## 8-byte Reload
-; CHECK-APPLE-NEXT:    movq %rbx, %r12
+; CHECK-APPLE-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %r13 ## 8-byte Reload
+; CHECK-APPLE-NEXT:    movq (%rsp), %r12 ## 8-byte Reload
 ; CHECK-APPLE-NEXT:    callq _params_and_return_in_reg2
-; CHECK-APPLE-NEXT:    movq %rax, %r14
-; CHECK-APPLE-NEXT:    movq %rdx, %r15
-; CHECK-APPLE-NEXT:    movq %rcx, %rbp
-; CHECK-APPLE-NEXT:    movq %r8, %rbx
+; CHECK-APPLE-NEXT:    movq %rax, %rbx
+; CHECK-APPLE-NEXT:    movq %rdx, %r14
+; CHECK-APPLE-NEXT:    movq %rcx, %r15
+; CHECK-APPLE-NEXT:    movq %r8, %rbp
 ; CHECK-APPLE-NEXT:    movq %r12, (%rsp) ## 8-byte Spill
 ; CHECK-APPLE-NEXT:    movl $1, %edi
 ; CHECK-APPLE-NEXT:    movl $2, %esi
 ; CHECK-APPLE-NEXT:    movl $3, %edx
 ; CHECK-APPLE-NEXT:    movl $4, %ecx
 ; CHECK-APPLE-NEXT:    movl $5, %r8d
 ; CHECK-APPLE-NEXT:    movl $6, %r9d
 ; CHECK-APPLE-NEXT:    xorl %r13d, %r13d
 ; CHECK-APPLE-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %r12 ## 8-byte Reload
 ; CHECK-APPLE-NEXT:    callq _params_in_reg2
-; CHECK-APPLE-NEXT:    movq %r14, %rax
-; CHECK-APPLE-NEXT:    movq %r15, %rdx
-; CHECK-APPLE-NEXT:    movq %rbp, %rcx
-; CHECK-APPLE-NEXT:    movq %rbx, %r8
+; CHECK-APPLE-NEXT:    movq %rbx, %rax
+; CHECK-APPLE-NEXT:    movq %r14, %rdx
+; CHECK-APPLE-NEXT:    movq %r15, %rcx
+; CHECK-APPLE-NEXT:    movq %rbp, %r8
 ; CHECK-APPLE-NEXT:    movq (%rsp), %r12 ## 8-byte Reload
 ; CHECK-APPLE-NEXT:    addq $48, %rsp
 ; CHECK-APPLE-NEXT:    popq %rbx
 ; CHECK-APPLE-NEXT:    popq %r13
 ; CHECK-APPLE-NEXT:    popq %r14
 ; CHECK-APPLE-NEXT:    popq %r15
 ; CHECK-APPLE-NEXT:    popq %rbp
 ; CHECK-APPLE-NEXT:    retq

llvm/test/CodeGen/X86/vector-shuffle-combining-avx512bwvl.ll

; X64-NEXT:    retq
   %shl = shl nuw <8 x i32> %zext, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
   %sel = select <8 x i1> %mask, <8 x i32> %shl, <8 x i32> zeroinitializer
   ret <8 x i32> %sel
 }
 
 define i64 @PR55050() {
 ; X86-LABEL: PR55050:
 ; X86:       # %bb.0: # %entry
+; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    xorl %eax, %eax
-; X86-NEXT:    testb %al, %al
+; X86-NEXT:    testb %dl, %dl
 ; X86-NEXT:    jne .LBB10_2
 ; X86-NEXT:  # %bb.1: # %if
 ; X86-NEXT:    xorl %eax, %eax
+; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:  .LBB10_2: # %exit
-; X86-NEXT:    movl %eax, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: PR55050:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    testb %al, %al
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    retq