[MachineCSE] Allow PRE of instructions that read physical registers

[john.brawn]

Summary

Currently MachineCSE forbids PRE when the instruction reads a physical register. Relax this so that it's allowed when the value being read is the same as what would be read in the place the instruction would be hoisted to.

This is being done in preparation for adding FPCR handling to the AArch64 backend, in order to prevent it to from worsening the generated code, but for targets that already have a similar register it should improve things.

This patch affects code generation in several tests. The new code looks better except for in Thumb2/LowOverheadLoops/memcall.ll where we perform PRE but the LowOverheadLoops transformation then undoes it. Also in AMDGPU/selectcc-opt.ll the CHECK makes things look worse, but actually the function as a whole is better (as a MOV is PRE'd).

[t.p.northover]

Thanks for working on this, I think this change is reasonable.

[mstorsjo]

Headsup, this caused miscompilations on arm for me.

To reproduce the issue, you can e.g. do this:

git clone git://git.videolan.org/ffmpeg.git
cd ffmpeg
./configure --arch=armv7 --cc="clang -target armv7-linux-gnueabihf .." --samples=$(pwd)/../samples
make fate-rsync
make -j$(nproc) fate-oma-demux

One preprocessed source file that differs in behaviour after this commit is this: https://martin.st/temp/id3v2-preproc.c Compiled with clang -target armv7-linux-gnueabihf id3v2-preproc.c -c -o libavformat/id3v2.o -O2.

Ok to revert until the issue has been resolved?

[john.brawn]

I've reverted. Looks like there's something going on with a flags-setting SUBS being hosted above a conditional branch.

[john.brawn]

Adjusted to find physical register uses and defs by iterating through all operands, as the defs list doesn't include implicit defs. Added a test based on the code in ffmpeg that was getting incorrectly optimised.

[vigneshv]

Hi @john.brawn

This change is causing miscompilations in armv7. I was looking into a chromium bug [1] and eventually figured out that it was a regression caused by a clang update in the chromium tree.

I bisected further within the llvm changes and it bisects to this change. If i manually revert this change at current tip-of-tree, the code compiles correctly and works as expected.

Steps to reproduce:

1. build libyuv with "-march=armv7-a -mfloat-abi=softfp -mfpu=neon -O2 -mthumb"
2. Build the attached code snippet from [2] with "-march=armv7-a -mfloat-abi=softfp -mfpu=neon -O2 -mthumb -lyuv"
3. The resulting binary is doing some of the computations incorrectly after this change. If i revert this change, it works as expected.

Please let me know if you need any more details. :)

[1] https://bugs.chromium.org/p/chromium/issues/detail?id=1424089
[2] https://gist.github.com/vigneshvg/09c8414b94187349429641171475ce47

[vigneshv]

1. build libyuv with "-march=armv7-a -mfloat-abi=softfp -mfpu=neon -O2 -mthumb"

link to libyuv: https://chromium.googlesource.com/libyuv/libyuv/

[john.brawn]

In D136675#4213750, @vigneshv wrote:

Please let me know if you need any more details. :)

I quickly tried cross-building that example for a bare-metal environment (because it's something I have the setup to do easily) and the output produced by the executable was identical in current trunk and with this patch reverted. So:

• I'm guessing this failure is happening in some kind of android environment?
• A description of what's going wrong that's more precise than "doing some of the computations incorrectly" would be helpful
• A smaller reproducer that doesn't depend on the host environment (e.g. preprocessed with -E) would also be helpful

Also: I'm on holiday tomorrow and next week, so won't be able to look at this until after that.

[vigneshv]

In D136675#4217132, @john.brawn wrote:

In D136675#4213750, @vigneshv wrote:

Please let me know if you need any more details. :)

I quickly tried cross-building that example for a bare-metal environment (because it's something I have the setup to do easily) and the output produced by the executable was identical in current trunk and with this patch reverted. So:

• I'm guessing this failure is happening in some kind of android environment?
• A description of what's going wrong that's more precise than "doing some of the computations incorrectly" would be helpful
• A smaller reproducer that doesn't depend on the host environment (e.g. preprocessed with -E) would also be helpful

Also: I'm on holiday tomorrow and next week, so won't be able to look at this until after that.

Thanks for the reply @john.brawn. Turns out it was a bug in libyuv's assembly code and how it was being optimized with -flto.

My bisect may just have been accidentally triggering the edge case in this commit. Sorry about that.

llvm/lib/CodeGen/MachineCSE.cpp

bool isProfitableToCSE(Register CSReg, Register Reg,
                            MachineBasicBlock *CSBB, MachineInstr *MI);
     void EnterScope(MachineBasicBlock *MBB);
     void ExitScope(MachineBasicBlock *MBB);
     bool ProcessBlockCSE(MachineBasicBlock *MBB);
     void ExitScopeIfDone(MachineDomTreeNode *Node,
                          DenseMap<MachineDomTreeNode*, unsigned> &OpenChildren);
     bool PerformCSE(MachineDomTreeNode *Node);
 
-    bool isPRECandidate(MachineInstr *MI);
+    bool isPRECandidate(MachineInstr *MI, SmallSet<MCRegister, 8> &PhysRefs);
     bool ProcessBlockPRE(MachineDominatorTree *MDT, MachineBasicBlock *MBB);
     bool PerformSimplePRE(MachineDominatorTree *DT);
     /// Heuristics to see if it's profitable to move common computations of MBB
     /// and MBB1 to CandidateBB.
     bool isProfitableToHoistInto(MachineBasicBlock *CandidateBB,
                                  MachineBasicBlock *MBB,
                                  MachineBasicBlock *MBB1);
   };
bool MachineCSE::PerformCSE(MachineDomTreeNode *Node) {
   }
 
   return Changed;
 }
 
 // We use stronger checks for PRE candidate rather than for CSE ones to embrace
 // checks inside ProcessBlockCSE(), not only inside isCSECandidate(). This helps
 // to exclude instrs created by PRE that won't be CSEed later.
-bool MachineCSE::isPRECandidate(MachineInstr *MI) {
+bool MachineCSE::isPRECandidate(MachineInstr *MI,
+                                SmallSet<MCRegister, 8> &PhysRefs) {
   if (!isCSECandidate(MI) ||
       MI->isNotDuplicable() ||
       MI->mayLoad() ||
       TII->isAsCheapAsAMove(*MI) ||
       MI->getNumDefs() != 1 ||
       MI->getNumExplicitDefs() != 1)
     return false;
 
-  for (const auto &def : MI->defs())
-    if (!Register::isVirtualRegister(def.getReg()))
-      return false;
-
-  for (const auto &use : MI->uses())
-    if (use.isReg() && !Register::isVirtualRegister(use.getReg()))
+  for (const MachineOperand &MO : MI->operands()) {
+    if (MO.isReg() && !Register::isVirtualRegister(MO.getReg())) {
+      if (MO.isDef())
         return false;
+      else
+        PhysRefs.insert(MO.getReg());
+    }
+  }
 
   return true;
 }
 
 bool MachineCSE::ProcessBlockPRE(MachineDominatorTree *DT,
                                  MachineBasicBlock *MBB) {
   bool Changed = false;
   for (MachineInstr &MI : llvm::make_early_inc_range(*MBB)) {
-    if (!isPRECandidate(&MI))
+    SmallSet<MCRegister, 8> PhysRefs;
+    if (!isPRECandidate(&MI, PhysRefs))
       continue;
 
     if (!PREMap.count(&MI)) {
       PREMap[&MI] = MBB;
       continue;
     }
 
     auto MBB1 = PREMap[&MI];
if (CMBB != MBB1) {
         // conditions, but also on fewer conditions. LLVM does not have a
         // MachineInstr attribute which expresses this extended definition, so
         // it's necessary to use `isConvergent` to prevent illegally PRE-ing the
         // subset of `isConvergent` instructions which do fall into this
         // extended definition.
         if (MI.isConvergent() && CMBB != MBB)
           continue;
 
+        // If this instruction uses physical registers then we can only do PRE
+        // if it's using the value that is live at the place we're hoisting to.
+        bool NonLocal;
+        PhysDefVector PhysDefs;
+        if (!PhysRefs.empty() &&
+            !PhysRegDefsReach(&*(CMBB->getFirstTerminator()), &MI, PhysRefs,
+                              PhysDefs, NonLocal))
+          continue;
+
         assert(MI.getOperand(0).isDef() &&
                "First operand of instr with one explicit def must be this def");
         Register VReg = MI.getOperand(0).getReg();
         Register NewReg = MRI->cloneVirtualRegister(VReg);
         if (!isProfitableToCSE(NewReg, VReg, CMBB, &MI))
           continue;
         MachineInstr &NewMI =
             TII->duplicate(*CMBB, CMBB->getFirstTerminator(), MI);

llvm/test/CodeGen/AMDGPU/GlobalISel/hip.extern.shared.array.ll

define amdgpu_kernel void @dynamic_shared_array_0(float addrspace(1)* %out) {
   %arrayidx0 = getelementptr inbounds [512 x float], [512 x float] addrspace(3)* @lds0, i32 0, i32 %tid.x
   %val0 = load float, float addrspace(3)* %arrayidx0, align 4
   %arrayidx1 = getelementptr inbounds [0 x float], [0 x float] addrspace(3)* @dynamic_shared0, i32 0, i32 %tid.x
   store float %val0, float addrspace(3)* %arrayidx1, align 4
   ret void
 }
 
 ; CHECK-LABEL: {{^}}dynamic_shared_array_1:
-; CHECK: v_lshlrev_b32_e32 {{v[0-9]+}}, 2, {{v[0-9]+}}
-; CHECK: v_lshlrev_b32_e32 {{v[0-9]+}}, 2, {{v[0-9]+}}
 ; CHECK: v_lshlrev_b32_e32 [[IDX:v[0-9]+]], 2, {{v[0-9]+}}
 ; CHECK: v_add_u32_e32 {{v[0-9]+}}, 0xc00, [[IDX]]
 define amdgpu_kernel void @dynamic_shared_array_1(float addrspace(1)* %out, i32 %cond) {
 entry:
   %tid.x = tail call i32 @llvm.amdgcn.workitem.id.x()
   %idx.0 = add nsw i32 %tid.x, 64
   %tmp = icmp eq i32 %cond, 0
   br i1 %tmp, label %if, label %else

llvm/test/CodeGen/AMDGPU/GlobalISel/udiv.i64.ll

 define amdgpu_ps i64 @s_udiv_i64(i64 inreg %num, i64 inreg %den) {
 ; CHECK-LABEL: s_udiv_i64:
 ; CHECK:       ; %bb.0:
 ; CHECK-NEXT:    s_or_b64 s[6:7], s[0:1], s[2:3]
 ; CHECK-NEXT:    s_mov_b32 s4, 0
 ; CHECK-NEXT:    s_mov_b32 s5, -1
 ; CHECK-NEXT:    s_and_b64 s[6:7], s[6:7], s[4:5]
 ; CHECK-NEXT:    v_cmp_ne_u64_e64 vcc, s[6:7], 0
+; CHECK-NEXT:    v_cvt_f32_u32_e32 v2, s2
 ; CHECK-NEXT:    s_cbranch_vccz .LBB1_2
 ; CHECK-NEXT:  ; %bb.1:
-; CHECK-NEXT:    v_cvt_f32_u32_e32 v0, s2
-; CHECK-NEXT:    v_mov_b32_e32 v1, s3
-; CHECK-NEXT:    v_cvt_f32_u32_e32 v2, s3
+; CHECK-NEXT:    v_mov_b32_e32 v0, s3
+; CHECK-NEXT:    v_cvt_f32_u32_e32 v1, s3
 ; CHECK-NEXT:    s_sub_u32 s4, 0, s2
 ; CHECK-NEXT:    v_mov_b32_e32 v3, s1
-; CHECK-NEXT:    v_mac_f32_e32 v0, 0x4f800000, v2
+; CHECK-NEXT:    v_madmk_f32 v1, v1, 0x4f800000, v2
 ; CHECK-NEXT:    s_subb_u32 s5, 0, s3
-; CHECK-NEXT:    v_rcp_iflag_f32_e32 v0, v0
-; CHECK-NEXT:    v_mul_f32_e32 v0, 0x5f7ffffc, v0
-; CHECK-NEXT:    v_mul_f32_e32 v2, 0x2f800000, v0
-; CHECK-NEXT:    v_trunc_f32_e32 v2, v2
-; CHECK-NEXT:    v_mac_f32_e32 v0, 0xcf800000, v2
-; CHECK-NEXT:    v_cvt_u32_f32_e32 v2, v2
-; CHECK-NEXT:    v_cvt_u32_f32_e32 v0, v0
-; CHECK-NEXT:    v_mul_lo_u32 v4, s4, v2
-; CHECK-NEXT:    v_mul_lo_u32 v5, s4, v0
-; CHECK-NEXT:    v_mul_lo_u32 v6, s5, v0
-; CHECK-NEXT:    v_mul_hi_u32 v7, s4, v0
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v6, v4
-; CHECK-NEXT:    v_mul_lo_u32 v6, v2, v5
-; CHECK-NEXT:    v_mul_hi_u32 v8, v0, v5
-; CHECK-NEXT:    v_mul_hi_u32 v5, v2, v5
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v7
-; CHECK-NEXT:    v_mul_lo_u32 v7, v0, v4
-; CHECK-NEXT:    v_mul_lo_u32 v9, v2, v4
-; CHECK-NEXT:    v_mul_hi_u32 v10, v0, v4
-; CHECK-NEXT:    v_mul_hi_u32 v4, v2, v4
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_rcp_iflag_f32_e32 v1, v1
+; CHECK-NEXT:    v_mul_f32_e32 v1, 0x5f7ffffc, v1
+; CHECK-NEXT:    v_mul_f32_e32 v4, 0x2f800000, v1
+; CHECK-NEXT:    v_trunc_f32_e32 v4, v4
+; CHECK-NEXT:    v_mac_f32_e32 v1, 0xcf800000, v4
+; CHECK-NEXT:    v_cvt_u32_f32_e32 v4, v4
+; CHECK-NEXT:    v_cvt_u32_f32_e32 v1, v1
+; CHECK-NEXT:    v_mul_lo_u32 v5, s4, v4
+; CHECK-NEXT:    v_mul_lo_u32 v6, s4, v1
+; CHECK-NEXT:    v_mul_lo_u32 v7, s5, v1
+; CHECK-NEXT:    v_mul_hi_u32 v8, s4, v1
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v7, v5
+; CHECK-NEXT:    v_mul_lo_u32 v7, v4, v6
+; CHECK-NEXT:    v_mul_hi_u32 v9, v1, v6
+; CHECK-NEXT:    v_mul_hi_u32 v6, v4, v6
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v8
+; CHECK-NEXT:    v_mul_lo_u32 v8, v1, v5
+; CHECK-NEXT:    v_mul_lo_u32 v10, v4, v5
+; CHECK-NEXT:    v_mul_hi_u32 v11, v1, v5
+; CHECK-NEXT:    v_mul_hi_u32 v5, v4, v5
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v8
+; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v10, v6
+; CHECK-NEXT:    v_cndmask_b32_e64 v10, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v9
 ; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v9, v5
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v11
 ; CHECK-NEXT:    v_cndmask_b32_e64 v9, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v8, v7
+; CHECK-NEXT:    v_add_i32_e32 v8, vcc, v10, v9
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v8, v7
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v7
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v6
+; CHECK-NEXT:    v_addc_u32_e32 v4, vcc, v4, v5, vcc
+; CHECK-NEXT:    v_mul_lo_u32 v5, s4, v1
+; CHECK-NEXT:    v_mul_lo_u32 v6, s5, v1
+; CHECK-NEXT:    v_mul_hi_u32 v7, s4, v1
+; CHECK-NEXT:    v_mul_lo_u32 v8, s4, v4
+; CHECK-NEXT:    v_mul_lo_u32 v9, v4, v5
+; CHECK-NEXT:    v_mul_hi_u32 v10, v1, v5
+; CHECK-NEXT:    v_mul_hi_u32 v5, v4, v5
 ; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v8
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v10
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_mul_lo_u32 v7, v1, v6
+; CHECK-NEXT:    v_mul_lo_u32 v8, v4, v6
+; CHECK-NEXT:    v_mul_hi_u32 v11, v1, v6
+; CHECK-NEXT:    v_mul_hi_u32 v6, v4, v6
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v9, v7
+; CHECK-NEXT:    v_cndmask_b32_e64 v9, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v8, v5
 ; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v7, v6
-; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v9, v8
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v7, v6
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v6
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v5
-; CHECK-NEXT:    v_addc_u32_e32 v2, vcc, v2, v4, vcc
-; CHECK-NEXT:    v_mul_lo_u32 v4, s4, v0
-; CHECK-NEXT:    v_mul_lo_u32 v5, s5, v0
-; CHECK-NEXT:    v_mul_hi_u32 v6, s4, v0
-; CHECK-NEXT:    v_mul_lo_u32 v7, s4, v2
-; CHECK-NEXT:    v_mul_lo_u32 v8, v2, v4
-; CHECK-NEXT:    v_mul_hi_u32 v9, v0, v4
-; CHECK-NEXT:    v_mul_hi_u32 v4, v2, v4
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v10
+; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v11
+; CHECK-NEXT:    v_cndmask_b32_e64 v10, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v9, v7
+; CHECK-NEXT:    v_add_i32_e32 v8, vcc, v8, v10
 ; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v7
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
-; CHECK-NEXT:    v_mul_lo_u32 v6, v0, v5
-; CHECK-NEXT:    v_mul_lo_u32 v7, v2, v5
-; CHECK-NEXT:    v_mul_hi_u32 v10, v0, v5
-; CHECK-NEXT:    v_mul_hi_u32 v5, v2, v5
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v8, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v7, v4
 ; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v9
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v10
-; CHECK-NEXT:    v_cndmask_b32_e64 v9, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v8, v6
-; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v9
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v7, v6
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v4
-; CHECK-NEXT:    v_addc_u32_e32 v2, vcc, v2, v5, vcc
-; CHECK-NEXT:    v_mul_lo_u32 v4, s1, v0
-; CHECK-NEXT:    v_mul_hi_u32 v5, s0, v0
-; CHECK-NEXT:    v_mul_hi_u32 v0, s1, v0
-; CHECK-NEXT:    v_mul_lo_u32 v6, s0, v2
-; CHECK-NEXT:    v_mul_lo_u32 v7, s1, v2
-; CHECK-NEXT:    v_mul_hi_u32 v8, s0, v2
-; CHECK-NEXT:    v_mul_hi_u32 v2, s1, v2
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v7, v0
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v8, v7
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v5
+; CHECK-NEXT:    v_addc_u32_e32 v4, vcc, v4, v6, vcc
+; CHECK-NEXT:    v_mul_lo_u32 v5, s1, v1
+; CHECK-NEXT:    v_mul_hi_u32 v6, s0, v1
+; CHECK-NEXT:    v_mul_hi_u32 v1, s1, v1
+; CHECK-NEXT:    v_mul_lo_u32 v7, s0, v4
+; CHECK-NEXT:    v_mul_lo_u32 v8, s1, v4
+; CHECK-NEXT:    v_mul_hi_u32 v9, s0, v4
+; CHECK-NEXT:    v_mul_hi_u32 v4, s1, v4
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v7
 ; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v5
-; CHECK-NEXT:    v_cndmask_b32_e64 v4, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v8
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v8, v1
+; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
 ; CHECK-NEXT:    v_cndmask_b32_e64 v5, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v6, v4
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v9
+; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
 ; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v7, v5
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v4
-; CHECK-NEXT:    v_cndmask_b32_e64 v4, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v5, v4
-; CHECK-NEXT:    v_mul_lo_u32 v5, s2, v0
-; CHECK-NEXT:    v_mul_lo_u32 v6, s3, v0
-; CHECK-NEXT:    v_mul_hi_u32 v7, s2, v0
-; CHECK-NEXT:    v_add_i32_e32 v8, vcc, 1, v0
-; CHECK-NEXT:    v_add_i32_e32 v2, vcc, v2, v4
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, 1, v8
-; CHECK-NEXT:    v_mul_lo_u32 v2, s2, v2
-; CHECK-NEXT:    v_add_i32_e32 v2, vcc, v6, v2
-; CHECK-NEXT:    v_add_i32_e32 v2, vcc, v2, v7
-; CHECK-NEXT:    v_sub_i32_e32 v5, vcc, s0, v5
-; CHECK-NEXT:    v_subb_u32_e64 v3, s[4:5], v3, v2, vcc
-; CHECK-NEXT:    v_sub_i32_e64 v2, s[4:5], s1, v2
-; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s2, v5
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, -1, s[4:5]
-; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s3, v3
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v8, v6
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v5
+; CHECK-NEXT:    v_cndmask_b32_e64 v5, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v6, v5
+; CHECK-NEXT:    v_mul_lo_u32 v6, s2, v1
+; CHECK-NEXT:    v_mul_lo_u32 v7, s3, v1
+; CHECK-NEXT:    v_mul_hi_u32 v8, s2, v1
+; CHECK-NEXT:    v_add_i32_e32 v9, vcc, 1, v1
+; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v5
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, 1, v9
+; CHECK-NEXT:    v_mul_lo_u32 v4, s2, v4
+; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v7, v4
+; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v8
+; CHECK-NEXT:    v_sub_i32_e32 v6, vcc, s0, v6
+; CHECK-NEXT:    v_subb_u32_e64 v3, s[4:5], v3, v4, vcc
+; CHECK-NEXT:    v_sub_i32_e64 v4, s[4:5], s1, v4
+; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s2, v6
 ; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, -1, s[4:5]
-; CHECK-NEXT:    v_subb_u32_e32 v1, vcc, v2, v1, vcc
+; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s3, v3
+; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, -1, s[4:5]
+; CHECK-NEXT:    v_subb_u32_e32 v0, vcc, v4, v0, vcc
 ; CHECK-NEXT:    v_cmp_eq_u32_e32 vcc, s3, v3
-; CHECK-NEXT:    v_cndmask_b32_e32 v2, v7, v6, vcc
-; CHECK-NEXT:    v_subrev_i32_e32 v3, vcc, s2, v5
-; CHECK-NEXT:    v_subbrev_u32_e32 v1, vcc, 0, v1, vcc
-; CHECK-NEXT:    v_cmp_le_u32_e32 vcc, s2, v3
-; CHECK-NEXT:    v_cndmask_b32_e64 v3, 0, -1, vcc
-; CHECK-NEXT:    v_cmp_le_u32_e32 vcc, s3, v1
-; CHECK-NEXT:    v_cndmask_b32_e64 v5, 0, -1, vcc
-; CHECK-NEXT:    v_cmp_eq_u32_e32 vcc, s3, v1
-; CHECK-NEXT:    v_cndmask_b32_e32 v1, v5, v3, vcc
-; CHECK-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v1
-; CHECK-NEXT:    v_cndmask_b32_e32 v1, v8, v4, vcc
-; CHECK-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v2
-; CHECK-NEXT:    v_cndmask_b32_e32 v0, v0, v1, vcc
+; CHECK-NEXT:    v_cndmask_b32_e32 v3, v8, v7, vcc
+; CHECK-NEXT:    v_subrev_i32_e32 v4, vcc, s2, v6
+; CHECK-NEXT:    v_subbrev_u32_e32 v0, vcc, 0, v0, vcc
+; CHECK-NEXT:    v_cmp_le_u32_e32 vcc, s2, v4
+; CHECK-NEXT:    v_cndmask_b32_e64 v4, 0, -1, vcc
+; CHECK-NEXT:    v_cmp_le_u32_e32 vcc, s3, v0
+; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, -1, vcc
+; CHECK-NEXT:    v_cmp_eq_u32_e32 vcc, s3, v0
+; CHECK-NEXT:    v_cndmask_b32_e32 v0, v6, v4, vcc
+; CHECK-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v0
+; CHECK-NEXT:    v_cndmask_b32_e32 v0, v9, v5, vcc
+; CHECK-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v3
+; CHECK-NEXT:    v_cndmask_b32_e32 v0, v1, v0, vcc
 ; CHECK-NEXT:    s_mov_b32 s5, 0
 ; CHECK-NEXT:    s_branch .LBB1_3
 ; CHECK-NEXT:  .LBB1_2:
 ; CHECK-NEXT:    ; implicit-def: $vgpr0_vgpr1
 ; CHECK-NEXT:  .LBB1_3: ; %Flow
 ; CHECK-NEXT:    s_xor_b32 s1, s5, -1
 ; CHECK-NEXT:    s_and_b32 s1, s1, 1
 ; CHECK-NEXT:    s_cmp_lg_u32 s1, 0
 ; CHECK-NEXT:    s_cbranch_scc1 .LBB1_5
 ; CHECK-NEXT:  ; %bb.4:
-; CHECK-NEXT:    v_cvt_f32_u32_e32 v0, s2
+; CHECK-NEXT:    v_rcp_iflag_f32_e32 v0, v2
 ; CHECK-NEXT:    s_sub_i32 s1, 0, s2
-; CHECK-NEXT:    v_rcp_iflag_f32_e32 v0, v0
 ; CHECK-NEXT:    v_mul_f32_e32 v0, 0x4f7ffffe, v0
 ; CHECK-NEXT:    v_cvt_u32_f32_e32 v0, v0
 ; CHECK-NEXT:    v_mul_lo_u32 v1, s1, v0
 ; CHECK-NEXT:    v_mul_hi_u32 v1, v0, v1
 ; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v1
 ; CHECK-NEXT:    v_mul_hi_u32 v0, s0, v0
 ; CHECK-NEXT:    v_mul_lo_u32 v1, v0, s2
 ; CHECK-NEXT:    v_add_i32_e32 v2, vcc, 1, v0

llvm/test/CodeGen/AMDGPU/GlobalISel/urem.i64.ll

 define amdgpu_ps i64 @s_urem_i64(i64 inreg %num, i64 inreg %den) {
 ; CHECK-LABEL: s_urem_i64:
 ; CHECK:       ; %bb.0:
 ; CHECK-NEXT:    s_or_b64 s[6:7], s[0:1], s[2:3]
 ; CHECK-NEXT:    s_mov_b32 s4, 0
 ; CHECK-NEXT:    s_mov_b32 s5, -1
 ; CHECK-NEXT:    s_and_b64 s[6:7], s[6:7], s[4:5]
 ; CHECK-NEXT:    v_cmp_ne_u64_e64 vcc, s[6:7], 0
+; CHECK-NEXT:    v_cvt_f32_u32_e32 v2, s2
 ; CHECK-NEXT:    s_cbranch_vccz .LBB1_2
 ; CHECK-NEXT:  ; %bb.1:
-; CHECK-NEXT:    v_cvt_f32_u32_e32 v0, s2
-; CHECK-NEXT:    v_mov_b32_e32 v1, s3
-; CHECK-NEXT:    v_cvt_f32_u32_e32 v2, s3
+; CHECK-NEXT:    v_mov_b32_e32 v0, s3
+; CHECK-NEXT:    v_cvt_f32_u32_e32 v1, s3
 ; CHECK-NEXT:    s_sub_u32 s4, 0, s2
 ; CHECK-NEXT:    v_mov_b32_e32 v3, s1
-; CHECK-NEXT:    v_mac_f32_e32 v0, 0x4f800000, v2
+; CHECK-NEXT:    v_madmk_f32 v1, v1, 0x4f800000, v2
 ; CHECK-NEXT:    s_subb_u32 s5, 0, s3
-; CHECK-NEXT:    v_rcp_iflag_f32_e32 v0, v0
-; CHECK-NEXT:    v_mul_f32_e32 v0, 0x5f7ffffc, v0
-; CHECK-NEXT:    v_mul_f32_e32 v2, 0x2f800000, v0
-; CHECK-NEXT:    v_trunc_f32_e32 v2, v2
-; CHECK-NEXT:    v_mac_f32_e32 v0, 0xcf800000, v2
-; CHECK-NEXT:    v_cvt_u32_f32_e32 v2, v2
-; CHECK-NEXT:    v_cvt_u32_f32_e32 v0, v0
-; CHECK-NEXT:    v_mul_lo_u32 v4, s4, v2
-; CHECK-NEXT:    v_mul_lo_u32 v5, s4, v0
-; CHECK-NEXT:    v_mul_lo_u32 v6, s5, v0
-; CHECK-NEXT:    v_mul_hi_u32 v7, s4, v0
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v6, v4
-; CHECK-NEXT:    v_mul_lo_u32 v6, v2, v5
-; CHECK-NEXT:    v_mul_hi_u32 v8, v0, v5
-; CHECK-NEXT:    v_mul_hi_u32 v5, v2, v5
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v7
-; CHECK-NEXT:    v_mul_lo_u32 v7, v0, v4
-; CHECK-NEXT:    v_mul_lo_u32 v9, v2, v4
-; CHECK-NEXT:    v_mul_hi_u32 v10, v0, v4
-; CHECK-NEXT:    v_mul_hi_u32 v4, v2, v4
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_rcp_iflag_f32_e32 v1, v1
+; CHECK-NEXT:    v_mul_f32_e32 v1, 0x5f7ffffc, v1
+; CHECK-NEXT:    v_mul_f32_e32 v4, 0x2f800000, v1
+; CHECK-NEXT:    v_trunc_f32_e32 v4, v4
+; CHECK-NEXT:    v_mac_f32_e32 v1, 0xcf800000, v4
+; CHECK-NEXT:    v_cvt_u32_f32_e32 v4, v4
+; CHECK-NEXT:    v_cvt_u32_f32_e32 v1, v1
+; CHECK-NEXT:    v_mul_lo_u32 v5, s4, v4
+; CHECK-NEXT:    v_mul_lo_u32 v6, s4, v1
+; CHECK-NEXT:    v_mul_lo_u32 v7, s5, v1
+; CHECK-NEXT:    v_mul_hi_u32 v8, s4, v1
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v7, v5
+; CHECK-NEXT:    v_mul_lo_u32 v7, v4, v6
+; CHECK-NEXT:    v_mul_hi_u32 v9, v1, v6
+; CHECK-NEXT:    v_mul_hi_u32 v6, v4, v6
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v8
+; CHECK-NEXT:    v_mul_lo_u32 v8, v1, v5
+; CHECK-NEXT:    v_mul_lo_u32 v10, v4, v5
+; CHECK-NEXT:    v_mul_hi_u32 v11, v1, v5
+; CHECK-NEXT:    v_mul_hi_u32 v5, v4, v5
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v8
+; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v10, v6
+; CHECK-NEXT:    v_cndmask_b32_e64 v10, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v9
 ; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v9, v5
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v11
 ; CHECK-NEXT:    v_cndmask_b32_e64 v9, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v8, v7
+; CHECK-NEXT:    v_add_i32_e32 v8, vcc, v10, v9
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v8, v7
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v7
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v6
+; CHECK-NEXT:    v_addc_u32_e32 v4, vcc, v4, v5, vcc
+; CHECK-NEXT:    v_mul_lo_u32 v5, s4, v1
+; CHECK-NEXT:    v_mul_lo_u32 v6, s5, v1
+; CHECK-NEXT:    v_mul_hi_u32 v7, s4, v1
+; CHECK-NEXT:    v_mul_lo_u32 v8, s4, v4
+; CHECK-NEXT:    v_mul_lo_u32 v9, v4, v5
+; CHECK-NEXT:    v_mul_hi_u32 v10, v1, v5
+; CHECK-NEXT:    v_mul_hi_u32 v5, v4, v5
 ; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v8
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v10
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_mul_lo_u32 v7, v1, v6
+; CHECK-NEXT:    v_mul_lo_u32 v8, v4, v6
+; CHECK-NEXT:    v_mul_hi_u32 v11, v1, v6
+; CHECK-NEXT:    v_mul_hi_u32 v6, v4, v6
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v9, v7
+; CHECK-NEXT:    v_cndmask_b32_e64 v9, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v8, v5
 ; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v7, v6
-; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v9, v8
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v7, v6
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v6
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v5
-; CHECK-NEXT:    v_addc_u32_e32 v2, vcc, v2, v4, vcc
-; CHECK-NEXT:    v_mul_lo_u32 v4, s4, v0
-; CHECK-NEXT:    v_mul_lo_u32 v5, s5, v0
-; CHECK-NEXT:    v_mul_hi_u32 v6, s4, v0
-; CHECK-NEXT:    v_mul_lo_u32 v7, s4, v2
-; CHECK-NEXT:    v_mul_lo_u32 v8, v2, v4
-; CHECK-NEXT:    v_mul_hi_u32 v9, v0, v4
-; CHECK-NEXT:    v_mul_hi_u32 v4, v2, v4
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v10
+; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v11
+; CHECK-NEXT:    v_cndmask_b32_e64 v10, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v9, v7
+; CHECK-NEXT:    v_add_i32_e32 v8, vcc, v8, v10
 ; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v7
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
-; CHECK-NEXT:    v_mul_lo_u32 v6, v0, v5
-; CHECK-NEXT:    v_mul_lo_u32 v7, v2, v5
-; CHECK-NEXT:    v_mul_hi_u32 v10, v0, v5
-; CHECK-NEXT:    v_mul_hi_u32 v5, v2, v5
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v8, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v7, v4
 ; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v9
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v10
-; CHECK-NEXT:    v_cndmask_b32_e64 v9, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v8, v6
-; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v7, v9
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v7, v6
-; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v4
-; CHECK-NEXT:    v_addc_u32_e32 v2, vcc, v2, v5, vcc
-; CHECK-NEXT:    v_mul_lo_u32 v4, s1, v0
-; CHECK-NEXT:    v_mul_hi_u32 v5, s0, v0
-; CHECK-NEXT:    v_mul_hi_u32 v0, s1, v0
-; CHECK-NEXT:    v_mul_lo_u32 v6, s0, v2
-; CHECK-NEXT:    v_mul_lo_u32 v7, s1, v2
-; CHECK-NEXT:    v_mul_hi_u32 v8, s0, v2
-; CHECK-NEXT:    v_mul_hi_u32 v2, s1, v2
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v6
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v7, v0
+; CHECK-NEXT:    v_add_i32_e32 v7, vcc, v8, v7
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v6, v7
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v5
+; CHECK-NEXT:    v_addc_u32_e32 v4, vcc, v4, v6, vcc
+; CHECK-NEXT:    v_mul_lo_u32 v5, s1, v1
+; CHECK-NEXT:    v_mul_hi_u32 v6, s0, v1
+; CHECK-NEXT:    v_mul_hi_u32 v1, s1, v1
+; CHECK-NEXT:    v_mul_lo_u32 v7, s0, v4
+; CHECK-NEXT:    v_mul_lo_u32 v8, s1, v4
+; CHECK-NEXT:    v_mul_hi_u32 v9, s0, v4
+; CHECK-NEXT:    v_mul_hi_u32 v4, s1, v4
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v7
 ; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v5
-; CHECK-NEXT:    v_cndmask_b32_e64 v4, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v8
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v8, v1
+; CHECK-NEXT:    v_cndmask_b32_e64 v8, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v5, v6
 ; CHECK-NEXT:    v_cndmask_b32_e64 v5, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v6, v4
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v9
+; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, 1, vcc
 ; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v7, v5
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v4
-; CHECK-NEXT:    v_cndmask_b32_e64 v4, 0, 1, vcc
-; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v5, v4
-; CHECK-NEXT:    v_mul_lo_u32 v5, s2, v0
-; CHECK-NEXT:    v_mul_lo_u32 v6, s3, v0
-; CHECK-NEXT:    v_mul_hi_u32 v0, s2, v0
-; CHECK-NEXT:    v_add_i32_e32 v2, vcc, v2, v4
-; CHECK-NEXT:    v_mul_lo_u32 v2, s2, v2
-; CHECK-NEXT:    v_add_i32_e32 v2, vcc, v6, v2
-; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v2, v0
-; CHECK-NEXT:    v_sub_i32_e32 v2, vcc, s0, v5
-; CHECK-NEXT:    v_subb_u32_e64 v3, s[4:5], v3, v0, vcc
-; CHECK-NEXT:    v_sub_i32_e64 v0, s[4:5], s1, v0
-; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s2, v2
-; CHECK-NEXT:    v_cndmask_b32_e64 v4, 0, -1, s[4:5]
-; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s3, v3
+; CHECK-NEXT:    v_add_i32_e32 v6, vcc, v8, v6
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v1, v5
+; CHECK-NEXT:    v_cndmask_b32_e64 v5, 0, 1, vcc
+; CHECK-NEXT:    v_add_i32_e32 v5, vcc, v6, v5
+; CHECK-NEXT:    v_mul_lo_u32 v6, s2, v1
+; CHECK-NEXT:    v_mul_lo_u32 v7, s3, v1
+; CHECK-NEXT:    v_mul_hi_u32 v1, s2, v1
+; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v4, v5
+; CHECK-NEXT:    v_mul_lo_u32 v4, s2, v4
+; CHECK-NEXT:    v_add_i32_e32 v4, vcc, v7, v4
+; CHECK-NEXT:    v_add_i32_e32 v1, vcc, v4, v1
+; CHECK-NEXT:    v_sub_i32_e32 v4, vcc, s0, v6
+; CHECK-NEXT:    v_subb_u32_e64 v3, s[4:5], v3, v1, vcc
+; CHECK-NEXT:    v_sub_i32_e64 v1, s[4:5], s1, v1
+; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s2, v4
 ; CHECK-NEXT:    v_cndmask_b32_e64 v5, 0, -1, s[4:5]
-; CHECK-NEXT:    v_subb_u32_e32 v0, vcc, v0, v1, vcc
+; CHECK-NEXT:    v_cmp_le_u32_e64 s[4:5], s3, v3
+; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, -1, s[4:5]
+; CHECK-NEXT:    v_subb_u32_e32 v0, vcc, v1, v0, vcc
 ; CHECK-NEXT:    v_cmp_eq_u32_e32 vcc, s3, v3
-; CHECK-NEXT:    v_cndmask_b32_e32 v1, v5, v4, vcc
-; CHECK-NEXT:    v_subrev_i32_e32 v3, vcc, s2, v2
+; CHECK-NEXT:    v_cndmask_b32_e32 v1, v6, v5, vcc
+; CHECK-NEXT:    v_subrev_i32_e32 v3, vcc, s2, v4
 ; CHECK-NEXT:    v_subbrev_u32_e32 v0, vcc, 0, v0, vcc
 ; CHECK-NEXT:    v_cmp_le_u32_e32 vcc, s2, v3
-; CHECK-NEXT:    v_cndmask_b32_e64 v4, 0, -1, vcc
-; CHECK-NEXT:    v_subrev_i32_e32 v5, vcc, s2, v3
+; CHECK-NEXT:    v_cndmask_b32_e64 v5, 0, -1, vcc
+; CHECK-NEXT:    v_subrev_i32_e32 v6, vcc, s2, v3
 ; CHECK-NEXT:    v_cmp_le_u32_e32 vcc, s3, v0
-; CHECK-NEXT:    v_cndmask_b32_e64 v6, 0, -1, vcc
+; CHECK-NEXT:    v_cndmask_b32_e64 v7, 0, -1, vcc
 ; CHECK-NEXT:    v_cmp_eq_u32_e32 vcc, s3, v0
-; CHECK-NEXT:    v_cndmask_b32_e32 v0, v6, v4, vcc
+; CHECK-NEXT:    v_cndmask_b32_e32 v0, v7, v5, vcc
 ; CHECK-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v0
-; CHECK-NEXT:    v_cndmask_b32_e32 v0, v3, v5, vcc
+; CHECK-NEXT:    v_cndmask_b32_e32 v0, v3, v6, vcc
 ; CHECK-NEXT:    v_cmp_ne_u32_e32 vcc, 0, v1
-; CHECK-NEXT:    v_cndmask_b32_e32 v0, v2, v0, vcc
+; CHECK-NEXT:    v_cndmask_b32_e32 v0, v4, v0, vcc
 ; CHECK-NEXT:    s_mov_b32 s5, 0
 ; CHECK-NEXT:    s_branch .LBB1_3
 ; CHECK-NEXT:  .LBB1_2:
 ; CHECK-NEXT:    ; implicit-def: $vgpr0_vgpr1
 ; CHECK-NEXT:  .LBB1_3: ; %Flow
 ; CHECK-NEXT:    s_xor_b32 s1, s5, -1
 ; CHECK-NEXT:    s_and_b32 s1, s1, 1
 ; CHECK-NEXT:    s_cmp_lg_u32 s1, 0
 ; CHECK-NEXT:    s_cbranch_scc1 .LBB1_5
 ; CHECK-NEXT:  ; %bb.4:
-; CHECK-NEXT:    v_cvt_f32_u32_e32 v0, s2
+; CHECK-NEXT:    v_rcp_iflag_f32_e32 v0, v2
 ; CHECK-NEXT:    s_sub_i32 s1, 0, s2
-; CHECK-NEXT:    v_rcp_iflag_f32_e32 v0, v0
 ; CHECK-NEXT:    v_mul_f32_e32 v0, 0x4f7ffffe, v0
 ; CHECK-NEXT:    v_cvt_u32_f32_e32 v0, v0
 ; CHECK-NEXT:    v_mul_lo_u32 v1, s1, v0
 ; CHECK-NEXT:    v_mul_hi_u32 v1, v0, v1
 ; CHECK-NEXT:    v_add_i32_e32 v0, vcc, v0, v1
 ; CHECK-NEXT:    v_mul_hi_u32 v0, s0, v0
 ; CHECK-NEXT:    v_mul_lo_u32 v0, v0, s2
 ; CHECK-NEXT:    v_sub_i32_e32 v0, vcc, s0, v0

llvm/test/CodeGen/AMDGPU/selectcc-opt.ll

ENDIF:
   ret void
 }
 
 ; Same as test_a, but the branch labels are swapped to produce the inverse cc
 ; for the icmp instruction
 
 ; EG-LABEL: {{^}}test_b:
 ; EG: SET{{[GTEQN]+}}_DX10
+; EG-NEXT: 0(0.000000e+00), 0(0.000000e+00)
 ; EG-NEXT: PRED_
 ; EG-NEXT: ALU clause starting
 define amdgpu_kernel void @test_b(i32 addrspace(1)* %out, float %in) {
 entry:
   %0 = fcmp olt float %in, 0.0
   %1 = select i1 %0, float 1.000000e+00, float 0.000000e+00
   %2 = fneg float %1
   %3 = fptosi float %2 to i32

llvm/test/CodeGen/ARM/machine-cse-cmp.ll

 store i32 %s3, i32* %offset, align 4
 %add.ptr = getelementptr inbounds i8, i8* %base, i32 %sub
 br label %return
 
 return:
 %retval.0 = phi i8* [ %add.ptr, %if.end ], [ null, %entry ]
 ret i8* %retval.0
 }
+
+; The cmp of %val should not be hoisted above the preceding conditional branch
+define void @f4(i32** %ptr1, i64* %ptr2, i64 %val) {
+entry:
+; CHECK-LABEL: f4:
+; CHECK: cmp
+; CHECK: movne
+; CHECK: strne
+; CHECK: orrs
+; CHECK-NOT: subs
+; CHECK-NOT: sbcs
+; CHECK: beq
+  %tobool.not = icmp eq i32** %ptr1, null
+  br i1 %tobool.not, label %if.end, label %if.then
+
+if.then:
+  store i32* null, i32** %ptr1, align 4
+  br label %if.end
+
+if.end:
+; CHECK: subs
+; CHECK: sbcs
+; CHECK: bxlt lr
+  %tobool1 = icmp ne i64 %val, 0
+  %cmp = icmp slt i64 %val, 10
+  %or.cond = and i1 %tobool1, %cmp
+  br i1 %or.cond, label %cleanup, label %if.end3
+
+if.end3:
+; CHECK: subs
+; CHECK: sbc
+  %sub = add nsw i64 %val, -10
+  store i64 %sub, i64* %ptr2, align 8
+  br label %cleanup
+
+cleanup:
+  ret void
+}

llvm/test/CodeGen/PowerPC/machine-cse-rm-pre.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc %s -o - -mtriple=powerpc-unknown-unknown -run-pass=machine-cse -verify-machineinstrs | FileCheck %s
+--- |
+  define void @can_pre() {
+  entry:
+    br label %for.body
+
+  for.body:
+    br i1 undef, label %if.then, label %if.else
+
+  if.then:
+    br label %if.end
+
+  if.else:
+    br label %if.end
+
+  if.end:
+    br label %for.body
+  }
+
+  define void @cannot_pre() {
+  entry:
+    br label %for.body
+
+  for.body:
+    br i1 undef, label %if.then, label %if.else
+
+  if.then:
+    br label %if.end
+
+  if.else:
+    br label %if.end
+
+  if.end:
+    br label %for.body
+  }
+...
+---
+name: can_pre
+registers:
+  - { id: 0, class: f8rc, preferred-register: '' }
+  - { id: 1, class: f8rc, preferred-register: '' }
+  - { id: 2, class: gprc, preferred-register: '' }
+  - { id: 3, class: gprc, preferred-register: '' }
+  - { id: 4, class: f8rc, preferred-register: '' }
+  - { id: 5, class: f8rc, preferred-register: '' }
+liveins:
+  - { reg: '$r1', virtual-reg: '%2' }
+  - { reg: '$r2', virtual-reg: '%3' }
+  - { reg: '$f1', virtual-reg: '%4' }
+  - { reg: '$f2', virtual-reg: '%5' }
+body:             |
+  ; CHECK-LABEL: name: can_pre
+  ; CHECK: bb.0.for.body:
+  ; CHECK-NEXT:   successors: %bb.1(0x40000000), %bb.2(0x40000000)
+  ; CHECK-NEXT:   liveins: $r1, $r2, $f1, $f2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:f8rc = COPY $f2
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:f8rc = COPY $f1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:gprc = COPY $r2
+  ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:gprc = COPY $r1
+  ; CHECK-NEXT:   $cr0 = CMPLWI [[COPY3]], 0
+  ; CHECK-NEXT:   %6:f8rc = nofpexcept FDIV [[COPY1]], [[COPY]], implicit $rm
+  ; CHECK-NEXT:   BCC 44, $cr0, %bb.1
+  ; CHECK-NEXT:   B %bb.2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.1.if.then:
+  ; CHECK-NEXT:   successors: %bb.3(0x80000000)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   B %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.if.else:
+  ; CHECK-NEXT:   successors: %bb.3(0x80000000)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.if.end:
+  ; CHECK-NEXT:   BLR implicit $lr, implicit $rm
+  bb.0.for.body:
+    successors: %bb.1(0x40000000), %bb.2(0x40000000)
+    liveins: $r1, $r2, $f1, $f2
+
+    %5:f8rc = COPY $f2
+    %4:f8rc = COPY $f1
+    %3:gprc = COPY $r2
+    %2:gprc = COPY $r1
+    $cr0 = CMPLWI %2, 0
+    BCC 44, $cr0, %bb.1
+    B %bb.2
+
+  bb.1.if.then:
+    successors: %bb.3(0x80000000)
+
+    %0:f8rc = nofpexcept FDIV %4, %5, implicit $rm
+    B %bb.3
+
+  bb.2.if.else:
+    successors: %bb.3(0x80000000)
+
+    %1:f8rc = nofpexcept FDIV %4, %5, implicit $rm
+
+  bb.3.if.end:
+    BLR implicit $lr, implicit $rm
+...
+---
+name: cannot_pre
+registers:
+  - { id: 0, class: f8rc, preferred-register: '' }
+  - { id: 1, class: f8rc, preferred-register: '' }
+  - { id: 2, class: gprc, preferred-register: '' }
+  - { id: 3, class: gprc, preferred-register: '' }
+  - { id: 4, class: f8rc, preferred-register: '' }
+  - { id: 5, class: f8rc, preferred-register: '' }
+  - { id: 6, class: f8rc, preferred-register: '' }
+liveins:
+  - { reg: '$r1', virtual-reg: '%2' }
+  - { reg: '$r2', virtual-reg: '%3' }
+  - { reg: '$f1', virtual-reg: '%4' }
+  - { reg: '$f2', virtual-reg: '%5' }
+body:             |
+  ; CHECK-LABEL: name: cannot_pre
+  ; CHECK: bb.0.for.body:
+  ; CHECK-NEXT:   successors: %bb.1(0x40000000), %bb.2(0x40000000)
+  ; CHECK-NEXT:   liveins: $r1, $r2, $f1, $f2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[COPY:%[0-9]+]]:f8rc = COPY $f2
+  ; CHECK-NEXT:   [[COPY1:%[0-9]+]]:f8rc = COPY $f1
+  ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:gprc = COPY $r2
+  ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:gprc = COPY $r1
+  ; CHECK-NEXT:   $cr0 = CMPLWI [[COPY3]], 0
+  ; CHECK-NEXT:   BCC 44, $cr0, %bb.1
+  ; CHECK-NEXT:   B %bb.2
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.1.if.then:
+  ; CHECK-NEXT:   successors: %bb.3(0x80000000)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   [[SETRND:%[0-9]+]]:f8rc = SETRND [[COPY2]], implicit-def $rm, implicit $rm
+  ; CHECK-NEXT:   %0:f8rc = nofpexcept FDIV [[COPY1]], [[COPY]], implicit $rm
+  ; CHECK-NEXT:   B %bb.3
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.2.if.else:
+  ; CHECK-NEXT:   successors: %bb.3(0x80000000)
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT:   %1:f8rc = nofpexcept FDIV [[COPY1]], [[COPY]], implicit $rm
+  ; CHECK-NEXT: {{  $}}
+  ; CHECK-NEXT: bb.3.if.end:
+  ; CHECK-NEXT:   BLR implicit $lr, implicit $rm
+  bb.0.for.body:
+    successors: %bb.1(0x40000000), %bb.2(0x40000000)
+    liveins: $r1, $r2, $f1, $f2
+
+    %5:f8rc = COPY $f2
+    %4:f8rc = COPY $f1
+    %3:gprc = COPY $r2
+    %2:gprc = COPY $r1
+    $cr0 = CMPLWI %2, 0
+    BCC 44, $cr0, %bb.1
+    B %bb.2
+
+  bb.1.if.then:
+    successors: %bb.3(0x80000000)
+
+    %6:f8rc = SETRND %3, implicit-def $rm, implicit $rm
+    %0:f8rc = nofpexcept FDIV %4, %5, implicit $rm
+    B %bb.3
+
+  bb.2.if.else:
+    successors: %bb.3(0x80000000)
+
+    %1:f8rc = nofpexcept FDIV %4, %5, implicit $rm
+
+  bb.3.if.end:
+    BLR implicit $lr, implicit $rm
+...

llvm/test/CodeGen/Thumb2/LowOverheadLoops/memcall.ll

for.body:                                         ; preds = %entry, %for.body
   %inc = add nuw nsw i32 %i.09, 1
   %exitcond.not = icmp eq i32 %inc, %n
   br i1 %exitcond.not, label %for.cond.cleanup, label %for.body
 }
 
 define void @test_memset_preheader(i8* %x, i8* %y, i32 %n) {
 ; CHECK-LABEL: test_memset_preheader:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
+; CHECK-NEXT:    .save {r4, lr}
+; CHECK-NEXT:    push {r4, lr}
 ; CHECK-NEXT:    cbz r2, .LBB6_5
 ; CHECK-NEXT:  @ %bb.1: @ %prehead
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
-; CHECK-NEXT:    mov r12, r0
+; CHECK-NEXT:    mov r4, r0
 ; CHECK-NEXT:    wlstp.8 lr, r2, .LBB6_3
 ; CHECK-NEXT:  .LBB6_2: @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    vstrb.8 q0, [r12], #16
+; CHECK-NEXT:    vstrb.8 q0, [r4], #16
 ; CHECK-NEXT:    letp lr, .LBB6_2
 ; CHECK-NEXT:  .LBB6_3: @ %prehead
 ; CHECK-NEXT:    dls lr, r2
-; CHECK-NEXT:    mov r12, r0
+; CHECK-NEXT:    mov r3, r0
 ; CHECK-NEXT:  .LBB6_4: @ %for.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    ldrb r3, [r12], #1
-; CHECK-NEXT:    strb r3, [r1], #1
+; CHECK-NEXT:    ldrb r4, [r3], #1
+; CHECK-NEXT:    strb r4, [r1], #1
 ; CHECK-NEXT:    le lr, .LBB6_4
 ; CHECK-NEXT:  .LBB6_5: @ %for.cond.cleanup
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
 ; CHECK-NEXT:    wlstp.8 lr, r2, .LBB6_7
 ; CHECK-NEXT:  .LBB6_6: @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vstrb.8 q0, [r0], #16
 ; CHECK-NEXT:    letp lr, .LBB6_6
 ; CHECK-NEXT:  .LBB6_7: @ %for.cond.cleanup
-; CHECK-NEXT:    pop {r7, pc}
+; CHECK-NEXT:    pop {r4, pc}
 entry:
   %cmp6 = icmp ne i32 %n, 0
   br i1 %cmp6, label %prehead, label %for.cond.cleanup
 
 prehead:
   call void @llvm.memset.p0i8.i32(i8* %x, i8 0, i32 %n, i1 false)
   br label %for.body