[MachineCopy] Enhance sub register machine copy propagation

[xiangzhangllvm]

Summary

The machine copy propagation logic has some problem:
When do machine copy propagation for subreg we need to consider the CopyDstReg and CopySrcReg's regclass and size.
(because we want to get SubReg from CopySrcReg by using SubregIdx which we got from CopyDstReg)
Take follow case for example:

MCP: Replacing $ecx
     with $k0
     in renamable $k1 = COPY renamable $ecx
     from renamable $rcx = COPY renamable $k0, implicit-def $ecx

We can not use SubregIdx (ecx of rcx) to get anything from k0 register.

The upper test case problem was fix/escaped by D146930
But there is still risk in it. More details PLS refer to comments in D146930
Now this patch is to enhance the safety in propagating SubReg.

[barannikov88]

Does it try to solve the same problem as D146930?

[xiangzhangllvm]

In D147031#4227472, @barannikov88 wrote:

Does it try to solve the same problem as D146930?

Very similar from the test case, I commented on D146930

[barannikov88]

It crashes on the test from D146930.

[barannikov88]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

672

All registers in a register class have the same sizes. The check for size above is redundant.

[lkail]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

631

This looks similar to what isForwardableRegClassCopy does? Can we extend isForwardableRegClassCopy to contain subregindex?

[xiangzhangllvm]

In D147031#4229099, @barannikov88 wrote:

It crashes on the test from D146930.

Sorry for late respond. Let me take a look on your test.
Anyway this bug affect our critical tests，I +1 for let D146930 first in.
I also comment the risk in D146930.

llvm/lib/CodeGen/MachineCopyPropagation.cpp

631

Sound good, we can check it in that function.

672

Make sense!

[xiangzhangllvm]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

631

isForwardableRegClassCopy already try handle cross-class case.
Looks we only need to enhance it:

diff --git a/llvm/lib/CodeGen/MachineCopyPropagation.cpp b/llvm/lib/CodeGen/MachineCopyPropagation.cpp
index 218f293a4085..fff2893e2148 100644
--- a/llvm/lib/CodeGen/MachineCopyPropagation.cpp
+++ b/llvm/lib/CodeGen/MachineCopyPropagation.cpp
@@ -539,6 +539,16 @@ bool MachineCopyPropagation::isForwardableRegClassCopy(const MachineInstr &Copy,
   /// so we have reduced the number of cross-class COPYs and potentially
   /// introduced a nop COPY that can be removed.

+  // We don't propagate for SubReg in cross-class COPYs.
+  // For example: (ecx is SubReg of rcx)
+  // renamable $rcx = COPY killed renamable $k0
+  // renamable $k1 = COPY renamable $ecx
+  Register CopyDstReg = CopyOperands->Destination->getReg();
+  Register UseSrcReg = UseICopyOperands->Source->getReg();
+  if (TRI->getRegSizeInBits(CopyDstReg, *MRI) !=
+      TRI->getRegSizeInBits(UseSrcReg, *MRI))
+    return false;
+
   // Allow forwarding if src and dst belong to any common class, so long as they
   // don't belong to any (possibly smaller) common class that requires copies to
   // go via a different class.

Let me do more testing, thanks!

[barannikov88]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

555

This method is not called for this example (look for "MCP: Copy source does not have sub-register" below).
$ecx is a sub-register of $rcx, and its sub-register index is sub_32bit. $k0 does not have sub-registers with index sub_32bit.
You can check my assertion by removing this code and making sure that the added test still passes.

[arsenm]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

427–433

You should be able to use existing, generated class relation queries

558–560

Shouldn't need to operate in terms of register sizes

[xiangzhangllvm]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

427–433

Can you gave more hint pls ?
I see the existing code use same way to compare RegClass, pls refer lambda
CheckCopyConstraint

555

Yes, after you commit D146930 the old test machine-copy-subreg.mir can pass without this patch. I need to create a new test.

This patch is not handle "MCP: Copy source does not have sub-register". It try to handle "we can get sub-register from copy source, but the sub-register is wrong"

As I comment in D146930
For different reg class, the following convert has risk.

B= COPY C
A= COPY SubB

==>
A = COPY SubC

Because the SubRegIdx of B (for SubB) may not match the SubRegIdx of C (for SubC).
(for example B has 2 subreg but C only has 4)

558–560

Yes, let me directly check SubReg relation here.

[xiangzhangllvm]

llvm/lib/CodeGen/MachineCopyPropagation.cpp

555

I think the risk is not existed.
Here the unsigned sub-register index imply both the offset and size ( sub_32bit)
Not only index of sub-register in one register 's subs.
Though @barannikov88 mentioned sub_32bit before, but I am not 100% make sure about it.
Let me close this patch.

llvm/lib/CodeGen/MachineCopyPropagation.cpp

bool isForwardableRegClassCopy(const MachineInstr &Copy,
                                  const MachineInstr &UseI, unsigned UseIdx);
   bool isBackwardPropagatableRegClassCopy(const MachineInstr &Copy,
                                           const MachineInstr &UseI,
                                           unsigned UseIdx);
   bool hasImplicitOverlap(const MachineInstr &MI, const MachineOperand &Use);
   bool hasOverlappingMultipleDef(const MachineInstr &MI,
                                  const MachineOperand &MODef, Register Def);
 
+  bool hasSameRegClass(Register Def, Register Src);
+
   /// Candidates for deletion.
   SmallSetVector<MachineInstr *, 8> MaybeDeadCopies;
 
   /// Multimap tracking debug users in current BB
   DenseMap<MachineInstr *, SmallSet<MachineInstr *, 2>> CopyDbgUsers;
 
   CopyTracker Tracker;
 
if (MachineInstr *Copy = Tracker.findCopyForUnit(*RUI, *TRI)) {
         MaybeDeadCopies.remove(Copy);
       } else {
         CopyDbgUsers[Copy].insert(&Reader);
       }
     }
   }
 }
 
+bool MachineCopyPropagation::hasSameRegClass(Register Def, Register Src) {
+  for (const TargetRegisterClass *RC : TRI->regclasses()) {
+    if (RC->contains(Def) && RC->contains(Src))
+      return true;
+  }
+  return false;
+}
+
 /// Return true if \p PreviousCopy did copy register \p Src to register \p Def.
 /// This fact may have been obscured by sub register usage or may not be true at
 /// all even though Src and Def are subregisters of the registers used in
 /// PreviousCopy. e.g.
 /// isNopCopy("ecx = COPY eax", AX, CX) == true
 /// isNopCopy("ecx = COPY eax", AH, CL) == false
 static bool isNopCopy(const MachineInstr &PreviousCopy, MCRegister Src,
                       MCRegister Def, const TargetRegisterInfo *TRI,
bool MachineCopyPropagation::isForwardableRegClassCopy(const MachineInstr &Copy,
   ///
   ///   RegClassA = COPY RegClassB
   ///   ...
   ///   RegClassB = COPY RegClassB
   ///
   /// so we have reduced the number of cross-class COPYs and potentially
   /// introduced a nop COPY that can be removed.
 
+  // We don't propagate for SubReg in cross-class COPYs.
+  // For example: (ecx is SubReg of rcx)
+  // renamable $rcx = COPY killed renamable $k0
+  // renamable $k1 = COPY renamable $ecx
+  Register CopyDstReg = CopyOperands->Destination->getReg();
+  Register UseSrcReg = UseICopyOperands->Source->getReg();
+  if ((TRI->getRegSizeInBits(CopyDstReg, *MRI) !=
+       TRI->getRegSizeInBits(UseSrcReg, *MRI)) &&
+      !hasSameRegClass(CopyDstReg, CopySrcReg))
+    return false;
+
   // Allow forwarding if src and dst belong to any common class, so long as they
   // don't belong to any (possibly smaller) common class that requires copies to
   // go via a different class.
   Register UseDstReg = UseICopyOperands->Destination->getReg();
   bool Found = false;
   bool IsCrossClass = false;
   for (const TargetRegisterClass *RC : TRI->regclasses()) {
     if (RC->contains(CopySrcReg) && RC->contains(UseDstReg)) {
       Found = true;
       if (TRI->getCrossCopyRegClass(RC) != RC) {
         IsCrossClass = true;
         break;
       }
     }
   }
   if (!Found)
     return false;
   if (!IsCrossClass)
     return true;
   // The forwarded copy would be cross-class. Only do this if the original copy
   // was also cross-class.
-  Register CopyDstReg = CopyOperands->Destination->getReg();
   for (const TargetRegisterClass *RC : TRI->regclasses()) {
     if (RC->contains(CopySrcReg) && RC->contains(CopyDstReg) &&
         TRI->getCrossCopyRegClass(RC) != RC)
       return true;
   }
   return false;
 }
 

 }
 
 /// Look for available copies whose destination register is used by \p MI and
 /// replace the use in \p MI with the copy's source register.
 void MachineCopyPropagation::forwardUses(MachineInstr &MI) {
   if (!Tracker.hasAnyCopies())
     return;
 
   // Look for non-tied explicit vreg uses that have an active COPY
   // instruction that defines the physical register allocated to them.
   // Replace the vreg with the source of the active COPY.
   for (unsigned OpIdx = 0, OpEnd = MI.getNumOperands(); OpIdx < OpEnd;
        ++OpIdx) {
     MachineOperand &MOUse = MI.getOperand(OpIdx);
     // Don't forward into undef use operands since doing so can cause problems
     // with the machine verifier, since it doesn't treat undef reads as reads,
     // so we can end up with a live range that ends on an undef read, leading to
for (unsigned OpIdx = 0, OpEnd = MI.getNumOperands(); OpIdx < OpEnd;
     Register CopySrcReg = CopySrc.getReg();
 
     Register ForwardedReg = CopySrcReg;
     // MI might use a sub-register of the Copy destination, in which case the
     // forwarded register is the matching sub-register of the Copy source.
     if (MOUse.getReg() != CopyDstReg) {
       unsigned SubRegIdx = TRI->getSubRegIndex(CopyDstReg, MOUse.getReg());
       assert(SubRegIdx &&
              "MI source is not a sub-register of Copy destination");
       ForwardedReg = TRI->getSubReg(CopySrcReg, SubRegIdx);
       if (!ForwardedReg) {
         LLVM_DEBUG(dbgs() << "MCP: Copy source does not have sub-register "
                           << TRI->getSubRegIndexName(SubRegIdx) << '\n');
         continue;
       }
     }
 
auto TryFoldSpillageCopies =
         for (const MachineInstr *Spill : make_range(SC.begin() + 1, SC.end()))
           if (CopySourceInvalid.count(Spill))
             return;
 
         for (const MachineInstr *Reload : make_range(RC.begin(), RC.end() - 1))
           if (CopySourceInvalid.count(Reload))
             return;
 
-        auto CheckCopyConstraint = [this](Register Def, Register Src) {
-          for (const TargetRegisterClass *RC : TRI->regclasses()) {
-            if (RC->contains(Def) && RC->contains(Src))
-              return true;
-          }
-          return false;
-        };
-
         auto UpdateReg = [](MachineInstr *MI, const MachineOperand *Old,
                             const MachineOperand *New) {
           for (MachineOperand &MO : MI->operands()) {
             if (&MO == Old)
               MO.setReg(New->getReg());
           }
         };
 
         std::optional<DestSourcePair> InnerMostSpillCopy =
             isCopyInstr(*SC[0], *TII, UseCopyInstr);
         std::optional<DestSourcePair> OuterMostSpillCopy =
             isCopyInstr(*SC.back(), *TII, UseCopyInstr);
         std::optional<DestSourcePair> InnerMostReloadCopy =
             isCopyInstr(*RC[0], *TII, UseCopyInstr);
         std::optional<DestSourcePair> OuterMostReloadCopy =
             isCopyInstr(*RC.back(), *TII, UseCopyInstr);
-        if (!CheckCopyConstraint(OuterMostSpillCopy->Source->getReg(),
-                                 InnerMostSpillCopy->Source->getReg()) ||
-            !CheckCopyConstraint(InnerMostReloadCopy->Destination->getReg(),
-                                 OuterMostReloadCopy->Destination->getReg()))
+        if (!hasSameRegClass(OuterMostSpillCopy->Source->getReg(),
+            !hasSameRegClass(InnerMostReloadCopy->Destination->getReg(),
           return;
 
         SpillageChainsLength += SC.size() + RC.size();
         NumSpillageChains += 1;
         UpdateReg(SC[0], InnerMostSpillCopy->Destination,
                   OuterMostSpillCopy->Source);
         UpdateReg(RC[0], InnerMostReloadCopy->Source,
                   OuterMostReloadCopy->Destination);

llvm/test/CodeGen/X86/machine-copy-subreg.mir

+# RUN: llc -mtriple=x86_64-- -run-pass=machine-cp %s -o - | FileCheck %s
+--- |
+  target triple = "x86_64-pc-windows-msvc19.29.30145"
+
+  define void @foo(<256 x double> %0, ptr %1) #0 {
+  DIR.OMP.PARALLEL.LOOP.2:
+    %2 = fcmp olt <256 x double> %0, zeroinitializer
+    %3 = extractelement <256 x i1> %2, i64 64
+    %4 = select i1 %3, double 0.000000e+00, double 1.000000e+00
+    %5 = extractelement <256 x i1> %2, i64 65
+    %6 = select i1 %5, double 0.000000e+00, double 1.000000e+00
+    store double %4, ptr %1, align 16
+    store double %6, ptr null, align 8
+    ret void
+  }
+
+  attributes #0 = { "target-cpu"="skylake-avx512" }
+
+...
+---
+# CHECK-LABEL: name: foo
+name:            foo
+alignment:       16
+exposesReturnsTwice: false
+legalized:       false
+regBankSelected: false
+selected:        false
+failedISel:      false
+tracksRegLiveness: true
+hasWinCFI:       false
+callsEHReturn:   false
+callsUnwindInit: false
+hasEHCatchret:   false
+hasEHScopes:     false
+hasEHFunclets:   false
+debugInstrRef:   true
+failsVerification: false
+tracksDebugUserValues: true
+registers:       []
+liveins:         []
+frameInfo:
+  isFrameAddressTaken: false
+  isReturnAddressTaken: false
+  hasStackMap:     false
+  hasPatchPoint:   false
+  stackSize:       0
+  offsetAdjustment: 0
+  maxAlignment:    8
+  adjustsStack:    false
+  hasCalls:        false
+  stackProtector:  ''
+  functionContext: ''
+  maxCallFrameSize: 4294967295
+  cvBytesOfCalleeSavedRegisters: 0
+  hasOpaqueSPAdjustment: false
+  hasVAStart:      false
+  hasMustTailInVarArgFunc: false
+  hasTailCall:     false
+  localFrameSize:  0
+  savePoint:       ''
+  restorePoint:    ''
+fixedStack:
+  - { id: 0, type: default, offset: 256, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 1, type: default, offset: 248, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 2, type: default, offset: 240, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 3, type: default, offset: 232, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 4, type: default, offset: 224, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 5, type: default, offset: 216, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 6, type: default, offset: 208, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 7, type: default, offset: 200, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 8, type: default, offset: 192, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 9, type: default, offset: 184, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 10, type: default, offset: 176, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 11, type: default, offset: 168, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 12, type: default, offset: 160, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 13, type: default, offset: 152, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 14, type: default, offset: 144, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 15, type: default, offset: 136, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 16, type: default, offset: 128, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 17, type: default, offset: 120, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 18, type: default, offset: 112, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 19, type: default, offset: 104, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 20, type: default, offset: 96, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 21, type: default, offset: 88, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 22, type: default, offset: 80, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 23, type: default, offset: 72, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 24, type: default, offset: 64, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 25, type: default, offset: 56, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 26, type: default, offset: 48, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 27, type: default, offset: 40, size: 8, alignment: 8, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+  - { id: 28, type: default, offset: 32, size: 8, alignment: 16, stack-id: default,
+      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true,
+      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
+stack:           []
+callSites:       []
+debugValueSubstitutions: []
+constants:
+  - id:              0
+    value:           'double 1.000000e+00'
+    alignment:       8
+    isTargetSpecific: false
+machineFunctionInfo: {}
+body:             |
+  bb.0.DIR.OMP.PARALLEL.LOOP.2:
+    ; CHECK:      renamable $k2 = KUNPCKBWrr killed renamable $k3, killed renamable $k2
+    ; CHECK-NEXT: renamable $k1 = KUNPCKWDrr killed renamable $k2, killed renamable $k1
+    ; CHECK-NEXT: renamable $k0 = KUNPCKDQrr killed renamable $k1, killed renamable $k0
+    ; CHECK-NEXT: renamable $rcx = COPY killed renamable $k0
+    ; CHECK-NEXT: renamable $k1 = COPY renamable $ecx
+
+    renamable $rax = MOV64rm %fixed-stack.0, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.0, align 16)
+    renamable $rcx = MOV64rm %fixed-stack.17, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.17)
+    renamable $rdx = MOV64rm %fixed-stack.18, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.18, align 16)
+    renamable $r8 = MOV64rm %fixed-stack.19, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.19)
+    renamable $r9 = MOV64rm %fixed-stack.20, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.20, align 16)
+    renamable $r10 = MOV64rm %fixed-stack.21, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.21)
+    renamable $r11 = MOV64rm %fixed-stack.22, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.22, align 16)
+    renamable $rsi = MOV64rm %fixed-stack.23, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.23)
+    renamable $rdi = MOV64rm %fixed-stack.24, 1, $noreg, 0, $noreg :: (load (s64) from %fixed-stack.24, align 16)
+    renamable $zmm0 = AVX512_512_SET0
+    renamable $k0 = nofpexcept VCMPPDZrmi renamable $zmm0, killed renamable $rdi, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k1 = nofpexcept VCMPPDZrmi renamable $zmm0, killed renamable $rsi, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k0 = KUNPCKBWrr killed renamable $k1, killed renamable $k0
+    renamable $k1 = nofpexcept VCMPPDZrmi renamable $zmm0, killed renamable $r11, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k2 = nofpexcept VCMPPDZrmi renamable $zmm0, killed renamable $r10, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k1 = KUNPCKBWrr killed renamable $k2, killed renamable $k1
+    renamable $k0 = KUNPCKWDrr killed renamable $k1, killed renamable $k0
+    renamable $k1 = nofpexcept VCMPPDZrmi renamable $zmm0, killed renamable $r9, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k2 = nofpexcept VCMPPDZrmi renamable $zmm0, killed renamable $r8, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k1 = KUNPCKBWrr killed renamable $k2, killed renamable $k1
+    renamable $k2 = nofpexcept VCMPPDZrmi renamable $zmm0, killed renamable $rdx, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k3 = nofpexcept VCMPPDZrmi killed renamable $zmm0, killed renamable $rcx, 1, $noreg, 0, $noreg, 14, implicit $mxcsr :: (load (s512))
+    renamable $k2 = KUNPCKBWrr killed renamable $k3, killed renamable $k2
+    renamable $k1 = KUNPCKWDrr killed renamable $k2, killed renamable $k1
+    renamable $k0 = KUNPCKDQrr killed renamable $k1, killed renamable $k0
+    renamable $rcx = COPY killed renamable $k0
+    renamable $k1 = COPY renamable $ecx
+    renamable $ecx = KILL renamable $ecx, implicit killed $rcx
+    renamable $ecx = AND32ri8 killed renamable $ecx, 2, implicit-def dead $eflags
+    renamable $ecx = SHR32r1 killed renamable $ecx, implicit-def dead $eflags
+    renamable $xmm0 = AVX512_FsFLD0SD
+    renamable $xmm1 = VMOVSDZrm_alt $rip, 1, $noreg, %const.0, $noreg :: (load (s64) from constant-pool)
+    renamable $xmm2 = COPY renamable $xmm1
+    renamable $xmm2 = VMOVSDZrrk killed renamable $xmm2, killed renamable $k1, undef renamable $xmm0, renamable $xmm0
+    renamable $k1 = COPY killed renamable $ecx
+    renamable $xmm1 = VMOVSDZrrk killed renamable $xmm1, killed renamable $k1, undef renamable $xmm0, killed renamable $xmm0
+    VMOVSDZmr killed renamable $rax, 1, $noreg, 0, $noreg, killed renamable $xmm2 :: (store (s64) into %ir.1, align 16)
+    VMOVSDZmr $noreg, 1, $noreg, 0, $noreg, killed renamable $xmm1 :: (store (s64) into `ptr null`)
+    RET 0
+
+...