[GSel]: Add a cleanup combiner to cleanup legalization artifacts

[aditya_nandakumar]

Summary

Added a combiner which can clean up truncs/extends that are created in order to make the types work during legalization.

Also moved the combineMerges to the LegalizeCombiner.

Looking forward to your feedback.

[aditya_nandakumar]

Added comment in the header section of LegalizeCombine.h

[aditya_nandakumar]

Previous update didn't have the comment section change staged and was missing from the diff I uploaded.

[volkan]

include/llvm/CodeGen/GlobalISel/LegalizerCombine.h

2

This doesn't match with the filename. Is it LegalizerCombiner.h or LegalizerCombine.h?

3

Could you remove the redundant dashes and format this line?

36

MI.getOpcode() != TargetOpcode::G_ANYEXT would be better here.

39

• Missing check: DefMI && ....
• Same here: DefMI->getOpcode() == TargetOpcode::G_TRUNC.

Since you check the same condition in the other functions too, I think it would be better if you could add a function for this.

40

What about DEBUG(dbgs() << ".. Combine MI: " << *MI);?

52

An empty line here would make the code easy to read.

53

Same as above.

77

An empty line here would make the code easy to read.

78

Same as above.

108

Same as above.

113

Could you replace this with MRI.getVRegDef(SrcReg) and check if(MRI.getVRegDef(SrcReg))?

179

An empty line here would make the code easy to read.

lib/CodeGen/GlobalISel/Legalizer.cpp

131

You do the combines after legalizing all of the recorded instructions, but a recorded instruction might be erased. So, I think we need to iterate over the MBB after the legalization.

[aditya_nandakumar]

Thanks Volkan for your feedback. Replied/Updated to most comments.

include/llvm/CodeGen/GlobalISel/LegalizerCombine.h

36

This comes from my habit of using the Constant on the left idiom (old habit so I don't accidentally do if (a =42) foo() which will silently compile where I meant to say ==, but with constant on the left, it won't accidentally compile). While I don't think one way is better than the other, I have updated as requested :).

39

I didn't think the DefMI check here is necessary as it's impossible (illegal) to generate a G_[AZS]_EXT instruction where the input operand is not a register defined from an instruction. I can probably add it for completeness. Maybe an assert here is better?

113

Is the check really necessary vs an assert?

lib/CodeGen/GlobalISel/Legalizer.cpp

131

I've added a check where if the legalization response is Legalized, we should assume it's erased and remove from the combine list. Is this what you had in mind? Is there some case where this check wouldn't be sufficient?

[aditya_nandakumar]

Updated based on feedback.

[volkan]

Thank you Aditya. I added a few more inline comments.

include/llvm/CodeGen/GlobalISel/LegalizerCombiner.h

34 ↗

(On Diff #112971)

Could you clang-format this line?

55 ↗

(On Diff #112971)

Same as above.

58 ↗

(On Diff #112971)

Same as above.

82 ↗

(On Diff #112971)

Could you update this too? Just to make it consistent with the others.

lib/CodeGen/GlobalISel/Legalizer.cpp

125

DEBUG(dbgs() << ".. .. New MI: " << *WorkList[I]);

131

The problem is LegalizerHelper::Legalized doesn't mean all of the inserted instruction are legal. Targets can insert illegal instructions as they are going to be legalized anyway.

137

Same as above: dbgs() << ... << *DeadMI.

148

Same as above.

[aditya_nandakumar]

lib/CodeGen/GlobalISel/Legalizer.cpp

131

For each iteration of legalizeInstr, we get a return value Legalized (if legalized) and we assume that it's dead and remove from the combine list. When the additional instructions are legalized, depending on the legalizeAction, it will be Legalized (and removed from the CombineList) - or else combined.

[aditya_nandakumar]

Updated based on feedback.

[volkan]

lib/CodeGen/GlobalISel/Legalizer.cpp

131

I see now, thanks for explaining. That check should be enough then.

167

Do we still need this code block? I think the tryCombineInstruction call above already handles this case.

[aditya_nandakumar]

Updated based on Volkan's Feedback.

[aditya_nandakumar]

lib/CodeGen/GlobalISel/Legalizer.cpp

167

Yes - the tryCombine is currently only called on newly created instructions (legalization artifacts) where as we want to try to combine already existing merges.

[aditya_nandakumar]

Added a TODO comment about considering moving the Merges combining into a combiner pass

[volkan]

Thank you Aditya, LGTM!

[aditya_nandakumar]

Committed in r312158

include/llvm/CodeGen/GlobalISel/Legalizer.h

public:
   MachineFunctionProperties getSetProperties() const override {
     return MachineFunctionProperties().set(
         MachineFunctionProperties::Property::Legalized);
   }
 
   bool combineExtracts(MachineInstr &MI, MachineRegisterInfo &MRI,
                        const TargetInstrInfo &TII);
 
-  bool combineMerges(MachineInstr &MI, MachineRegisterInfo &MRI,
-                     const TargetInstrInfo &TII, MachineIRBuilder &MIRBuilder);
-
   bool runOnMachineFunction(MachineFunction &MF) override;
 };
 } // End namespace llvm.
 
 #endif

include/llvm/CodeGen/GlobalISel/LegalizerCombine.h

+//===-- llvm/CodeGen/GlobalISel/LegalizerCombiner.h
+//-----------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/GlobalISel/Legalizer.h"
+#include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
+#include "llvm/CodeGen/GlobalISel/Utils.h"
+#include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/Support/Debug.h"
+
+#define DEBUG_TYPE "legalizer"
+
+namespace llvm {
+class LegalizerCombiner {
+  MachineIRBuilder &Builder;
+  MachineRegisterInfo &MRI;
+
+public:
+  LegalizerCombiner(MachineIRBuilder &B, MachineRegisterInfo &MRI)
+      : Builder(B), MRI(MRI) {}
+
+  bool tryCombineAnyExt(MachineInstr &MI,
+                        SmallVectorImpl<MachineInstr *> &DeadInsts) {
+    if (TargetOpcode::G_ANYEXT != MI.getOpcode())
+      return false;
+    MachineInstr *DefMI = MRI.getVRegDef(MI.getOperand(1).getReg());
+    if (TargetOpcode::G_TRUNC == DefMI->getOpcode()) {
+      DEBUG(dbgs() << ".. Combine MI: "; MI.print(dbgs()));
+      unsigned DstReg = MI.getOperand(0).getReg();
+      unsigned SrcReg = DefMI->getOperand(1).getReg();
+      Builder.setInstr(MI);
+      // We get a copy/trunc/extend depending on the sizes
+      Builder.buildAnyExtOrTrunc(DstReg, SrcReg);
+      MI.eraseFromParent();
+      if (MRI.use_empty(DefMI->getOperand(0).getReg()))
+        DeadInsts.push_back(DefMI);
+      return true;
+    }
+    return false;
+  }
+  bool tryCombineZExt(MachineInstr &MI,
+                      SmallVectorImpl<MachineInstr *> &DeadInsts) {
+    if (TargetOpcode::G_ZEXT != MI.getOpcode())
+      return false;
+    MachineInstr *DefMI = MRI.getVRegDef(MI.getOperand(1).getReg());
+    if (TargetOpcode::G_TRUNC == DefMI->getOpcode()) {
+      DEBUG(dbgs() << ".. Combine MI: "; MI.print(dbgs()));
+      Builder.setInstr(MI);
+      unsigned DstReg = MI.getOperand(0).getReg();
+      unsigned ZExtSrc = MI.getOperand(1).getReg();
+      LLT ZExtSrcTy = MRI.getType(ZExtSrc);
+      LLT DstTy = MRI.getType(DstReg);
+      APInt Mask = APInt::getAllOnesValue(ZExtSrcTy.getSizeInBits());
+      auto MaskCstMIB = Builder.buildConstant(DstTy, Mask.getZExtValue());
+      unsigned TruncSrc = DefMI->getOperand(1).getReg();
+      // We get a copy/trunc/extend depending on the sizes
+      auto SrcCopyOrTrunc = Builder.buildAnyExtOrTrunc(DstTy, TruncSrc);
+      Builder.buildAnd(DstReg, SrcCopyOrTrunc, MaskCstMIB);
+      MI.eraseFromParent();
+      if (MRI.use_empty(DefMI->getOperand(0).getReg()))
+        DeadInsts.push_back(DefMI);
+      return true;
+    }
+    return false;
+  }
+  bool tryCombineSExt(MachineInstr &MI,
+                      SmallVectorImpl<MachineInstr *> &DeadInsts) {
+    if (TargetOpcode::G_SEXT != MI.getOpcode())
+      return false;
+    MachineInstr *DefMI = MRI.getVRegDef(MI.getOperand(1).getReg());
+    if (TargetOpcode::G_TRUNC == DefMI->getOpcode()) {
+      DEBUG(dbgs() << ".. Combine MI: "; MI.print(dbgs()));
+      Builder.setInstr(MI);
+      unsigned DstReg = MI.getOperand(0).getReg();
+      LLT DstTy = MRI.getType(DstReg);
+      unsigned SExtSrc = MI.getOperand(1).getReg();
+      LLT SExtSrcTy = MRI.getType(SExtSrc);
+      unsigned SizeDiff = DstTy.getSizeInBits() - SExtSrcTy.getSizeInBits();
+      auto SizeDiffMIB = Builder.buildConstant(DstTy, SizeDiff);
+      unsigned TruncSrcReg = DefMI->getOperand(1).getReg();
+      // We get a copy/trunc/extend depending on the sizes
+      auto SrcCopyExtOrTrunc = Builder.buildAnyExtOrTrunc(DstTy, TruncSrcReg);
+      auto ShlMIB = Builder.buildInstr(TargetOpcode::G_SHL, DstTy,
+                                       SrcCopyExtOrTrunc, SizeDiffMIB);
+      Builder.buildInstr(TargetOpcode::G_ASHR, DstReg, ShlMIB, SizeDiffMIB);
+      MI.eraseFromParent();
+      if (MRI.use_empty(DefMI->getOperand(0).getReg()))
+        DeadInsts.push_back(DefMI);
+      return true;
+    }
+    return false;
+  }
+
+  bool tryCombineMerges(MachineInstr &MI,
+                        SmallVectorImpl<MachineInstr *> &DeadInsts) {
+    if (MI.getOpcode() != TargetOpcode::G_UNMERGE_VALUES)
+      return false;
+
+    unsigned NumDefs = MI.getNumOperands() - 1;
+    unsigned SrcReg = MI.getOperand(NumDefs).getReg();
+    MachineInstr &MergeI = *MRI.def_instr_begin(SrcReg);
+    if (MergeI.getOpcode() != TargetOpcode::G_MERGE_VALUES)
+      return false;
+
+    const unsigned NumMergeRegs = MergeI.getNumOperands() - 1;
+
+    if (NumMergeRegs < NumDefs) {
+      if (NumDefs % NumMergeRegs != 0)
+        return false;
+
+      Builder.setInstr(MI);
+      // Transform to UNMERGEs, for example
+      //   %1 = G_MERGE_VALUES %4, %5
+      //   %9, %10, %11, %12 = G_UNMERGE_VALUES %1
+      // to
+      //   %9, %10 = G_UNMERGE_VALUES %4
+      //   %11, %12 = G_UNMERGE_VALUES %5
+
+      const unsigned NewNumDefs = NumDefs / NumMergeRegs;
+      for (unsigned Idx = 0; Idx < NumMergeRegs; ++Idx) {
+        SmallVector<unsigned, 2> DstRegs;
+        for (unsigned j = 0, DefIdx = Idx * NewNumDefs; j < NewNumDefs;
+             ++j, ++DefIdx)
+          DstRegs.push_back(MI.getOperand(DefIdx).getReg());
+
+        Builder.buildUnmerge(DstRegs, MergeI.getOperand(Idx + 1).getReg());
+      }
+
+    } else if (NumMergeRegs > NumDefs) {
+      if (NumMergeRegs % NumDefs != 0)
+        return false;
+
+      Builder.setInstr(MI);
+      // Transform to MERGEs
+      //   %6 = G_MERGE_VALUES %17, %18, %19, %20
+      //   %7, %8 = G_UNMERGE_VALUES %6
+      // to
+      //   %7 = G_MERGE_VALUES %17, %18
+      //   %8 = G_MERGE_VALUES %19, %20
+
+      const unsigned NumRegs = NumMergeRegs / NumDefs;
+      for (unsigned DefIdx = 0; DefIdx < NumDefs; ++DefIdx) {
+        SmallVector<unsigned, 2> Regs;
+        for (unsigned j = 0, Idx = NumRegs * DefIdx + 1; j < NumRegs;
+             ++j, ++Idx)
+          Regs.push_back(MergeI.getOperand(Idx).getReg());
+
+        Builder.buildMerge(MI.getOperand(DefIdx).getReg(), Regs);
+      }
+
+    } else {
+      // FIXME: is a COPY appropriate if the types mismatch? We know both
+      // registers are allocatable by now.
+      if (MRI.getType(MI.getOperand(0).getReg()) !=
+          MRI.getType(MergeI.getOperand(1).getReg()))
+        return false;
+
+      for (unsigned Idx = 0; Idx < NumDefs; ++Idx)
+        MRI.replaceRegWith(MI.getOperand(Idx).getReg(),
+                           MergeI.getOperand(Idx + 1).getReg());
+    }
+
+    MI.eraseFromParent();
+    if (MRI.use_empty(MergeI.getOperand(0).getReg()))
+      DeadInsts.push_back(&MergeI);
+    return true;
+  }
+  /// Try to combine away MI.
+  /// Returns true if it combined away the MI.
+  /// Caller should not rely in MI existing as it may be deleted.
+  /// Adds instructions that are dead as a result of the combine
+  // into DeadInsts
+  bool tryCombineInstruction(MachineInstr &MI,
+                             SmallVectorImpl<MachineInstr *> &DeadInsts) {
+    switch (MI.getOpcode()) {
+    default:
+      return false;
+    case TargetOpcode::G_ANYEXT:
+      return tryCombineAnyExt(MI, DeadInsts);
+    case TargetOpcode::G_ZEXT:
+      return tryCombineZExt(MI, DeadInsts);
+    case TargetOpcode::G_SEXT:
+      return tryCombineSExt(MI, DeadInsts);
+    case TargetOpcode::G_UNMERGE_VALUES:
+      return tryCombineMerges(MI, DeadInsts);
+    }
+  }
+};
+
+} // namespace llvm

include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h

class MachineIRBuilder {
   }
 
   void addUsesFromArgs(MachineInstrBuilder &MIB) { }
   template<typename UseArgTy, typename ... UseArgsTy>
   void addUsesFromArgs(MachineInstrBuilder &MIB, UseArgTy &&Arg1, UseArgsTy &&... Args) {
     addUseFromArg(MIB, Arg1);
     addUsesFromArgs(MIB, std::forward<UseArgsTy>(Args)...);
   }
+
+  unsigned getRegFromArg(unsigned Reg) { return Reg; }
+  unsigned getRegFromArg(MachineInstrBuilder &MIB) {
+    return MIB->getOperand(0).getReg();
+  }
+
 public:
   /// Some constructors for easy use.
   MachineIRBuilder() = default;
   MachineIRBuilder(MachineFunction &MF) { setMF(MF); }
   MachineIRBuilder(MachineInstr &MI)
       : MachineIRBuilder(*MI.getParent()->getParent()) {
     setInstr(MI);
   }
public:
   /// G_AND sets \p Res to the bitwise and of integer parameters \p Op0 and \p
   /// Op1.
   ///
   /// \pre setBasicBlock or setMI must have been called.
   /// \pre \p Res, \p Op0 and \p Op1 must be generic virtual registers
   ///      with the same (scalar or vector) type).
   ///
   /// \return a MachineInstrBuilder for the newly created instruction.
+  template <typename DstTy, typename... UseArgsTy>
+  MachineInstrBuilder buildAnd(DstTy &&Dst, UseArgsTy &&... UseArgs) {
+    return buildAnd(getDestFromArg(Dst), getRegFromArg(UseArgs)...);
+  }
   MachineInstrBuilder buildAnd(unsigned Res, unsigned Op0,
                                unsigned Op1);
 
   /// Build and insert \p Res<def> = G_OR \p Op0, \p Op1
   ///
   /// G_OR sets \p Res to the bitwise or of integer parameters \p Op0 and \p
   /// Op1.
   ///
public:
   /// Build and insert \p Res<def> = G_ZEXT \p Op, \p Res = G_TRUNC \p Op, or
   /// \p Res = COPY \p Op depending on the differing sizes of \p Res and \p Op.
   ///  ///
   /// \pre setBasicBlock or setMI must have been called.
   /// \pre \p Res must be a generic virtual register with scalar or vector type.
   /// \pre \p Op must be a generic virtual register with scalar or vector type.
   ///
   /// \return The newly created instruction.
+  template <typename DstTy, typename UseArgTy>
+  MachineInstrBuilder buildZExtOrTrunc(DstTy &&Dst, UseArgTy &&Use) {
+    return buildZExtOrTrunc(getDestFromArg(Dst), getRegFromArg(Use));
+  }
   MachineInstrBuilder buildZExtOrTrunc(unsigned Res, unsigned Op);
 
+  /// Build and insert \p Res<def> = G_ANYEXT \p Op, \p Res = G_TRUNC \p Op, or
+  /// \p Res = COPY \p Op depending on the differing sizes of \p Res and \p Op.
+  ///  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  /// \pre \p Res must be a generic virtual register with scalar or vector type.
+  /// \pre \p Op must be a generic virtual register with scalar or vector type.
+  ///
+  /// \return The newly created instruction.
+  template <typename DstTy, typename UseArgTy>
+  MachineInstrBuilder buildAnyExtOrTrunc(DstTy &&Dst, UseArgTy &&Use) {
+    return buildAnyExtOrTrunc(getDestFromArg(Dst), getRegFromArg(Use));
+  }
+  MachineInstrBuilder buildAnyExtOrTrunc(unsigned Res, unsigned Op);
+
+  /// Build and insert \p Res<def> = \p ExtOpc, \p Res = G_TRUNC \p
+  /// Op, or \p Res = COPY \p Op depending on the differing sizes of \p Res and
+  /// \p Op.
+  ///  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  /// \pre \p Res must be a generic virtual register with scalar or vector type.
+  /// \pre \p Op must be a generic virtual register with scalar or vector type.
+  ///
+  /// \return The newly created instruction.
+  MachineInstrBuilder buildExtOrTrunc(unsigned ExtOpc, unsigned Res,
+                                      unsigned Op);
+
   /// Build and insert an appropriate cast between two registers of equal size.
   MachineInstrBuilder buildCast(unsigned Dst, unsigned Src);
 
   /// Build and insert G_BR \p Dest
   ///
   /// G_BR is an unconditional branch to \p Dest.
   ///
   /// \pre setBasicBlock or setMI must have been called.

lib/CodeGen/GlobalISel/Legalizer.cpp

 //===-- llvm/CodeGen/GlobalISel/Legalizer.cpp -----------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 /// \file This file implements the LegalizerHelper class to legalize individual
 /// instructions and the LegalizePass wrapper pass for the primary
 /// legalization.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/CodeGen/GlobalISel/Legalizer.h"
+#include "llvm/ADT/SetVector.h"
+#include "llvm/CodeGen/GlobalISel/LegalizerCombine.h"
 #include "llvm/CodeGen/GlobalISel/LegalizerHelper.h"
 #include "llvm/CodeGen/GlobalISel/Utils.h"
 #include "llvm/CodeGen/MachineOptimizationRemarkEmitter.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/TargetPassConfig.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Target/TargetInstrInfo.h"
 #include "llvm/Target/TargetSubtargetInfo.h"

 void Legalizer::getAnalysisUsage(AnalysisUsage &AU) const {
   AU.addRequired<TargetPassConfig>();
   MachineFunctionPass::getAnalysisUsage(AU);
 }
 
 void Legalizer::init(MachineFunction &MF) {
 }
 
-bool Legalizer::combineMerges(MachineInstr &MI, MachineRegisterInfo &MRI,
-                              const TargetInstrInfo &TII,
-                              MachineIRBuilder &MIRBuilder) {
-  if (MI.getOpcode() != TargetOpcode::G_UNMERGE_VALUES)
-    return false;
-
-  unsigned NumDefs = MI.getNumOperands() - 1;
-  unsigned SrcReg = MI.getOperand(NumDefs).getReg();
-  MachineInstr &MergeI = *MRI.def_instr_begin(SrcReg);
-  if (MergeI.getOpcode() != TargetOpcode::G_MERGE_VALUES)
-    return false;
-
-  const unsigned NumMergeRegs = MergeI.getNumOperands() - 1;
-
-  if (NumMergeRegs < NumDefs) {
-    if (NumDefs % NumMergeRegs != 0)
-      return false;
-
-    MIRBuilder.setInstr(MI);
-    // Transform to UNMERGEs, for example
-    //   %1 = G_MERGE_VALUES %4, %5
-    //   %9, %10, %11, %12 = G_UNMERGE_VALUES %1
-    // to
-    //   %9, %10 = G_UNMERGE_VALUES %4
-    //   %11, %12 = G_UNMERGE_VALUES %5
-
-    const unsigned NewNumDefs = NumDefs / NumMergeRegs;
-    for (unsigned Idx = 0; Idx < NumMergeRegs; ++Idx) {
-      SmallVector<unsigned, 2> DstRegs;
-      for (unsigned j = 0, DefIdx = Idx * NewNumDefs; j < NewNumDefs;
-           ++j, ++DefIdx)
-        DstRegs.push_back(MI.getOperand(DefIdx).getReg());
-
-      MIRBuilder.buildUnmerge(DstRegs, MergeI.getOperand(Idx + 1).getReg());
-    }
-
-  } else if (NumMergeRegs > NumDefs) {
-    if (NumMergeRegs % NumDefs != 0)
-      return false;
-
-    MIRBuilder.setInstr(MI);
-    // Transform to MERGEs
-    //   %6 = G_MERGE_VALUES %17, %18, %19, %20
-    //   %7, %8 = G_UNMERGE_VALUES %6
-    // to
-    //   %7 = G_MERGE_VALUES %17, %18
-    //   %8 = G_MERGE_VALUES %19, %20
-
-    const unsigned NumRegs = NumMergeRegs / NumDefs;
-    for (unsigned DefIdx = 0; DefIdx < NumDefs; ++DefIdx) {
-      SmallVector<unsigned, 2> Regs;
-      for (unsigned j = 0, Idx = NumRegs * DefIdx + 1; j < NumRegs; ++j, ++Idx)
-        Regs.push_back(MergeI.getOperand(Idx).getReg());
-
-      MIRBuilder.buildMerge(MI.getOperand(DefIdx).getReg(), Regs);
-    }
-
-  } else {
-    // FIXME: is a COPY appropriate if the types mismatch? We know both
-    // registers are allocatable by now.
-    if (MRI.getType(MI.getOperand(0).getReg()) !=
-        MRI.getType(MergeI.getOperand(1).getReg()))
-      return false;
-
-    for (unsigned Idx = 0; Idx < NumDefs; ++Idx)
-      MRI.replaceRegWith(MI.getOperand(Idx).getReg(),
-                         MergeI.getOperand(Idx + 1).getReg());
-  }
-
-  MI.eraseFromParent();
-  if (MRI.use_empty(MergeI.getOperand(0).getReg()))
-    MergeI.eraseFromParent();
-  return true;
-}
-
 bool Legalizer::runOnMachineFunction(MachineFunction &MF) {
   // If the ISel pipeline failed, do not bother running that pass.
   if (MF.getProperties().hasProperty(
           MachineFunctionProperties::Property::FailedISel))
     return false;
   DEBUG(dbgs() << "Legalize Machine IR for: " << MF.getName() << '\n');
   init(MF);
   const TargetPassConfig &TPC = getAnalysis<TargetPassConfig>();
for (auto MI = MBB.begin(); MI != MBB.end(); MI = NextMI) {
       // good chance MI will be deleted.
       NextMI = std::next(MI);
 
       // Only legalize pre-isel generic instructions: others don't have types
       // and are assumed to be legal.
       if (!isPreISelGenericOpcode(MI->getOpcode()))
         continue;
       unsigned NumNewInsns = 0;
-      SmallVector<MachineInstr *, 4> WorkList;
+      using VecType = SetVector<MachineInstr *, SmallVector<MachineInstr *, 8>>;
+      VecType WorkList;
+      VecType CombineList;
       Helper.MIRBuilder.recordInsertions([&](MachineInstr *MI) {
         // Only legalize pre-isel generic instructions.
         // Legalization process could generate Target specific pseudo
         // instructions with generic types. Don't record them
         if (isPreISelGenericOpcode(MI->getOpcode())) {
           ++NumNewInsns;
-          WorkList.push_back(MI);
+          WorkList.insert(MI);
+          CombineList.insert(MI);
         }
       });
-      WorkList.push_back(&*MI);
-
+      WorkList.insert(&*MI);
+      LegalizerCombiner C(Helper.MIRBuilder, MF.getRegInfo());
       bool Changed = false;
       LegalizerHelper::LegalizeResult Res;
-      unsigned Idx = 0;
       do {
-        Res = Helper.legalizeInstrStep(*WorkList[Idx]);
+        assert(!WorkList.empty() && "Expecting illegal ops");
+        while (!WorkList.empty()) {
+          NumNewInsns = 0;
+          MachineInstr *CurrInst = WorkList.pop_back_val();
+          if (!CurrInst)
+            continue;
+          Res = Helper.legalizeInstrStep(*CurrInst);
           // Error out if we couldn't legalize this instruction. We may want to
           // fall back to DAG ISel instead in the future.
           if (Res == LegalizerHelper::UnableToLegalize) {
             Helper.MIRBuilder.stopRecordingInsertions();
             if (Res == LegalizerHelper::UnableToLegalize) {
               reportGISelFailure(MF, TPC, MORE, "gisel-legalize",
-                               "unable to legalize instruction",
-                               *WorkList[Idx]);
+                                 "unable to legalize instruction", *CurrInst);
               return false;
             }
           }
           Changed |= Res == LegalizerHelper::Legalized;
-        ++Idx;
-
 #ifndef NDEBUG
-        if (NumNewInsns) {
-          DEBUG(dbgs() << ".. .. Emitted " << NumNewInsns << " insns\n");
-          for (auto I = WorkList.end() - NumNewInsns, E = WorkList.end();
+          if (NumNewInsns)
+            for (unsigned I = WorkList.size() - NumNewInsns,
+                          E = WorkList.size();
                  I != E; ++I)
-            DEBUG(dbgs() << ".. .. New MI: "; (*I)->print(dbgs()));
+              DEBUG(dbgs() << ".. .. New MI: "; WorkList[I]->print(dbgs()));
+#endif
+        }
+        // Do the combines.
+        while (!CombineList.empty()) {
           NumNewInsns = 0;
+          MachineInstr *CurrInst = CombineList.pop_back_val();
+          if (!CurrInst)
+            continue;
+          SmallVector<MachineInstr *, 4> DeadInstructions;
+          Changed |= C.tryCombineInstruction(*CurrInst, DeadInstructions);
+          for (auto *DeadMI : DeadInstructions) {
+            DEBUG(dbgs() << ".. Erasing Dead Instruction ";
+                  DeadMI->print(dbgs()));
+            CombineList.remove(DeadMI);
+            WorkList.remove(DeadMI);
+            DeadMI->eraseFromParent();
           }
+#ifndef NDEBUG
+          if (NumNewInsns)
+            for (unsigned I = CombineList.size() - NumNewInsns,
+                          E = CombineList.size();
+                 I != E; ++I)
+              DEBUG(dbgs() << ".. .. Combine New MI: ";
+                    CombineList[I]->print(dbgs()));
 #endif
-      } while (Idx < WorkList.size());
+        }
+      } while (!WorkList.empty());
 
       Helper.MIRBuilder.stopRecordingInsertions();
     }
   }
 
   MachineRegisterInfo &MRI = MF.getRegInfo();
-  const TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo();
+  MachineIRBuilder MIRBuilder(MF);
+  LegalizerCombiner C(MIRBuilder, MRI);
   for (auto &MBB : MF) {
     for (auto MI = MBB.begin(); MI != MBB.end(); MI = NextMI) {
       // Get the next Instruction before we try to legalize, because there's a
       // good chance MI will be deleted.
       NextMI = std::next(MI);
-      Changed |= combineMerges(*MI, MRI, TII, Helper.MIRBuilder);
+      SmallVector<MachineInstr *, 4> DeadInsts;
+      Changed |= C.tryCombineMerges(*MI, DeadInsts);
+      for (auto *DeadMI : DeadInsts)
+        DeadMI->eraseFromParent();
     }
   }
 
   return Changed;
 }

lib/CodeGen/GlobalISel/MachineIRBuilder.cpp

MachineInstrBuilder MachineIRBuilder::buildSExt(unsigned Res, unsigned Op) {
   return buildInstr(TargetOpcode::G_SEXT).addDef(Res).addUse(Op);
 }
 
 MachineInstrBuilder MachineIRBuilder::buildZExt(unsigned Res, unsigned Op) {
   validateTruncExt(Res, Op, true);
   return buildInstr(TargetOpcode::G_ZEXT).addDef(Res).addUse(Op);
 }
 
-MachineInstrBuilder MachineIRBuilder::buildSExtOrTrunc(unsigned Res,
-                                                       unsigned Op) {
+MachineInstrBuilder
+MachineIRBuilder::buildExtOrTrunc(unsigned ExtOpc, unsigned Res, unsigned Op) {
+  assert((TargetOpcode::G_ANYEXT == ExtOpc || TargetOpcode::G_ZEXT == ExtOpc ||
+          TargetOpcode::G_SEXT == ExtOpc) &&
+         "Expecting Extending Opc");
   assert(MRI->getType(Res).isScalar() || MRI->getType(Res).isVector());
   assert(MRI->getType(Res).isScalar() == MRI->getType(Op).isScalar());
 
   unsigned Opcode = TargetOpcode::COPY;
   if (MRI->getType(Res).getSizeInBits() > MRI->getType(Op).getSizeInBits())
-    Opcode = TargetOpcode::G_SEXT;
+    Opcode = ExtOpc;
   else if (MRI->getType(Res).getSizeInBits() < MRI->getType(Op).getSizeInBits())
     Opcode = TargetOpcode::G_TRUNC;
   else
     assert(MRI->getType(Res) == MRI->getType(Op));
 
   return buildInstr(Opcode).addDef(Res).addUse(Op);
 }
 
-MachineInstrBuilder MachineIRBuilder::buildZExtOrTrunc(unsigned Res,
+MachineInstrBuilder MachineIRBuilder::buildSExtOrTrunc(unsigned Res,
                                                        unsigned Op) {
-  assert(MRI->getType(Res).isScalar() || MRI->getType(Res).isVector());
-  assert(MRI->getType(Res).isScalar() == MRI->getType(Op).isScalar());
+  return buildExtOrTrunc(TargetOpcode::G_SEXT, Res, Op);
+}
 
-  unsigned Opcode = TargetOpcode::COPY;
-  if (MRI->getType(Res).getSizeInBits() > MRI->getType(Op).getSizeInBits())
-    Opcode = TargetOpcode::G_ZEXT;
-  else if (MRI->getType(Res).getSizeInBits() < MRI->getType(Op).getSizeInBits())
-    Opcode = TargetOpcode::G_TRUNC;
-  else
-    assert(MRI->getType(Res) == MRI->getType(Op));
+MachineInstrBuilder MachineIRBuilder::buildZExtOrTrunc(unsigned Res,
+                                                       unsigned Op) {
+  return buildExtOrTrunc(TargetOpcode::G_ZEXT, Res, Op);
+}
 
-  return buildInstr(Opcode).addDef(Res).addUse(Op);
+MachineInstrBuilder MachineIRBuilder::buildAnyExtOrTrunc(unsigned Res,
+                                                         unsigned Op) {
+  return buildExtOrTrunc(TargetOpcode::G_ANYEXT, Res, Op);
 }
 
 MachineInstrBuilder MachineIRBuilder::buildCast(unsigned Dst, unsigned Src) {
   LLT SrcTy = MRI->getType(Src);
   LLT DstTy = MRI->getType(Dst);
   if (SrcTy == DstTy)
     return buildCopy(Dst, Src);
 

test/CodeGen/AArch64/GlobalISel/legalize-add.mir

registers:
   - { id: 2, class: _ }
   - { id: 3, class: _ }
   - { id: 4, class: _ }
   - { id: 5, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_scalar_add_small
-    ; CHECK: [[OP0:%.*]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[OP1:%.*]](s32) = G_ANYEXT %3(s8)
+    ; CHECK: [[A:%.*]](s64) = COPY %x0
+    ; CHECK: [[B:%.*]](s64) = COPY %x1
+    ; CHECK: [[OP0:%.*]](s32) = G_TRUNC [[A]]
+    ; CHECK: [[OP1:%.*]](s32) = G_TRUNC [[B]]
     ; CHECK: [[RES32:%.*]](s32) = G_ADD [[OP0]], [[OP1]]
     ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[RES32]](s32)
 
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
     %4(s8) = G_ADD %2, %3

test/CodeGen/AArch64/GlobalISel/legalize-and.mir

registers:
   - { id: 2, class: _ }
   - { id: 3, class: _ }
   - { id: 4, class: _ }
   - { id: 5, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_scalar_and_small
-    ; CHECK: [[OP0:%.*]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[OP1:%.*]](s32) = G_ANYEXT %3(s8)
+    ; CHECK: [[A:%.*]](s64) = COPY %x0
+    ; CHECK: [[B:%.*]](s64) = COPY %x1
+    ; CHECK: [[OP0:%.*]](s32) = G_TRUNC [[A]]
+    ; CHECK: [[OP1:%.*]](s32) = G_TRUNC [[B]]
     ; CHECK: [[RES32:%.*]](s32) = G_AND [[OP0]], [[OP1]]
     ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[RES32]](s32)
 
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
     %4(s8) = G_AND %2, %3
     %5(s64) = G_ANYEXT %2
     %x0 = COPY %5
 ...

test/CodeGen/AArch64/GlobalISel/legalize-cmp.mir

bb.0.entry:
 
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
 
     ; CHECK: [[CMP1:%[0-9]+]](s32) = G_ICMP intpred(sge), %0(s64), %1
     ; CHECK: [[CMP_T1:%[0-9]+]](s1) = G_TRUNC [[CMP1]]
     %4(s1) = G_ICMP intpred(sge), %0, %1
 
-    ; CHECK: [[LHS32:%[0-9]+]](s32) = G_ZEXT %2
-    ; CHECK: [[RHS32:%[0-9]+]](s32) = G_ZEXT %3
-    ; CHECK: [[CMP2:%[0-9]+]](s32) = G_ICMP intpred(ult), [[LHS32]](s32), [[RHS32]]
+    ; CHECK: [[CSTMASK1:%[0-9]+]](s32) = G_CONSTANT i32 255
+    ; CHECK: [[T1:%[0-9]+]](s32) = G_TRUNC %0(s64)
+    ; CHECK: [[AND1:%[0-9]+]](s32) = G_AND [[T1]], [[CSTMASK1]]
+    ; CHECK: [[CSTMASK2:%[0-9]+]](s32) = G_CONSTANT i32 255
+    ; CHECK: [[T2:%[0-9]+]](s32) = G_TRUNC %1(s64)
+    ; CHECK: [[AND2:%[0-9]+]](s32) = G_AND [[T2]], [[CSTMASK2]]
+    ; CHECK: [[CMP2:%[0-9]+]](s32) = G_ICMP intpred(ult), [[AND1]](s32), [[AND2]]
     ; CHECK: [[CMP_T2:%[0-9]+]](s1) = G_TRUNC [[CMP2]]
     %8(s1) = G_ICMP intpred(ult), %2, %3
 
     %9(p0) = G_INTTOPTR %0(s64)
     %10(s1) = G_ICMP intpred(eq), %9(p0), %9(p0)
 ...

test/CodeGen/AArch64/GlobalISel/legalize-div.mir

 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
 
+    ; CHECK: [[A:%.*]](s64) = COPY %x0
+    ; CHECK: [[B:%.*]](s64) = COPY %x1
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 24
+    ; CHECK: [[S1:%.*]](s32) = G_TRUNC [[A]]
+    ; CHECK: [[SHL1:%.*]](s32) = G_SHL [[S1]], [[C1]]
+    ; CHECK: [[SEXT1:%.*]](s32) = G_ASHR [[SHL1]], [[C1]]
+    ; CHECK: [[C2:%.*]](s32) = G_CONSTANT i32 24
+    ; CHECK: [[S2:%.*]](s32) = G_TRUNC [[B]]
+    ; CHECK: [[SHL2:%.*]](s32) = G_SHL [[S2]], [[C2]]
+    ; CHECK: [[SEXT2:%.*]](s32) = G_ASHR [[SHL2]], [[C2]]
+    ; CHECK: [[DIV:%.*]](s32) = G_SDIV [[SEXT1]], [[SEXT2]]
+    ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[DIV]]
 
-    ; CHECK: [[LHS32:%[0-9]+]](s32) = G_SEXT %2
-    ; CHECK: [[RHS32:%[0-9]+]](s32) = G_SEXT %3
-    ; CHECK: [[QUOT32:%[0-9]+]](s32) = G_SDIV [[LHS32]], [[RHS32]]
-    ; CHECK: [[RES:%[0-9]+]](s8) = G_TRUNC [[QUOT32]]
     %4(s8) = G_SDIV %2, %3
 
-    ; CHECK: [[LHS32:%[0-9]+]](s32) = G_ZEXT %2
-    ; CHECK: [[RHS32:%[0-9]+]](s32) = G_ZEXT %3
+
+    ; CHECK: [[CMASK1:%.*]](s32) = G_CONSTANT i32 255
+    ; CHECK: [[T1:%.*]](s32) = G_TRUNC [[A]]
+    ; CHECK: [[LHS32:%.*]](s32) = G_AND [[T1]], [[CMASK1]]
+    ; CHECK: [[CMASK2:%.*]](s32) = G_CONSTANT i32 255
+    ; CHECK: [[T2:%.*]](s32) = G_TRUNC [[B]]
+    ; CHECK: [[RHS32:%.*]](s32) = G_AND [[T2]], [[CMASK2]]
     ; CHECK: [[QUOT32:%[0-9]+]](s32) = G_UDIV [[LHS32]], [[RHS32]]
     ; CHECK: [[RES:%[0-9]+]](s8) = G_TRUNC [[QUOT32]]
     %5(s8) = G_UDIV %2, %3
 
 ...

test/CodeGen/AArch64/GlobalISel/legalize-gep.mir

registers:
   - { id: 0, class: _ }
   - { id: 1, class: _ }
   - { id: 2, class: _ }
   - { id: 3, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_gep_small
-    ; CHECK: [[OFFSET_EXT:%[0-9]+]](s64) = G_SEXT %2(s8)
-    ; CHECK: %3(p0) = G_GEP %0, [[OFFSET_EXT]](s64)
+    ; CHECK: [[A:%.*]](p0) = COPY %x0
+    ; CHECK: [[B:%.*]](s64) = COPY %x1
+    ; CHECK: [[C:%.*]](s64) = G_CONSTANT i64 56
+    ; CHECK: [[SRC:%.*]](s64) = COPY [[B]](s64)
+    ; CHECK: [[SHL:%.*]](s64) = G_SHL [[SRC]], [[C]]
+    ; CHECK: [[SEXT:%.*]](s64) = G_ASHR [[SHL]], [[C]]
+    ; CHECK: G_GEP [[A]], [[SEXT]]
+
 
     %0(p0) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %1
     %3(p0) = G_GEP %0, %2(s8)
     %x0 = COPY %3
 ...

test/CodeGen/AArch64/GlobalISel/legalize-itofp.mir

 name:            test_sitofp_s32_s1
 body: |
   bb.0:
     liveins: %w0
     %0:_(s32) = COPY %w0
     %1:_(s1) = G_TRUNC %0
 
     ; CHECK-LABEL: name: test_sitofp_s32_s1
-    ; CHECK: %3(s32) = G_SEXT %1
-    ; CHECK: %2(s32) = G_SITOFP %3
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 31
+    ; CHECK: [[SRC:%.*]](s32) = COPY %0(s32)
+    ; CHECK: [[SHL1:%.*]](s32) = G_SHL [[SRC]], [[C1]]
+    ; CHECK: [[SEXT:%.*]](s32) = G_ASHR [[SHL1]], [[C1]]
+    ; CHECK: %2(s32) = G_SITOFP [[SEXT]]
     %2:_(s32) = G_SITOFP %1
 ...
 
 ---
 name:            test_uitofp_s32_s1
 body: |
   bb.0:
     liveins: %w0
     %0:_(s32) = COPY %w0
     %1:_(s1) = G_TRUNC %0
 
     ; CHECK-LABEL: name: test_uitofp_s32_s1
-    ; CHECK: %3(s32) = G_ZEXT %1
-    ; CHECK: %2(s32) = G_UITOFP %3
+    ; CHECK: [[C:%.*]](s32) = G_CONSTANT i32 1
+    ; CHECK: [[SRC:%.*]](s32) = COPY %0(s32)
+    ; CHECK: [[ZEXT:%.*]](s32) = G_AND [[SRC]], [[C]]
+    ; CHECK: [[RES:%.*]](s32) = G_UITOFP [[ZEXT]]
     %2:_(s32) = G_UITOFP %1
 ...
 
 ---
 name:            test_sitofp_s64_s8
 body: |
   bb.0:
     liveins: %w0
     %0:_(s32) = COPY %w0
     %1:_(s8) = G_TRUNC %0
 
     ; CHECK-LABEL: name: test_sitofp_s64_s8
-    ; CHECK: %3(s32) = G_SEXT %1
-    ; CHECK: %2(s64) = G_SITOFP %3
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 24
+    ; CHECK: [[SRC:%.*]](s32) = COPY %0(s32)
+    ; CHECK: [[SHL1:%.*]](s32) = G_SHL [[SRC]], [[C1]]
+    ; CHECK: [[SEXT:%.*]](s32) = G_ASHR [[SHL1]], [[C1]]
+    ; CHECK: %2(s64) = G_SITOFP [[SEXT]]
     %2:_(s64) = G_SITOFP %1
 ...
 
 ---
 name:            test_uitofp_s64_s8
 body: |
   bb.0:
     liveins: %w0
     %0:_(s32) = COPY %w0
     %1:_(s8) = G_TRUNC %0
 
     ; CHECK-LABEL: name: test_uitofp_s64_s8
-    ; CHECK: %3(s32) = G_ZEXT %1
-    ; CHECK: %2(s64) = G_UITOFP %3
+    ; CHECK: [[C:%.*]](s32) = G_CONSTANT i32 255
+    ; CHECK: [[SRC:%.*]](s32) = COPY %0(s32)
+    ; CHECK: [[ZEXT:%.*]](s32) = G_AND [[SRC]], [[C]]
+    ; CHECK: %2(s64) = G_UITOFP [[ZEXT]]
     %2:_(s64) = G_UITOFP %1
 ...
 
 ---
 name:            test_sitofp_s32_s16
 body: |
   bb.0:
     liveins: %w0
     %0:_(s32) = COPY %w0
     %1:_(s16) = G_TRUNC %0
 
     ; CHECK-LABEL: name: test_sitofp_s32_s16
-    ; CHECK: %3(s32) = G_SEXT %1
-    ; CHECK: %2(s32) = G_SITOFP %3
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 16
+    ; CHECK: [[SRC:%.*]](s32) = COPY %0(s32)
+    ; CHECK: [[SHL1:%.*]](s32) = G_SHL [[SRC]], [[C1]]
+    ; CHECK: [[SEXT:%.*]](s32) = G_ASHR [[SHL1]], [[C1]]
+    ; CHECK: %2(s32) = G_SITOFP [[SEXT]]
     %2:_(s32) = G_SITOFP %1
 ...
 
 ---
 name:            test_uitofp_s32_s16
 body: |
   bb.0:
     liveins: %w0
     %0:_(s32) = COPY %w0
     %1:_(s16) = G_TRUNC %0
 
     ; CHECK-LABEL: name: test_uitofp_s32_s16
-    ; CHECK: %3(s32) = G_ZEXT %1
-    ; CHECK: %2(s32) = G_UITOFP %3
+    ; CHECK: [[C:%.*]](s32) = G_CONSTANT i32 65535
+    ; CHECK: [[SRC:%.*]](s32) = COPY %0(s32)
+    ; CHECK: [[ZEXT:%.*]](s32) = G_AND [[SRC]], [[C]]
+    ; CHECK: [[RES:%.*]](s32) = G_UITOFP [[ZEXT]]
     %2:_(s32) = G_UITOFP %1
 ...

test/CodeGen/AArch64/GlobalISel/legalize-load-store.mir

 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_store
 
     %0(p0) = COPY %x0
     %1(s32) = COPY %w1
 
-    ; CHECK: [[BIT8:%[0-9]+]](s8) = G_ZEXT %2(s1)
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 1
+    ; CHECK: [[B:%.*]](s32) = COPY %1(s32)
+    ; CHECK: [[COPY_C1:%.*]](s32) = COPY [[C1]]
+    ; CHECK: [[AND:%.*]](s32) = G_AND [[B]], [[COPY_C1]]
+    ; CHECK: [[BIT8:%.*]](s8) = G_TRUNC [[AND]]
+
+
     ; CHECK: G_STORE [[BIT8]](s8), %0(p0) :: (store 1 into %ir.addr)
     %2(s1) = G_TRUNC %1
     G_STORE %2, %0 :: (store 1 into %ir.addr)
 
     ; CHECK: G_STORE %3(s8), %0(p0) :: (store 1 into %ir.addr)
     %3(s8) = G_TRUNC %1
     G_STORE %3, %0 :: (store 1 into %ir.addr)
 

test/CodeGen/AArch64/GlobalISel/legalize-mul.mir

registers:
   - { id: 2, class: _ }
   - { id: 3, class: _ }
   - { id: 4, class: _ }
   - { id: 5, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_scalar_mul_small
-    ; CHECK: [[OP0:%.*]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[OP1:%.*]](s32) = G_ANYEXT %3(s8)
+    ; CHECK: [[OP0:%.*]](s32) = G_TRUNC %0
+    ; CHECK: [[OP1:%.*]](s32) = G_TRUNC %1
     ; CHECK: [[RES32:%.*]](s32) = G_MUL [[OP0]], [[OP1]]
     ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[RES32]](s32)
 
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
     %4(s8) = G_MUL %2, %3

test/CodeGen/AArch64/GlobalISel/legalize-or.mir

registers:
   - { id: 2, class: _ }
   - { id: 3, class: _ }
   - { id: 4, class: _ }
   - { id: 5, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_scalar_or_small
-    ; CHECK: [[OP0:%.*]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[OP1:%.*]](s32) = G_ANYEXT %3(s8)
+    ; CHECK: [[OP0:%.*]](s32) = G_TRUNC %0
+    ; CHECK: [[OP1:%.*]](s32) = G_TRUNC %1
     ; CHECK: [[RES32:%.*]](s32) = G_OR [[OP0]], [[OP1]]
     ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[RES32]](s32)
 
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
     %4(s8) = G_OR %2, %3
     %5(s64) = G_ANYEXT %2
     %x0 = COPY %5
 ...

test/CodeGen/AArch64/GlobalISel/legalize-rem.mir

 # RUN: llc -O0 -run-pass=legalizer -global-isel %s -o - | FileCheck %s
 
 --- |
   target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
   target triple = "aarch64--"
-  define void @test_rem() {
+  define void @test_urem_64() {
+  entry:
+    ret void
+  }
+  define void @test_srem_32() {
+  entry:
+    ret void
+  }
+  define void @test_srem_8() {
+  entry:
+    ret void
+  }
+  define void @test_frem() {
   entry:
     ret void
   }
 ...
 
 ---
-name:            test_rem
+name:            test_urem_64
 registers:
   - { id: 0, class: _ }
   - { id: 1, class: _ }
   - { id: 2, class: _ }
-  - { id: 3, class: _ }
-  - { id: 4, class: _ }
-  - { id: 5, class: _ }
-  - { id: 6, class: _ }
-  - { id: 7, class: _ }
-  - { id: 8, class: _ }
-  - { id: 9, class: _ }
-  - { id: 10, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
 
+    ; CHECK-LABEL: name: test_urem_64
     ; CHECK: [[QUOT:%[0-9]+]](s64) = G_UDIV %0, %1
     ; CHECK: [[PROD:%[0-9]+]](s64) = G_MUL [[QUOT]], %1
     ; CHECK: [[RES:%[0-9]+]](s64) = G_SUB %0, [[PROD]]
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s64) = G_UREM %0, %1
 
-    ; CHECK: [[QUOT:%[0-9]+]](s32) = G_SDIV %3, %4
-    ; CHECK: [[PROD:%[0-9]+]](s32) = G_MUL [[QUOT]], %4
-    ; CHECK: [[RES:%[0-9]+]](s32) = G_SUB %3, [[PROD]]
+
+...
+---
+name:            test_srem_32
+registers:
+  - { id: 0, class: _ }
+  - { id: 1, class: _ }
+  - { id: 3, class: _ }
+  - { id: 4, class: _ }
+  - { id: 5, class: _ }
+body: |
+  bb.0.entry:
+    liveins: %x0, %x1, %x2, %x3
+    ; CHECK-LABEL: name: test_srem_32
+    ; CHECK: [[T1:%.*]](s32) = G_TRUNC %0(s64)
+    ; CHECK: [[T2:%.*]](s32) = G_TRUNC %1(s64)
+    ; CHECK: [[DIV:%.*]](s32) = G_SDIV [[T1]], [[T2]]
+    ; CHECK: [[MUL:%.*]](s32) = G_MUL [[DIV]], [[T2]]
+    ; CHECK: [[RES:%.*]](s32) = G_SUB [[T1]], [[MUL]]
+
+    %0(s64) = COPY %x0
+    %1(s64) = COPY %x1
     %3(s32) = G_TRUNC %0
     %4(s32) = G_TRUNC %1
     %5(s32) = G_SREM %3, %4
 
-    ; CHECK: [[LHS32:%[0-9]+]](s32) = G_SEXT %6
-    ; CHECK: [[RHS32:%[0-9]+]](s32) = G_SEXT %7
-    ; CHECK: [[QUOT32:%[0-9]+]](s32) = G_SDIV [[LHS32]], [[RHS32]]
-    ; CHECK: [[QUOT:%[0-9]+]](s8) = G_TRUNC [[QUOT32]]
-
-    ; CHECK: [[QUOT32_2:%.*]](s32) = G_ANYEXT [[QUOT]](s8)
-    ; CHECK: [[RHS32_2:%.*]](s32) = G_ANYEXT %7(s8)
-    ; CHECK: [[PROD32:%.*]](s32) = G_MUL [[QUOT32_2]], [[RHS32_2]]
-    ; CHECK: [[PROD:%.*]](s8) = G_TRUNC [[PROD32]](s32)
-
-    ; CHECK: [[LHS32_2:%.*]](s32) = G_ANYEXT %6(s8)
-    ; CHECK: [[PROD32_2:%.*]](s32) = G_ANYEXT [[PROD]](s8)
-    ; CHECK: [[RES:%[0-9]+]](s32) = G_SUB [[LHS32_2]], [[PROD32_2]]
+...
+---
+name:            test_srem_8
+registers:
+  - { id: 0, class: _ }
+  - { id: 1, class: _ }
+  - { id: 6, class: _ }
+  - { id: 7, class: _ }
+  - { id: 8, class: _ }
+body: |
+  bb.0.entry:
+    liveins: %x0, %x1, %x2, %x3
+    ; CHECK-LABEL: name: test_srem_8
+
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 24
+    ; CHECK: [[SRC1:%.*]](s32) = G_TRUNC %0(s64)
+    ; CHECK: [[SHL1:%.*]](s32) = G_SHL [[SRC1]], [[C1]]
+    ; CHECK: [[LHS_SEXT:%.*]](s32) = G_ASHR [[SHL1]], [[C1]]
+    ; CHECK: [[C2:%.*]](s32) = G_CONSTANT i32 24
+    ; CHECK: [[SRC2:%.*]](s32) = G_TRUNC %1(s64)
+    ; CHECK: [[SHL2:%.*]](s32) = G_SHL [[SRC2]], [[C2]]
+    ; CHECK: [[RHS_SEXT:%.*]](s32) = G_ASHR [[SHL2]], [[C2]]
+    ; CHECK: [[SDIV:%.*]](s32) = G_SDIV [[LHS_SEXT]], [[RHS_SEXT]]
+    ; CHECK: [[A:%.*]](s32) = COPY [[SDIV]]
+    ; CHECK: [[SRC3:%.*]](s32) = G_TRUNC %1(s64)
+    ; CHECK: [[MUL:%.*]](s32) = G_MUL [[A]], [[SRC3]]
+    ; CHECK: [[SRC4:%.*]](s32) = G_TRUNC %0(s64)
+    ; CHECK: [[SRC5:%.*]](s32) = COPY [[MUL]]
+    ; CHECK: [[SUB:%.*]](s32) = G_SUB [[SRC4]], [[SRC5]]
+    ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[SUB]]
+
+    %0(s64) = COPY %x0
+    %1(s64) = COPY %x1
     %6(s8) = G_TRUNC %0
     %7(s8) = G_TRUNC %1
     %8(s8) = G_SREM %6, %7
+...
+---
+name:            test_frem
+registers:
+  - { id: 0, class: _ }
+  - { id: 1, class: _ }
+  - { id: 2, class: _ }
+  - { id: 3, class: _ }
+  - { id: 4, class: _ }
+  - { id: 5, class: _ }
+body: |
+  bb.0.entry:
+    liveins: %x0, %x1, %x2, %x3
+    ; CHECK-LABEL: name: test_frem
 
     ; CHECK: %d0 = COPY %0
     ; CHECK: %d1 = COPY %1
     ; CHECK: BL $fmod, csr_aarch64_aapcs, implicit-def %lr, implicit %sp, implicit %d0, implicit %d1, implicit-def %d0
-    ; CHECK: %9(s64) = COPY %d0
-    %9(s64) = G_FREM %0, %1
+    ; CHECK: [[RES:%.*]](s64) = COPY %d0
+    %0(s64) = COPY %x0
+    %1(s64) = COPY %x1
+    %2(s64) = G_FREM %0, %1
 
     ; CHECK: %s0 = COPY %3
     ; CHECK: %s1 = COPY %4
     ; CHECK: BL $fmodf, csr_aarch64_aapcs, implicit-def %lr, implicit %sp, implicit %s0, implicit %s1, implicit-def %s0
-    ; CHECK: %10(s32) = COPY %s0
-    %10(s32) = G_FREM %3, %4
-
-...
+    ; CHECK: [[RES:%.*]](s32) = COPY %s0
+    %3(s32) = G_TRUNC %0
+    %4(s32) = G_TRUNC %1
+    %5(s32) = G_FREM %3, %4

test/CodeGen/AArch64/GlobalISel/legalize-shift.mir

 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
 
-    ; CHECK: [[LHS32:%[0-9]+]](s32) = G_SEXT %2
-    ; CHECK: [[RHS32:%[0-9]+]](s32) = G_SEXT %3
-    ; CHECK: [[RES32:%[0-9]+]](s32) = G_ASHR [[LHS32]], [[RHS32]]
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 24
+    ; CHECK: [[SRC:%.*]](s32) = G_TRUNC %0(s64)
+    ; CHECK: [[SHL1:%.*]](s32) = G_SHL [[SRC]], [[C1]]
+    ; CHECK: [[SEXT1:%.*]](s32) = G_ASHR [[SHL1]], [[C1]]
+    ; CHECK: [[C2:%.*]](s32) = G_CONSTANT i32 24
+    ; CHECK: [[SRC2:%.*]](s32) = G_TRUNC %1(s64)
+    ; CHECK: [[SHL2:%.*]](s32) = G_SHL [[SRC2]], [[C2]]
+    ; CHECK: [[SEXT2:%.*]](s32) = G_ASHR [[SHL2]], [[C2]]
+    ; CHECK: [[RES32:%[0-9]+]](s32) = G_ASHR [[SEXT1]], [[SEXT2]]
     ; CHECK: %4(s8) = G_TRUNC [[RES32]]
     %4(s8) = G_ASHR %2, %3
 
-    ; CHECK: [[LHS32:%[0-9]+]](s32) = G_ZEXT %2
-    ; CHECK: [[RHS32:%[0-9]+]](s32) = G_ZEXT %3
-    ; CHECK: [[RES32:%[0-9]+]](s32) = G_LSHR [[LHS32]], [[RHS32]]
+
+    ; CHECK: [[C1:%.*]](s32) = G_CONSTANT i32 255
+    ; CHECK: [[SRC:%.*]](s32) = G_TRUNC %0(s64)
+    ; CHECK: [[ZEXT:%.*]](s32) = G_AND [[SRC]], [[C1]]
+    ; CHECK: [[C2:%.*]](s32) = G_CONSTANT i32 255
+    ; CHECK: [[SRC2:%.*]](s32) = G_TRUNC %1(s64)
+    ; CHECK: [[ZEXT2:%.*]](s32) = G_AND [[SRC2]], [[C2]]
+    ; CHECK: [[RES32:%[0-9]+]](s32) = G_LSHR [[ZEXT]], [[ZEXT2]]
     ; CHECK: %5(s8) = G_TRUNC [[RES32]]
     %5(s8) = G_LSHR %2, %3
 
-    ; CHECK: [[OP0:%.*]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[OP1:%.*]](s32) = G_ANYEXT %3(s8)
+    ; CHECK: [[OP0:%.*]](s32) = G_TRUNC %0
+    ; CHECK: [[OP1:%.*]](s32) = G_TRUNC %1
     ; CHECK: [[RES32:%.*]](s32) = G_SHL [[OP0]], [[OP1]]
     ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[RES32]](s32)
     %6(s8) = G_SHL %2, %3
 ...

test/CodeGen/AArch64/GlobalISel/legalize-simple.mir

bb.0.entry:
     %5(p0) = G_INTTOPTR %0
     %6(s64) = G_PTRTOINT %5
 
     ; CHECK: G_BRCOND %1(s1), %bb.1.next
     G_BRCOND %1, %bb.1.next
 
   bb.1.next:
 
-    ; CHECK: [[LHS:%[0-9]+]](s32) = G_ANYEXT %1(s1)
-    ; CHECK: [[RHS:%[0-9]+]](s32) = G_ANYEXT %1(s1)
+    ; CHECK: [[LHS:%[0-9]+]](s32) = G_TRUNC %0
+    ; CHECK: [[RHS:%[0-9]+]](s32) = G_TRUNC %0
     ; CHECK: [[RES:%[0-9]+]](s32) = G_SELECT %1(s1), [[LHS]], [[RHS]]
     ; CHECK: %7(s1) = G_TRUNC [[RES]](s32)
     %7(s1) = G_SELECT %1, %1, %1
 
-    ; CHECK: [[LHS:%[0-9]+]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[RHS:%[0-9]+]](s32) = G_ANYEXT %2(s8)
+    ; CHECK: [[LHS:%[0-9]+]](s32) = G_TRUNC %0
+    ; CHECK: [[RHS:%[0-9]+]](s32) = G_TRUNC %0
     ; CHECK: [[RES:%[0-9]+]](s32) = G_SELECT %1(s1), [[LHS]], [[RHS]]
     ; CHECK: %8(s8) = G_TRUNC [[RES]](s32)
     %8(s8) = G_SELECT %1, %2, %2
 
-    ; CHECK: [[LHS:%[0-9]+]](s32) = G_ANYEXT %3(s16)
-    ; CHECK: [[RHS:%[0-9]+]](s32) = G_ANYEXT %3(s16)
+    ; CHECK: [[LHS:%[0-9]+]](s32) = G_TRUNC %0
+    ; CHECK: [[RHS:%[0-9]+]](s32) = G_TRUNC %0
     ; CHECK: [[RES:%[0-9]+]](s32) = G_SELECT %1(s1), [[LHS]], [[RHS]]
     ; CHECK: %9(s16) = G_TRUNC [[RES]](s32)
     %9(s16) = G_SELECT %1, %3, %3
 
     %10(s32) = G_SELECT %1, %4, %4
     %11(s64) = G_SELECT %1, %0, %0
 
     ; CHECK: %12(<2 x s32>) = G_BITCAST %0

test/CodeGen/AArch64/GlobalISel/legalize-sub.mir

registers:
   - { id: 2, class: _ }
   - { id: 3, class: _ }
   - { id: 4, class: _ }
   - { id: 5, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_scalar_sub_small
-    ; CHECK: [[OP0:%.*]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[OP1:%.*]](s32) = G_ANYEXT %3(s8)
+    ; CHECK: [[OP0:%.*]](s32) = G_TRUNC %0
+    ; CHECK: [[OP1:%.*]](s32) = G_TRUNC %1
     ; CHECK: [[RES32:%.*]](s32) = G_SUB [[OP0]], [[OP1]]
     ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[RES32]](s32)
 
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
     %4(s8) = G_SUB %2, %3
     %5(s64) = G_ANYEXT %2
     %x0 = COPY %5
 ...

test/CodeGen/AArch64/GlobalISel/legalize-xor.mir

registers:
   - { id: 2, class: _ }
   - { id: 3, class: _ }
   - { id: 4, class: _ }
   - { id: 5, class: _ }
 body: |
   bb.0.entry:
     liveins: %x0, %x1, %x2, %x3
     ; CHECK-LABEL: name: test_scalar_xor_small
-    ; CHECK: [[OP0:%.*]](s32) = G_ANYEXT %2(s8)
-    ; CHECK: [[OP1:%.*]](s32) = G_ANYEXT %3(s8)
+    ; CHECK: [[OP0:%.*]](s32) = G_TRUNC %0
+    ; CHECK: [[OP1:%.*]](s32) = G_TRUNC %1
     ; CHECK: [[RES32:%.*]](s32) = G_XOR [[OP0]], [[OP1]]
     ; CHECK: [[RES:%.*]](s8) = G_TRUNC [[RES32]](s32)
 
     %0(s64) = COPY %x0
     %1(s64) = COPY %x1
     %2(s8) = G_TRUNC %0
     %3(s8) = G_TRUNC %1
     %4(s8) = G_XOR %2, %3
     %5(s64) = G_ANYEXT %2
     %x0 = COPY %5
 ...

test/CodeGen/ARM/GlobalISel/arm-legalize-fp.mir

bb.0:
     ; SOFT-NOT: G_FCMP
     ; SOFT: ADJCALLSTACKDOWN
     ; SOFT-DAG: %r0 = COPY [[X]]
     ; SOFT-DAG: %r1 = COPY [[Y]]
     ; SOFT-AEABI: BLX $__aeabi_fcmpgt, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__gtsf2, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT: [[RET1:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R1:%[0-9]+]](s1) = G_TRUNC [[RET1]]
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R1:%[0-9]+]](s1) = G_ICMP intpred(sgt), [[RET1]](s32), [[ZERO]]
     ; SOFT-NOT: G_FCMP
     ; SOFT: ADJCALLSTACKDOWN
     ; SOFT-DAG: %r0 = COPY [[X]]
     ; SOFT-DAG: %r1 = COPY [[Y]]
     ; SOFT-AEABI: BLX $__aeabi_fcmplt, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__ltsf2, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT: [[RET2:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R2:%[0-9]+]](s1) = G_TRUNC [[RET2]](s32)
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R2:%[0-9]+]](s1) = G_ICMP intpred(slt), [[RET2]](s32), [[ZERO]]
-    ; SOFT-DAG: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
-    ; SOFT-DAG: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
+    ; SOFT-AEABI: [[R1EXT:%[0-9]+]](s32) = COPY [[RET1]]
+    ; SOFT-AEABI: [[R2EXT:%[0-9]+]](s32) = COPY [[RET2]]
+    ; SOFT-DEFAULT: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
+    ; SOFT-DEFAULT: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
     ; SOFT: [[REXT:%[0-9]+]](s32) = G_OR [[R1EXT]], [[R2EXT]]
     ; SOFT: [[R:%[0-9]+]](s1) = G_TRUNC [[REXT]]
     ; SOFT-NOT: G_FCMP
     %3(s32) = G_ZEXT %2(s1)
     ; CHECK: [[REXT:%[0-9]+]](s32) = G_ZEXT [[R]](s1)
     %r0 = COPY %3(s32)
     ; CHECK: %r0 = COPY [[REXT]]
     BX_RET 14, _, implicit %r0
bb.0:
     ; SOFT-NOT: G_FCMP
     ; SOFT: ADJCALLSTACKDOWN
     ; SOFT-DAG: %r0 = COPY [[X]]
     ; SOFT-DAG: %r1 = COPY [[Y]]
     ; SOFT-AEABI: BLX $__aeabi_fcmpeq, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__eqsf2, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT: [[RET1:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R1:%[0-9]+]](s1) = G_TRUNC [[RET1]]
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R1:%[0-9]+]](s1) = G_ICMP intpred(eq), [[RET1]](s32), [[ZERO]]
     ; SOFT-NOT: G_FCMP
     ; SOFT: ADJCALLSTACKDOWN
     ; SOFT-DAG: %r0 = COPY [[X]]
     ; SOFT-DAG: %r1 = COPY [[Y]]
     ; SOFT-AEABI: BLX $__aeabi_fcmpun, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__unordsf2, {{.*}}, implicit %r0, implicit %r1, implicit-def %r0
     ; SOFT: [[RET2:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R2:%[0-9]+]](s1) = G_TRUNC [[RET2]](s32)
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R2:%[0-9]+]](s1) = G_ICMP intpred(ne), [[RET2]](s32), [[ZERO]]
-    ; SOFT-DAG: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
-    ; SOFT-DAG: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
+    ; SOFT-AEABI: [[R1EXT:%[0-9]+]](s32) = COPY [[RET1]]
+    ; SOFT-AEABI: [[R2EXT:%[0-9]+]](s32) = COPY [[RET2]]
+    ; SOFT-DEFAULT: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
+    ; SOFT-DEFAULT: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
     ; SOFT: [[REXT:%[0-9]+]](s32) = G_OR [[R1EXT]], [[R2EXT]]
     ; SOFT: [[R:%[0-9]+]](s1) = G_TRUNC [[REXT]]
     ; SOFT-NOT: G_FCMP
     %3(s32) = G_ZEXT %2(s1)
     ; CHECK: [[REXT:%[0-9]+]](s32) = G_ZEXT [[R]](s1)
     %r0 = COPY %3(s32)
     ; CHECK: %r0 = COPY [[REXT]]
     BX_RET 14, _, implicit %r0
bb.0:
     ; SOFT-DAG: %r0 = COPY [[X0]]
     ; SOFT-DAG: %r1 = COPY [[X1]]
     ; SOFT-DAG: %r2 = COPY [[Y0]]
     ; SOFT-DAG: %r3 = COPY [[Y1]]
     ; SOFT-AEABI: BLX $__aeabi_dcmpgt, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__gtdf2, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT: [[RET1:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R1:%[0-9]+]](s1) = G_TRUNC [[RET1]](s32)
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R1:%[0-9]+]](s1) = G_ICMP intpred(sgt), [[RET1]](s32), [[ZERO]]
     ; SOFT-NOT: G_FCMP
     ; SOFT: ADJCALLSTACKDOWN
     ; SOFT-DAG: %r0 = COPY [[X0]]
     ; SOFT-DAG: %r1 = COPY [[X1]]
     ; SOFT-DAG: %r2 = COPY [[Y0]]
     ; SOFT-DAG: %r3 = COPY [[Y1]]
     ; SOFT-AEABI: BLX $__aeabi_dcmplt, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__ltdf2, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT: [[RET2:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R2:%[0-9]+]](s1) = G_TRUNC [[RET2]](s32)
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R2:%[0-9]+]](s1) = G_ICMP intpred(slt), [[RET2]](s32), [[ZERO]]
-    ; SOFT-DAG: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
-    ; SOFT-DAG: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
+    ; SOFT-AEABI: [[R1EXT:%[0-9]+]](s32) = COPY [[RET1]]
+    ; SOFT-AEABI: [[R2EXT:%[0-9]+]](s32) = COPY [[RET2]]
+    ; SOFT-DEFAULT: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
+    ; SOFT-DEFAULT: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
     ; SOFT: [[REXT:%[0-9]+]](s32) = G_OR [[R1EXT]], [[R2EXT]]
     ; SOFT: [[R:%[0-9]+]](s1) = G_TRUNC [[REXT]]
     ; SOFT-NOT: G_FCMP
     %7(s32) = G_ZEXT %6(s1)
     ; CHECK: [[REXT:%[0-9]+]](s32) = G_ZEXT [[R]](s1)
     %r0 = COPY %7(s32)
     ; CHECK: %r0 = COPY [[REXT]]
     BX_RET 14, _, implicit %r0
bb.0:
     ; SOFT-DAG: %r0 = COPY [[X0]]
     ; SOFT-DAG: %r1 = COPY [[X1]]
     ; SOFT-DAG: %r2 = COPY [[Y0]]
     ; SOFT-DAG: %r3 = COPY [[Y1]]
     ; SOFT-AEABI: BLX $__aeabi_dcmpeq, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__eqdf2, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT: [[RET1:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R1:%[0-9]+]](s1) = G_TRUNC [[RET1]](s32)
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R1:%[0-9]+]](s1) = G_ICMP intpred(eq), [[RET1]](s32), [[ZERO]]
     ; SOFT-NOT: G_FCMP
     ; SOFT: ADJCALLSTACKDOWN
     ; SOFT-DAG: %r0 = COPY [[X0]]
     ; SOFT-DAG: %r1 = COPY [[X1]]
     ; SOFT-DAG: %r2 = COPY [[Y0]]
     ; SOFT-DAG: %r3 = COPY [[Y1]]
     ; SOFT-AEABI: BLX $__aeabi_dcmpun, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT-DEFAULT: BLX $__unorddf2, {{.*}}, implicit %r0, implicit %r1, implicit %r2, implicit %r3, implicit-def %r0
     ; SOFT: [[RET2:%[0-9]+]](s32) = COPY %r0
     ; SOFT: ADJCALLSTACKUP
-    ; SOFT-AEABI: [[R2:%[0-9]+]](s1) = G_TRUNC [[RET2]](s32)
     ; SOFT-DEFAULT: [[ZERO:%[0-9]+]](s32) = G_CONSTANT i32 0
     ; SOFT-DEFAULT: [[R2:%[0-9]+]](s1) = G_ICMP intpred(ne), [[RET2]](s32), [[ZERO]]
-    ; SOFT-DAG: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
-    ; SOFT-DAG: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
+    ; SOFT-AEABI: [[R1EXT:%[0-9]+]](s32) = COPY [[RET1]]
+    ; SOFT-AEABI: [[R2EXT:%[0-9]+]](s32) = COPY [[RET2]]
+    ; SOFT-DEFAULT: [[R1EXT:%[0-9]+]](s32) = G_ANYEXT [[R1]]
+    ; SOFT-DEFAULT: [[R2EXT:%[0-9]+]](s32) = G_ANYEXT [[R2]]
     ; SOFT: [[REXT:%[0-9]+]](s32) = G_OR [[R1EXT]], [[R2EXT]]
     ; SOFT: [[R:%[0-9]+]](s1) = G_TRUNC [[REXT]]
     ; SOFT-NOT: G_FCMP
     %7(s32) = G_ZEXT %6(s1)
     ; CHECK: [[REXT:%[0-9]+]](s32) = G_ZEXT [[R]](s1)
     %r0 = COPY %7(s32)
     ; CHECK: %r0 = COPY [[REXT]]
     BX_RET 14, _, implicit %r0
 ...