[AArch64] Adds SUBS and ADDS instructions to the MIPeepholeOpt.

[red1bluelost]

Summary

Implements ADDS/SUBS 24-bit immediate optimization using the
MIPeepholeOpt pass. This follows the pattern:

Optimize ([adds|subs] r, imm) -> ([ADDS|SUBS] ([ADD|SUB] r, #imm0, lsl #12), #imm1),
if imm == (imm0<<12)+imm1. and both imm0 and imm1 are non-zero 12-bit unsigned
integers.

Optimize ([adds|subs] r, imm) -> ([SUBS|ADDS] ([SUB|ADD] r, #imm0, lsl #12), #imm1),
if imm == -(imm0<<12)-imm1, and both imm0 and imm1 are non-zero 12-bit unsigned
integers.

The SplitAndOpcFunc type had to change the return type to an Opcode pair so that
the first add/sub is the regular instruction and the second is the flag setting
instruction. This required updating the code in the AND case.

Testing:

I ran a two stage bootstrap with this code.
Using the second stage compiler, I verified that the negation of an ADDS to SUBS
or vice versa is a valid optimization. Example V == -0x111111.

[dmgreen]

Nice patch, but I think we might need to be checking the condition that is used too. If we split a single SUBS into two, the second of which is setting the flags, then out of NZCV - N (negative) and Z (zero) will be valid as they produce the same results as before, but C (carry) and V (overflow) might be different than the original, given just the wrong input that does overflow/carry on the first SUB but doesn't on the second SUBS.

So I think all the tests here are valid, because they all use eq or ne conditions, but other conditions might not as be.

llvm/lib/Target/AArch64/AArch64MIPeepholeOpt.cpp

381

Can we give these better register class names? A quick comment looks like it would be useful too, explaining the instruction we are adding and which registers they will use.

// NewTmpReg = Opcode.first SrcReg
// NewDstReg = Opcode.second NewTmpReg

393

This is technically assuming the two instructions have the same RegClass for Operand 1? I think that will always be the case, so maybe that's OK to keep as-is.

llvm/test/CodeGen/AArch64/arm64-instruction-mix-remarks.ll

14

Should the ldr still be present? And the orr was already present before? It looks like the test was already missing some input, but we might as well keep the ldr around.

[benshi001]

llvm/test/CodeGen/AArch64/addsub.ll

410

Do we need to pre-commit these tests, and check the difference between before/after ?

[red1bluelost]

Ensures that optimizations only occur on EQ and NE.

Updates the peephole optimization to check subsequent instructions to ensure that
the condition is only used for EQ and NE cases. This is done after checking the
immediate value to avoid running unnecessarily.

Updates comments and naming in the code.

Adds tests that verify various instructions that would use the condition code.
Also a test for when the condition code is used in multiple instructions.

[red1bluelost]

llvm/lib/Target/AArch64/AArch64MIPeepholeOpt.cpp

338

I scan the instructions which read NZCV after the Adds/Subs and check that the condition code is just EQ or NE.

These are the instructions that I that I was able to generate test cases for. Let me know if you think that there other instructions that I should add.

393

I added it just to be safe.

[dmgreen]

Someone once wrote examineCFlagsUse for doing something very similar in the optimizeCompareInstr (because these class as compare instructions). Can some of that be re-used here?

The alternative might be to scan backward through instructions in the main loop, keeping a track of which parts of NZCV are "alive" as we go, so we always have that info available.

llvm/lib/Target/AArch64/AArch64MIPeepholeOpt.cpp

308

Will this always be AArch64::NZCV?

343

We can stop scanning as soon as the NZCV operand is marked as Dead too, I think.

346

On rare occasion the NZCV might be live-out of the block (which is probably enough reason to just return false, as opposed to trying to scan further blocks).

[red1bluelost]

Uses examineCFlagsUse to check NZCV usages.

I was able to use the helper function but I had to slightly modify it to make
it general to any use of NZCV.

[red1bluelost]

In D118663#3300642, @dmgreen wrote:

Someone once wrote examineCFlagsUse for doing something very similar in the optimizeCompareInstr (because these class as compare instructions). Can some of that be re-used here?

The alternative might be to scan backward through instructions in the main loop, keeping a track of which parts of NZCV are "alive" as we go, so we always have that info available.

Thanks for the suggestion! I was able to use examineCFlagsUse. I had to give it external linkage so that I could access it in MIPeepholeOpt. I also made it general to any NZCV case which required slight changes to the preexisting call sites.

llvm/lib/Target/AArch64/AArch64MIPeepholeOpt.cpp

308

examineCFlagsUse makes this no longer relevant.

343

examineCFlagsUse makes this no longer relevant.

346

examineCFlagsUse takes care of this.

[dmgreen]

Thanks for the updates. Nice work.

This LGTM as far as I can see.

[red1bluelost]

In D118663#3315078, @dmgreen wrote:

Thanks for the updates. Nice work.

This LGTM as far as I can see.

Thanks! Could you commit this on my behalf when you have a chance. (Micah Weston - [e-mail])

[nathanchance]

I have reverted this in 22eb1dae3fb20ca8ada865de1d95baab0e08a060, as it causes assertion failures when building the Linux kernel, which has caused our CI to go red:

https://github.com/ClangBuiltLinux/continuous-integration2/actions/runs/1836780929
https://builds.tuxbuild.com/253UOEmSLgpZWbJd3IJsK96YDq5/build.log

A reduced reproducer from cvise:

$ cat neighbour.i
neigh_periodic_work_tbl_1() {
  if ((long)neigh_periodic_work_tbl_1 + 300 * 250 < 0)
    for (;;)
      ;
}

$ clang -O2 --target=aarch64-linux-gnu -c -o /dev/null neighbour.i
...
clang: /home/nathan/cbl/src/llvm-project/llvm/include/llvm/CodeGen/Register.h:78: static unsigned int llvm::Register::virtReg2Index(llvm::Register): Assertion `isVirtualRegister(Reg) && "Not a virtual register"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: clang -O2 --target=aarch64-linux-gnu -c -o /dev/null neighbour.i
1.      <eof> parser at end of file
2.      Code generation
3.      Running pass 'Function Pass Manager' on module 'neighbour.i'.
4.      Running pass 'AArch64 MI Peephole Optimization pass' on function '@neigh_periodic_work_tbl_1'
...

If there is any other information that I can provide or patches I can test, please let me know.

[red1bluelost]

In D118663#3317794, @nathanchance wrote:

I have reverted this in 22eb1dae3fb20ca8ada865de1d95baab0e08a060, as it causes assertion failures when building the Linux kernel, which has caused our CI to go red:

...

If there is any other information that I can provide or patches I can test, please let me know.

Thank you for reverting! I will look into this.

[red1bluelost]

Fixes issue found in ClangBuiltLinux.

Turns out a scenario was encountered where DstReg is physical which causes
MRI->getRegClass(DstReg) to have an assertion failure. This was verified with
a new test case based on the ClangBuiltLinux IR output.

As a solution, DstReg only constrains the new NewDstReg if DstReg is virtual.
The times where it isn't virtual is when the output is going to XZR or WZR.

[nathanchance]

Thank you for the quick fix! Unfortunately, I still an assertion failure with this patch while building at least allnoconfig and allmodconfig + ThinLTO kernels, albeit a different one. See below and let me know if there are any problems with reproducing:

$ cat random.i
jiffies, primary_crng, input_pool;
_extract_crng_crng() {
  if ((long)_extract_crng_crng < 0 ||
      (long)(_extract_crng_crng + 300 * 250 - jiffies) < 0)
    crng_reseed(primary_crng ? &input_pool : 0);
}

$ clang -O2 --target=aarch64-linux-gnu -c -o /dev/null random.i
...
clang: /home/nathan/cbl/src/llvm-project/llvm/lib/CodeGen/LiveVariables.cpp:111: void llvm::LiveVariables::MarkVirtRegAliveInBlock(llvm::LiveVariables::VarInfo &, llvm::MachineBasicBlock *, llvm::MachineBasicBlock *, SmallVectorImpl<llvm:
:MachineBasicBlock *> &): Assertion `MBB != &MF->front() && "Can't find reaching def for virtreg"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: clang -O2 --target=aarch64-linux-gnu -c -o /dev/null random.i
1.      <eof> parser at end of file
2.      Code generation
3.      Running pass 'Function Pass Manager' on module 'random.i'.
4.      Running pass 'Live Variable Analysis' on function '@_extract_crng_crng'
...

[red1bluelost]

Fixes cross basic block found in ClangBuiltLinux.

The issue was the physical register was being replaced which screws up coherency
across basic blocks. The scenario was added as a test case. Before, the XZR register
was replaced every where with a new virtual register screwing up the MIR.

In the case where DstReg is physical, we should reuse it rather than making a new
virtual register. This fix ensures that a physical DstReg is preversed while
a virtual DstReg is replaced with a new virtual register.

Thank you for testing and finding the errors!

[nathanchance]

The latest revision passes all of my AArch64 Linux kernel builds with assertions enabled and boots on bare metal, thank you for fixing all the issues!

[red1bluelost]

In D118663#3323737, @nathanchance wrote:

The latest revision passes all of my AArch64 Linux kernel builds with assertions enabled and boots on bare metal, thank you for fixing all the issues!

That's great to hear! Thank you for testing it on bare metal!

If the changes looks good to the reviewers, then it just needs to be recommitted on my behalf (Micah Weston - micahsweston@gmail).

[dmgreen]

Sorry - I nearly missed this. Yeah it sounds OK to me. It makes sense we would need to treat physical registers differently. It's a little surprising that didn't come up from the existing tests.

LGTM. I'll commit this now.

llvm/lib/Target/AArch64/AArch64MIPeepholeOpt.cpp

AArch64MIPeepholeOpt() : MachineFunctionPass(ID) {
     initializeAArch64MIPeepholeOptPass(*PassRegistry::getPassRegistry());
   }
 
   const AArch64InstrInfo *TII;
   const AArch64RegisterInfo *TRI;
   MachineLoopInfo *MLI;
   MachineRegisterInfo *MRI;
 
+  using OpcodePair = std::pair<unsigned, unsigned>;
   template <typename T>
   using SplitAndOpcFunc =
-      std::function<Optional<unsigned>(T, unsigned, T &, T &)>;
+      std::function<Optional<OpcodePair>(T, unsigned, T &, T &)>;
   using BuildMIFunc =
-      std::function<void(MachineInstr &, unsigned, unsigned, unsigned, Register,
-                         Register, Register)>;
+      std::function<void(MachineInstr &, OpcodePair, unsigned, unsigned,
+                         Register, Register, Register)>;
 
   /// For instructions where an immediate operand could be split into two
   /// separate immediate instructions, use the splitTwoPartImm two handle the
   /// optimization.
   ///
   /// To implement, the following function types must be passed to
   /// splitTwoPartImm. A SplitAndOpcFunc must be implemented that determines if
   /// splitting the immediate is valid and returns the associated new opcode. A
   /// BuildMIFunc must be implemented to build the two immediate instructions.
   ///
   /// Example Pattern (where IMM would require 2+ MOV instructions):
   ///     %dst = <Instr>rr %src IMM [...]
   /// becomes:
   ///     %tmp = <Instr>ri %src (encode half IMM) [...]
   ///     %dst = <Instr>ri %tmp (encode half IMM) [...]
   template <typename T>
   bool splitTwoPartImm(MachineInstr &MI,
                        SmallSetVector<MachineInstr *, 8> &ToBeRemoved,
                        SplitAndOpcFunc<T> SplitAndOpc, BuildMIFunc BuildInstr);
 
   bool checkMovImmInstr(MachineInstr &MI, MachineInstr *&MovMI,
                         MachineInstr *&SubregToRegMI);
 
   template <typename T>
+  bool visitGenericADDSUB(OpcodePair PosOpcs, OpcodePair NegOpcs,
+                          MachineInstr &MI,
+                          SmallSetVector<MachineInstr *, 8> &ToBeRemoved);
+  template <typename T>
   bool visitADDSUB(unsigned PosOpc, unsigned NegOpc, MachineInstr &MI,
                    SmallSetVector<MachineInstr *, 8> &ToBeRemoved);
   template <typename T>
   bool visitAND(unsigned Opc, MachineInstr &MI,
                 SmallSetVector<MachineInstr *, 8> &ToBeRemoved);
   bool visitORR(MachineInstr &MI,
                 SmallSetVector<MachineInstr *, 8> &ToBeRemoved);
   bool runOnMachineFunction(MachineFunction &MF) override;
bool AArch64MIPeepholeOpt::visitAND(
   //
   // The mov pseudo instruction could be expanded to multiple mov instructions
   // later. Let's try to split the constant operand of mov instruction into two
   // bitmask immediates. It makes only two AND instructions intead of multiple
   // mov + and instructions.
 
   return splitTwoPartImm<T>(
       MI, ToBeRemoved,
-      [Opc](T Imm, unsigned RegSize, T &Imm0, T &Imm1) -> Optional<unsigned> {
+      [Opc](T Imm, unsigned RegSize, T &Imm0, T &Imm1) -> Optional<OpcodePair> {
         if (splitBitmaskImm(Imm, RegSize, Imm0, Imm1))
-          return Opc;
+          return std::make_pair(Opc, Opc);
         return None;
       },
-      [&TII = TII](MachineInstr &MI, unsigned Opcode, unsigned Imm0,
+      [&TII = TII](MachineInstr &MI, OpcodePair Opcode, unsigned Imm0,
                    unsigned Imm1, Register SrcReg, Register NewTmpReg,
                    Register NewDstReg) {
         DebugLoc DL = MI.getDebugLoc();
         MachineBasicBlock *MBB = MI.getParent();
-        BuildMI(*MBB, MI, DL, TII->get(Opcode), NewTmpReg)
+        BuildMI(*MBB, MI, DL, TII->get(Opcode.first), NewTmpReg)
             .addReg(SrcReg)
             .addImm(Imm0);
-        BuildMI(*MBB, MI, DL, TII->get(Opcode), NewDstReg)
+        BuildMI(*MBB, MI, DL, TII->get(Opcode.second), NewDstReg)
             .addReg(NewTmpReg)
             .addImm(Imm1);
       });
 }
 
 bool AArch64MIPeepholeOpt::visitORR(
     MachineInstr &MI, SmallSetVector<MachineInstr *, 8> &ToBeRemoved) {
   // Check this ORR comes from below zero-extend pattern.
static bool splitAddSubImm(T Imm, unsigned RegSize, T &Imm0, T &Imm1) {
 
   // Split Imm into (Imm0 << 12) + Imm1;
   Imm0 = (Imm >> 12) & 0xfff;
   Imm1 = Imm & 0xfff;
   return true;
 }
 
 template <typename T>
-bool AArch64MIPeepholeOpt::visitADDSUB(
-    unsigned PosOpc, unsigned NegOpc, MachineInstr &MI,
+bool AArch64MIPeepholeOpt::visitGenericADDSUB(
+    OpcodePair PosOpcs, OpcodePair NegOpcs, MachineInstr &MI,
     SmallSetVector<MachineInstr *, 8> &ToBeRemoved) {
   // Try below transformation.
   //
   // MOVi32imm + ADDWrr ==> ADDWri + ADDWri
   // MOVi64imm + ADDXrr ==> ADDXri + ADDXri
   //
   // MOVi32imm + SUBWrr ==> SUBWri + SUBWri
   // MOVi64imm + SUBXrr ==> SUBXri + SUBXri
   //
   // The mov pseudo instruction could be expanded to multiple mov instructions
   // later. Let's try to split the constant operand of mov instruction into two
   // legal add/sub immediates. It makes only two ADD/SUB instructions intead of
   // multiple `mov` + `and/sub` instructions.
 
   return splitTwoPartImm<T>(
       MI, ToBeRemoved,
-      [PosOpc, NegOpc](T Imm, unsigned RegSize, T &Imm0,
-                       T &Imm1) -> Optional<unsigned> {
+      [PosOpcs, NegOpcs](T Imm, unsigned RegSize, T &Imm0,
+                         T &Imm1) -> Optional<OpcodePair> {
         if (splitAddSubImm(Imm, RegSize, Imm0, Imm1))
-          return PosOpc;
+          return PosOpcs;
         if (splitAddSubImm(-Imm, RegSize, Imm0, Imm1))
-          return NegOpc;
+          return NegOpcs;
         return None;
       },
-      [&TII = TII](MachineInstr &MI, unsigned Opcode, unsigned Imm0,
+      [&TII = TII](MachineInstr &MI, OpcodePair Opcode, unsigned Imm0,
                    unsigned Imm1, Register SrcReg, Register NewTmpReg,
                    Register NewDstReg) {
         DebugLoc DL = MI.getDebugLoc();
         MachineBasicBlock *MBB = MI.getParent();
-        BuildMI(*MBB, MI, DL, TII->get(Opcode), NewTmpReg)
+        BuildMI(*MBB, MI, DL, TII->get(Opcode.first), NewTmpReg)
             .addReg(SrcReg)
             .addImm(Imm0)
             .addImm(12);
-        BuildMI(*MBB, MI, DL, TII->get(Opcode), NewDstReg)
+        BuildMI(*MBB, MI, DL, TII->get(Opcode.second), NewDstReg)
             .addReg(NewTmpReg)
             .addImm(Imm1)
             .addImm(0);
       });
 }
 
+template <typename T>
+bool AArch64MIPeepholeOpt::visitADDSUB(
+    unsigned PosOpc, unsigned NegOpc, MachineInstr &MI,
+    SmallSetVector<MachineInstr *, 8> &ToBeRemoved) {
+  return visitGenericADDSUB<T>({PosOpc, PosOpc}, {NegOpc, NegOpc}, MI,
+                               ToBeRemoved);
+}
+
 // Checks if the corresponding MOV immediate instruction is applicable for
 // this peephole optimization.
 bool AArch64MIPeepholeOpt::checkMovImmInstr(MachineInstr &MI,
                                             MachineInstr *&MovMI,
                                             MachineInstr *&SubregToRegMI) {
   // Check whether current MBB is in loop and the AND is loop invariant.
   MachineBasicBlock *MBB = MI.getParent();
   MachineLoop *L = MLI->getLoopFor(MBB);
bool AArch64MIPeepholeOpt::checkMovImmInstr(MachineInstr &MI,
   if (MovMI->getOpcode() == TargetOpcode::SUBREG_TO_REG) {
     SubregToRegMI = MovMI;
     MovMI = MRI->getUniqueVRegDef(MovMI->getOperand(2).getReg());
     if (!MovMI)
       return false;
   }
 
   if (MovMI->getOpcode() != AArch64::MOVi32imm &&
       MovMI->getOpcode() != AArch64::MOVi64imm)
     return false;
 
   // If the MOV has multiple uses, do not split the immediate because it causes
   // more instructions.
   if (!MRI->hasOneUse(MovMI->getOperand(0).getReg()))
     return false;
   if (SubregToRegMI && !MRI->hasOneUse(SubregToRegMI->getOperand(0).getReg()))
     return false;
 
   // It is OK to perform this peephole optimization.
   return true;
 }
 
 template <typename T>
 bool AArch64MIPeepholeOpt::splitTwoPartImm(
     MachineInstr &MI, SmallSetVector<MachineInstr *, 8> &ToBeRemoved,
bool AArch64MIPeepholeOpt::splitTwoPartImm(
   // Split the immediate to Imm0 and Imm1, and calculate the Opcode.
   T Imm = static_cast<T>(MovMI->getOperand(1).getImm()), Imm0, Imm1;
   // For the 32 bit form of instruction, the upper 32 bits of the destination
   // register are set to zero. If there is SUBREG_TO_REG, set the upper 32 bits
   // of Imm to zero. This is essential if the Immediate value was a negative
   // number since it was sign extended when we assign to the 64-bit Imm.
   if (SubregToRegMI)
     Imm &= 0xFFFFFFFF;
-  unsigned Opcode;
+  OpcodePair Opcode;
   if (auto R = SplitAndOpc(Imm, RegSize, Imm0, Imm1))
     Opcode = R.getValue();
   else
     return false;
 
   // Create new ADD/SUB MIs.
   MachineFunction *MF = MI.getMF();
-  const TargetRegisterClass *RC =
-      TII->getRegClass(TII->get(Opcode), 0, TRI, *MF);
+  const TargetRegisterClass *RC0 =
+      TII->getRegClass(TII->get(Opcode.first), 0, TRI, *MF);
+  const TargetRegisterClass *RC1 =
+      TII->getRegClass(TII->get(Opcode.second), 0, TRI, *MF);
   const TargetRegisterClass *ORC =
-      TII->getRegClass(TII->get(Opcode), 1, TRI, *MF);
+      TII->getRegClass(TII->get(Opcode.first), 1, TRI, *MF);
   Register DstReg = MI.getOperand(0).getReg();
   Register SrcReg = MI.getOperand(1).getReg();
-  Register NewTmpReg = MRI->createVirtualRegister(RC);
-  Register NewDstReg = MRI->createVirtualRegister(RC);
+  Register NewTmpReg = MRI->createVirtualRegister(RC0);
+  Register NewDstReg = MRI->createVirtualRegister(RC1);
 
-  MRI->constrainRegClass(SrcReg, RC);
+  MRI->constrainRegClass(SrcReg, ORC);
   MRI->constrainRegClass(NewTmpReg, ORC);
   MRI->constrainRegClass(NewDstReg, MRI->getRegClass(DstReg));
 
   BuildInstr(MI, Opcode, Imm0, Imm1, SrcReg, NewTmpReg, NewDstReg);
 
   MRI->replaceRegWith(DstReg, NewDstReg);
   // replaceRegWith changes MI's definition register. Keep it for SSA form until
   // deleting MI.
   MI.getOperand(0).setReg(DstReg);
for (MachineInstr &MI : MBB) {
       case AArch64::ADDXrr:
         Changed = visitADDSUB<uint64_t>(AArch64::ADDXri, AArch64::SUBXri, MI,
                                         ToBeRemoved);
         break;
       case AArch64::SUBXrr:
         Changed = visitADDSUB<uint64_t>(AArch64::SUBXri, AArch64::ADDXri, MI,
                                         ToBeRemoved);
         break;
+      case AArch64::ADDSWrr:
+        Changed = visitGenericADDSUB<uint32_t>(
+            {AArch64::ADDWri, AArch64::ADDSWri},
+            {AArch64::SUBWri, AArch64::SUBSWri}, MI, ToBeRemoved);
+        break;
+      case AArch64::SUBSWrr:
+        Changed = visitGenericADDSUB<uint32_t>(
+            {AArch64::SUBWri, AArch64::SUBSWri},
+            {AArch64::ADDWri, AArch64::ADDSWri}, MI, ToBeRemoved);
+        break;
+      case AArch64::ADDSXrr:
+        Changed = visitGenericADDSUB<uint64_t>(
+            {AArch64::ADDXri, AArch64::ADDSXri},
+            {AArch64::SUBXri, AArch64::SUBSXri}, MI, ToBeRemoved);
+        break;
+      case AArch64::SUBSXrr:
+        Changed = visitGenericADDSUB<uint64_t>(
+            {AArch64::SUBXri, AArch64::SUBSXri},
+            {AArch64::ADDXri, AArch64::ADDSXri}, MI, ToBeRemoved);
+        break;
       }
     }
   }
 
   for (MachineInstr *MI : ToBeRemoved)
     MI->eraseFromParent();
 
   return Changed;
 }
 
 FunctionPass *llvm::createAArch64MIPeepholeOptPass() {
   return new AArch64MIPeepholeOpt();
 }

llvm/test/CodeGen/AArch64/addsub.ll

 ; CHECK-NEXT:    mov w8, #48576
 ; CHECK-NEXT:    movk w8, #65520, lsl #16
 ; CHECK-NEXT:    add x0, x0, x8
 ; CHECK-NEXT:    ret
   %b = add i64 %a, 4293967296
   ret i64 %b
 }
 
-; TODO: adds/subs
+; ADDS and SUBS Optimizations
+define i1 @eq_i(i32 %0) {
+; CHECK-LABEL: eq_i:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    sub w8, w0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmp w8, #273
+; CHECK-NEXT:    cset w0, eq
+; CHECK-NEXT:    ret
+  %2 = icmp eq i32 %0, 1118481
+  ret i1 %2
+}
+
+define i1 @eq_l(i64 %0) {
+; CHECK-LABEL: eq_l:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    sub x8, x0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmp x8, #273
+; CHECK-NEXT:    cset w0, eq
+; CHECK-NEXT:    ret
+  %2 = icmp eq i64 %0, 1118481
+  ret i1 %2
+}
+
+define i1 @ne_i(i32 %0) {
+; CHECK-LABEL: ne_i:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    sub w8, w0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmp w8, #273
+; CHECK-NEXT:    cset w0, ne
+; CHECK-NEXT:    ret
+  %2 = icmp ne i32 %0, 1118481
+  ret i1 %2
+}
+
+define i1 @ne_l(i64 %0) {
+; CHECK-LABEL: ne_l:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    sub x8, x0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmp x8, #273
+; CHECK-NEXT:    cset w0, ne
+; CHECK-NEXT:    ret
+  %2 = icmp ne i64 %0, 1118481
+  ret i1 %2
+}
+
+define i1 @eq_in(i32 %0) {
+; CHECK-LABEL: eq_in:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    add w8, w0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmn w8, #273
+; CHECK-NEXT:    cset w0, eq
+; CHECK-NEXT:    ret
+  %2 = icmp eq i32 %0, -1118481
+  ret i1 %2
+}
+
+define i1 @eq_ln(i64 %0) {
+; CHECK-LABEL: eq_ln:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    add x8, x0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmn x8, #273
+; CHECK-NEXT:    cset w0, eq
+; CHECK-NEXT:    ret
+  %2 = icmp eq i64 %0, -1118481
+  ret i1 %2
+}
+
+define i1 @ne_in(i32 %0) {
+; CHECK-LABEL: ne_in:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    add w8, w0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmn w8, #273
+; CHECK-NEXT:    cset w0, ne
+; CHECK-NEXT:    ret
+  %2 = icmp ne i32 %0, -1118481
+  ret i1 %2
+}
+
+define i1 @ne_ln(i64 %0) {
+; CHECK-LABEL: ne_ln:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    add x8, x0, #273, lsl #12 // =1118208
+; CHECK-NEXT:    cmn x8, #273
+; CHECK-NEXT:    cset w0, ne
+; CHECK-NEXT:    ret
+  %2 = icmp ne i64 %0, -1118481
+  ret i1 %2
+}
+
+define i1 @reject_eq(i32 %0) {
+; CHECK-LABEL: reject_eq:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    mov w8, #51712
+; CHECK-NEXT:    movk w8, #15258, lsl #16
+; CHECK-NEXT:    cmp w0, w8
+; CHECK-NEXT:    cset w0, eq
+; CHECK-NEXT:    ret
+  %2 = icmp eq i32 %0, 1000000000
+  ret i1 %2
+}

llvm/test/CodeGen/AArch64/arm64-instruction-mix-remarks.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=arm64-apple-ios7.0 | FileCheck %s
 
 ; RUN: llc -mtriple=arm64-apple-ios7.0 -pass-remarks-output=%t -pass-remarks=asm-printer -o - %s
 ; RUN: FileCheck --input-file=%t --check-prefix=YAML %s
 
 ; YAML:      Name:            InstructionMix
 ; YAML-NEXT: DebugLoc:        { File: arm64-instruction-mix-remarks.ll, Line: 10, Column: 10 }
 ; YAML-NEXT: Function:        foo
 ; YAML-NEXT: Args:
 ; YAML:      - BasicBlock:  entry
 ; YAML:      - INST_add:    '2'
 ; YAML:      - INST_b.:     '1'
-; YAML:      - INST_ldr:    '1'
-; YAML:      - INST_movk:   '1'
-; YAML:      - INST_movz:   '1'
+; YAML:      - INST_orr:    '1'
+; YAML:      - INST_sub:   '1'
 ; YAML:      - INST_subs:   '1'
 
 ; YAML:      Name:            InstructionMix
 ; YAML-NEXT: DebugLoc:        { File: arm64-instruction-mix-remarks.ll, Line: 30, Column: 30 }
 ; YAML-NEXT: Function:        foo
 ; YAML-NEXT: Args:
 ; YAML:       - BasicBlock:  else
 ; YAML:       - INST_madd:   '2'
 ; YAML:       - INST_movz:   '1'
 ; YAML:       - INST_str:    '1'
 define i32 @foo(i32* %ptr, i32 %x, i64 %y) !dbg !3 {
 ; CHECK-LABEL: foo:
 ; CHECK:       ; %bb.0: ; %entry
-; CHECK-NEXT:    ldr w10, [x0]
+; CHECK-NEXT:    ldr w9, [x0]
 ; CHECK-NEXT:    mov x8, x0
-; CHECK-NEXT:    mov w9, #16959
-; CHECK-NEXT:    movk w9, #15, lsl #16
-; CHECK-NEXT:    add w0, w10, w1
-; CHECK-NEXT:    add x10, x0, x2
-; CHECK-NEXT:    cmp x10, x9
+; CHECK-NEXT:    add w0, w9, w1
+; CHECK-NEXT:    add x9, x0, x2
+; CHECK-NEXT:    sub x9, x9, #244, lsl #12 ; =999424
+; CHECK-NEXT:    cmp x9, #575
 ; CHECK-NEXT:    b.eq LBB0_2
 ; CHECK-NEXT:  ; %bb.1: ; %else
 ; CHECK-NEXT:    mul w9, w0, w1
 ; CHECK-NEXT:    mov w10, #10
 ; CHECK-NEXT:    mul w0, w9, w1
 ; CHECK-NEXT:    str w10, [x8]
 ; CHECK-NEXT:  LBB0_2: ; %common.ret
 ; CHECK-NEXT:    ; kill: def $w0 killed $w0 killed $x0

llvm/test/CodeGen/AArch64/srem-seteq-illegal-types.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=aarch64-unknown-linux-gnu < %s | FileCheck %s
 
 define i1 @test_srem_odd(i29 %X) nounwind {
 ; CHECK-LABEL: test_srem_odd:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    mov w8, #33099
 ; CHECK-NEXT:    mov w9, #24493
 ; CHECK-NEXT:    movk w8, #8026, lsl #16
 ; CHECK-NEXT:    movk w9, #41, lsl #16
 ; CHECK-NEXT:    madd w8, w0, w8, w9
-; CHECK-NEXT:    mov w9, #48987
-; CHECK-NEXT:    movk w9, #82, lsl #16
 ; CHECK-NEXT:    and w8, w8, #0x1fffffff
-; CHECK-NEXT:    cmp w8, w9
+; CHECK-NEXT:    sub w8, w8, #1323, lsl #12 // =5419008
+; CHECK-NEXT:    cmp w8, #3931
 ; CHECK-NEXT:    cset w0, lo
 ; CHECK-NEXT:    ret
   %srem = srem i29 %X, 99
   %cmp = icmp eq i29 %srem, 0
   ret i1 %cmp
 }
 
 define i1 @test_srem_even(i4 %X) nounwind {

llvm/test/CodeGen/AArch64/urem-seteq-illegal-types.ll

 ; CHECK-LABEL: test_urem_even:
 ; CHECK:       // %bb.0:
 ; CHECK-NEXT:    mov w8, #28087
 ; CHECK-NEXT:    movk w8, #1755, lsl #16
 ; CHECK-NEXT:    mul w8, w0, w8
 ; CHECK-NEXT:    lsl w9, w8, #26
 ; CHECK-NEXT:    bfxil w9, w8, #1, #26
 ; CHECK-NEXT:    and w8, w9, #0x7ffffff
-; CHECK-NEXT:    mov w9, #18725
-; CHECK-NEXT:    movk w9, #146, lsl #16
-; CHECK-NEXT:    cmp w8, w9
+; CHECK-NEXT:    sub w8, w8, #2340, lsl #12 // =9584640
+; CHECK-NEXT:    cmp w8, #2341
 ; CHECK-NEXT:    cset w0, lo
 ; CHECK-NEXT:    ret
   %urem = urem i27 %X, 14
   %cmp = icmp eq i27 %urem, 0
   ret i1 %cmp
 }
 
 define i1 @test_urem_odd_setne(i4 %X) nounwind {