[AVR] Generate ELPM for loading byte/word from extended program memory

[benshi001]

This comment was removed by benshi001.

[aykevl]

For my context, I believe this is for devices that have a larger flash memory, such as the ATmega2560?

I haven't looked too deeply at the code yet.

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

829

There is actually a much more elegant way to do this: by giving ELPMRdZ (and similar) instructions an extra LD8 input and output register. This makes sure the register allocator allocates a spare register for this instruction. The scavengeGPR8 method is really just a hack to find an unused register and it can fail if no register is available. In addition, by telling the register allocator you need an LD8 register, you don't need the if/else below.

[aykevl]

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

829

I recently implemented this, in a work-in-progress change. It looks like this in my case:

+let Constraints = "@earlyclobber $scratch" in
 class AtomicLoad<PatFrag Op, RegisterClass DRC,
                  RegisterClass PTRRC> :
-  Pseudo<(outs DRC:$rd), (ins PTRRC:$rr), "atomic_op",
-         [(set DRC:$rd, (Op i16:$rr))]>;
+  Pseudo<(outs DRC:$rd, GPR8:$scratch), (ins PTRRC:$rr), "atomic_op",
+         [(set DRC:$rd, (Op i16:$rr)), (set GPR8:$scratch, SREG)]>;

So essentially, this adds a new output with the "earlyclobber" flag set. This scratch register can then be used however you like.

[aykevl]

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

829

Another option is to make the Bank operand a register operand instead of an immediate. This would make the code generator put the value there without an extra ldi instruction, and allows optimizations such as using the same value multiple times for multiple elpm instructions.

[benshi001]

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

829

That is a good idea, Thanks. I will try it later this week.

[benshi001]

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

829

Done. I have created an extra LDI machine node, which I think is the best way to revmove calling of scavengeGPR8 in my previous patch revision.

[aykevl]

This looks pretty good. Thanks for the improvements!

Can you add a test where you read twice from the same extended address space? It should only have one ldi instruction to load the address space number.

We should also save RAMPZ in interrupts on devices that support ELPM. But I guess that can happen in a separate patch.
See for example: https://godbolt.org/z/x9Tx1jzMc

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

833

I don't think this necessarily kills the register. A later instruction might use the same value.

892

Same here.

llvm/lib/Target/AVR/AVRISelDAGToDAG.cpp

366–368

I think it is better to use report_fatal_error here instead of assert. Asserts are removed in release builds, so this check will not be performed in release builds, while I think it should always be checked.

The same is true for the assert above (Subtarget->hasLPM()) but that is outside this patch.

In general, asserts should only be used for things that are always true unless there are bugs in the code. In this case, it is possible to trigger the assert without a bug in the AVR backend.

385–387

I don't really understand this comment?
The instruction selector should always use a register class that is supported by all the instruction operands for the virtual register. In this case, that would be LD8.

[aykevl]

I found out that avr-gcc does not support reusing the same ldi instruction for multiple elpm instructions, so if we can do that, we're better than avr-gcc :)
https://godbolt.org/z/Yzhaj7e54

[benshi001]

llvm/lib/Target/AVR/AVRISelDAGToDAG.cpp

366–368

I also change the above assert to report_fatal_error.

385–387

My previous intention is to say

do not combine LDI into ELPM, since ELPM may be allocated a register in R0-R15, and LDI can not access.

Anyway, I change the comment to a more clear expression.

[benshi001]

I also have added a new test foob4 in the new test elpm.ll, which will show the case you mentioned:

read twice from the same extended address space, and there is only one LDI instruction to load the address space number.

In D116493#3251291, @aykevl wrote:

This looks pretty good. Thanks for the improvements!

Can you add a test where you read twice from the same extended address space? It should only have one ldi instruction to load the address space number.

We should also save RAMPZ in interrupts on devices that support ELPM. But I guess that can happen in a separate patch.
See for example: https://godbolt.org/z/x9Tx1jzMc

[aykevl]

Looks good to me!

llvm/lib/Target/AVR/AVR.h

 
 inline bool isProgramMemoryAccess(MemSDNode const *N) {
   auto *V = N->getMemOperand()->getValue();
   if (V != nullptr && isProgramMemoryAddress(V))
     return true;
   return false;
 }
 
+// Get the index of the program memory bank.
+//  -1: not program memory
+//   0: ordinary program memory
+// 1~5: extended program memory
+inline int getProgramMemoryBank(MemSDNode const *N) {
+  auto *V = N->getMemOperand()->getValue();
+  if (V == nullptr || !isProgramMemoryAddress(V))
+    return -1;
+  AddressSpace AS = getAddressSpace(V);
+  assert(ProgramMemory <= AS && AS <= ProgramMemory5);
+  return static_cast<int>(AS - ProgramMemory);
+}
+
 } // end of namespace AVR
 
 } // end namespace llvm
 
 #endif // LLVM_AVR_H

llvm/lib/Target/AVR/AVRExpandPseudoInsts.cpp

private:
   bool expandLSLW4Rd(Block &MBB, BlockIt MBBI);
   bool expandLSRW4Rd(Block &MBB, BlockIt MBBI);
   bool expandLSLW8Rd(Block &MBB, BlockIt MBBI);
   bool expandLSRW8Rd(Block &MBB, BlockIt MBBI);
   bool expandASRW8Rd(Block &MBB, BlockIt MBBI);
   bool expandLSLW12Rd(Block &MBB, BlockIt MBBI);
   bool expandLSRW12Rd(Block &MBB, BlockIt MBBI);
 
+  // Common implementation of LPMWRdZ and ELPMWRdZ.
+  bool expandLPMWELPMW(Block &MBB, BlockIt MBBI, bool IsExt);
+
   /// Scavenges a free GPR8 register for use.
   Register scavengeGPR8(MachineInstr &MI);
 };
 
 char AVRExpandPseudo::ID = 0;
 
 bool AVRExpandPseudo::expandMBB(MachineBasicBlock &MBB) {
   bool Modified = false;
bool AVRExpandPseudo::expand<AVR::LDDWRdPtrQ>(Block &MBB, BlockIt MBBI) {
 
   MIBLO.setMemRefs(MI.memoperands());
   MIBHI.setMemRefs(MI.memoperands());
 
   MI.eraseFromParent();
   return true;
 }
 
-template <>
-bool AVRExpandPseudo::expand<AVR::LPMWRdZ>(Block &MBB, BlockIt MBBI) {
+bool AVRExpandPseudo::expandLPMWELPMW(Block &MBB, BlockIt MBBI, bool IsExt) {
   MachineInstr &MI = *MBBI;
   Register DstLoReg, DstHiReg;
   Register DstReg = MI.getOperand(0).getReg();
   Register TmpReg = 0; // 0 for no temporary register
   Register SrcReg = MI.getOperand(1).getReg();
   bool SrcIsKill = MI.getOperand(1).isKill();
-  unsigned OpLo = AVR::LPMRdZPi;
-  unsigned OpHi = AVR::LPMRdZ;
+  unsigned OpLo = IsExt ? AVR::ELPMRdZPi : AVR::LPMRdZPi;
+  unsigned OpHi = IsExt ? AVR::ELPMRdZ : AVR::LPMRdZ;
   TRI->splitReg(DstReg, DstLoReg, DstHiReg);
 
+  // Set the I/O register RAMPZ for ELPM.
+  if (IsExt) {
+    const AVRSubtarget &STI = MBB.getParent()->getSubtarget<AVRSubtarget>();
+    Register Bank = MI.getOperand(2).getReg();
+    // out RAMPZ, rtmp
+    buildMI(MBB, MBBI, AVR::OUTARr).addImm(STI.getIORegRAMPZ()).addReg(Bank);
+  }
+
   // Use a temporary register if src and dst registers are the same.
   if (DstReg == SrcReg)
     TmpReg = scavengeGPR8(MI);
 
   Register CurDstLoReg = (DstReg == SrcReg) ? TmpReg : DstLoReg;
   Register CurDstHiReg = (DstReg == SrcReg) ? TmpReg : DstHiReg;
 
   // Load low byte.
bool AVRExpandPseudo::expandLPMWELPMW(Block &MBB, BlockIt MBBI, bool IsExt) {
   MIBLO.setMemRefs(MI.memoperands());
   MIBHI.setMemRefs(MI.memoperands());
 
   MI.eraseFromParent();
   return true;
 }
 
 template <>
+bool AVRExpandPseudo::expand<AVR::LPMWRdZ>(Block &MBB, BlockIt MBBI) {
+  return expandLPMWELPMW(MBB, MBBI, false);
+}
+
+template <>
+bool AVRExpandPseudo::expand<AVR::ELPMWRdZ>(Block &MBB, BlockIt MBBI) {
+  return expandLPMWELPMW(MBB, MBBI, true);
+}
+
+template <>
+bool AVRExpandPseudo::expand<AVR::ELPMBRdZ>(Block &MBB, BlockIt MBBI) {
+  MachineInstr &MI = *MBBI;
+  Register DstReg = MI.getOperand(0).getReg();
+  Register SrcReg = MI.getOperand(1).getReg();
+  Register BankReg = MI.getOperand(2).getReg();
+  bool SrcIsKill = MI.getOperand(1).isKill();
+  const AVRSubtarget &STI = MBB.getParent()->getSubtarget<AVRSubtarget>();
+
+  // Set the I/O register RAMPZ for ELPM (out RAMPZ, rtmp).
+  buildMI(MBB, MBBI, AVR::OUTARr).addImm(STI.getIORegRAMPZ()).addReg(BankReg);
+
+  // Load byte.
+  auto MILB = buildMI(MBB, MBBI, AVR::ELPMRdZ)
+                  .addReg(DstReg, RegState::Define)
+                  .addReg(SrcReg, getKillRegState(SrcIsKill));
+
+  MILB.setMemRefs(MI.memoperands());
+
+  MI.eraseFromParent();
+  return true;
+}
+
+template <>
 bool AVRExpandPseudo::expand<AVR::LPMWRdZPi>(Block &MBB, BlockIt MBBI) {
-  llvm_unreachable("wide LPMPi is unimplemented");
+  llvm_unreachable("16-bit LPMPi is unimplemented");
+}
+
+template <>
+bool AVRExpandPseudo::expand<AVR::ELPMBRdZPi>(Block &MBB, BlockIt MBBI) {
+  llvm_unreachable("byte ELPMPi is unimplemented");
+}
+
+template <>
+bool AVRExpandPseudo::expand<AVR::ELPMWRdZPi>(Block &MBB, BlockIt MBBI) {
+  llvm_unreachable("16-bit ELPMPi is unimplemented");
 }
 
 template <typename Func>
 bool AVRExpandPseudo::expandAtomic(Block &MBB, BlockIt MBBI, Func f) {
   // Remove the pseudo instruction.
   MachineInstr &MI = *MBBI;
 
   // Store the SREG.
switch (Opcode) {
     EXPAND(AVR::LDSWRdK);
     EXPAND(AVR::LDWRdPtr);
     EXPAND(AVR::LDWRdPtrPi);
     EXPAND(AVR::LDWRdPtrPd);
   case AVR::LDDWRdYQ: //: FIXME: remove this once PR13375 gets fixed
     EXPAND(AVR::LDDWRdPtrQ);
     EXPAND(AVR::LPMWRdZ);
     EXPAND(AVR::LPMWRdZPi);
+    EXPAND(AVR::ELPMBRdZ);
+    EXPAND(AVR::ELPMWRdZ);
+    EXPAND(AVR::ELPMBRdZPi);
+    EXPAND(AVR::ELPMWRdZPi);
     EXPAND(AVR::AtomicLoad8);
     EXPAND(AVR::AtomicLoad16);
     EXPAND(AVR::AtomicStore8);
     EXPAND(AVR::AtomicStore16);
     EXPAND(AVR::AtomicLoadAdd8);
     EXPAND(AVR::AtomicLoadAdd16);
     EXPAND(AVR::AtomicLoadSub8);
     EXPAND(AVR::AtomicLoadSub16);

llvm/lib/Target/AVR/AVRISelDAGToDAG.cpp

StringRef getPassName() const override {
     return "AVR DAG->DAG Instruction Selection";
   }
 
   bool runOnMachineFunction(MachineFunction &MF) override;
 
   bool SelectAddr(SDNode *Op, SDValue N, SDValue &Base, SDValue &Disp);
 
   bool selectIndexedLoad(SDNode *N);
-  unsigned selectIndexedProgMemLoad(const LoadSDNode *LD, MVT VT);
+  unsigned selectIndexedProgMemLoad(const LoadSDNode *LD, MVT VT, int Bank);
 
   bool SelectInlineAsmMemoryOperand(const SDValue &Op, unsigned ConstraintCode,
                                     std::vector<SDValue> &OutOps) override;
 
 // Include the pieces autogenerated from the target description.
 #include "AVRGenDAGISel.inc"
 
 private:
SDNode *ResNode =
       CurDAG->getMachineNode(Opcode, SDLoc(N), VT, PtrVT, MVT::Other,
                              LD->getBasePtr(), LD->getChain());
   ReplaceUses(N, ResNode);
   CurDAG->RemoveDeadNode(N);
 
   return true;
 }
 
-unsigned AVRDAGToDAGISel::selectIndexedProgMemLoad(const LoadSDNode *LD,
-                                                   MVT VT) {
-  ISD::MemIndexedMode AM = LD->getAddressingMode();
-
+unsigned AVRDAGToDAGISel::selectIndexedProgMemLoad(const LoadSDNode *LD, MVT VT,
+                                                   int Bank) {
   // Progmem indexed loads only work in POSTINC mode.
-  if (LD->getExtensionType() != ISD::NON_EXTLOAD || AM != ISD::POST_INC) {
+  if (LD->getExtensionType() != ISD::NON_EXTLOAD ||
+      LD->getAddressingMode() != ISD::POST_INC)
     return 0;
-  }
+
+  // Feature ELPM is needed for loading from extended program memory.
+  assert((Bank == 0 || Subtarget->hasELPM()) &&
+         "cannot load from extended program memory on this mcu");
 
   unsigned Opcode = 0;
   int Offs = cast<ConstantSDNode>(LD->getOffset())->getSExtValue();
 
   switch (VT.SimpleTy) {
-  case MVT::i8: {
-    if (Offs != 1) {
-      return 0;
-    }
-    Opcode = AVR::LPMRdZPi;
+  case MVT::i8:
+    if (Offs == 1)
+      Opcode = Bank > 0 ? AVR::ELPMBRdZPi : AVR::LPMRdZPi;
     break;
-  }
-  case MVT::i16: {
-    if (Offs != 2) {
-      return 0;
-    }
-    Opcode = AVR::LPMWRdZPi;
+  case MVT::i16:
+    if (Offs == 2)
+      Opcode = Bank > 0 ? AVR::ELPMWRdZPi : AVR::LPMWRdZPi;
     break;
-  }
   default:
-    return 0;
+    break;
   }
 
   return Opcode;
 }
 
 bool AVRDAGToDAGISel::SelectInlineAsmMemoryOperand(
     const SDValue &Op, unsigned ConstraintCode, std::vector<SDValue> &OutOps) {
   assert((ConstraintCode == InlineAsm::Constraint_m ||

 
 template <> bool AVRDAGToDAGISel::select<ISD::LOAD>(SDNode *N) {
   const LoadSDNode *LD = cast<LoadSDNode>(N);
   if (!AVR::isProgramMemoryAccess(LD)) {
     // Check if the opcode can be converted into an indexed load.
     return selectIndexedLoad(N);
   }
 
-  assert(Subtarget->hasLPM() && "cannot load from program memory on this mcu");
+  if (!Subtarget->hasLPM())
+    report_fatal_error("cannot load from program memory on this mcu");
+
+  int ProgMemBank = AVR::getProgramMemoryBank(LD);
+  if (ProgMemBank < 0 || ProgMemBank > 5)
+    report_fatal_error("unexpected program memory bank");
 
   // This is a flash memory load, move the pointer into R31R30 and emit
   // the lpm instruction.
   MVT VT = LD->getMemoryVT().getSimpleVT();
   SDValue Chain = LD->getChain();
   SDValue Ptr = LD->getBasePtr();
   SDNode *ResNode;
   SDLoc DL(N);
 
   Chain = CurDAG->getCopyToReg(Chain, DL, AVR::R31R30, Ptr, SDValue());
   Ptr = CurDAG->getCopyFromReg(Chain, DL, AVR::R31R30, MVT::i16,
                                Chain.getValue(1));
 
-  SDValue RegZ = CurDAG->getRegister(AVR::R31R30, MVT::i16);
-
   // Check if the opcode can be converted into an indexed load.
-  if (unsigned LPMOpc = selectIndexedProgMemLoad(LD, VT)) {
+  if (unsigned LPMOpc = selectIndexedProgMemLoad(LD, VT, ProgMemBank)) {
     // It is legal to fold the load into an indexed load.
+    if (ProgMemBank == 0) {
       ResNode =
-        CurDAG->getMachineNode(LPMOpc, DL, VT, MVT::i16, MVT::Other, Ptr, RegZ);
-    ReplaceUses(SDValue(N, 1), SDValue(ResNode, 1));
+          CurDAG->getMachineNode(LPMOpc, DL, VT, MVT::i16, MVT::Other, Ptr);
+    } else {
+      // Do not combine the LDI instruction into the ELPM pseudo instruction,
+      // since it may be reused by other ELPM pseudo instructions.
+      SDValue NC = CurDAG->getTargetConstant(ProgMemBank, DL, MVT::i8);
+      auto *NP = CurDAG->getMachineNode(AVR::LDIRdK, DL, MVT::i8, NC);
+      ResNode = CurDAG->getMachineNode(LPMOpc, DL, VT, MVT::i16, MVT::Other,
+                                       Ptr, SDValue(NP, 0));
+    }
   } else {
     // Selecting an indexed load is not legal, fallback to a normal load.
     switch (VT.SimpleTy) {
     case MVT::i8:
-      ResNode = CurDAG->getMachineNode(AVR::LPMRdZ, DL, MVT::i8, MVT::Other,
-                                       Ptr, RegZ);
+      if (ProgMemBank == 0) {
+        ResNode =
+            CurDAG->getMachineNode(AVR::LPMRdZ, DL, MVT::i8, MVT::Other, Ptr);
+      } else {
+        // Do not combine the LDI instruction into the ELPM pseudo instruction,
+        // since it may be reused by other ELPM pseudo instructions.
+        SDValue NC = CurDAG->getTargetConstant(ProgMemBank, DL, MVT::i8);
+        auto *NP = CurDAG->getMachineNode(AVR::LDIRdK, DL, MVT::i8, NC);
+        ResNode = CurDAG->getMachineNode(AVR::ELPMBRdZ, DL, MVT::i8, MVT::Other,
+                                         Ptr, SDValue(NP, 0));
+      }
       break;
     case MVT::i16:
-      ResNode = CurDAG->getMachineNode(AVR::LPMWRdZ, DL, MVT::i16, MVT::Other,
-                                       Ptr, RegZ);
-      ReplaceUses(SDValue(N, 1), SDValue(ResNode, 1));
+      if (ProgMemBank == 0) {
+        ResNode =
+            CurDAG->getMachineNode(AVR::LPMWRdZ, DL, MVT::i16, MVT::Other, Ptr);
+      } else {
+        // Do not combine the LDI instruction into the ELPM pseudo instruction,
+        // since LDI requires the destination register in range R16~R31.
+        SDValue NC = CurDAG->getTargetConstant(ProgMemBank, DL, MVT::i8);
+        auto *NP = CurDAG->getMachineNode(AVR::LDIRdK, DL, MVT::i8, NC);
+        ResNode = CurDAG->getMachineNode(AVR::ELPMWRdZ, DL, MVT::i16,
+                                         MVT::Other, Ptr, SDValue(NP, 0));
+      }
       break;
     default:
       llvm_unreachable("Unsupported VT!");
     }
   }
 
   // Transfer memory operands.
   CurDAG->setNodeMemRefs(cast<MachineSDNode>(ResNode), {LD->getMemOperand()});

llvm/lib/Target/AVR/AVRInstrInfo.td

 
 // Extended load program memory operations.
 let mayLoad = 1, hasSideEffects = 0 in {
   let Defs = [R0],
       Uses = [R31R30] in def ELPM
       : F16<0b1001010111011000, (outs), (ins), "elpm", []>,
       Requires<[HasELPM]>;
 
-  def ELPMRdZ : FLPMX<1, 0,
-                      (outs GPR8
-                       : $dst),
-                      (ins ZREG
-                       : $z),
+  def ELPMRdZ : FLPMX<1, 0, (outs GPR8:$dst), (ins ZREG:$z),
                       "elpm\t$dst, $z", []>,
                 Requires<[HasELPMX]>;
 
-  let Defs = [R31R30] in def ELPMRdZPi : FLPMX<1, 1,
-                                               (outs GPR8
-                                                : $dst),
-                                               (ins ZREG
-                                                : $z),
-                                               "elpm\t$dst, $z+", []>,
+  let Defs = [R31R30] in {
+    def ELPMRdZPi : FLPMX<1, 1, (outs GPR8:$dst), (ins ZREG:$z),
                     Requires<[HasELPMX]>;
   }
 
+  // These pseudos are combination of the OUT and ELPM instructions.
+  let Defs = [R31R30], hasSideEffects = 1 in {
+    def ELPMBRdZ : Pseudo<(outs GPR8:$dst), (ins ZREG:$z, LD8:$p),
+                          "elpmb\t$dst, $z, $p", []>,
+                   Requires<[HasELPMX]>;
+
+    def ELPMWRdZ : Pseudo<(outs DREGS:$dst), (ins ZREG:$z, LD8:$p),
+                          "elpmw\t$dst, $z, $p", []>,
+                   Requires<[HasELPMX]>;
+
+    def ELPMBRdZPi : Pseudo<(outs GPR8:$dst), (ins ZREG:$z, LD8:$p),
+                            "elpmb\t$dst, $z+, $p", []>,
+                     Requires<[HasELPMX]>;
+
+    def ELPMWRdZPi : Pseudo<(outs DREGS:$dst), (ins ZREG:$z, LD8:$p),
+                            "elpmw\t$dst, $z+, $p", []>,
+                     Requires<[HasELPMX]>;
+  }
+}
+
 // Store program memory operations.
 let Uses = [R1, R0] in {
   let Uses = [R31R30, R1, R0] in def SPM
       : F16<0b1001010111101000, (outs), (ins), "spm", []>,
       Requires<[HasSPM]>;
 
   let Defs = [R31R30] in def SPMZPi : F16<0b1001010111111000, (outs),
                                           (ins ZREG

llvm/lib/Target/AVR/AVRSubtarget.h

public:
   /// Gets the ELF architecture for the e_flags field
   /// of an ELF object file.
   unsigned getELFArch() const {
     assert(ELFArch != 0 &&
            "every device must have an associate ELF architecture");
     return ELFArch;
   }
 
+  /// Get I/O register address.
+  int getIORegRAMPZ(void) const { return 0x3b; }
+
 private:
   /// The ELF e_flags architecture.
   unsigned ELFArch;
 
   // Subtarget feature settings
   // See AVR.td for details.
   bool m_hasSRAM;
   bool m_hasJMPCALL;

llvm/test/CodeGen/AVR/elpm.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=avr --mcpu=atmega2560 -verify-machineinstrs | FileCheck %s
+
+@arr0 = addrspace(1) constant [4 x i16] [i16 123, i16 24, i16 56, i16 37], align 1
+@arr1 = addrspace(2) constant [4 x i16] [i16 123, i16 34, i16 46, i16 27], align 1
+@arr2 = addrspace(3) constant [4 x i16] [i16 123, i16 23, i16 45, i16 17], align 1
+
+define i16 @foo0(i16 %a, i16 %b) {
+; CHECK-LABEL: foo0:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    lsl r22
+; CHECK-NEXT:    rol r23
+; CHECK-NEXT:    subi r22, -lo8(arr0)
+; CHECK-NEXT:    sbci r23, -hi8(arr0)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    lpm r18, Z+
+; CHECK-NEXT:    lpm r19, Z
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    rol r25
+; CHECK-NEXT:    subi r24, -lo8(arr0)
+; CHECK-NEXT:    sbci r25, -hi8(arr0)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    lpm r24, Z+
+; CHECK-NEXT:    lpm r25, Z
+; CHECK-NEXT:    sub r24, r18
+; CHECK-NEXT:    sbc r25, r19
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i16], [4 x i16] addrspace(1)* @arr0, i16 0, i16 %a
+  %0 = load i16, i16 addrspace(1)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i16], [4 x i16] addrspace(1)* @arr0, i16 0, i16 %b
+  %1 = load i16, i16 addrspace(1)* %arrayidx1, align 1
+  %sub = sub nsw i16 %0, %1
+  ret i16 %sub
+}
+
+define i16 @foo1(i16 %a, i16 %b) {
+; CHECK-LABEL: foo1:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    lsl r22
+; CHECK-NEXT:    rol r23
+; CHECK-NEXT:    subi r22, -lo8(arr1)
+; CHECK-NEXT:    sbci r23, -hi8(arr1)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    ldi r18, 1
+; CHECK-NEXT:    out 59, r18
+; CHECK-NEXT:    elpm r18, Z+
+; CHECK-NEXT:    elpm r19, Z
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    rol r25
+; CHECK-NEXT:    subi r24, -lo8(arr0)
+; CHECK-NEXT:    sbci r25, -hi8(arr0)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    lpm r24, Z+
+; CHECK-NEXT:    lpm r25, Z
+; CHECK-NEXT:    sub r24, r18
+; CHECK-NEXT:    sbc r25, r19
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i16], [4 x i16] addrspace(1)* @arr0, i16 0, i16 %a
+  %0 = load i16, i16 addrspace(1)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i16], [4 x i16] addrspace(2)* @arr1, i16 0, i16 %b
+  %1 = load i16, i16 addrspace(2)* %arrayidx1, align 1
+  %sub = sub nsw i16 %0, %1
+  ret i16 %sub
+}
+
+define i16 @foo2(i16 %a, i16 %b) {
+; CHECK-LABEL: foo2:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    rol r25
+; CHECK-NEXT:    subi r24, -lo8(arr2)
+; CHECK-NEXT:    sbci r25, -hi8(arr2)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    ldi r24, 2
+; CHECK-NEXT:    out 59, r24
+; CHECK-NEXT:    elpm r24, Z+
+; CHECK-NEXT:    elpm r25, Z
+; CHECK-NEXT:    lsl r22
+; CHECK-NEXT:    rol r23
+; CHECK-NEXT:    subi r22, -lo8(arr0)
+; CHECK-NEXT:    sbci r23, -hi8(arr0)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    lpm r18, Z+
+; CHECK-NEXT:    lpm r19, Z
+; CHECK-NEXT:    sub r24, r18
+; CHECK-NEXT:    sbc r25, r19
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i16], [4 x i16] addrspace(3)* @arr2, i16 0, i16 %a
+  %0 = load i16, i16 addrspace(3)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i16], [4 x i16] addrspace(1)* @arr0, i16 0, i16 %b
+  %1 = load i16, i16 addrspace(1)* %arrayidx1, align 1
+  %sub = sub nsw i16 %0, %1
+  ret i16 %sub
+}
+
+define i16 @foo3(i16 %a, i16 %b) {
+; CHECK-LABEL: foo3:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    lsl r22
+; CHECK-NEXT:    rol r23
+; CHECK-NEXT:    subi r22, -lo8(arr1)
+; CHECK-NEXT:    sbci r23, -hi8(arr1)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    ldi r18, 1
+; CHECK-NEXT:    out 59, r18
+; CHECK-NEXT:    elpm r18, Z+
+; CHECK-NEXT:    elpm r19, Z
+; CHECK-NEXT:    lsl r24
+; CHECK-NEXT:    rol r25
+; CHECK-NEXT:    subi r24, -lo8(arr2)
+; CHECK-NEXT:    sbci r25, -hi8(arr2)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    ldi r24, 2
+; CHECK-NEXT:    out 59, r24
+; CHECK-NEXT:    elpm r24, Z+
+; CHECK-NEXT:    elpm r25, Z
+; CHECK-NEXT:    sub r24, r18
+; CHECK-NEXT:    sbc r25, r19
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i16], [4 x i16] addrspace(3)* @arr2, i16 0, i16 %a
+  %0 = load i16, i16 addrspace(3)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i16], [4 x i16] addrspace(2)* @arr1, i16 0, i16 %b
+  %1 = load i16, i16 addrspace(2)* %arrayidx1, align 1
+  %sub = sub nsw i16 %0, %1
+  ret i16 %sub
+}
+
+@arrb1 = addrspace(1) constant [4 x i8] c"{\188%", align 1
+@arrb3 = addrspace(3) constant [4 x i8] c"{\22.\1B", align 1
+@arrb5 = addrspace(5) constant [4 x i8] c"{\17-\11", align 1
+
+define signext i8 @foob0(i16 %a, i16 %b) {
+; CHECK-LABEL: foob0:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    subi r22, -lo8(arrb1)
+; CHECK-NEXT:    sbci r23, -hi8(arrb1)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    lpm r18, Z
+; CHECK-NEXT:    subi r24, -lo8(arrb1)
+; CHECK-NEXT:    sbci r25, -hi8(arrb1)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    lpm r24, Z
+; CHECK-NEXT:    sub r24, r18
+; CHECK-NEXT:    mov r25, r24
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    sbc r25, r25
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i8], [4 x i8] addrspace(1)* @arrb1, i16 0, i16 %a
+  %0 = load i8, i8 addrspace(1)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i8], [4 x i8] addrspace(1)* @arrb1, i16 0, i16 %b
+  %1 = load i8, i8 addrspace(1)* %arrayidx1, align 1
+  %sub = sub i8 %0, %1
+  ret i8 %sub
+}
+
+define signext i8 @foob1(i16 %a, i16 %b) {
+; CHECK-LABEL: foob1:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    subi r22, -lo8(arrb3)
+; CHECK-NEXT:    sbci r23, -hi8(arrb3)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    ldi r18, 2
+; CHECK-NEXT:    out 59, r18
+; CHECK-NEXT:    elpm r18, Z
+; CHECK-NEXT:    subi r24, -lo8(arrb1)
+; CHECK-NEXT:    sbci r25, -hi8(arrb1)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    lpm r24, Z
+; CHECK-NEXT:    sub r24, r18
+; CHECK-NEXT:    mov r25, r24
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    sbc r25, r25
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i8], [4 x i8] addrspace(1)* @arrb1, i16 0, i16 %a
+  %0 = load i8, i8 addrspace(1)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i8], [4 x i8] addrspace(3)* @arrb3, i16 0, i16 %b
+  %1 = load i8, i8 addrspace(3)* %arrayidx1, align 1
+  %sub = sub i8 %0, %1
+  ret i8 %sub
+}
+
+define signext i8 @foob2(i16 %a, i16 %b) {
+; CHECK-LABEL: foob2:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    subi r24, -lo8(arrb5)
+; CHECK-NEXT:    sbci r25, -hi8(arrb5)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    ldi r24, 4
+; CHECK-NEXT:    out 59, r24
+; CHECK-NEXT:    elpm r24, Z
+; CHECK-NEXT:    subi r22, -lo8(arrb1)
+; CHECK-NEXT:    sbci r23, -hi8(arrb1)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    lpm r25, Z
+; CHECK-NEXT:    sub r24, r25
+; CHECK-NEXT:    mov r25, r24
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    sbc r25, r25
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i8], [4 x i8] addrspace(5)* @arrb5, i16 0, i16 %a
+  %0 = load i8, i8 addrspace(5)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i8], [4 x i8] addrspace(1)* @arrb1, i16 0, i16 %b
+  %1 = load i8, i8 addrspace(1)* %arrayidx1, align 1
+  %sub = sub i8 %0, %1
+  ret i8 %sub
+}
+
+define signext i8 @foob3(i16 %a, i16 %b) {
+; CHECK-LABEL: foob3:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    subi r22, -lo8(arrb5)
+; CHECK-NEXT:    sbci r23, -hi8(arrb5)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    ldi r18, 4
+; CHECK-NEXT:    out 59, r18
+; CHECK-NEXT:    elpm r18, Z
+; CHECK-NEXT:    subi r24, -lo8(arrb3)
+; CHECK-NEXT:    sbci r25, -hi8(arrb3)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    ldi r24, 2
+; CHECK-NEXT:    out 59, r24
+; CHECK-NEXT:    elpm r24, Z
+; CHECK-NEXT:    sub r24, r18
+; CHECK-NEXT:    mov r25, r24
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    sbc r25, r25
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i8], [4 x i8] addrspace(3)* @arrb3, i16 0, i16 %a
+  %0 = load i8, i8 addrspace(3)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i8], [4 x i8] addrspace(5)* @arrb5, i16 0, i16 %b
+  %1 = load i8, i8 addrspace(5)* %arrayidx1, align 1
+  %sub = sub i8 %0, %1
+  ret i8 %sub
+}
+
+define signext i8 @foob4(i16 %a, i16 %b) {
+; CHECK-LABEL: foob4:
+; CHECK:       ; %bb.0: ; %entry
+; CHECK-NEXT:    subi r22, -lo8(arrb3)
+; CHECK-NEXT:    sbci r23, -hi8(arrb3)
+; CHECK-NEXT:    movw r30, r22
+; CHECK-NEXT:    ldi r18, 2
+; CHECK-NEXT:    out 59, r18
+; CHECK-NEXT:    elpm r19, Z
+; CHECK-NEXT:    subi r24, -lo8(arrb3)
+; CHECK-NEXT:    sbci r25, -hi8(arrb3)
+; CHECK-NEXT:    movw r30, r24
+; CHECK-NEXT:    out 59, r18
+; CHECK-NEXT:    elpm r24, Z
+; CHECK-NEXT:    sub r24, r19
+; CHECK-NEXT:    mov r25, r24
+; CHECK-NEXT:    lsl r25
+; CHECK-NEXT:    sbc r25, r25
+; CHECK-NEXT:    ret
+entry:
+  %arrayidx = getelementptr inbounds [4 x i8], [4 x i8] addrspace(3)* @arrb3, i16 0, i16 %a
+  %0 = load i8, i8 addrspace(3)* %arrayidx, align 1
+  %arrayidx1 = getelementptr inbounds [4 x i8], [4 x i8] addrspace(3)* @arrb3, i16 0, i16 %b
+  %1 = load i8, i8 addrspace(3)* %arrayidx1, align 1
+  %sub = sub i8 %0, %1
+  ret i8 %sub
+}

llvm/test/CodeGen/AVR/lpmx.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=avr --mcpu=atmega328 -O0 -verify-machineinstrs \
+; RUN:     | FileCheck -check-prefix=CHECK-O0 %s
+; RUN: llc < %s -mtriple=avr --mcpu=atmega328 -O3 -verify-machineinstrs \
+; RUN:     | FileCheck -check-prefix=CHECK-O3 %s
+
+@arr0 = addrspace(1) constant [4 x i16] [i16 123, i16 234, i16 456, i16 67], align 1
+@arr1 = addrspace(1) constant [4 x i8] c"ABCD", align 1
+
+define i16 @foo0(i16 %a) addrspace(1) {
+; CHECK-O0-LABEL: foo0:
+; CHECK-O0:       ; %bb.0: ; %entry
+; CHECK-O0-NEXT:    push r28
+; CHECK-O0-NEXT:    push r29
+; CHECK-O0-NEXT:    in r28, 61
+; CHECK-O0-NEXT:    in r29, 62
+; CHECK-O0-NEXT:    sbiw r28, 2
+; CHECK-O0-NEXT:    in r0, 63
+; CHECK-O0-NEXT:    cli
+; CHECK-O0-NEXT:    out 62, r29
+; CHECK-O0-NEXT:    out 63, r0
+; CHECK-O0-NEXT:    out 61, r28
+; CHECK-O0-NEXT:    std Y+1, r24
+; CHECK-O0-NEXT:    std Y+2, r25
+; CHECK-O0-NEXT:    ldd r24, Y+1
+; CHECK-O0-NEXT:    ldd r25, Y+2
+; CHECK-O0-NEXT:    lsl r24
+; CHECK-O0-NEXT:    rol r25
+; CHECK-O0-NEXT:    subi r24, -lo8(arr0)
+; CHECK-O0-NEXT:    sbci r25, -hi8(arr0)
+; CHECK-O0-NEXT:    movw r30, r24
+; CHECK-O0-NEXT:    lpm r24, Z+
+; CHECK-O0-NEXT:    lpm r25, Z
+; CHECK-O0-NEXT:    adiw r28, 2
+; CHECK-O0-NEXT:    in r0, 63
+; CHECK-O0-NEXT:    cli
+; CHECK-O0-NEXT:    out 62, r29
+; CHECK-O0-NEXT:    out 63, r0
+; CHECK-O0-NEXT:    out 61, r28
+; CHECK-O0-NEXT:    pop r29
+; CHECK-O0-NEXT:    pop r28
+; CHECK-O0-NEXT:    ret
+;
+; CHECK-O3-LABEL: foo0:
+; CHECK-O3:       ; %bb.0: ; %entry
+; CHECK-O3-NEXT:    push r28
+; CHECK-O3-NEXT:    push r29
+; CHECK-O3-NEXT:    in r28, 61
+; CHECK-O3-NEXT:    in r29, 62
+; CHECK-O3-NEXT:    sbiw r28, 2
+; CHECK-O3-NEXT:    in r0, 63
+; CHECK-O3-NEXT:    cli
+; CHECK-O3-NEXT:    out 62, r29
+; CHECK-O3-NEXT:    out 63, r0
+; CHECK-O3-NEXT:    out 61, r28
+; CHECK-O3-NEXT:    std Y+1, r24
+; CHECK-O3-NEXT:    std Y+2, r25
+; CHECK-O3-NEXT:    lsl r24
+; CHECK-O3-NEXT:    rol r25
+; CHECK-O3-NEXT:    subi r24, -lo8(arr0)
+; CHECK-O3-NEXT:    sbci r25, -hi8(arr0)
+; CHECK-O3-NEXT:    movw r30, r24
+; CHECK-O3-NEXT:    lpm r24, Z+
+; CHECK-O3-NEXT:    lpm r25, Z
+; CHECK-O3-NEXT:    adiw r28, 2
+; CHECK-O3-NEXT:    in r0, 63
+; CHECK-O3-NEXT:    cli
+; CHECK-O3-NEXT:    out 62, r29
+; CHECK-O3-NEXT:    out 63, r0
+; CHECK-O3-NEXT:    out 61, r28
+; CHECK-O3-NEXT:    pop r29
+; CHECK-O3-NEXT:    pop r28
+; CHECK-O3-NEXT:    ret
+entry:
+  %a.addr = alloca i16, align 1
+  store i16 %a, i16* %a.addr, align 1
+  %0 = load i16, i16* %a.addr, align 1
+  %arrayidx = getelementptr inbounds [4 x i16], [4 x i16] addrspace(1)* @arr0, i16 0, i16 %0
+  %1 = load i16, i16 addrspace(1)* %arrayidx, align 1
+  ret i16 %1
+}
+
+define i8 @foo1(i16 %a) addrspace(1) {
+; CHECK-O0-LABEL: foo1:
+; CHECK-O0:       ; %bb.0: ; %entry
+; CHECK-O0-NEXT:    push r28
+; CHECK-O0-NEXT:    push r29
+; CHECK-O0-NEXT:    in r28, 61
+; CHECK-O0-NEXT:    in r29, 62
+; CHECK-O0-NEXT:    sbiw r28, 2
+; CHECK-O0-NEXT:    in r0, 63
+; CHECK-O0-NEXT:    cli
+; CHECK-O0-NEXT:    out 62, r29
+; CHECK-O0-NEXT:    out 63, r0
+; CHECK-O0-NEXT:    out 61, r28
+; CHECK-O0-NEXT:    std Y+1, r24
+; CHECK-O0-NEXT:    std Y+2, r25
+; CHECK-O0-NEXT:    ldd r24, Y+1
+; CHECK-O0-NEXT:    ldd r25, Y+2
+; CHECK-O0-NEXT:    subi r24, -lo8(arr1)
+; CHECK-O0-NEXT:    sbci r25, -hi8(arr1)
+; CHECK-O0-NEXT:    movw r30, r24
+; CHECK-O0-NEXT:    lpm r24, Z
+; CHECK-O0-NEXT:    adiw r28, 2
+; CHECK-O0-NEXT:    in r0, 63
+; CHECK-O0-NEXT:    cli
+; CHECK-O0-NEXT:    out 62, r29
+; CHECK-O0-NEXT:    out 63, r0
+; CHECK-O0-NEXT:    out 61, r28
+; CHECK-O0-NEXT:    pop r29
+; CHECK-O0-NEXT:    pop r28
+; CHECK-O0-NEXT:    ret
+;
+; CHECK-O3-LABEL: foo1:
+; CHECK-O3:       ; %bb.0: ; %entry
+; CHECK-O3-NEXT:    push r28
+; CHECK-O3-NEXT:    push r29
+; CHECK-O3-NEXT:    in r28, 61
+; CHECK-O3-NEXT:    in r29, 62
+; CHECK-O3-NEXT:    sbiw r28, 2
+; CHECK-O3-NEXT:    in r0, 63
+; CHECK-O3-NEXT:    cli
+; CHECK-O3-NEXT:    out 62, r29
+; CHECK-O3-NEXT:    out 63, r0
+; CHECK-O3-NEXT:    out 61, r28
+; CHECK-O3-NEXT:    std Y+1, r24
+; CHECK-O3-NEXT:    std Y+2, r25
+; CHECK-O3-NEXT:    subi r24, -lo8(arr1)
+; CHECK-O3-NEXT:    sbci r25, -hi8(arr1)
+; CHECK-O3-NEXT:    movw r30, r24
+; CHECK-O3-NEXT:    lpm r24, Z
+; CHECK-O3-NEXT:    adiw r28, 2
+; CHECK-O3-NEXT:    in r0, 63
+; CHECK-O3-NEXT:    cli
+; CHECK-O3-NEXT:    out 62, r29
+; CHECK-O3-NEXT:    out 63, r0
+; CHECK-O3-NEXT:    out 61, r28
+; CHECK-O3-NEXT:    pop r29
+; CHECK-O3-NEXT:    pop r28
+; CHECK-O3-NEXT:    ret
+entry:
+  %a.addr = alloca i16, align 1
+  store i16 %a, i16* %a.addr, align 1
+  %0 = load i16, i16* %a.addr, align 1
+  %arrayidx = getelementptr inbounds [4 x i8], [4 x i8] addrspace(1)* @arr1, i16 0, i16 %0
+  %1 = load i8, i8 addrspace(1)* %arrayidx, align 1
+  ret i8 %1
+}