[AggressiveInstCombine] Load merge the reverse load pattern of consecutive loads.

[bipmis]

Summary

This patch extends the load merge/widen in AggressiveInstCombine() defined in
https://reviews.llvm.org/D127392

to handle reverse load patterns.

[dmgreen]

Nice addition. I find the recursion a bit difficult to reason through, so forgive my questions below.

llvm/lib/Transforms/AggressiveInstCombine/AggressiveInstCombine.cpp

713

Offset2.slt(Offset1) will prevent the conversion to i64.

725

What happens if the load sizes are not the same? That sounds like it was protecting against a number of things.

743

This looks like it just needs a Start and End, that are LI1/LI2 depending on which comesBefore. That could hopefully avoid the duplication.

[spatel]

llvm/lib/Transforms/AggressiveInstCombine/AggressiveInstCombine.cpp

725

Related question: can we make this one-line change independently of the rest of the patch (either before or after)? That would reduce risk that we uncover some difficult-to-debug corner case.

[bipmis]

llvm/lib/Transforms/AggressiveInstCombine/AggressiveInstCombine.cpp

725

This is from the review comments from initial Implementation to enable mixed size merge. Some of the scenarios as added in the test seems to be working fine. However, in doing so we still make sure that the entire chain is reduced to a single load.
I am reverting this change for now. If needed this can go independently in a patch.

725

That is a good suggestion. I think we can do this last bit towards the end if needed.

[bipmis]

Revert mixed sized load merge and handle review comments.

[spatel]

This improves the pattern-matching, but it's still not complete, right? Ie, we should have a PhaseOrdering test with all 24 (4!) combinations of four i8-loads spliced into a 32-bit value, so we know the pattern is matched no matter what order the values are put back together with 'or'. We can probably rely on instcombine to canonicalize half of those patterns, but I'm not sure there's anything else there to reduce the matching space.

define i32 @loadCombine_4consecutive_1234(ptr %p) {
  %p1 = getelementptr i8, ptr %p, i32 1
  %p2 = getelementptr i8, ptr %p, i32 2
  %p3 = getelementptr i8, ptr %p, i32 3
  %l1 = load i8, ptr %p
  %l2 = load i8, ptr %p1
  %l3 = load i8, ptr %p2
  %l4 = load i8, ptr %p3

  %e1 = zext i8 %l1 to i32
  %e2 = zext i8 %l2 to i32
  %e3 = zext i8 %l3 to i32
  %e4 = zext i8 %l4 to i32

  %s2 = shl i32 %e2, 8
  %s3 = shl i32 %e3, 16
  %s4 = shl i32 %e4, 24

  %o1 = or i32 %e1, %s2
  %o2 = or i32 %o1, %s3
  %o3 = or i32 %o2, %s4
  ret i32 %o3
}

define i32 @loadCombine_4consecutive_1243(ptr %p) {
  %p1 = getelementptr i8, ptr %p, i32 1
  %p2 = getelementptr i8, ptr %p, i32 2
  %p3 = getelementptr i8, ptr %p, i32 3
  %l1 = load i8, ptr %p
  %l2 = load i8, ptr %p1
  %l3 = load i8, ptr %p2
  %l4 = load i8, ptr %p3

  %e1 = zext i8 %l1 to i32
  %e2 = zext i8 %l2 to i32
  %e3 = zext i8 %l3 to i32
  %e4 = zext i8 %l4 to i32

  %s2 = shl i32 %e2, 8
  %s3 = shl i32 %e3, 16
  %s4 = shl i32 %e4, 24

  %o1 = or i32 %e1, %s2
  %o2 = or i32 %o1, %s4
  %o3 = or i32 %o2, %s3
  ret i32 %o3
}

define i32 @loadCombine_4consecutive_1324(ptr %p) {
  %p1 = getelementptr i8, ptr %p, i32 1
  %p2 = getelementptr i8, ptr %p, i32 2
  %p3 = getelementptr i8, ptr %p, i32 3
  %l1 = load i8, ptr %p
  %l2 = load i8, ptr %p1
  %l3 = load i8, ptr %p2
  %l4 = load i8, ptr %p3

  %e1 = zext i8 %l1 to i32
  %e2 = zext i8 %l2 to i32
  %e3 = zext i8 %l3 to i32
  %e4 = zext i8 %l4 to i32

  %s2 = shl i32 %e2, 8
  %s3 = shl i32 %e3, 16
  %s4 = shl i32 %e4, 24

  %o1 = or i32 %e1, %s3
  %o2 = or i32 %o1, %s2
  %o3 = or i32 %o2, %s4
  ret i32 %o3
}

define i32 @loadCombine_4consecutive_1342(ptr %p) {

...

[bipmis]

In D135137#3856034, @spatel wrote:

This improves the pattern-matching, but it's still not complete, right? Ie, we should have a PhaseOrdering test with all 24 (4!) combinations of four i8-loads spliced into a 32-bit value, so we know the pattern is matched no matter what order the values are put back together with 'or'. We can probably rely on instcombine to canonicalize half of those patterns, but I'm not sure there's anything else there to reduce the matching space.

Right but won't these cover the most commonly seen patterns in the real application scenario, if not all considering it belongs to a pattern which generates a wider load.
InstCombine can possibly canonicalize the or-chain of loads in an ascending/descending order of load indexes. The fact that it is called multiple times should get us the pattern expected by AggressiveInstCombine. However, I am not sure if this is the right thing to do.

[spatel]

In D135137#3858404, @bipmis wrote:

In D135137#3856034, @spatel wrote:

This improves the pattern-matching, but it's still not complete, right? Ie, we should have a PhaseOrdering test with all 24 (4!) combinations of four i8-loads spliced into a 32-bit value, so we know the pattern is matched no matter what order the values are put back together with 'or'. We can probably rely on instcombine to canonicalize half of those patterns, but I'm not sure there's anything else there to reduce the matching space.

Right but won't these cover the most commonly seen patterns in the real application scenario, if not all considering it belongs to a pattern which generates a wider load.

In dealing with the related case of trying to match bswap patterns, we've found that eventually we will see every possible pattern in source code somewhere.
I don't disagree that the in-order/reverse are the most likely (and I won't hold up improvements from this patch), but I'd like to have tests in place that acknowledge that we are aware of the limitations of the current implementation. For users, having inconsistent optimization can be almost as frustrating as having no optimization.

InstCombine can possibly canonicalize the or-chain of loads in an ascending/descending order of load indexes. The fact that it is called multiple times should get us the pattern expected by AggressiveInstCombine. However, I am not sure if this is the right thing to do.

I expect that we'll end up using cooperation between "reassociate", "instcombine", and "aggressive-instcombine" to get the optimal loads. That's why I suggest PhaseOrdering tests - it's not clear which pass will be responsible for canonicalizing everything, but we do want to make sure that a complete trip through the optimizer will catch all patterns. There's also an open question of when and how often to invoke aggressive-instcombine (for example, it's only enabled at -O3 currently).

[bipmis]

Add PhaseOrdering tests for loadCombine.

[spatel]

The tests for completeness look as expected, but we want those to live in tests/Transforms/PhaseOrdering and use "RUN: opt -O3 -S < %s" or similar. That way, we can verify that the various passes that are expected to alter this IR are working cooperatively (and some seemingly unrelated change in another pass won't break the optimization that we want to enable with this patch).

[bipmis]

Move the loadCombine.ll which has combinations of four i8-loads spliced into a 32-bit value test to Transforms/PhaseOrdering

[spatel]

llvm/test/Transforms/PhaseOrdering/loadcombine.ll

2 ↗

(On Diff #468508)

I forgot to mention if the test uses a triple, then it needs to go in the X86 directory under this dir. Otherwise, you'll get test bot failures.

Please pre-commit the baseline test file (no pre-commit review needed to add tests). That way, we'll be sure it is working as expected before the code patch goes in.

[bipmis]

Repatch with test loadCombine commited.

[spatel]

LGTM.
I'm not sure if the transforms are completely reliable, but the PhaseOrdering tests show that we're getting 8 of the 24 patterns. Another 4 are partially folded, so those would presumably improve with the one-line enhancement that was originally in this patch. That leaves 12 that are escaping, but those might be canonicalized enough that we don't need to deal with every possible ordering.

llvm/lib/Transforms/AggressiveInstCombine/AggressiveInstCombine.cpp

712

typo: Offest -> Offset

[bipmis]

In D135137#3865593, @spatel wrote:

LGTM.
I'm not sure if the transforms are completely reliable, but the PhaseOrdering tests show that we're getting 8 of the 24 patterns. Another 4 are partially folded, so those would presumably improve with the one-line enhancement that was originally in this patch. That leaves 12 that are escaping, but those might be canonicalized enough that we don't need to deal with every possible ordering.

Thanks.
Looks like Reassociate does convert all the cases to the pattern needed. Just that it occurs late.

[spatel]

In D135137#3867684, @bipmis wrote:

In D135137#3865593, @spatel wrote:

LGTM.
I'm not sure if the transforms are completely reliable, but the PhaseOrdering tests show that we're getting 8 of the 24 patterns. Another 4 are partially folded, so those would presumably improve with the one-line enhancement that was originally in this patch. That leaves 12 that are escaping, but those might be canonicalized enough that we don't need to deal with every possible ordering.

Thanks.
Looks like Reassociate does convert all the cases to the pattern needed. Just that it occurs late.

Thanks for checking. So there are 3 possible follow-ups to make the transform robust:

1. Re-order the passes so AIC is after Reassociate
2. Add a late run of AIC.
3. Enhance the pattern matching in AIC to capture the alternate patterns.

[joanahalili]

Heads up! We are having some tests break due to this patch. We are working on a reproducer on our end and will post as soon as we get it.

[eaeltsin]

Here is the reproducer - https://godbolt.org/z/7bcjzfYK4

The following program returns 33 on trunk and 0 on clang 14:

#include <utility>

template <typename ValueType>
class Cell {
 public:
  Cell() = default;

  void Set(ValueType value) {
    value_ = value;
    has_data_ = true;
  }
  std::pair<ValueType, bool> GetAndMarkRead() {
    bool fresh = has_data_;
    has_data_ = false;
    return std::make_pair(value_, fresh);
  }
  bool GetAndMarkRead2() {
    bool fresh = has_data_;
    has_data_ = false;
    return fresh;
  }

  ValueType value_;
  bool has_data_ = false;
};

bool foo() {
  Cell<bool> cell;
  cell.Set(true);
  return cell.GetAndMarkRead().second;
}

int main() {
    if (foo()) return 0;
    return 33;
}

GetAndMarkRead returns false instead of 'true`. Returning std::pair matters.

[bipmis]

In D135137#3899915, @eaeltsin wrote:

Here is the reproducer - https://godbolt.org/z/7bcjzfYK4

Thanks for the test case. The issue likely is inserting the new merged load at a right insert point. I have done a patch to handle this. Can you please test the patch and confirm if this works correctly.

[asmok-g]

In D135137#3900447, @bipmis wrote:

In D135137#3899915, @eaeltsin wrote:

Here is the reproducer - https://godbolt.org/z/7bcjzfYK4

Thanks for the test case. The issue likely is inserting the new merged load at a right insert point. I have done a patch to handle this. Can you please test the patch and confirm if this works correctly.

Hello,

This is blocking our releases in Google. I see the fix patch didn't have an update since Tuesday. Can you please give it higher priority ?

[dmgreen]

Hello, as mentioned above there is a fix in https://reviews.llvm.org/D137201. We found another case that was incorrect though, so were looking into fixing that too. If it does take too long, perhaps a revert is in order in the meantime.

Did you get a chance to verify the fix worked for your case? Thanks.

[bipmis]

In D135137#3904719, @dmgreen wrote:

Hello, as mentioned above there is a fix in https://reviews.llvm.org/D137201. We found another case that was incorrect though, so were looking into fixing that too. If it does take too long, perhaps a revert is in order in the meantime.

Did you get a chance to verify the fix worked for your case? Thanks.

I think as a quick fix we can switch off load merge when we encounter stores and eventually come with a proper fix with additional test cases. I will do the patch on the same so that this does not block us.

[asmok-g]

Did you get a chance to verify the fix worked for your case? Thanks.

We unblocked ourselves for now. Will test the fix once it lands.

[dmgreen]

We unblocked ourselves for now. Will test the fix once it lands.

OK sounds good. @bipmis was on training this week, but did manage to put https://reviews.llvm.org/D137333 together which as far as we understand should fix all the known issues. Hopefully that will mean no more issues your end and we can work on a more substantial fix as we have more time. Please let us know if more issues do come up. Thanks.

[asmok-g]

In D135137#3905210, @asmok-g wrote:

Did you get a chance to verify the fix worked for your case? Thanks.

We unblocked ourselves for now. Will test the fix once it lands.

The test passed with the fix. Thanks!

llvm/lib/Transforms/AggressiveInstCombine/AggressiveInstCombine.cpp

static bool foldLoadsRecursive(Value *V, LoadOps &LOps, const DataLayout &DL,
   Instruction *L1, *L2;
 
   // Go to the last node with loads.
   if (match(V, m_OneUse(m_c_Or(
                    m_Value(X),
                    m_OneUse(m_Shl(m_OneUse(m_ZExt(m_OneUse(m_Instruction(L2)))),
                                   m_Value(ShAmt2)))))) ||
       match(V, m_OneUse(m_Or(m_Value(X),
-                             m_OneUse(m_ZExt(m_OneUse(m_Instruction(L2))))))))
-    foldLoadsRecursive(X, LOps, DL, AA);
-  else
+                             m_OneUse(m_ZExt(m_OneUse(m_Instruction(L2)))))))) {
+    if (!foldLoadsRecursive(X, LOps, DL, AA) && LOps.FoundRoot)
+      // Avoid Partial chain merge.
+      return false;
+  } else
     return false;
 
   // Check if the pattern has loads
   LoadInst *LI1 = LOps.Root;
   Value *ShAmt1 = LOps.Shift;
   if (LOps.FoundRoot == false &&
       (match(X, m_OneUse(m_ZExt(m_Instruction(L1)))) ||
        match(X, m_OneUse(m_Shl(m_OneUse(m_ZExt(m_OneUse(m_Instruction(L1)))),
                                m_Value(ShAmt1)))))) {
     LI1 = dyn_cast<LoadInst>(L1);
   }
   LoadInst *LI2 = dyn_cast<LoadInst>(L2);
 
   // Check if loads are same, atomic, volatile and having same address space.
   if (LI1 == LI2 || !LI1 || !LI2 || !LI1->isSimple() || !LI2->isSimple() ||
       LI1->getPointerAddressSpace() != LI2->getPointerAddressSpace())
     return false;
 
   // Check if Loads come from same BB.
   if (LI1->getParent() != LI2->getParent())
     return false;
 
-  // Swap loads if LI1 comes later as we handle only forward loads.
-  // This is done as InstCombine folds lowest node forward loads to reverse.
-  // The implementation will be subsequently extended to handle all reverse
-  // loads.
-  if (!LI1->comesBefore(LI2)) {
-    if (LOps.FoundRoot == false) {
-      std::swap(LI1, LI2);
-      std::swap(ShAmt1, ShAmt2);
-    } else
-      return false;
-  }
-
   // Find the data layout
   bool IsBigEndian = DL.isBigEndian();
 
   // Check if loads are consecutive and same size.
   Value *Load1Ptr = LI1->getPointerOperand();
   APInt Offset1(DL.getIndexTypeSizeInBits(Load1Ptr->getType()), 0);
   Load1Ptr =
       Load1Ptr->stripAndAccumulateConstantOffsets(DL, Offset1,
                                                   /* AllowNonInbounds */ true);
 
   Value *Load2Ptr = LI2->getPointerOperand();
   APInt Offset2(DL.getIndexTypeSizeInBits(Load2Ptr->getType()), 0);
   Load2Ptr =
       Load2Ptr->stripAndAccumulateConstantOffsets(DL, Offset2,
                                                   /* AllowNonInbounds */ true);
 
+  // Make sure Load with lower Offest is at LI1
+  bool Reverse = false;
+  if (Offset2.slt(Offset1)) {
+    std::swap(LI1, LI2);
+    std::swap(ShAmt1, ShAmt2);
+    std::swap(Offset1, Offset2);
+    std::swap(Load1Ptr, Load2Ptr);
+    Reverse = true;
+  }
+
   // Verify if both loads have same base pointers and load sizes are same.
   uint64_t LoadSize1 = LI1->getType()->getPrimitiveSizeInBits();
   uint64_t LoadSize2 = LI2->getType()->getPrimitiveSizeInBits();
   if (Load1Ptr != Load2Ptr || LoadSize1 != LoadSize2)
     return false;
 
   // Support Loadsizes greater or equal to 8bits and only power of 2.
   if (LoadSize1 < 8 || !isPowerOf2_64(LoadSize1))
     return false;
 
   // Alias Analysis to check for store b/w the loads.
-  MemoryLocation Loc = MemoryLocation::get(LI2);
+  LoadInst *Start = LI1, *End = LI2;
+  if (!LI1->comesBefore(LI2))
+    std::swap(Start, End);
+  MemoryLocation Loc = MemoryLocation::get(End);
   unsigned NumScanned = 0;
-  for (Instruction &Inst : make_range(LI1->getIterator(), LI2->getIterator())) {
+  for (Instruction &Inst :
+       make_range(Start->getIterator(), End->getIterator())) {
     if (Inst.mayWriteToMemory() && isModSet(AA.getModRefInfo(&Inst, Loc)))
       return false;
     if (++NumScanned > MaxInstrsToScan)
       return false;
   }
 
   // Big endian swap the shifts
   if (IsBigEndian)
     std::swap(ShAmt1, ShAmt2);
 
   // Find Shifts values.
   const APInt *Temp;
   uint64_t Shift1 = 0, Shift2 = 0;
   if (ShAmt1 && match(ShAmt1, m_APInt(Temp)))
     Shift1 = Temp->getZExtValue();
   if (ShAmt2 && match(ShAmt2, m_APInt(Temp)))
     Shift2 = Temp->getZExtValue();
 
   // First load is always LI1. This is where we put the new load.
-  // Use the merged load size available from LI1, if we already combined loads.
-  if (LOps.FoundRoot)
+  // Use the merged load size available from LI1 for forward loads.
+  if (LOps.FoundRoot) {
+    if (!Reverse)
       LoadSize1 = LOps.LoadSize;
+    else
+      LoadSize2 = LOps.LoadSize;
+  }
 
   // Verify if shift amount and load index aligns and verifies that loads
   // are consecutive.
   uint64_t ShiftDiff = IsBigEndian ? LoadSize2 : LoadSize1;
   uint64_t PrevSize =
       DL.getTypeStoreSize(IntegerType::get(LI1->getContext(), LoadSize1));
   if ((Shift2 - Shift1) != ShiftDiff || (Offset2 - Offset1) != PrevSize)
     return false;
 
   // Update LOps
   AAMDNodes AATags1 = LOps.AATags;
   AAMDNodes AATags2 = LI2->getAAMetadata();
   if (LOps.FoundRoot == false) {
     LOps.FoundRoot = true;
-    LOps.LoadSize = LoadSize1 + LoadSize2;
     AATags1 = LI1->getAAMetadata();
-  } else
-    LOps.LoadSize = LOps.LoadSize + LoadSize2;
+  }
+  LOps.LoadSize = LoadSize1 + LoadSize2;
 
   // Concatenate the AATags of the Merged Loads.
   LOps.AATags = AATags1.concat(AATags2);
 
   LOps.Root = LI1;
   LOps.Shift = ShAmt1;
   LOps.ZextType = X->getType();
   return true;

llvm/test/Transforms/AggressiveInstCombine/AArch64/or-load.ll

;
   %s4 = shl i16 %e4, 12
   %o1 = or i16 %e1, %s2
   %o2 = or i16 %o1, %s3
   %o3 = or i16 %o2, %s4
   ret i16 %o3
 }
 
 define i32 @loadCombine_4consecutive_rev(ptr %p) {
-; ALL-LABEL: @loadCombine_4consecutive_rev(
-; ALL-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; ALL-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; ALL-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
-; ALL-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; ALL-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; ALL-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
-; ALL-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
-; ALL-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i32
-; ALL-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i32
-; ALL-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i32
-; ALL-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i32
-; ALL-NEXT:    [[S2:%.*]] = shl i32 [[E2]], 8
-; ALL-NEXT:    [[S3:%.*]] = shl i32 [[E3]], 16
-; ALL-NEXT:    [[S4:%.*]] = shl i32 [[E4]], 24
-; ALL-NEXT:    [[O1:%.*]] = or i32 [[S4]], [[S3]]
-; ALL-NEXT:    [[O2:%.*]] = or i32 [[O1]], [[S2]]
-; ALL-NEXT:    [[O3:%.*]] = or i32 [[O2]], [[E1]]
-; ALL-NEXT:    ret i32 [[O3]]
+; LE-LABEL: @loadCombine_4consecutive_rev(
+; LE-NEXT:    [[L1:%.*]] = load i32, ptr [[P:%.*]], align 1
+; LE-NEXT:    ret i32 [[L1]]
+;
+; BE-LABEL: @loadCombine_4consecutive_rev(
+; BE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
+; BE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
+; BE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
+; BE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
+; BE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
+; BE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
+; BE-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
+; BE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i32
+; BE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i32
+; BE-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i32
+; BE-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i32
+; BE-NEXT:    [[S2:%.*]] = shl i32 [[E2]], 8
+; BE-NEXT:    [[S3:%.*]] = shl i32 [[E3]], 16
+; BE-NEXT:    [[S4:%.*]] = shl i32 [[E4]], 24
+; BE-NEXT:    [[O1:%.*]] = or i32 [[S4]], [[S3]]
+; BE-NEXT:    [[O2:%.*]] = or i32 [[O1]], [[S2]]
+; BE-NEXT:    [[O3:%.*]] = or i32 [[O2]], [[E1]]
+; BE-NEXT:    ret i32 [[O3]]
 ;
   %p1 = getelementptr i8, ptr %p, i32 1
   %p2 = getelementptr i8, ptr %p, i32 2
   %p3 = getelementptr i8, ptr %p, i32 3
   %l1 = load i8, ptr %p
   %l2 = load i8, ptr %p1
   %l3 = load i8, ptr %p2
   %l4 = load i8, ptr %p3
;
 
   %o1 = or i32 %s4, %s3
   %o2 = or i32 %o1, %s2
   %o3 = or i32 %o2, %e1
   ret i32 %o3
 }
 
 define i64 @loadCombine_8consecutive_rev(ptr %p) {
-; ALL-LABEL: @loadCombine_8consecutive_rev(
-; ALL-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; ALL-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; ALL-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
-; ALL-NEXT:    [[P4:%.*]] = getelementptr i8, ptr [[P]], i32 4
-; ALL-NEXT:    [[P5:%.*]] = getelementptr i8, ptr [[P]], i32 5
-; ALL-NEXT:    [[P6:%.*]] = getelementptr i8, ptr [[P]], i32 6
-; ALL-NEXT:    [[P7:%.*]] = getelementptr i8, ptr [[P]], i32 7
-; ALL-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; ALL-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; ALL-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
-; ALL-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
-; ALL-NEXT:    [[L5:%.*]] = load i8, ptr [[P4]], align 1
-; ALL-NEXT:    [[L6:%.*]] = load i8, ptr [[P5]], align 1
-; ALL-NEXT:    [[L7:%.*]] = load i8, ptr [[P6]], align 1
-; ALL-NEXT:    [[L8:%.*]] = load i8, ptr [[P7]], align 1
-; ALL-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i64
-; ALL-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i64
-; ALL-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i64
-; ALL-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i64
-; ALL-NEXT:    [[E5:%.*]] = zext i8 [[L5]] to i64
-; ALL-NEXT:    [[E6:%.*]] = zext i8 [[L6]] to i64
-; ALL-NEXT:    [[E7:%.*]] = zext i8 [[L7]] to i64
-; ALL-NEXT:    [[E8:%.*]] = zext i8 [[L8]] to i64
-; ALL-NEXT:    [[S2:%.*]] = shl i64 [[E2]], 8
-; ALL-NEXT:    [[S3:%.*]] = shl i64 [[E3]], 16
-; ALL-NEXT:    [[S4:%.*]] = shl i64 [[E4]], 24
-; ALL-NEXT:    [[S5:%.*]] = shl i64 [[E5]], 32
-; ALL-NEXT:    [[S6:%.*]] = shl i64 [[E6]], 40
-; ALL-NEXT:    [[S7:%.*]] = shl i64 [[E7]], 48
-; ALL-NEXT:    [[S8:%.*]] = shl i64 [[E8]], 56
-; ALL-NEXT:    [[O7:%.*]] = or i64 [[S8]], [[S7]]
-; ALL-NEXT:    [[O6:%.*]] = or i64 [[O7]], [[S6]]
-; ALL-NEXT:    [[O5:%.*]] = or i64 [[O6]], [[S5]]
-; ALL-NEXT:    [[O4:%.*]] = or i64 [[O5]], [[S4]]
-; ALL-NEXT:    [[O3:%.*]] = or i64 [[O4]], [[S3]]
-; ALL-NEXT:    [[O2:%.*]] = or i64 [[O3]], [[S2]]
-; ALL-NEXT:    [[O1:%.*]] = or i64 [[O2]], [[E1]]
-; ALL-NEXT:    ret i64 [[O1]]
+; LE-LABEL: @loadCombine_8consecutive_rev(
+; LE-NEXT:    [[L1:%.*]] = load i64, ptr [[P:%.*]], align 1
+; LE-NEXT:    ret i64 [[L1]]
+;
+; BE-LABEL: @loadCombine_8consecutive_rev(
+; BE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
+; BE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
+; BE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
+; BE-NEXT:    [[P4:%.*]] = getelementptr i8, ptr [[P]], i32 4
+; BE-NEXT:    [[P5:%.*]] = getelementptr i8, ptr [[P]], i32 5
+; BE-NEXT:    [[P6:%.*]] = getelementptr i8, ptr [[P]], i32 6
+; BE-NEXT:    [[P7:%.*]] = getelementptr i8, ptr [[P]], i32 7
+; BE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
+; BE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
+; BE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
+; BE-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
+; BE-NEXT:    [[L5:%.*]] = load i8, ptr [[P4]], align 1
+; BE-NEXT:    [[L6:%.*]] = load i8, ptr [[P5]], align 1
+; BE-NEXT:    [[L7:%.*]] = load i8, ptr [[P6]], align 1
+; BE-NEXT:    [[L8:%.*]] = load i8, ptr [[P7]], align 1
+; BE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i64
+; BE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i64
+; BE-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i64
+; BE-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i64
+; BE-NEXT:    [[E5:%.*]] = zext i8 [[L5]] to i64
+; BE-NEXT:    [[E6:%.*]] = zext i8 [[L6]] to i64
+; BE-NEXT:    [[E7:%.*]] = zext i8 [[L7]] to i64
+; BE-NEXT:    [[E8:%.*]] = zext i8 [[L8]] to i64
+; BE-NEXT:    [[S2:%.*]] = shl i64 [[E2]], 8
+; BE-NEXT:    [[S3:%.*]] = shl i64 [[E3]], 16
+; BE-NEXT:    [[S4:%.*]] = shl i64 [[E4]], 24
+; BE-NEXT:    [[S5:%.*]] = shl i64 [[E5]], 32
+; BE-NEXT:    [[S6:%.*]] = shl i64 [[E6]], 40
+; BE-NEXT:    [[S7:%.*]] = shl i64 [[E7]], 48
+; BE-NEXT:    [[S8:%.*]] = shl i64 [[E8]], 56
+; BE-NEXT:    [[O7:%.*]] = or i64 [[S8]], [[S7]]
+; BE-NEXT:    [[O6:%.*]] = or i64 [[O7]], [[S6]]
+; BE-NEXT:    [[O5:%.*]] = or i64 [[O6]], [[S5]]
+; BE-NEXT:    [[O4:%.*]] = or i64 [[O5]], [[S4]]
+; BE-NEXT:    [[O3:%.*]] = or i64 [[O4]], [[S3]]
+; BE-NEXT:    [[O2:%.*]] = or i64 [[O3]], [[S2]]
+; BE-NEXT:    [[O1:%.*]] = or i64 [[O2]], [[E1]]
+; BE-NEXT:    ret i64 [[O1]]
 ;
   %p1 = getelementptr i8, ptr %p, i32 1
   %p2 = getelementptr i8, ptr %p, i32 2
   %p3 = getelementptr i8, ptr %p, i32 3
   %p4 = getelementptr i8, ptr %p, i32 4
   %p5 = getelementptr i8, ptr %p, i32 5
   %p6 = getelementptr i8, ptr %p, i32 6
   %p7 = getelementptr i8, ptr %p, i32 7
;
   %o4 = or i64 %o5, %s4
   %o3 = or i64 %o4, %s3
   %o2 = or i64 %o3, %s2
   %o1 = or i64 %o2, %e1
   ret i64 %o1
 }
 
 define i64 @loadCombine_8consecutive_rev_BE(ptr %p) {
-; ALL-LABEL: @loadCombine_8consecutive_rev_BE(
-; ALL-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; ALL-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; ALL-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
-; ALL-NEXT:    [[P4:%.*]] = getelementptr i8, ptr [[P]], i32 4
-; ALL-NEXT:    [[P5:%.*]] = getelementptr i8, ptr [[P]], i32 5
-; ALL-NEXT:    [[P6:%.*]] = getelementptr i8, ptr [[P]], i32 6
-; ALL-NEXT:    [[P7:%.*]] = getelementptr i8, ptr [[P]], i32 7
-; ALL-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; ALL-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; ALL-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
-; ALL-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
-; ALL-NEXT:    [[L5:%.*]] = load i8, ptr [[P4]], align 1
-; ALL-NEXT:    [[L6:%.*]] = load i8, ptr [[P5]], align 1
-; ALL-NEXT:    [[L7:%.*]] = load i8, ptr [[P6]], align 1
-; ALL-NEXT:    [[L8:%.*]] = load i8, ptr [[P7]], align 1
-; ALL-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i64
-; ALL-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i64
-; ALL-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i64
-; ALL-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i64
-; ALL-NEXT:    [[E5:%.*]] = zext i8 [[L5]] to i64
-; ALL-NEXT:    [[E6:%.*]] = zext i8 [[L6]] to i64
-; ALL-NEXT:    [[E7:%.*]] = zext i8 [[L7]] to i64
-; ALL-NEXT:    [[E8:%.*]] = zext i8 [[L8]] to i64
-; ALL-NEXT:    [[S1:%.*]] = shl i64 [[E1]], 56
-; ALL-NEXT:    [[S2:%.*]] = shl i64 [[E2]], 48
-; ALL-NEXT:    [[S3:%.*]] = shl i64 [[E3]], 40
-; ALL-NEXT:    [[S4:%.*]] = shl i64 [[E4]], 32
-; ALL-NEXT:    [[S5:%.*]] = shl i64 [[E5]], 24
-; ALL-NEXT:    [[S6:%.*]] = shl i64 [[E6]], 16
-; ALL-NEXT:    [[S7:%.*]] = shl i64 [[E7]], 8
-; ALL-NEXT:    [[O7:%.*]] = or i64 [[E8]], [[S7]]
-; ALL-NEXT:    [[O6:%.*]] = or i64 [[O7]], [[S6]]
-; ALL-NEXT:    [[O5:%.*]] = or i64 [[O6]], [[S5]]
-; ALL-NEXT:    [[O4:%.*]] = or i64 [[O5]], [[S4]]
-; ALL-NEXT:    [[O3:%.*]] = or i64 [[O4]], [[S3]]
-; ALL-NEXT:    [[O2:%.*]] = or i64 [[O3]], [[S2]]
-; ALL-NEXT:    [[O1:%.*]] = or i64 [[O2]], [[S1]]
-; ALL-NEXT:    ret i64 [[O1]]
+; LE-LABEL: @loadCombine_8consecutive_rev_BE(
+; LE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
+; LE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
+; LE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
+; LE-NEXT:    [[P4:%.*]] = getelementptr i8, ptr [[P]], i32 4
+; LE-NEXT:    [[P5:%.*]] = getelementptr i8, ptr [[P]], i32 5
+; LE-NEXT:    [[P6:%.*]] = getelementptr i8, ptr [[P]], i32 6
+; LE-NEXT:    [[P7:%.*]] = getelementptr i8, ptr [[P]], i32 7
+; LE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
+; LE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
+; LE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
+; LE-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
+; LE-NEXT:    [[L5:%.*]] = load i8, ptr [[P4]], align 1
+; LE-NEXT:    [[L6:%.*]] = load i8, ptr [[P5]], align 1
+; LE-NEXT:    [[L7:%.*]] = load i8, ptr [[P6]], align 1
+; LE-NEXT:    [[L8:%.*]] = load i8, ptr [[P7]], align 1
+; LE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i64
+; LE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i64
+; LE-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i64
+; LE-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i64
+; LE-NEXT:    [[E5:%.*]] = zext i8 [[L5]] to i64
+; LE-NEXT:    [[E6:%.*]] = zext i8 [[L6]] to i64
+; LE-NEXT:    [[E7:%.*]] = zext i8 [[L7]] to i64
+; LE-NEXT:    [[E8:%.*]] = zext i8 [[L8]] to i64
+; LE-NEXT:    [[S1:%.*]] = shl i64 [[E1]], 56
+; LE-NEXT:    [[S2:%.*]] = shl i64 [[E2]], 48
+; LE-NEXT:    [[S3:%.*]] = shl i64 [[E3]], 40
+; LE-NEXT:    [[S4:%.*]] = shl i64 [[E4]], 32
+; LE-NEXT:    [[S5:%.*]] = shl i64 [[E5]], 24
+; LE-NEXT:    [[S6:%.*]] = shl i64 [[E6]], 16
+; LE-NEXT:    [[S7:%.*]] = shl i64 [[E7]], 8
+; LE-NEXT:    [[O7:%.*]] = or i64 [[E8]], [[S7]]
+; LE-NEXT:    [[O6:%.*]] = or i64 [[O7]], [[S6]]
+; LE-NEXT:    [[O5:%.*]] = or i64 [[O6]], [[S5]]
+; LE-NEXT:    [[O4:%.*]] = or i64 [[O5]], [[S4]]
+; LE-NEXT:    [[O3:%.*]] = or i64 [[O4]], [[S3]]
+; LE-NEXT:    [[O2:%.*]] = or i64 [[O3]], [[S2]]
+; LE-NEXT:    [[O1:%.*]] = or i64 [[O2]], [[S1]]
+; LE-NEXT:    ret i64 [[O1]]
+;
+; BE-LABEL: @loadCombine_8consecutive_rev_BE(
+; BE-NEXT:    [[L1:%.*]] = load i64, ptr [[P:%.*]], align 1
+; BE-NEXT:    ret i64 [[L1]]
 ;
   %p1 = getelementptr i8, ptr %p, i32 1
   %p2 = getelementptr i8, ptr %p, i32 2
   %p3 = getelementptr i8, ptr %p, i32 3
   %p4 = getelementptr i8, ptr %p, i32 4
   %p5 = getelementptr i8, ptr %p, i32 5
   %p6 = getelementptr i8, ptr %p, i32 6
   %p7 = getelementptr i8, ptr %p, i32 7
;
   %o4 = or i64 %o5, %s4
   %o3 = or i64 %o4, %s3
   %o2 = or i64 %o3, %s2
   %o1 = or i64 %o2, %s1
   ret i64 %o1
 }
 
 define i64 @eggs(ptr noundef readonly %arg) {
-; ALL-LABEL: @eggs(
-; ALL-NEXT:    [[TMP3:%.*]] = load i8, ptr [[ARG:%.*]], align 1
-; ALL-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 1
-; ALL-NEXT:    [[TMP5:%.*]] = load i8, ptr [[TMP4]], align 1
-; ALL-NEXT:    [[TMP6:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 2
-; ALL-NEXT:    [[TMP7:%.*]] = load i8, ptr [[TMP6]], align 1
-; ALL-NEXT:    [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 3
-; ALL-NEXT:    [[TMP9:%.*]] = load i8, ptr [[TMP8]], align 1
-; ALL-NEXT:    [[TMP10:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 4
-; ALL-NEXT:    [[TMP11:%.*]] = load i8, ptr [[TMP10]], align 1
-; ALL-NEXT:    [[TMP12:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 5
-; ALL-NEXT:    [[TMP13:%.*]] = load i8, ptr [[TMP12]], align 1
-; ALL-NEXT:    [[TMP14:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 6
-; ALL-NEXT:    [[TMP15:%.*]] = load i8, ptr [[TMP14]], align 1
-; ALL-NEXT:    [[TMP16:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 7
-; ALL-NEXT:    [[TMP17:%.*]] = load i8, ptr [[TMP16]], align 1
-; ALL-NEXT:    [[TMP18:%.*]] = zext i8 [[TMP17]] to i64
-; ALL-NEXT:    [[TMP19:%.*]] = shl nuw i64 [[TMP18]], 56
-; ALL-NEXT:    [[TMP20:%.*]] = zext i8 [[TMP15]] to i64
-; ALL-NEXT:    [[TMP21:%.*]] = shl nuw nsw i64 [[TMP20]], 48
-; ALL-NEXT:    [[TMP22:%.*]] = or i64 [[TMP19]], [[TMP21]]
-; ALL-NEXT:    [[TMP23:%.*]] = zext i8 [[TMP13]] to i64
-; ALL-NEXT:    [[TMP24:%.*]] = shl nuw nsw i64 [[TMP23]], 40
-; ALL-NEXT:    [[TMP25:%.*]] = or i64 [[TMP22]], [[TMP24]]
-; ALL-NEXT:    [[TMP26:%.*]] = zext i8 [[TMP11]] to i64
-; ALL-NEXT:    [[TMP27:%.*]] = shl nuw nsw i64 [[TMP26]], 32
-; ALL-NEXT:    [[TMP28:%.*]] = or i64 [[TMP25]], [[TMP27]]
-; ALL-NEXT:    [[TMP29:%.*]] = zext i8 [[TMP9]] to i64
-; ALL-NEXT:    [[TMP30:%.*]] = shl nuw nsw i64 [[TMP29]], 24
-; ALL-NEXT:    [[TMP31:%.*]] = or i64 [[TMP28]], [[TMP30]]
-; ALL-NEXT:    [[TMP32:%.*]] = zext i8 [[TMP7]] to i64
-; ALL-NEXT:    [[TMP33:%.*]] = shl nuw nsw i64 [[TMP32]], 16
-; ALL-NEXT:    [[TMP34:%.*]] = zext i8 [[TMP5]] to i64
-; ALL-NEXT:    [[TMP35:%.*]] = shl nuw nsw i64 [[TMP34]], 8
-; ALL-NEXT:    [[TMP36:%.*]] = or i64 [[TMP31]], [[TMP33]]
-; ALL-NEXT:    [[TMP37:%.*]] = zext i8 [[TMP3]] to i64
-; ALL-NEXT:    [[TMP38:%.*]] = or i64 [[TMP36]], [[TMP35]]
-; ALL-NEXT:    [[TMP39:%.*]] = or i64 [[TMP38]], [[TMP37]]
-; ALL-NEXT:    ret i64 [[TMP39]]
+; LE-LABEL: @eggs(
+; LE-NEXT:    [[TMP3:%.*]] = load i64, ptr [[ARG:%.*]], align 1
+; LE-NEXT:    ret i64 [[TMP3]]
+;
+; BE-LABEL: @eggs(
+; BE-NEXT:    [[TMP3:%.*]] = load i8, ptr [[ARG:%.*]], align 1
+; BE-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 1
+; BE-NEXT:    [[TMP5:%.*]] = load i8, ptr [[TMP4]], align 1
+; BE-NEXT:    [[TMP6:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 2
+; BE-NEXT:    [[TMP7:%.*]] = load i8, ptr [[TMP6]], align 1
+; BE-NEXT:    [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 3
+; BE-NEXT:    [[TMP9:%.*]] = load i8, ptr [[TMP8]], align 1
+; BE-NEXT:    [[TMP10:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 4
+; BE-NEXT:    [[TMP11:%.*]] = load i8, ptr [[TMP10]], align 1
+; BE-NEXT:    [[TMP12:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 5
+; BE-NEXT:    [[TMP13:%.*]] = load i8, ptr [[TMP12]], align 1
+; BE-NEXT:    [[TMP14:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 6
+; BE-NEXT:    [[TMP15:%.*]] = load i8, ptr [[TMP14]], align 1
+; BE-NEXT:    [[TMP16:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 7
+; BE-NEXT:    [[TMP17:%.*]] = load i8, ptr [[TMP16]], align 1
+; BE-NEXT:    [[TMP18:%.*]] = zext i8 [[TMP17]] to i64
+; BE-NEXT:    [[TMP19:%.*]] = shl nuw i64 [[TMP18]], 56
+; BE-NEXT:    [[TMP20:%.*]] = zext i8 [[TMP15]] to i64
+; BE-NEXT:    [[TMP21:%.*]] = shl nuw nsw i64 [[TMP20]], 48
+; BE-NEXT:    [[TMP22:%.*]] = or i64 [[TMP19]], [[TMP21]]
+; BE-NEXT:    [[TMP23:%.*]] = zext i8 [[TMP13]] to i64
+; BE-NEXT:    [[TMP24:%.*]] = shl nuw nsw i64 [[TMP23]], 40
+; BE-NEXT:    [[TMP25:%.*]] = or i64 [[TMP22]], [[TMP24]]
+; BE-NEXT:    [[TMP26:%.*]] = zext i8 [[TMP11]] to i64
+; BE-NEXT:    [[TMP27:%.*]] = shl nuw nsw i64 [[TMP26]], 32
+; BE-NEXT:    [[TMP28:%.*]] = or i64 [[TMP25]], [[TMP27]]
+; BE-NEXT:    [[TMP29:%.*]] = zext i8 [[TMP9]] to i64
+; BE-NEXT:    [[TMP30:%.*]] = shl nuw nsw i64 [[TMP29]], 24
+; BE-NEXT:    [[TMP31:%.*]] = or i64 [[TMP28]], [[TMP30]]
+; BE-NEXT:    [[TMP32:%.*]] = zext i8 [[TMP7]] to i64
+; BE-NEXT:    [[TMP33:%.*]] = shl nuw nsw i64 [[TMP32]], 16
+; BE-NEXT:    [[TMP34:%.*]] = zext i8 [[TMP5]] to i64
+; BE-NEXT:    [[TMP35:%.*]] = shl nuw nsw i64 [[TMP34]], 8
+; BE-NEXT:    [[TMP36:%.*]] = or i64 [[TMP31]], [[TMP33]]
+; BE-NEXT:    [[TMP37:%.*]] = zext i8 [[TMP3]] to i64
+; BE-NEXT:    [[TMP38:%.*]] = or i64 [[TMP36]], [[TMP35]]
+; BE-NEXT:    [[TMP39:%.*]] = or i64 [[TMP38]], [[TMP37]]
+; BE-NEXT:    ret i64 [[TMP39]]
 ;
   %tmp3 = load i8, ptr %arg, align 1
   %tmp4 = getelementptr inbounds i8, ptr %arg, i64 1
   %tmp5 = load i8, ptr %tmp4, align 1
   %tmp6 = getelementptr inbounds i8, ptr %arg, i64 2
   %tmp7 = load i8, ptr %tmp6, align 1
   %tmp8 = getelementptr inbounds i8, ptr %arg, i64 3
   %tmp9 = load i8, ptr %tmp8, align 1
;
   %s3 = shl i32 %e3, 16
 
   %o1 = or i32 %e1, %s2
   %o2 = or i32 %o1, %s3
   ret i32 %o2
 }
 
 define i32 @loadCombine_4consecutive_lower_index_comes_before(ptr %p) {
-; ALL-LABEL: @loadCombine_4consecutive_lower_index_comes_before(
-; ALL-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; ALL-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; ALL-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
-; ALL-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
-; ALL-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
-; ALL-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; ALL-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; ALL-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i32
-; ALL-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i32
-; ALL-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i32
-; ALL-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i32
-; ALL-NEXT:    [[S2:%.*]] = shl i32 [[E2]], 8
-; ALL-NEXT:    [[S3:%.*]] = shl i32 [[E3]], 16
-; ALL-NEXT:    [[S4:%.*]] = shl i32 [[E4]], 24
-; ALL-NEXT:    [[O1:%.*]] = or i32 [[E1]], [[S2]]
-; ALL-NEXT:    [[O2:%.*]] = or i32 [[O1]], [[S3]]
-; ALL-NEXT:    [[O3:%.*]] = or i32 [[O2]], [[S4]]
-; ALL-NEXT:    ret i32 [[O3]]
+; LE-LABEL: @loadCombine_4consecutive_lower_index_comes_before(
+; LE-NEXT:    [[L1:%.*]] = load i32, ptr [[P:%.*]], align 1
+; LE-NEXT:    ret i32 [[L1]]
+;
+; BE-LABEL: @loadCombine_4consecutive_lower_index_comes_before(
+; BE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
+; BE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
+; BE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
+; BE-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
+; BE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
+; BE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
+; BE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
+; BE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i32
+; BE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i32
+; BE-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i32
+; BE-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i32
+; BE-NEXT:    [[S2:%.*]] = shl i32 [[E2]], 8
+; BE-NEXT:    [[S3:%.*]] = shl i32 [[E3]], 16
+; BE-NEXT:    [[S4:%.*]] = shl i32 [[E4]], 24
+; BE-NEXT:    [[O1:%.*]] = or i32 [[E1]], [[S2]]
+; BE-NEXT:    [[O2:%.*]] = or i32 [[O1]], [[S3]]
+; BE-NEXT:    [[O3:%.*]] = or i32 [[O2]], [[S4]]
+; BE-NEXT:    ret i32 [[O3]]
 ;
   %p1 = getelementptr i8, ptr %p, i32 1
   %p2 = getelementptr i8, ptr %p, i32 2
   %p3 = getelementptr i8, ptr %p, i32 3
   %l4 = load i8, ptr %p3
   %l3 = load i8, ptr %p2
   %l2 = load i8, ptr %p1
   %l1 = load i8, ptr %p

llvm/test/Transforms/AggressiveInstCombine/X86/or-load.ll

;
   %o1 = or i16 %e1, %s2
   %o2 = or i16 %o1, %s3
   %o3 = or i16 %o2, %s4
   ret i16 %o3
 }
 
 define i32 @loadCombine_4consecutive_rev(ptr %p) {
 ; LE-LABEL: @loadCombine_4consecutive_rev(
-; LE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; LE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; LE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; LE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; LE-NEXT:    [[L3:%.*]] = load i16, ptr [[P2]], align 1
-; LE-NEXT:    [[TMP1:%.*]] = zext i16 [[L3]] to i32
-; LE-NEXT:    [[TMP2:%.*]] = shl i32 [[TMP1]], 16
-; LE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i32
-; LE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i32
-; LE-NEXT:    [[S2:%.*]] = shl i32 [[E2]], 8
-; LE-NEXT:    [[O2:%.*]] = or i32 [[TMP2]], [[S2]]
-; LE-NEXT:    [[O3:%.*]] = or i32 [[O2]], [[E1]]
-; LE-NEXT:    ret i32 [[O3]]
+; LE-NEXT:    [[L1:%.*]] = load i32, ptr [[P:%.*]], align 1
+; LE-NEXT:    ret i32 [[L1]]
 ;
 ; BE-LABEL: @loadCombine_4consecutive_rev(
 ; BE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
 ; BE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
 ; BE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
 ; BE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
 ; BE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
 ; BE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
;
   %o1 = or i32 %s4, %s3
   %o2 = or i32 %o1, %s2
   %o3 = or i32 %o2, %e1
   ret i32 %o3
 }
 
 define i64 @loadCombine_8consecutive_rev(ptr %p) {
 ; LE-LABEL: @loadCombine_8consecutive_rev(
-; LE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; LE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; LE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
-; LE-NEXT:    [[P4:%.*]] = getelementptr i8, ptr [[P]], i32 4
-; LE-NEXT:    [[P5:%.*]] = getelementptr i8, ptr [[P]], i32 5
-; LE-NEXT:    [[P6:%.*]] = getelementptr i8, ptr [[P]], i32 6
-; LE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; LE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; LE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
-; LE-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
-; LE-NEXT:    [[L5:%.*]] = load i8, ptr [[P4]], align 1
-; LE-NEXT:    [[L6:%.*]] = load i8, ptr [[P5]], align 1
-; LE-NEXT:    [[L7:%.*]] = load i16, ptr [[P6]], align 1
-; LE-NEXT:    [[TMP1:%.*]] = zext i16 [[L7]] to i64
-; LE-NEXT:    [[TMP2:%.*]] = shl i64 [[TMP1]], 48
-; LE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i64
-; LE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i64
-; LE-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i64
-; LE-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i64
-; LE-NEXT:    [[E5:%.*]] = zext i8 [[L5]] to i64
-; LE-NEXT:    [[E6:%.*]] = zext i8 [[L6]] to i64
-; LE-NEXT:    [[S2:%.*]] = shl i64 [[E2]], 8
-; LE-NEXT:    [[S3:%.*]] = shl i64 [[E3]], 16
-; LE-NEXT:    [[S4:%.*]] = shl i64 [[E4]], 24
-; LE-NEXT:    [[S5:%.*]] = shl i64 [[E5]], 32
-; LE-NEXT:    [[S6:%.*]] = shl i64 [[E6]], 40
-; LE-NEXT:    [[O6:%.*]] = or i64 [[TMP2]], [[S6]]
-; LE-NEXT:    [[O5:%.*]] = or i64 [[O6]], [[S5]]
-; LE-NEXT:    [[O4:%.*]] = or i64 [[O5]], [[S4]]
-; LE-NEXT:    [[O3:%.*]] = or i64 [[O4]], [[S3]]
-; LE-NEXT:    [[O2:%.*]] = or i64 [[O3]], [[S2]]
-; LE-NEXT:    [[O1:%.*]] = or i64 [[O2]], [[E1]]
-; LE-NEXT:    ret i64 [[O1]]
+; LE-NEXT:    [[L1:%.*]] = load i64, ptr [[P:%.*]], align 1
+; LE-NEXT:    ret i64 [[L1]]
 ;
 ; BE-LABEL: @loadCombine_8consecutive_rev(
 ; BE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
 ; BE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
 ; BE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
 ; BE-NEXT:    [[P4:%.*]] = getelementptr i8, ptr [[P]], i32 4
 ; BE-NEXT:    [[P5:%.*]] = getelementptr i8, ptr [[P]], i32 5
 ; BE-NEXT:    [[P6:%.*]] = getelementptr i8, ptr [[P]], i32 6

 ; LE-NEXT:    [[O5:%.*]] = or i64 [[O6]], [[S5]]
 ; LE-NEXT:    [[O4:%.*]] = or i64 [[O5]], [[S4]]
 ; LE-NEXT:    [[O3:%.*]] = or i64 [[O4]], [[S3]]
 ; LE-NEXT:    [[O2:%.*]] = or i64 [[O3]], [[S2]]
 ; LE-NEXT:    [[O1:%.*]] = or i64 [[O2]], [[S1]]
 ; LE-NEXT:    ret i64 [[O1]]
 ;
 ; BE-LABEL: @loadCombine_8consecutive_rev_BE(
-; BE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; BE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; BE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
-; BE-NEXT:    [[P4:%.*]] = getelementptr i8, ptr [[P]], i32 4
-; BE-NEXT:    [[P5:%.*]] = getelementptr i8, ptr [[P]], i32 5
-; BE-NEXT:    [[P6:%.*]] = getelementptr i8, ptr [[P]], i32 6
-; BE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; BE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; BE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
-; BE-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
-; BE-NEXT:    [[L5:%.*]] = load i8, ptr [[P4]], align 1
-; BE-NEXT:    [[L6:%.*]] = load i8, ptr [[P5]], align 1
-; BE-NEXT:    [[L7:%.*]] = load i16, ptr [[P6]], align 1
-; BE-NEXT:    [[TMP1:%.*]] = zext i16 [[L7]] to i64
-; BE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i64
-; BE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i64
-; BE-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i64
-; BE-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i64
-; BE-NEXT:    [[E5:%.*]] = zext i8 [[L5]] to i64
-; BE-NEXT:    [[E6:%.*]] = zext i8 [[L6]] to i64
-; BE-NEXT:    [[S1:%.*]] = shl i64 [[E1]], 56
-; BE-NEXT:    [[S2:%.*]] = shl i64 [[E2]], 48
-; BE-NEXT:    [[S3:%.*]] = shl i64 [[E3]], 40
-; BE-NEXT:    [[S4:%.*]] = shl i64 [[E4]], 32
-; BE-NEXT:    [[S5:%.*]] = shl i64 [[E5]], 24
-; BE-NEXT:    [[S6:%.*]] = shl i64 [[E6]], 16
-; BE-NEXT:    [[O6:%.*]] = or i64 [[TMP1]], [[S6]]
-; BE-NEXT:    [[O5:%.*]] = or i64 [[O6]], [[S5]]
-; BE-NEXT:    [[O4:%.*]] = or i64 [[O5]], [[S4]]
-; BE-NEXT:    [[O3:%.*]] = or i64 [[O4]], [[S3]]
-; BE-NEXT:    [[O2:%.*]] = or i64 [[O3]], [[S2]]
-; BE-NEXT:    [[O1:%.*]] = or i64 [[O2]], [[S1]]
-; BE-NEXT:    ret i64 [[O1]]
+; BE-NEXT:    [[L1:%.*]] = load i64, ptr [[P:%.*]], align 1
+; BE-NEXT:    ret i64 [[L1]]
 ;
   %p1 = getelementptr i8, ptr %p, i32 1
   %p2 = getelementptr i8, ptr %p, i32 2
   %p3 = getelementptr i8, ptr %p, i32 3
   %p4 = getelementptr i8, ptr %p, i32 4
   %p5 = getelementptr i8, ptr %p, i32 5
   %p6 = getelementptr i8, ptr %p, i32 6
   %p7 = getelementptr i8, ptr %p, i32 7
;
   %o3 = or i64 %o4, %s3
   %o2 = or i64 %o3, %s2
   %o1 = or i64 %o2, %s1
   ret i64 %o1
 }
 
 define i64 @eggs(ptr noundef readonly %arg) {
 ; LE-LABEL: @eggs(
-; LE-NEXT:    [[TMP3:%.*]] = load i8, ptr [[ARG:%.*]], align 1
-; LE-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 1
-; LE-NEXT:    [[TMP5:%.*]] = load i8, ptr [[TMP4]], align 1
-; LE-NEXT:    [[TMP6:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 2
-; LE-NEXT:    [[TMP7:%.*]] = load i8, ptr [[TMP6]], align 1
-; LE-NEXT:    [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 3
-; LE-NEXT:    [[TMP9:%.*]] = load i8, ptr [[TMP8]], align 1
-; LE-NEXT:    [[TMP10:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 4
-; LE-NEXT:    [[TMP11:%.*]] = load i8, ptr [[TMP10]], align 1
-; LE-NEXT:    [[TMP12:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 5
-; LE-NEXT:    [[TMP13:%.*]] = load i8, ptr [[TMP12]], align 1
-; LE-NEXT:    [[TMP14:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 6
-; LE-NEXT:    [[TMP15:%.*]] = load i16, ptr [[TMP14]], align 1
-; LE-NEXT:    [[TMP1:%.*]] = zext i16 [[TMP15]] to i64
-; LE-NEXT:    [[TMP2:%.*]] = shl i64 [[TMP1]], 48
-; LE-NEXT:    [[TMP23:%.*]] = zext i8 [[TMP13]] to i64
-; LE-NEXT:    [[TMP24:%.*]] = shl nuw nsw i64 [[TMP23]], 40
-; LE-NEXT:    [[TMP25:%.*]] = or i64 [[TMP2]], [[TMP24]]
-; LE-NEXT:    [[TMP26:%.*]] = zext i8 [[TMP11]] to i64
-; LE-NEXT:    [[TMP27:%.*]] = shl nuw nsw i64 [[TMP26]], 32
-; LE-NEXT:    [[TMP28:%.*]] = or i64 [[TMP25]], [[TMP27]]
-; LE-NEXT:    [[TMP29:%.*]] = zext i8 [[TMP9]] to i64
-; LE-NEXT:    [[TMP30:%.*]] = shl nuw nsw i64 [[TMP29]], 24
-; LE-NEXT:    [[TMP31:%.*]] = or i64 [[TMP28]], [[TMP30]]
-; LE-NEXT:    [[TMP32:%.*]] = zext i8 [[TMP7]] to i64
-; LE-NEXT:    [[TMP33:%.*]] = shl nuw nsw i64 [[TMP32]], 16
-; LE-NEXT:    [[TMP34:%.*]] = zext i8 [[TMP5]] to i64
-; LE-NEXT:    [[TMP35:%.*]] = shl nuw nsw i64 [[TMP34]], 8
-; LE-NEXT:    [[TMP36:%.*]] = or i64 [[TMP31]], [[TMP33]]
-; LE-NEXT:    [[TMP37:%.*]] = zext i8 [[TMP3]] to i64
-; LE-NEXT:    [[TMP38:%.*]] = or i64 [[TMP36]], [[TMP35]]
-; LE-NEXT:    [[TMP39:%.*]] = or i64 [[TMP38]], [[TMP37]]
-; LE-NEXT:    ret i64 [[TMP39]]
+; LE-NEXT:    [[TMP3:%.*]] = load i64, ptr [[ARG:%.*]], align 1
+; LE-NEXT:    ret i64 [[TMP3]]
 ;
 ; BE-LABEL: @eggs(
 ; BE-NEXT:    [[TMP3:%.*]] = load i8, ptr [[ARG:%.*]], align 1
 ; BE-NEXT:    [[TMP4:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 1
 ; BE-NEXT:    [[TMP5:%.*]] = load i8, ptr [[TMP4]], align 1
 ; BE-NEXT:    [[TMP6:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 2
 ; BE-NEXT:    [[TMP7:%.*]] = load i8, ptr [[TMP6]], align 1
 ; BE-NEXT:    [[TMP8:%.*]] = getelementptr inbounds i8, ptr [[ARG]], i64 3
;
   %s3 = shl i32 %e3, 16
 
   %o1 = or i32 %e1, %s2
   %o2 = or i32 %o1, %s3
   ret i32 %o2
 }
 
 define i32 @loadCombine_4consecutive_lower_index_comes_before(ptr %p) {
-; ALL-LABEL: @loadCombine_4consecutive_lower_index_comes_before(
-; ALL-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
-; ALL-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
-; ALL-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
-; ALL-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
-; ALL-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
-; ALL-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
-; ALL-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
-; ALL-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i32
-; ALL-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i32
-; ALL-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i32
-; ALL-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i32
-; ALL-NEXT:    [[S2:%.*]] = shl i32 [[E2]], 8
-; ALL-NEXT:    [[S3:%.*]] = shl i32 [[E3]], 16
-; ALL-NEXT:    [[S4:%.*]] = shl i32 [[E4]], 24
-; ALL-NEXT:    [[O1:%.*]] = or i32 [[E1]], [[S2]]
-; ALL-NEXT:    [[O2:%.*]] = or i32 [[O1]], [[S3]]
-; ALL-NEXT:    [[O3:%.*]] = or i32 [[O2]], [[S4]]
-; ALL-NEXT:    ret i32 [[O3]]
+; LE-LABEL: @loadCombine_4consecutive_lower_index_comes_before(
+; LE-NEXT:    [[L1:%.*]] = load i32, ptr [[P:%.*]], align 1
+; LE-NEXT:    ret i32 [[L1]]
+;
+; BE-LABEL: @loadCombine_4consecutive_lower_index_comes_before(
+; BE-NEXT:    [[P1:%.*]] = getelementptr i8, ptr [[P:%.*]], i32 1
+; BE-NEXT:    [[P2:%.*]] = getelementptr i8, ptr [[P]], i32 2
+; BE-NEXT:    [[P3:%.*]] = getelementptr i8, ptr [[P]], i32 3
+; BE-NEXT:    [[L4:%.*]] = load i8, ptr [[P3]], align 1
+; BE-NEXT:    [[L3:%.*]] = load i8, ptr [[P2]], align 1
+; BE-NEXT:    [[L2:%.*]] = load i8, ptr [[P1]], align 1
+; BE-NEXT:    [[L1:%.*]] = load i8, ptr [[P]], align 1
+; BE-NEXT:    [[E1:%.*]] = zext i8 [[L1]] to i32
+; BE-NEXT:    [[E2:%.*]] = zext i8 [[L2]] to i32
+; BE-NEXT:    [[E3:%.*]] = zext i8 [[L3]] to i32
+; BE-NEXT:    [[E4:%.*]] = zext i8 [[L4]] to i32
+; BE-NEXT:    [[S2:%.*]] = shl i32 [[E2]], 8
+; BE-NEXT:    [[S3:%.*]] = shl i32 [[E3]], 16
+; BE-NEXT:    [[S4:%.*]] = shl i32 [[E4]], 24
+; BE-NEXT:    [[O1:%.*]] = or i32 [[E1]], [[S2]]
+; BE-NEXT:    [[O2:%.*]] = or i32 [[O1]], [[S3]]
+; BE-NEXT:    [[O3:%.*]] = or i32 [[O2]], [[S4]]
+; BE-NEXT:    ret i32 [[O3]]
 ;
   %p1 = getelementptr i8, ptr %p, i32 1
   %p2 = getelementptr i8, ptr %p, i32 2
   %p3 = getelementptr i8, ptr %p, i32 3
   %l4 = load i8, ptr %p3
   %l3 = load i8, ptr %p2
   %l2 = load i8, ptr %p1
   %l1 = load i8, ptr %p