[DAGCombine] Remove AND in SETCC if we can prove they are unneeded

[dmgreen]

Summary

This is a possible solution for the remainder of PR35875. The idea is that if
we have a SETCC of two ANDs with the same mask, and we can calculate
that the non-mask bits of the operands of the AND are the same, we can
drop the ANDs from the SETCC.
i.e (x&C < y&C) => x<y for any s/u </>/=/etc.

Here is an attempt to prove such a thing at the IR level:
https://rise4fun.com/Alive/ha2

So that seems to be fine. Except that the code in question, the ANDs come
from type legalising a SETCC node, with values that come from anyext loads.
These don't provide upper known bits, even though they are likely to become
zext loads.

Hence all this plumbing to find and return anyext loads, treating them as
zeroext loads during computeKnownBits and adjusting the loads to zeroext
if that is useful for this case.

The left hand side of these tests are not yet checked in. As you can hopefully
see, it helps out quite a bit here. If there are better ways to solve this
problem, I'm all ears.

[samparker]

include/llvm/CodeGen/SelectionDAG.h

1385

If you reorder the default parameters, you could reduce the number of changes needed in the backends.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

2682

Is this safe? Shouldn't the load be checked for a single use or that all the uses on your known bits path?

[spatel]

FWIW - I haven't looked closely at this patch (it might be good independently of PR35875), but I think we can optimize the motivating case even more than what's shown here by reducing the IR:
https://bugs.llvm.org/show_bug.cgi?id=35875#c5

[RKSimon]

I was asked not to add DemandedElts to individual cases in computeKnownBits until you have test coverage - this should probably be the same for AnyToZeroExtLoads. That will be easier if you move the AnyToZeroExtLoads arg to the end to make use of the default nullptr.

[RKSimon]

Random idea - would a more general approach be for us to compute undefined bits as well as one/zero bits - either adding them to the KnownBits struct or add it separately?

[dmgreen]

Thanks all for the reviews. I'm currently hoping that the transform Sanjay suggested will take away my main motivation for this patch. Here are some responses in any case:

I was asked not to add DemandedElts to individual cases in computeKnownBits until you have test coverage - this should probably be the same for AnyToZeroExtLoads.

OK. Noted. By this you mean unit tests for computeKnownBits? Sounds like a sensible idea. I'm guessing the nature of ISel makes this difficult?

Random idea - would a more general approach be for us to compute undefined bits as well as one/zero bits - either adding them to the KnownBits struct or add it separately?

In this specific case (SETCC(AND(%x, C) , AND(%y, C)) where we know that the ~C bits of %x == %y, then the AND can be removed) - we need to know the bits are a specific value. We can't have the two sides being undef and then choosing to differently zero/signext for example. In the case I was looking, x and y were both and(xor(8bit load, -1), 255), so the top bits are all 1 on both sides, so long as we choose to zeroext. I think we need to know the bits, and know which nodes to change to force those bits to be specified. It might well be a good idea for other situations.

include/llvm/CodeGen/SelectionDAG.h

1385

Yeah, makes sense. That's probably best. I was trying to keep depth as the last argument for some reason.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

2682

I don't know a huge amount about the internals of selection dag. My understanding was that if they are anyext then we cannot presume the values of the bits, but then neither can anything else. So long as we set the values of the bits (as we do here), then all other optimisations have to then use the bits as 0.

If I could have just changed the existing LD's to zeroextends, that would be simpler. But I believe the nodes are immutable.

So I don't think we have to check for single uses as the only bits we are changing are the upper bits from unspecified to 0's. Which isn't something that anything else can rely upon, without itself doing the unspecified -> set transform.

[RKSimon]

In D44043#1031702, @dmgreen wrote:

I was asked not to add DemandedElts to individual cases in computeKnownBits until you have test coverage - this should probably be the same for AnyToZeroExtLoads.

OK. Noted. By this you mean unit tests for computeKnownBits? Sounds like a sensible idea. I'm guessing the nature of ISel makes this difficult?

Most of my tests for these cases come from combines that make use of SelectionDAG::SignBitIsZero which uses computeKnownBits - converting uitofp to sitofp etc. in x86 combine-*.ll tests have some good examples.

[RKSimon]

@dmgreen Do you still need this patch or should it be abandoned?

[dmgreen]

Yep sure, with 52177, I no longer have a motivating case for this.

include/llvm/CodeGen/SelectionDAG.h

public:
   bool MaskedValueIsZero(SDValue Op, const APInt &Mask, unsigned Depth = 0)
     const;
 
   /// Determine which bits of Op are known to be either zero or one and return
   /// them in Known. For vectors, the known bits are those that are shared by
   /// every vector element.
   /// Targets can implement the computeKnownBitsForTargetNode method in the
   /// TargetLowering class to allow target nodes to be understood.
-  void computeKnownBits(SDValue Op, KnownBits &Known, unsigned Depth = 0) const;
+  void
+  computeKnownBits(SDValue Op, KnownBits &Known,
+                   SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads = nullptr,
+                   unsigned Depth = 0) const;
 
   /// Determine which bits of Op are known to be either zero or one and return
   /// them in Known. The DemandedElts argument allows us to only collect the
   /// known bits that are shared by the requested vector elements.
   /// Targets can implement the computeKnownBitsForTargetNode method in the
   /// TargetLowering class to allow target nodes to be understood.
-  void computeKnownBits(SDValue Op, KnownBits &Known, const APInt &DemandedElts,
-                        unsigned Depth = 0) const;
+  void
+  computeKnownBits(SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+                   SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads = nullptr,
 
   /// Used to represent the possible overflow behavior of an operation.
   /// Never: the operation cannot overflow.
   /// Always: the operation will always overflow.
   /// Sometime: the operation may or may not overflow.
   enum OverflowKind {
     OFK_Never,
     OFK_Sometime,

include/llvm/CodeGen/TargetLowering.h

public:
   bool SimplifyDemandedVectorElts(SDValue Op, const APInt &DemandedElts,
                                   APInt &KnownUndef, APInt &KnownZero,
                                   DAGCombinerInfo &DCI) const;
 
   /// Determine which of the bits specified in Mask are known to be either zero
   /// or one and return them in the KnownZero/KnownOne bitsets. The DemandedElts
   /// argument allows us to only collect the known bits that are shared by the
   /// requested vector elements.
-  virtual void computeKnownBitsForTargetNode(const SDValue Op,
-                                             KnownBits &Known,
-                                             const APInt &DemandedElts,
-                                             const SelectionDAG &DAG,
-                                             unsigned Depth = 0) const;
+  virtual void computeKnownBitsForTargetNode(
+      const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+      const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
   /// Determine which of the bits of FrameIndex \p FIOp are known to be 0.
   /// Default implementation computes low bits based on alignment
   /// information. This should preserve known bits passed into it.
   virtual void computeKnownBitsForFrameIndex(const SDValue FIOp,
                                              KnownBits &Known,
                                              const APInt &DemandedElts,
                                              const SelectionDAG &DAG,

lib/CodeGen/SelectionDAG/SelectionDAG.cpp

 }
 
 /// MaskedValueIsZero - Return true if 'V & Mask' is known to be zero.  We use
 /// this predicate to simplify operations downstream.  Mask is known to be zero
 /// for bits that V cannot have.
 bool SelectionDAG::MaskedValueIsZero(SDValue Op, const APInt &Mask,
                                      unsigned Depth) const {
   KnownBits Known;
-  computeKnownBits(Op, Known, Depth);
+  computeKnownBits(Op, Known, nullptr, Depth);
   return Mask.isSubsetOf(Known.Zero);
 }
 
 /// Helper function that checks to see if a node is a constant or a
 /// build vector of splat constants at least within the demanded elts.
 static ConstantSDNode *isConstOrDemandedConstSplat(SDValue N,
                                                    const APInt &DemandedElts) {
   if (ConstantSDNode *CN = dyn_cast<ConstantSDNode>(N))
if (ShAmt.ult(V.getScalarValueSizeInBits()))
       return &ShAmt;
   }
   return nullptr;
 }
 
 /// Determine which bits of Op are known to be either zero or one and return
 /// them in Known. For vectors, the known bits are those that are shared by
 /// every vector element.
-void SelectionDAG::computeKnownBits(SDValue Op, KnownBits &Known,
-                                    unsigned Depth) const {
+void SelectionDAG::computeKnownBits(
+    SDValue Op, KnownBits &Known,
+    SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads, unsigned Depth) const {
   EVT VT = Op.getValueType();
   APInt DemandedElts = VT.isVector()
                            ? APInt::getAllOnesValue(VT.getVectorNumElements())
                            : APInt(1, 1);
-  computeKnownBits(Op, Known, DemandedElts, Depth);
+  computeKnownBits(Op, Known, DemandedElts, AnyToZeroExtLoads, Depth);
 }
 
 /// Determine which bits of Op are known to be either zero or one and return
 /// them in Known. The DemandedElts argument allows us to only collect the known
-/// bits that are shared by the requested vector elements.
-void SelectionDAG::computeKnownBits(SDValue Op, KnownBits &Known,
-                                    const APInt &DemandedElts,
-                                    unsigned Depth) const {
+/// bits that are shared by the requested vector elements. AnyToZeroExtLoads
+/// optionally allows anyext loads to be treated as zeroext (top bits zero) and
+/// returned in the SmallPtrSet. They should be converted to zeroext loads for
+/// the returned KnownBits to be valid.
+void SelectionDAG::computeKnownBits(
+    SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads, unsigned Depth) const {
   unsigned BitWidth = Op.getScalarValueSizeInBits();
 
   Known = KnownBits(BitWidth);   // Don't know anything.
 
   if (auto *C = dyn_cast<ConstantSDNode>(Op)) {
     // We know all of the bits for a constant!
     Known.One = C->getAPIntValue();
     Known.Zero = ~Known.One;
case ISD::BUILD_VECTOR:
     assert(NumElts == Op.getValueType().getVectorNumElements() &&
            "Unexpected vector size");
     Known.Zero.setAllBits(); Known.One.setAllBits();
     for (unsigned i = 0, e = Op.getNumOperands(); i != e; ++i) {
       if (!DemandedElts[i])
         continue;
 
       SDValue SrcOp = Op.getOperand(i);
-      computeKnownBits(SrcOp, Known2, Depth + 1);
+      computeKnownBits(SrcOp, Known2, AnyToZeroExtLoads, Depth + 1);
 
       // BUILD_VECTOR can implicitly truncate sources, we must handle this.
       if (SrcOp.getValueSizeInBits() != BitWidth) {
         assert(SrcOp.getValueSizeInBits() > BitWidth &&
                "Expected BUILD_VECTOR implicit truncation");
         Known2 = Known2.trunc(BitWidth);
       }
 
for (unsigned i = 0; i != NumElts; ++i) {
       if ((unsigned)M < NumElts)
         DemandedLHS.setBit((unsigned)M % NumElts);
       else
         DemandedRHS.setBit((unsigned)M % NumElts);
     }
     // Known bits are the values that are shared by every demanded element.
     if (!!DemandedLHS) {
       SDValue LHS = Op.getOperand(0);
-      computeKnownBits(LHS, Known2, DemandedLHS, Depth + 1);
+      computeKnownBits(LHS, Known2, DemandedLHS, AnyToZeroExtLoads, Depth + 1);
       Known.One &= Known2.One;
       Known.Zero &= Known2.Zero;
     }
     // If we don't know any bits, early out.
     if (Known.isUnknown())
       break;
     if (!!DemandedRHS) {
       SDValue RHS = Op.getOperand(1);
-      computeKnownBits(RHS, Known2, DemandedRHS, Depth + 1);
+      computeKnownBits(RHS, Known2, DemandedRHS, AnyToZeroExtLoads, Depth + 1);
       Known.One &= Known2.One;
       Known.Zero &= Known2.Zero;
     }
     break;
   }
   case ISD::CONCAT_VECTORS: {
     // Split DemandedElts and test each of the demanded subvectors.
     Known.Zero.setAllBits(); Known.One.setAllBits();
     EVT SubVectorVT = Op.getOperand(0).getValueType();
     unsigned NumSubVectorElts = SubVectorVT.getVectorNumElements();
     unsigned NumSubVectors = Op.getNumOperands();
     for (unsigned i = 0; i != NumSubVectors; ++i) {
       APInt DemandedSub = DemandedElts.lshr(i * NumSubVectorElts);
       DemandedSub = DemandedSub.trunc(NumSubVectorElts);
       if (!!DemandedSub) {
         SDValue Sub = Op.getOperand(i);
-        computeKnownBits(Sub, Known2, DemandedSub, Depth + 1);
+        computeKnownBits(Sub, Known2, DemandedSub, AnyToZeroExtLoads,
+                         Depth + 1);
         Known.One &= Known2.One;
         Known.Zero &= Known2.Zero;
       }
       // If we don't know any bits, early out.
       if (Known.isUnknown())
         break;
     }
     break;
   }
   case ISD::INSERT_SUBVECTOR: {
     // If we know the element index, demand any elements from the subvector and
     // the remainder from the src its inserted into, otherwise demand them all.
     SDValue Src = Op.getOperand(0);
     SDValue Sub = Op.getOperand(1);
     ConstantSDNode *SubIdx = dyn_cast<ConstantSDNode>(Op.getOperand(2));
     unsigned NumSubElts = Sub.getValueType().getVectorNumElements();
     if (SubIdx && SubIdx->getAPIntValue().ule(NumElts - NumSubElts)) {
       Known.One.setAllBits();
       Known.Zero.setAllBits();
       uint64_t Idx = SubIdx->getZExtValue();
       APInt DemandedSubElts = DemandedElts.extractBits(NumSubElts, Idx);
       if (!!DemandedSubElts) {
-        computeKnownBits(Sub, Known, DemandedSubElts, Depth + 1);
+        computeKnownBits(Sub, Known, DemandedSubElts, AnyToZeroExtLoads,
+                         Depth + 1);
         if (Known.isUnknown())
           break; // early-out.
       }
       APInt SubMask = APInt::getBitsSet(NumElts, Idx, Idx + NumSubElts);
       APInt DemandedSrcElts = DemandedElts & ~SubMask;
       if (!!DemandedSrcElts) {
-        computeKnownBits(Src, Known2, DemandedSrcElts, Depth + 1);
+        computeKnownBits(Src, Known2, DemandedSrcElts, AnyToZeroExtLoads,
+                         Depth + 1);
         Known.One &= Known2.One;
         Known.Zero &= Known2.Zero;
       }
     } else {
-      computeKnownBits(Sub, Known, Depth + 1);
+      computeKnownBits(Sub, Known, AnyToZeroExtLoads, Depth + 1);
       if (Known.isUnknown())
         break; // early-out.
-      computeKnownBits(Src, Known2, Depth + 1);
+      computeKnownBits(Src, Known2, AnyToZeroExtLoads, Depth + 1);
       Known.One &= Known2.One;
       Known.Zero &= Known2.Zero;
     }
     break;
   }
   case ISD::EXTRACT_SUBVECTOR: {
     // If we know the element index, just demand that subvector elements,
     // otherwise demand them all.
     SDValue Src = Op.getOperand(0);
     ConstantSDNode *SubIdx = dyn_cast<ConstantSDNode>(Op.getOperand(1));
     unsigned NumSrcElts = Src.getValueType().getVectorNumElements();
     if (SubIdx && SubIdx->getAPIntValue().ule(NumSrcElts - NumElts)) {
       // Offset the demanded elts by the subvector index.
       uint64_t Idx = SubIdx->getZExtValue();
       APInt DemandedSrc = DemandedElts.zext(NumSrcElts).shl(Idx);
-      computeKnownBits(Src, Known, DemandedSrc, Depth + 1);
+      computeKnownBits(Src, Known, DemandedSrc, AnyToZeroExtLoads, Depth + 1);
     } else {
-      computeKnownBits(Src, Known, Depth + 1);
+      computeKnownBits(Src, Known, AnyToZeroExtLoads, Depth + 1);
     }
     break;
   }
   case ISD::BITCAST: {
     SDValue N0 = Op.getOperand(0);
     EVT SubVT = N0.getValueType();
     unsigned SubBitWidth = SubVT.getScalarSizeInBits();
 
     // Ignore bitcasts from unsupported types.
     if (!(SubVT.isInteger() || SubVT.isFloatingPoint()))
       break;
 
     // Fast handling of 'identity' bitcasts.
     if (BitWidth == SubBitWidth) {
-      computeKnownBits(N0, Known, DemandedElts, Depth + 1);
+      computeKnownBits(N0, Known, DemandedElts, AnyToZeroExtLoads, Depth + 1);
       break;
     }
 
     // Support big-endian targets when it becomes useful.
     bool IsLE = getDataLayout().isLittleEndian();
     if (!IsLE)
       break;
 
     // Bitcast 'small element' vector to 'large element' scalar/vector.
     if ((BitWidth % SubBitWidth) == 0) {
       assert(N0.getValueType().isVector() && "Expected bitcast from vector");
 
       // Collect known bits for the (larger) output by collecting the known
       // bits from each set of sub elements and shift these into place.
       // We need to separately call computeKnownBits for each set of
       // sub elements as the knownbits for each is likely to be different.
       unsigned SubScale = BitWidth / SubBitWidth;
       APInt SubDemandedElts(NumElts * SubScale, 0);
       for (unsigned i = 0; i != NumElts; ++i)
         if (DemandedElts[i])
           SubDemandedElts.setBit(i * SubScale);
 
       for (unsigned i = 0; i != SubScale; ++i) {
-        computeKnownBits(N0, Known2, SubDemandedElts.shl(i),
+        computeKnownBits(N0, Known2, SubDemandedElts.shl(i), AnyToZeroExtLoads,
                          Depth + 1);
         Known.One |= Known2.One.zext(BitWidth).shl(SubBitWidth * i);
         Known.Zero |= Known2.Zero.zext(BitWidth).shl(SubBitWidth * i);
       }
     }
 
     // Bitcast 'large element' scalar/vector to 'small element' vector.
     if ((SubBitWidth % BitWidth) == 0) {
       assert(Op.getValueType().isVector() && "Expected bitcast to vector");
 
       // Collect known bits for the (smaller) output by collecting the known
       // bits from the overlapping larger input elements and extracting the
       // sub sections we actually care about.
       unsigned SubScale = SubBitWidth / BitWidth;
       APInt SubDemandedElts(NumElts / SubScale, 0);
       for (unsigned i = 0; i != NumElts; ++i)
         if (DemandedElts[i])
           SubDemandedElts.setBit(i / SubScale);
 
-      computeKnownBits(N0, Known2, SubDemandedElts, Depth + 1);
+      computeKnownBits(N0, Known2, SubDemandedElts, AnyToZeroExtLoads,
+                       Depth + 1);
 
       Known.Zero.setAllBits(); Known.One.setAllBits();
       for (unsigned i = 0; i != NumElts; ++i)
         if (DemandedElts[i]) {
           unsigned Offset = (i % SubScale) * BitWidth;
           Known.One &= Known2.One.lshr(Offset).trunc(BitWidth);
           Known.Zero &= Known2.Zero.lshr(Offset).trunc(BitWidth);
           // If we don't know any bits, early out.
           if (Known.isUnknown())
             break;
         }
     }
     break;
   }
   case ISD::AND:
     // If either the LHS or the RHS are Zero, the result is zero.
-    computeKnownBits(Op.getOperand(1), Known, DemandedElts, Depth + 1);
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // Output known-1 bits are only known if set in both the LHS & RHS.
     Known.One &= Known2.One;
     // Output known-0 are known to be clear if zero in either the LHS | RHS.
     Known.Zero |= Known2.Zero;
     break;
   case ISD::OR:
-    computeKnownBits(Op.getOperand(1), Known, DemandedElts, Depth + 1);
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // Output known-0 bits are only known if clear in both the LHS & RHS.
     Known.Zero &= Known2.Zero;
     // Output known-1 are known to be set if set in either the LHS | RHS.
     Known.One |= Known2.One;
     break;
   case ISD::XOR: {
-    computeKnownBits(Op.getOperand(1), Known, DemandedElts, Depth + 1);
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // Output known-0 bits are known if clear or set in both the LHS & RHS.
     APInt KnownZeroOut = (Known.Zero & Known2.Zero) | (Known.One & Known2.One);
     // Output known-1 are known to be set if set in only one of the LHS, RHS.
     Known.One = (Known.Zero & Known2.One) | (Known.One & Known2.Zero);
     Known.Zero = KnownZeroOut;
     break;
   }
   case ISD::MUL: {
-    computeKnownBits(Op.getOperand(1), Known, DemandedElts, Depth + 1);
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // If low bits are zero in either operand, output low known-0 bits.
     // Also compute a conservative estimate for high known-0 bits.
     // More trickiness is possible, but this is sufficient for the
     // interesting case of alignment computation.
     unsigned TrailZ = Known.countMinTrailingZeros() +
                       Known2.countMinTrailingZeros();
     unsigned LeadZ =  std::max(Known.countMinLeadingZeros() +
                                Known2.countMinLeadingZeros(),
                                BitWidth) - BitWidth;
 
     Known.resetAll();
     Known.Zero.setLowBits(std::min(TrailZ, BitWidth));
     Known.Zero.setHighBits(std::min(LeadZ, BitWidth));
     break;
   }
   case ISD::UDIV: {
     // For the purposes of computing leading zeros we can conservatively
     // treat a udiv as a logical right shift by the power of 2 known to
     // be less than the denominator.
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     unsigned LeadZ = Known2.countMinLeadingZeros();
 
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     unsigned RHSMaxLeadingZeros = Known2.countMaxLeadingZeros();
     if (RHSMaxLeadingZeros != BitWidth)
       LeadZ = std::min(BitWidth, LeadZ + BitWidth - RHSMaxLeadingZeros - 1);
 
     Known.Zero.setHighBits(LeadZ);
     break;
   }
   case ISD::SELECT:
   case ISD::VSELECT:
-    computeKnownBits(Op.getOperand(2), Known, DemandedElts, Depth+1);
+    computeKnownBits(Op.getOperand(2), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     // If we don't know any bits, early out.
     if (Known.isUnknown())
       break;
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, Depth+1);
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // Only known if known in both the LHS and RHS.
     Known.One &= Known2.One;
     Known.Zero &= Known2.Zero;
     break;
   case ISD::SELECT_CC:
-    computeKnownBits(Op.getOperand(3), Known, DemandedElts, Depth+1);
+    computeKnownBits(Op.getOperand(3), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     // If we don't know any bits, early out.
     if (Known.isUnknown())
       break;
-    computeKnownBits(Op.getOperand(2), Known2, DemandedElts, Depth+1);
+    computeKnownBits(Op.getOperand(2), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // Only known if known in both the LHS and RHS.
     Known.One &= Known2.One;
     Known.Zero &= Known2.Zero;
     break;
   case ISD::SMULO:
   case ISD::UMULO:
   case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS:
case ISD::SETCC:
     // If we know the result of a setcc has the top bits zero, use this info.
     if (TLI->getBooleanContents(Op.getOperand(0).getValueType()) ==
             TargetLowering::ZeroOrOneBooleanContent &&
         BitWidth > 1)
       Known.Zero.setBitsFrom(1);
     break;
   case ISD::SHL:
     if (const APInt *ShAmt = getValidShiftAmountConstant(Op)) {
-      computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+      computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                       Depth + 1);
       unsigned Shift = ShAmt->getZExtValue();
       Known.Zero <<= Shift;
       Known.One <<= Shift;
       // Low bits are known zero.
       Known.Zero.setLowBits(Shift);
     }
     break;
   case ISD::SRL:
     if (const APInt *ShAmt = getValidShiftAmountConstant(Op)) {
-      computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+      computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                       Depth + 1);
       unsigned Shift = ShAmt->getZExtValue();
       Known.Zero.lshrInPlace(Shift);
       Known.One.lshrInPlace(Shift);
       // High bits are known zero.
       Known.Zero.setHighBits(Shift);
     } else if (auto *BV = dyn_cast<BuildVectorSDNode>(Op.getOperand(1))) {
       // If the shift amount is a vector of constants see if we can bound
       // the number of upper zero bits.
if (const APInt *ShAmt = getValidShiftAmountConstant(Op)) {
         break;
       }
 
       Known.Zero.setHighBits(ShiftAmountMin);
     }
     break;
   case ISD::SRA:
     if (const APInt *ShAmt = getValidShiftAmountConstant(Op)) {
-      computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+      computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                       Depth + 1);
       unsigned Shift = ShAmt->getZExtValue();
       // Sign extend known zero/one bit (else is unknown).
       Known.Zero.ashrInPlace(Shift);
       Known.One.ashrInPlace(Shift);
     }
     break;
   case ISD::SIGN_EXTEND_INREG: {
     EVT EVT = cast<VTSDNode>(Op.getOperand(1))->getVT();
     unsigned EBits = EVT.getScalarSizeInBits();
 
     // Sign extension.  Compute the demanded bits in the result that are not
     // present in the input.
     APInt NewBits = APInt::getHighBitsSet(BitWidth, BitWidth - EBits);
 
     APInt InSignMask = APInt::getSignMask(EBits);
     APInt InputDemandedBits = APInt::getLowBitsSet(BitWidth, EBits);
 
     // If the sign extended bits are demanded, we know that the sign
     // bit is demanded.
     InSignMask = InSignMask.zext(BitWidth);
     if (NewBits.getBoolValue())
       InputDemandedBits |= InSignMask;
 
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     Known.One &= InputDemandedBits;
     Known.Zero &= InputDemandedBits;
 
     // If the sign bit of the input is known set or clear, then we know the
     // top bits of the result.
     if (Known.Zero.intersects(InSignMask)) {        // Input sign bit known clear
       Known.Zero |= NewBits;
       Known.One  &= ~NewBits;
     } else if (Known.One.intersects(InSignMask)) {  // Input sign bit known set
       Known.One  |= NewBits;
       Known.Zero &= ~NewBits;
     } else {                              // Input sign bit unknown
       Known.Zero &= ~NewBits;
       Known.One  &= ~NewBits;
     }
     break;
   }
   case ISD::CTTZ:
   case ISD::CTTZ_ZERO_UNDEF: {
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     // If we have a known 1, its position is our upper bound.
     unsigned PossibleTZ = Known2.countMaxTrailingZeros();
     unsigned LowBits = Log2_32(PossibleTZ) + 1;
     Known.Zero.setBitsFrom(LowBits);
     break;
   }
   case ISD::CTLZ:
   case ISD::CTLZ_ZERO_UNDEF: {
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     // If we have a known 1, its position is our upper bound.
     unsigned PossibleLZ = Known2.countMaxLeadingZeros();
     unsigned LowBits = Log2_32(PossibleLZ) + 1;
     Known.Zero.setBitsFrom(LowBits);
     break;
   }
   case ISD::CTPOP: {
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     // If we know some of the bits are zero, they can't be one.
     unsigned PossibleOnes = Known2.countMaxPopulation();
     Known.Zero.setBitsFrom(Log2_32(PossibleOnes) + 1);
     break;
   }
   case ISD::LOAD: {
     LoadSDNode *LD = cast<LoadSDNode>(Op);
     // If this is a ZEXTLoad and we are looking at the loaded value.
     if (ISD::isZEXTLoad(Op.getNode()) && Op.getResNo() == 0) {
       EVT VT = LD->getMemoryVT();
       unsigned MemBits = VT.getScalarSizeInBits();
       Known.Zero.setBitsFrom(MemBits);
+    // If this is a EXTLoad and we are passed AnyToZeroExtLoads, treat
+    // the load as zero extended.
+    } else if (AnyToZeroExtLoads && ISD::isEXTLoad(Op.getNode()) &&
+               Op.getResNo() == 0) {
+      EVT VT = LD->getMemoryVT();
+      unsigned MemBits = VT.getScalarSizeInBits();
+      Known.Zero.setBitsFrom(MemBits);
+      AnyToZeroExtLoads->insert(LD);
     } else if (const MDNode *Ranges = LD->getRanges()) {
       if (LD->getExtensionType() == ISD::NON_EXTLOAD)
         computeKnownBitsFromRangeMetadata(*Ranges, Known);
     }
     break;
   }
   case ISD::ZERO_EXTEND_VECTOR_INREG: {
     EVT InVT = Op.getOperand(0).getValueType();
     APInt InDemandedElts = DemandedElts.zext(InVT.getVectorNumElements());
-    computeKnownBits(Op.getOperand(0), Known, InDemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, InDemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     Known = Known.zext(BitWidth);
     Known.Zero.setBitsFrom(InVT.getScalarSizeInBits());
     break;
   }
   case ISD::ZERO_EXTEND: {
     EVT InVT = Op.getOperand(0).getValueType();
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     Known = Known.zext(BitWidth);
     Known.Zero.setBitsFrom(InVT.getScalarSizeInBits());
     break;
   }
   // TODO ISD::SIGN_EXTEND_VECTOR_INREG
   case ISD::SIGN_EXTEND: {
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     // If the sign bit is known to be zero or one, then sext will extend
     // it to the top bits, else it will just zext.
     Known = Known.sext(BitWidth);
     break;
   }
   case ISD::ANY_EXTEND: {
-    computeKnownBits(Op.getOperand(0), Known, Depth+1);
+    computeKnownBits(Op.getOperand(0), Known, AnyToZeroExtLoads, Depth + 1);
     Known = Known.zext(BitWidth);
     break;
   }
   case ISD::TRUNCATE: {
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     Known = Known.trunc(BitWidth);
     break;
   }
   case ISD::AssertZext: {
     EVT VT = cast<VTSDNode>(Op.getOperand(1))->getVT();
     APInt InMask = APInt::getLowBitsSet(BitWidth, VT.getSizeInBits());
-    computeKnownBits(Op.getOperand(0), Known, Depth+1);
+    computeKnownBits(Op.getOperand(0), Known, AnyToZeroExtLoads, Depth + 1);
     Known.Zero |= (~InMask);
     Known.One  &= (~Known.Zero);
     break;
   }
   case ISD::FGETSIGN:
     // All bits are zero except the low bit.
     Known.Zero.setBitsFrom(1);
     break;
if (ConstantSDNode *CLHS = isConstOrConstSplat(Op.getOperand(0))) {
       // We know that the top bits of C-X are clear if X contains less bits
       // than C (i.e. no wrap-around can happen).  For example, 20-X is
       // positive if we can prove that X is >= 0 and < 16.
       if (CLHS->getAPIntValue().isNonNegative()) {
         unsigned NLZ = (CLHS->getAPIntValue()+1).countLeadingZeros();
         // NLZ can't be BitWidth with no sign bit
         APInt MaskV = APInt::getHighBitsSet(BitWidth, NLZ+1);
         computeKnownBits(Op.getOperand(1), Known2, DemandedElts,
-                         Depth + 1);
+                         AnyToZeroExtLoads, Depth + 1);
 
         // If all of the MaskV bits are known to be zero, then we know the
         // output top bits are zero, because we now know that the output is
         // from [0-C].
         if ((Known2.Zero & MaskV) == MaskV) {
           unsigned NLZ2 = CLHS->getAPIntValue().countLeadingZeros();
           // Top bits known zero.
           Known.Zero.setHighBits(NLZ2);
         }
       }
     }
 
     // If low bits are know to be zero in both operands, then we know they are
     // going to be 0 in the result. Both addition and complement operations
     // preserve the low zero bits.
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     unsigned KnownZeroLow = Known2.countMinTrailingZeros();
     if (KnownZeroLow == 0)
       break;
 
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     KnownZeroLow = std::min(KnownZeroLow, Known2.countMinTrailingZeros());
     Known.Zero.setLowBits(KnownZeroLow);
     break;
   }
   case ISD::UADDO:
   case ISD::SADDO:
   case ISD::ADDCARRY:
     if (Op.getResNo() == 1) {
void SelectionDAG::computeKnownBits(
   case ISD::ADDE: {
     // Output known-0 bits are known if clear or set in both the low clear bits
     // common to both LHS & RHS.  For example, 8+(X<<3) is known to have the
     // low 3 bits clear.
     // Output known-0 bits are also known if the top bits of each input are
     // known to be clear. For example, if one input has the top 10 bits clear
     // and the other has the top 8 bits clear, we know the top 7 bits of the
     // output must be clear.
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     unsigned KnownZeroHigh = Known2.countMinLeadingZeros();
     unsigned KnownZeroLow = Known2.countMinTrailingZeros();
 
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts,
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
                      Depth + 1);
     KnownZeroHigh = std::min(KnownZeroHigh, Known2.countMinLeadingZeros());
     KnownZeroLow = std::min(KnownZeroLow, Known2.countMinTrailingZeros());
 
     if (Opcode == ISD::ADDE || Opcode == ISD::ADDCARRY) {
       // With ADDE and ADDCARRY, a carry bit may be added in, so we can only
       // use this information if we know (at least) that the low two bits are
       // clear. We then return to the caller that the low bit is unknown but
       // that other bits are known zero.
       if (KnownZeroLow >= 2)
         Known.Zero.setBits(1, KnownZeroLow);
       break;
     }
 
     Known.Zero.setLowBits(KnownZeroLow);
     if (KnownZeroHigh > 1)
       Known.Zero.setHighBits(KnownZeroHigh - 1);
     break;
   }
   case ISD::SREM:
     if (ConstantSDNode *Rem = isConstOrConstSplat(Op.getOperand(1))) {
       const APInt &RA = Rem->getAPIntValue().abs();
       if (RA.isPowerOf2()) {
         APInt LowBits = RA - 1;
-        computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+        computeKnownBits(Op.getOperand(0), Known2, DemandedElts,
+                         AnyToZeroExtLoads, Depth + 1);
 
         // The low bits of the first operand are unchanged by the srem.
         Known.Zero = Known2.Zero & LowBits;
         Known.One = Known2.One & LowBits;
 
         // If the first operand is non-negative or has all low bits zero, then
         // the upper bits are all zero.
         if (Known2.Zero[BitWidth-1] || ((Known2.Zero & LowBits) == LowBits))
           Known.Zero |= ~LowBits;
 
         // If the first operand is negative and not all low bits are zero, then
         // the upper bits are all one.
         if (Known2.One[BitWidth-1] && ((Known2.One & LowBits) != 0))
           Known.One |= ~LowBits;
         assert((Known.Zero & Known.One) == 0&&"Bits known to be one AND zero?");
       }
     }
     break;
   case ISD::UREM: {
     if (ConstantSDNode *Rem = isConstOrConstSplat(Op.getOperand(1))) {
       const APInt &RA = Rem->getAPIntValue();
       if (RA.isPowerOf2()) {
         APInt LowBits = (RA - 1);
-        computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+        computeKnownBits(Op.getOperand(0), Known2, DemandedElts,
+                         AnyToZeroExtLoads, Depth + 1);
 
         // The upper bits are all zero, the lower ones are unchanged.
         Known.Zero = Known2.Zero | ~LowBits;
         Known.One = Known2.One & LowBits;
         break;
       }
     }
 
     // Since the result is less than or equal to either operand, any leading
     // zero bits in either operand must also exist in the result.
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     uint32_t Leaders =
         std::max(Known.countMinLeadingZeros(), Known2.countMinLeadingZeros());
     Known.resetAll();
     Known.Zero.setHighBits(Leaders);
     break;
   }
   case ISD::EXTRACT_ELEMENT: {
-    computeKnownBits(Op.getOperand(0), Known, Depth+1);
+    computeKnownBits(Op.getOperand(0), Known, AnyToZeroExtLoads, Depth + 1);
     const unsigned Index = Op.getConstantOperandVal(1);
     const unsigned BitWidth = Op.getValueSizeInBits();
 
     // Remove low part of known bits mask
     Known.Zero = Known.Zero.getHiBits(Known.Zero.getBitWidth() - Index * BitWidth);
     Known.One = Known.One.getHiBits(Known.One.getBitWidth() - Index * BitWidth);
 
     // Remove high part of known bit mask
case ISD::EXTRACT_VECTOR_ELT: {
     // anything about the extended bits.
     if (BitWidth > EltBitWidth)
       Known = Known.trunc(EltBitWidth);
     ConstantSDNode *ConstEltNo = dyn_cast<ConstantSDNode>(EltNo);
     if (ConstEltNo && ConstEltNo->getAPIntValue().ult(NumSrcElts)) {
       // If we know the element index, just demand that vector element.
       unsigned Idx = ConstEltNo->getZExtValue();
       APInt DemandedElt = APInt::getOneBitSet(NumSrcElts, Idx);
-      computeKnownBits(InVec, Known, DemandedElt, Depth + 1);
+      computeKnownBits(InVec, Known, DemandedElt, AnyToZeroExtLoads, Depth + 1);
     } else {
       // Unknown element index, so ignore DemandedElts and demand them all.
-      computeKnownBits(InVec, Known, Depth + 1);
+      computeKnownBits(InVec, Known, AnyToZeroExtLoads, Depth + 1);
     }
     if (BitWidth > EltBitWidth)
       Known = Known.zext(BitWidth);
     break;
   }
   case ISD::INSERT_VECTOR_ELT: {
     SDValue InVec = Op.getOperand(0);
     SDValue InVal = Op.getOperand(1);
     SDValue EltNo = Op.getOperand(2);
 
     ConstantSDNode *CEltNo = dyn_cast<ConstantSDNode>(EltNo);
     if (CEltNo && CEltNo->getAPIntValue().ult(NumElts)) {
       // If we know the element index, split the demand between the
       // source vector and the inserted element.
       Known.Zero = Known.One = APInt::getAllOnesValue(BitWidth);
       unsigned EltIdx = CEltNo->getZExtValue();
 
       // If we demand the inserted element then add its common known bits.
       if (DemandedElts[EltIdx]) {
-        computeKnownBits(InVal, Known2, Depth + 1);
+        computeKnownBits(InVal, Known2, AnyToZeroExtLoads, Depth + 1);
         Known.One &= Known2.One.zextOrTrunc(Known.One.getBitWidth());
         Known.Zero &= Known2.Zero.zextOrTrunc(Known.Zero.getBitWidth());
       }
 
       // If we demand the source vector then add its common known bits, ensuring
       // that we don't demand the inserted element.
       APInt VectorElts = DemandedElts & ~(APInt::getOneBitSet(NumElts, EltIdx));
       if (!!VectorElts) {
-        computeKnownBits(InVec, Known2, VectorElts, Depth + 1);
+        computeKnownBits(InVec, Known2, VectorElts, AnyToZeroExtLoads,
+                         Depth + 1);
         Known.One &= Known2.One;
         Known.Zero &= Known2.Zero;
       }
     } else {
       // Unknown element index, so ignore DemandedElts and demand them all.
-      computeKnownBits(InVec, Known, Depth + 1);
-      computeKnownBits(InVal, Known2, Depth + 1);
+      computeKnownBits(InVec, Known, AnyToZeroExtLoads, Depth + 1);
+      computeKnownBits(InVal, Known2, AnyToZeroExtLoads, Depth + 1);
       Known.One &= Known2.One.zextOrTrunc(Known.One.getBitWidth());
       Known.Zero &= Known2.Zero.zextOrTrunc(Known.Zero.getBitWidth());
     }
     break;
   }
   case ISD::BITREVERSE: {
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     Known.Zero = Known2.Zero.reverseBits();
     Known.One = Known2.One.reverseBits();
     break;
   }
   case ISD::BSWAP: {
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     Known.Zero = Known2.Zero.byteSwap();
     Known.One = Known2.One.byteSwap();
     break;
   }
   case ISD::ABS: {
-    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // If the source's MSB is zero then we know the rest of the bits already.
     if (Known2.isNonNegative()) {
       Known.Zero = Known2.Zero;
       Known.One = Known2.One;
       break;
     }
 
     // We only know that the absolute values's MSB will be zero iff there is
     // a set bit that isn't the sign bit (otherwise it could be INT_MIN).
     Known2.One.clearSignBit();
     if (Known2.One.getBoolValue()) {
       Known.Zero = APInt::getSignMask(BitWidth);
       break;
     }
     break;
   }
   case ISD::UMIN: {
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
 
     // UMIN - we know that the result will have the maximum of the
     // known zero leading bits of the inputs.
     unsigned LeadZero = Known.countMinLeadingZeros();
     LeadZero = std::max(LeadZero, Known2.countMinLeadingZeros());
 
     Known.Zero &= Known2.Zero;
     Known.One &= Known2.One;
     Known.Zero.setHighBits(LeadZero);
     break;
   }
   case ISD::UMAX: {
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts,
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
                      Depth + 1);
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, Depth + 1);
 
     // UMAX - we know that the result will have the maximum of the
     // known one leading bits of the inputs.
     unsigned LeadOne = Known.countMinLeadingOnes();
     LeadOne = std::max(LeadOne, Known2.countMinLeadingOnes());
 
     Known.Zero &= Known2.Zero;
     Known.One &= Known2.One;
if (CstLow && CstHigh) {
         if (ValueLow.isNonNegative() && ValueHigh.isNonNegative()) {
           Known.Zero.setHighBits(MinSignBits);
           break;
         }
       }
     }
 
     // Fallback - just get the shared known bits of the operands.
-    computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(0), Known, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     if (Known.isUnknown()) break; // Early-out
-    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, Depth + 1);
+    computeKnownBits(Op.getOperand(1), Known2, DemandedElts, AnyToZeroExtLoads,
+                     Depth + 1);
     Known.Zero &= Known2.Zero;
     Known.One &= Known2.One;
     break;
   }
   case ISD::FrameIndex:
   case ISD::TargetFrameIndex:
     TLI->computeKnownBitsForFrameIndex(Op, Known, DemandedElts, *this, Depth);
     break;
 
   default:
     if (Opcode < ISD::BUILTIN_OP_END)
       break;
     LLVM_FALLTHROUGH;
   case ISD::INTRINSIC_WO_CHAIN:
   case ISD::INTRINSIC_W_CHAIN:
   case ISD::INTRINSIC_VOID:
     // Allow the target to implement this method for its nodes.
-    TLI->computeKnownBitsForTargetNode(Op, Known, DemandedElts, *this, Depth);
+    TLI->computeKnownBitsForTargetNode(Op, Known, DemandedElts, *this,
+                                       AnyToZeroExtLoads, Depth);
     break;
   }
 
   assert(!Known.hasConflict() && "Bits known to be one AND zero?");
 }
 
 SelectionDAG::OverflowKind SelectionDAG::computeOverflowKind(SDValue N0,
                                                              SDValue N1) const {
case ISD::ADDC:
     // is, at worst, one more bit than the inputs.
     Tmp = ComputeNumSignBits(Op.getOperand(0), Depth+1);
     if (Tmp == 1) return 1;  // Early out.
 
     // Special case decrementing a value (ADD X, -1):
     if (ConstantSDNode *CRHS = dyn_cast<ConstantSDNode>(Op.getOperand(1)))
       if (CRHS->isAllOnesValue()) {
         KnownBits Known;
-        computeKnownBits(Op.getOperand(0), Known, Depth+1);
+        computeKnownBits(Op.getOperand(0), Known, nullptr, Depth + 1);
 
         // If the input is known to be 0 or 1, the output is 0/-1, which is all
         // sign bits set.
         if ((Known.Zero | 1).isAllOnesValue())
           return VTBits;
 
         // If we are subtracting one from a positive number, there is no carry
         // out of the result.
         if (Known.isNonNegative())
           return Tmp;
       }
 
     Tmp2 = ComputeNumSignBits(Op.getOperand(1), Depth+1);
     if (Tmp2 == 1) return 1;
     return std::min(Tmp, Tmp2)-1;
 
   case ISD::SUB:
     Tmp2 = ComputeNumSignBits(Op.getOperand(1), Depth+1);
     if (Tmp2 == 1) return 1;
 
     // Handle NEG.
     if (ConstantSDNode *CLHS = isConstOrConstSplat(Op.getOperand(0)))
       if (CLHS->isNullValue()) {
         KnownBits Known;
-        computeKnownBits(Op.getOperand(1), Known, Depth+1);
+        computeKnownBits(Op.getOperand(1), Known, nullptr, Depth + 1);
         // If the input is known to be 0 or 1, the output is 0/-1, which is all
         // sign bits set.
         if ((Known.Zero | 1).isAllOnesValue())
           return VTBits;
 
         // If the input is known to be positive (the sign bit is known clear),
         // the output of the NEG has the same number of sign bits as the input.
         if (Known.isNonNegative())
unsigned NumBits =
         TLI->ComputeNumSignBitsForTargetNode(Op, DemandedElts, *this, Depth);
     if (NumBits > 1)
       FirstAnswer = std::max(FirstAnswer, NumBits);
   }
 
   // Finally, if we can prove that the top bits of the result are 0's or 1's,
   // use this information.
   KnownBits Known;
-  computeKnownBits(Op, Known, DemandedElts, Depth);
+  computeKnownBits(Op, Known, DemandedElts, nullptr, Depth);
 
   APInt Mask;
   if (Known.isNonNegative()) {        // sign bit is 0
     Mask = Known.Zero;
   } else if (Known.isNegative()) {  // sign bit is 1;
     Mask = Known.One;
   } else {
     // Nothing known.

lib/CodeGen/SelectionDAG/TargetLowering.cpp

bool TargetLowering::SimplifyDemandedBits(SDValue Op,
   }
 
   // Other users may use these bits.
   EVT VT = Op.getValueType();
   if (!Op.getNode()->hasOneUse() && !AssumeSingleUse) {
     if (Depth != 0) {
       // If not at the root, Just compute the Known bits to
       // simplify things downstream.
-      TLO.DAG.computeKnownBits(Op, Known, Depth);
+      TLO.DAG.computeKnownBits(Op, Known, nullptr, Depth);
       return false;
     }
     // If this is the root being simplified, allow it to have multiple uses,
     // just set the NewMask to all bits.
     NewMask = APInt::getAllOnesValue(BitWidth);
   } else if (DemandedMask == 0) {
     // Not demanding any bits from Op.
     if (!Op.isUndef())
case ISD::AND:
     // If the RHS is a constant, check to see if the LHS would be zero without
     // using the bits from the RHS.  Below, we use knowledge about the RHS to
     // simplify the LHS, here we're using information from the LHS to simplify
     // the RHS.
     if (ConstantSDNode *RHSC = isConstOrConstSplat(Op.getOperand(1))) {
       SDValue Op0 = Op.getOperand(0);
       KnownBits LHSKnown;
       // Do not increment Depth here; that can cause an infinite loop.
-      TLO.DAG.computeKnownBits(Op0, LHSKnown, Depth);
+      TLO.DAG.computeKnownBits(Op0, LHSKnown, nullptr, Depth);
       // If the LHS already has zeros where RHSC does, this 'and' is dead.
       if ((LHSKnown.Zero & NewMask) == (~RHSC->getAPIntValue() & NewMask))
         return TLO.CombineTo(Op, Op0);
 
       // If any of the set bits in the RHS are known zero on the LHS, shrink
       // the constant.
       if (ShrinkDemandedConstant(Op, ~LHSKnown.Zero & NewMask, TLO))
         return true;
if (!TLO.LegalOperations() && !VT.isVector() &&
         unsigned ShVal = Op.getValueSizeInBits() - 1;
         SDValue ShAmt = TLO.DAG.getConstant(ShVal, dl, VT);
         return TLO.CombineTo(Op, TLO.DAG.getNode(ISD::SHL, dl, VT, Sign, ShAmt));
       }
     }
     // If this is a bitcast, let computeKnownBits handle it.  Only do this on a
     // recursive call where Known may be useful to the caller.
     if (Depth > 0) {
-      TLO.DAG.computeKnownBits(Op, Known, Depth);
+      TLO.DAG.computeKnownBits(Op, Known, nullptr, Depth);
       return false;
     }
     break;
   case ISD::ADD:
   case ISD::MUL:
   case ISD::SUB: {
     // Add, Sub, and Mul don't demand any bits in positions beyond that
     // of the highest bit demanded of them.
if (C && !C->isAllOnesValue() && !C->isOne() &&
       SDValue NewOp = TLO.DAG.getNode(Op.getOpcode(), dl, VT, Op0, Neg1, Flags);
       return TLO.CombineTo(Op, NewOp);
     }
 
     LLVM_FALLTHROUGH;
   }
   default:
     // Just use computeKnownBits to compute output bits.
-    TLO.DAG.computeKnownBits(Op, Known, Depth);
+    TLO.DAG.computeKnownBits(Op, Known, nullptr, Depth);
     break;
   }
 
   // If we know the value of all of the demanded bits, return this as a
   // constant.
   if (NewMask.isSubsetOf(Known.Zero|Known.One)) {
     // Avoid folding to a constant if any OpaqueConstant is involved.
     const SDNode *N = Op.getNode();
bool TargetLowering::SimplifyDemandedVectorElts(
   }
 
   assert((KnownUndef & KnownZero) == 0 && "Elements flagged as undef AND zero");
   return false;
 }
 
 /// Determine which of the bits specified in Mask are known to be either zero or
 /// one and return them in the Known.
-void TargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
-                                                   KnownBits &Known,
-                                                   const APInt &DemandedElts,
-                                                   const SelectionDAG &DAG,
-                                                   unsigned Depth) const {
+void TargetLowering::computeKnownBitsForTargetNode(
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
   assert((Op.getOpcode() >= ISD::BUILTIN_OP_END ||
           Op.getOpcode() == ISD::INTRINSIC_WO_CHAIN ||
           Op.getOpcode() == ISD::INTRINSIC_W_CHAIN ||
           Op.getOpcode() == ISD::INTRINSIC_VOID) &&
          "Should use MaskedValueIsZero if you don't know whether Op"
          " is a target node!");
   Known.resetAll();
 }
if (VT.getScalarType() != MVT::i1) {
         DCI.AddToWorklist(N0.getNode());
       // FIXME: If running after legalize, we probably can't do this.
       ISD::NodeType ExtendCode = getExtendForContent(getBooleanContents(OpVT));
       N0 = DAG.getNode(ExtendCode, dl, VT, N0);
     }
     return N0;
   }
 
+  // Test if both operands are AND's, with the same mask, and the
+  // non-mask bits are the same (often all zero). In this case we
+  // can drop the ands.
+  if (N0.getOpcode() == ISD::AND && N1.getOpcode() == ISD::AND &&
+      N0.getNode()->hasOneUse() && N1.getNode()->hasOneUse()) {
+    SDValue N0LHS = N0.getOperand(0);
+    SDValue N0RHS = N0.getOperand(1);
+    SDValue N1LHS = N1.getOperand(0);
+    SDValue N1RHS = N1.getOperand(1);
+
+    if (isa<ConstantSDNode>(N0LHS))
+      std::swap(N0LHS, N0RHS);
+    if (isa<ConstantSDNode>(N1LHS))
+      std::swap(N1LHS, N1RHS);
+
+    if (!isa<ConstantSDNode>(N0RHS) || !isa<ConstantSDNode>(N1RHS))
+      return SDValue();
+
+    APInt AndMask = cast<ConstantSDNode>(N0RHS)->getAPIntValue();
+    if (cast<ConstantSDNode>(N1RHS)->getAPIntValue() != AndMask)
+      return SDValue();
+
+    KnownBits KB0, KB1;
+    llvm::SmallPtrSet<LoadSDNode *, 4> AnyToZeroExtLoads;
+    DAG.computeKnownBits(N0LHS, KB0, &AnyToZeroExtLoads);
+    DAG.computeKnownBits(N1LHS, KB1, &AnyToZeroExtLoads);
+
+    // Check we know something about all bits
+    if (!(KB0.Zero | KB0.One | AndMask).isAllOnesValue())
+      return SDValue();
+
+    // All non-mask bits must be the same from N0 and N1.
+    if ((KB0.Zero & ~AndMask) != (KB1.Zero & ~AndMask) ||
+        (KB0.One & ~AndMask) != (KB1.One & ~AndMask))
+      return SDValue();
+
+    // Transform anyext loads -> zeroext loads
+    for (auto *LD : AnyToZeroExtLoads) {
+      SDValue Load = DAG.getExtLoad(
+          ISD::ZEXTLOAD, SDLoc(LD), LD->getValueType(0), LD->getChain(),
+          LD->getBasePtr(), LD->getPointerInfo(), LD->getMemoryVT(),
+          LD->getAlignment(), LD->getMemOperand()->getFlags(), LD->getAAInfo());
+      DAG.ReplaceAllUsesOfValueWith(SDValue(LD, 0), Load);
+      if (LD == N0LHS.getNode())
+        N0LHS = Load;
+      if (LD == N1LHS.getNode())
+        N1LHS = Load;
+    }
+
+    return DAG.getSetCC(dl, VT, N0LHS, N1LHS, Cond);
+  }
+
   // Could not fold it.
   return SDValue();
 }
 
 /// Returns true (and the GlobalValue and the offset) if the node is a
 /// GlobalAddress + offset.
 bool TargetLowering::isGAPlusOffset(SDNode *N, const GlobalValue *&GA,
                                     int64_t &Offset) const {

lib/Target/AArch64/AArch64ISelLowering.h

public:
   /// Selects the correct CCAssignFn for a given CallingConvention value.
   CCAssignFn *CCAssignFnForCall(CallingConv::ID CC, bool IsVarArg) const;
 
   /// Selects the correct CCAssignFn for a given CallingConvention value.
   CCAssignFn *CCAssignFnForReturn(CallingConv::ID CC) const;
 
   /// Determine which of the bits specified in Mask are known to be either zero
   /// or one and return them in the KnownZero/KnownOne bitsets.
-  void computeKnownBitsForTargetNode(const SDValue Op, KnownBits &Known,
-                                     const APInt &DemandedElts,
-                                     const SelectionDAG &DAG,
-                                     unsigned Depth = 0) const override;
+  void computeKnownBitsForTargetNode(
+      const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+      const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
   bool targetShrinkDemandedConstant(SDValue Op, const APInt &Demanded,
                                     TargetLoweringOpt &TLO) const override;
 
   MVT getScalarShiftAmountTy(const DataLayout &DL, EVT) const override;
 
   /// Returns true if the target allows unaligned memory accesses of the
   /// specified type.

lib/Target/AArch64/AArch64ISelLowering.cpp

if (!C)
     return false;
   uint64_t Imm = C->getZExtValue();
   return optimizeLogicalImm(Op, Size, Imm, Demanded, TLO, NewOpc);
 }
 
 /// computeKnownBitsForTargetNode - Determine which of the bits specified in
 /// Mask are known to be either zero or one and return them Known.
 void AArch64TargetLowering::computeKnownBitsForTargetNode(
-    const SDValue Op, KnownBits &Known,
-    const APInt &DemandedElts, const SelectionDAG &DAG, unsigned Depth) const {
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
+    unsigned Depth) const {
   switch (Op.getOpcode()) {
   default:
     break;
   case AArch64ISD::CSEL: {
     KnownBits Known2;
-    DAG.computeKnownBits(Op->getOperand(0), Known, Depth + 1);
-    DAG.computeKnownBits(Op->getOperand(1), Known2, Depth + 1);
+    DAG.computeKnownBits(Op->getOperand(0), Known, AnyToZeroExtLoads,
+                         Depth + 1);
+    DAG.computeKnownBits(Op->getOperand(1), Known2, AnyToZeroExtLoads,
+                         Depth + 1);
     Known.Zero &= Known2.Zero;
     Known.One &= Known2.One;
     break;
   }
   case ISD::INTRINSIC_W_CHAIN: {
     ConstantSDNode *CN = cast<ConstantSDNode>(Op->getOperand(1));
     Intrinsic::ID IntID = static_cast<Intrinsic::ID>(CN->getZExtValue());
     switch (IntID) {

lib/Target/AMDGPU/AMDGPUISelLowering.h

SDValue getRecipEstimate(SDValue Operand, SelectionDAG &DAG, int Enabled,
                            int &RefinementSteps) const override;
 
   virtual SDNode *PostISelFolding(MachineSDNode *N,
                                   SelectionDAG &DAG) const = 0;
 
   /// \brief Determine which of the bits specified in \p Mask are known to be
   /// either zero or one and return them in the \p KnownZero and \p KnownOne
   /// bitsets.
-  void computeKnownBitsForTargetNode(const SDValue Op,
-                                     KnownBits &Known,
-                                     const APInt &DemandedElts,
-                                     const SelectionDAG &DAG,
-                                     unsigned Depth = 0) const override;
+  void computeKnownBitsForTargetNode(
+      const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+      const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
   unsigned ComputeNumSignBitsForTargetNode(SDValue Op, const APInt &DemandedElts,
                                            const SelectionDAG &DAG,
                                            unsigned Depth = 0) const override;
 
   /// \brief Helper function that adds Reg to the LiveIn list of the DAG's
   /// MachineFunction.
   ///

lib/Target/AMDGPU/AMDGPUISelLowering.cpp

SDValue AMDGPUTargetLowering::getRecipEstimate(SDValue Operand,
 
   // TODO: There is also f64 rcp instruction, but the documentation is less
   // clear on its precision.
 
   return SDValue();
 }
 
 void AMDGPUTargetLowering::computeKnownBitsForTargetNode(
-    const SDValue Op, KnownBits &Known,
-    const APInt &DemandedElts, const SelectionDAG &DAG, unsigned Depth) const {
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
+    unsigned Depth) const {
 
   Known.resetAll(); // Don't know anything.
 
   unsigned Opc = Op.getOpcode();
 
   switch (Opc) {
   default:
     break;
case AMDGPUISD::FP16_ZEXT: {
 
     // High bits are zero.
     Known.Zero = APInt::getHighBitsSet(BitWidth, BitWidth - 16);
     break;
   }
   case AMDGPUISD::MUL_U24:
   case AMDGPUISD::MUL_I24: {
     KnownBits LHSKnown, RHSKnown;
-    DAG.computeKnownBits(Op.getOperand(0), LHSKnown, Depth + 1);
-    DAG.computeKnownBits(Op.getOperand(1), RHSKnown, Depth + 1);
+    DAG.computeKnownBits(Op.getOperand(0), LHSKnown, AnyToZeroExtLoads,
+                         Depth + 1);
+    DAG.computeKnownBits(Op.getOperand(1), RHSKnown, AnyToZeroExtLoads,
+                         Depth + 1);
 
     unsigned TrailZ = LHSKnown.countMinTrailingZeros() +
                       RHSKnown.countMinTrailingZeros();
     Known.Zero.setLowBits(std::min(TrailZ, 32u));
 
     unsigned LHSValBits = 32 - std::max(LHSKnown.countMinSignBits(), 8u);
     unsigned RHSValBits = 32 - std::max(RHSKnown.countMinSignBits(), 8u);
     unsigned MaxValBits = std::min(LHSValBits + RHSValBits, 32u);

lib/Target/ARM/ARMISelLowering.h

public:
 
     /// getPostIndexedAddressParts - returns true by value, base pointer and
     /// offset pointer and addressing mode by reference if this node can be
     /// combined with a load / store to form a post-indexed load / store.
     bool getPostIndexedAddressParts(SDNode *N, SDNode *Op, SDValue &Base,
                                     SDValue &Offset, ISD::MemIndexedMode &AM,
                                     SelectionDAG &DAG) const override;
 
-    void computeKnownBitsForTargetNode(const SDValue Op, KnownBits &Known,
-                                       const APInt &DemandedElts,
-                                       const SelectionDAG &DAG,
-                                       unsigned Depth) const override;
+    void computeKnownBitsForTargetNode(
+        const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+        SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
-
     bool ExpandInlineAsm(CallInst *CI) const override;
 
     ConstraintType getConstraintType(StringRef Constraint) const override;
 
     /// Examine constraint string and operand type and determine a weight value.
     /// The operand object must already have been set up with the operand type.
     ConstraintWeight getSingleConstraintMatchWeight(
       AsmOperandInfo &info, const char *constraint) const override;

lib/Target/ARM/ARMISelLowering.cpp

if (Ptr != Base) {
     if (Ptr != Base)
       return false;
   }
 
   AM = isInc ? ISD::POST_INC : ISD::POST_DEC;
   return true;
 }
 
-void ARMTargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
-                                                      KnownBits &Known,
-                                                      const APInt &DemandedElts,
-                                                      const SelectionDAG &DAG,
-                                                      unsigned Depth) const {
+void ARMTargetLowering::computeKnownBitsForTargetNode(
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
   unsigned BitWidth = Known.getBitWidth();
   Known.resetAll();
   switch (Op.getOpcode()) {
   default: break;
   case ARMISD::ADDC:
   case ARMISD::ADDE:
   case ARMISD::SUBC:
   case ARMISD::SUBE:
     // Special cases when we convert a carry to a boolean.
     if (Op.getResNo() == 0) {
       SDValue LHS = Op.getOperand(0);
       SDValue RHS = Op.getOperand(1);
       // (ADDE 0, 0, C) will give us a single bit.
       if (Op->getOpcode() == ARMISD::ADDE && isNullConstant(LHS) &&
           isNullConstant(RHS)) {
         Known.Zero |= APInt::getHighBitsSet(BitWidth, BitWidth - 1);
         return;
       }
     }
     break;
   case ARMISD::CMOV: {
     // Bits are known zero/one if known on the LHS and RHS.
-    DAG.computeKnownBits(Op.getOperand(0), Known, Depth+1);
+    DAG.computeKnownBits(Op.getOperand(0), Known, AnyToZeroExtLoads, Depth + 1);
     if (Known.isUnknown())
       return;
 
     KnownBits KnownRHS;
-    DAG.computeKnownBits(Op.getOperand(1), KnownRHS, Depth+1);
+    DAG.computeKnownBits(Op.getOperand(1), KnownRHS, AnyToZeroExtLoads,
+                         Depth + 1);
     Known.Zero &= KnownRHS.Zero;
     Known.One  &= KnownRHS.One;
     return;
   }
   case ISD::INTRINSIC_W_CHAIN: {
     ConstantSDNode *CN = cast<ConstantSDNode>(Op->getOperand(1));
     Intrinsic::ID IntID = static_cast<Intrinsic::ID>(CN->getZExtValue());
     switch (IntID) {
     default: return;
     case Intrinsic::arm_ldaex:
     case Intrinsic::arm_ldrex: {
       EVT VT = cast<MemIntrinsicSDNode>(Op)->getMemoryVT();
       unsigned MemBits = VT.getScalarSizeInBits();
       Known.Zero |= APInt::getHighBitsSet(BitWidth, BitWidth - MemBits);
       return;
     }
     }
   }
   case ARMISD::BFI: {
     // Conservatively, we can recurse down the first operand
     // and just mask out all affected bits.
-    DAG.computeKnownBits(Op.getOperand(0), Known, Depth + 1);
+    DAG.computeKnownBits(Op.getOperand(0), Known, AnyToZeroExtLoads, Depth + 1);
 
     // The operand to BFI is already a mask suitable for removing the bits it
     // sets.
     ConstantSDNode *CI = cast<ConstantSDNode>(Op.getOperand(2));
     const APInt &Mask = CI->getAPIntValue();
     Known.Zero &= Mask;
     Known.One &= Mask;
     return;

lib/Target/Lanai/LanaiISelLowering.h

public:
   getSingleConstraintMatchWeight(AsmOperandInfo &Info,
                                  const char *Constraint) const override;
   void LowerAsmOperandForConstraint(SDValue Op, std::string &Constraint,
                                     std::vector<SDValue> &Ops,
                                     SelectionDAG &DAG) const override;
 
   SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
 
-  void computeKnownBitsForTargetNode(const SDValue Op, KnownBits &Known,
-                                     const APInt &DemandedElts,
-                                     const SelectionDAG &DAG,
-                                     unsigned Depth = 0) const override;
+  void computeKnownBitsForTargetNode(
+      const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+      const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
 private:
   SDValue LowerCCCCallTo(SDValue Chain, SDValue Callee,
                          CallingConv::ID CallConv, bool IsVarArg,
                          bool IsTailCall,
                          const SmallVectorImpl<ISD::OutputArg> &Outs,
                          const SmallVectorImpl<SDValue> &OutVals,
                          const SmallVectorImpl<ISD::InputArg> &Ins,

lib/Target/Lanai/LanaiISelLowering.cpp

case ISD::SUB:
     return PerformSUBCombine(N, DCI);
   }
 
   return SDValue();
 }
 
 void LanaiTargetLowering::computeKnownBitsForTargetNode(
     const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
-    const SelectionDAG &DAG, unsigned Depth) const {
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
+    unsigned Depth) const {
   unsigned BitWidth = Known.getBitWidth();
   switch (Op.getOpcode()) {
   default:
     break;
   case LanaiISD::SETCC:
     Known = KnownBits(BitWidth);
     Known.Zero.setBits(1, BitWidth);
     break;
   case LanaiISD::SELECT_CC:
     KnownBits Known2;
-    DAG.computeKnownBits(Op->getOperand(0), Known, Depth + 1);
-    DAG.computeKnownBits(Op->getOperand(1), Known2, Depth + 1);
+    DAG.computeKnownBits(Op->getOperand(0), Known, AnyToZeroExtLoads,
+                         Depth + 1);
+    DAG.computeKnownBits(Op->getOperand(1), Known2, AnyToZeroExtLoads,
+                         Depth + 1);
     Known.Zero &= Known2.Zero;
     Known.One &= Known2.One;
     break;
   }
 }

lib/Target/PowerPC/PPCISelLowering.h

public:
     SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
 
     SDValue BuildSDIVPow2(SDNode *N, const APInt &Divisor, SelectionDAG &DAG,
                           std::vector<SDNode *> *Created) const override;
 
     unsigned getRegisterByName(const char* RegName, EVT VT,
                                SelectionDAG &DAG) const override;
 
-    void computeKnownBitsForTargetNode(const SDValue Op,
-                                       KnownBits &Known,
-                                       const APInt &DemandedElts,
-                                       const SelectionDAG &DAG,
-                                       unsigned Depth = 0) const override;
+    void computeKnownBitsForTargetNode(
+        const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+        SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
     unsigned getPrefLoopAlignment(MachineLoop *ML) const override;
 
     bool shouldInsertFencesForAtomic(const Instruction *I) const override {
       return true;
     }
 
     Instruction *emitLeadingFence(IRBuilder<> &Builder, Instruction *Inst,

lib/Target/PowerPC/PPCISelLowering.cpp

PPCTargetLowering::BuildSDIVPow2(SDNode *N, const APInt &Divisor,
 
   return Op;
 }
 
 //===----------------------------------------------------------------------===//
 // Inline Assembly Support
 //===----------------------------------------------------------------------===//
 
-void PPCTargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
-                                                      KnownBits &Known,
-                                                      const APInt &DemandedElts,
-                                                      const SelectionDAG &DAG,
-                                                      unsigned Depth) const {
+void PPCTargetLowering::computeKnownBitsForTargetNode(
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
   Known.resetAll();
   switch (Op.getOpcode()) {
   default: break;
   case PPCISD::LBRX: {
     // lhbrx is known to have the top bits cleared out.
     if (cast<VTSDNode>(Op.getOperand(2))->getVT() == MVT::i16)
       Known.Zero = 0xFFFF0000;
     break;

lib/Target/Sparc/SparcISelLowering.h

public:
     SparcTargetLowering(const TargetMachine &TM, const SparcSubtarget &STI);
     SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
     
     bool useSoftFloat() const override;
     
     /// computeKnownBitsForTargetNode - Determine which of the bits specified
     /// in Mask are known to be either zero or one and return them in the
     /// KnownZero/KnownOne bitsets.
-    void computeKnownBitsForTargetNode(const SDValue Op,
-                                       KnownBits &Known,
-                                       const APInt &DemandedElts,
-                                       const SelectionDAG &DAG,
-                                       unsigned Depth = 0) const override;
+    void computeKnownBitsForTargetNode(
+        const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+        SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
     MachineBasicBlock *
     EmitInstrWithCustomInserter(MachineInstr &MI,
                                 MachineBasicBlock *MBB) const override;
 
     const char *getTargetNodeName(unsigned Opcode) const override;
 
     ConstraintType getConstraintType(StringRef Constraint) const override;

lib/Target/Sparc/SparcISelLowering.cpp

EVT SparcTargetLowering::getSetCCResultType(const DataLayout &, LLVMContext &,
   if (!VT.isVector())
     return MVT::i32;
   return VT.changeVectorElementTypeToInteger();
 }
 
 /// isMaskedValueZeroForTargetNode - Return true if 'Op & Mask' is known to
 /// be zero. Op is expected to be a target specific node. Used by DAG
 /// combiner.
-void SparcTargetLowering::computeKnownBitsForTargetNode
-                                (const SDValue Op,
-                                 KnownBits &Known,
-                                 const APInt &DemandedElts,
-                                 const SelectionDAG &DAG,
-                                 unsigned Depth) const {
+void SparcTargetLowering::computeKnownBitsForTargetNode(
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
   KnownBits Known2;
   Known.resetAll();
 
   switch (Op.getOpcode()) {
   default: break;
   case SPISD::SELECT_ICC:
   case SPISD::SELECT_XCC:
   case SPISD::SELECT_FCC:
-    DAG.computeKnownBits(Op.getOperand(1), Known, Depth+1);
-    DAG.computeKnownBits(Op.getOperand(0), Known2, Depth+1);
+    DAG.computeKnownBits(Op.getOperand(1), Known, AnyToZeroExtLoads, Depth + 1);
+    DAG.computeKnownBits(Op.getOperand(0), Known2, AnyToZeroExtLoads,
+                         Depth + 1);
 
     // Only known if known in both the LHS and RHS.
     Known.One &= Known2.One;
     Known.Zero &= Known2.Zero;
     break;
   }
 }
 

lib/Target/SystemZ/SystemZISelLowering.h

public:
   SDValue LowerReturn(SDValue Chain, CallingConv::ID CallConv, bool IsVarArg,
                       const SmallVectorImpl<ISD::OutputArg> &Outs,
                       const SmallVectorImpl<SDValue> &OutVals, const SDLoc &DL,
                       SelectionDAG &DAG) const override;
   SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
 
   /// Determine which of the bits specified in Mask are known to be either
   /// zero or one and return them in the KnownZero/KnownOne bitsets.
-  void computeKnownBitsForTargetNode(const SDValue Op,
-                                     KnownBits &Known,
-                                     const APInt &DemandedElts,
-                                     const SelectionDAG &DAG,
-                                     unsigned Depth = 0) const override;
+  void computeKnownBitsForTargetNode(
+      const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+      const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
   ISD::NodeType getExtendForAtomicOps() const override {
     return ISD::ANY_EXTEND;
   }
 
   bool supportSwiftError() const override {
     return true;
   }

lib/Target/SystemZ/SystemZISelLowering.cpp

SDValue SystemZTargetLowering::PerformDAGCombine(SDNode *N,
   case ISD::ROTL:               return combineSHIFTROT(N, DCI);
   case SystemZISD::BR_CCMASK:   return combineBR_CCMASK(N, DCI);
   case SystemZISD::SELECT_CCMASK: return combineSELECT_CCMASK(N, DCI);
   }
 
   return SDValue();
 }
 
-void
-SystemZTargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
-                                                     KnownBits &Known,
-                                                     const APInt &DemandedElts,
-                                                     const SelectionDAG &DAG,
-                                                     unsigned Depth) const {
+void SystemZTargetLowering::computeKnownBitsForTargetNode(
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
   unsigned BitWidth = Known.getBitWidth();
 
   Known.resetAll();
   switch (Op.getOpcode()) {
   case SystemZISD::SELECT_CCMASK: {
     KnownBits TrueKnown(BitWidth), FalseKnown(BitWidth);
-    DAG.computeKnownBits(Op.getOperand(0), TrueKnown, Depth + 1);
-    DAG.computeKnownBits(Op.getOperand(1), FalseKnown, Depth + 1);
+    DAG.computeKnownBits(Op.getOperand(0), TrueKnown, AnyToZeroExtLoads,
+                         Depth + 1);
+    DAG.computeKnownBits(Op.getOperand(1), FalseKnown, AnyToZeroExtLoads,
+                         Depth + 1);
     Known.Zero = TrueKnown.Zero & FalseKnown.Zero;
     Known.One = TrueKnown.One & FalseKnown.One;
     break;
   }
 
   default:
     break;
   }

lib/Target/X86/X86ISelLowering.h

public:
     EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
                            EVT VT) const override;
 
     bool targetShrinkDemandedConstant(SDValue Op, const APInt &Demanded,
                                       TargetLoweringOpt &TLO) const override;
 
     /// Determine which of the bits specified in Mask are known to be either
     /// zero or one and return them in the KnownZero/KnownOne bitsets.
-    void computeKnownBitsForTargetNode(const SDValue Op,
-                                       KnownBits &Known,
-                                       const APInt &DemandedElts,
-                                       const SelectionDAG &DAG,
-                                       unsigned Depth = 0) const override;
+    void computeKnownBitsForTargetNode(
+        const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+        SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
     /// Determine the number of bits in the operation that are sign bits.
     unsigned ComputeNumSignBitsForTargetNode(SDValue Op,
                                              const APInt &DemandedElts,
                                              const SelectionDAG &DAG,
                                              unsigned Depth) const override;
 
     SDValue unwrapAddress(SDValue N) const override;

lib/Target/X86/X86ISelLowering.cpp

X86TargetLowering::targetShrinkDemandedConstant(SDValue Op,
 
   // Replace the constant with the zero extend mask.
   SDLoc DL(Op);
   SDValue NewC = TLO.DAG.getConstant(ZeroExtendMask, DL, VT);
   SDValue NewOp = TLO.DAG.getNode(ISD::AND, DL, VT, Op.getOperand(0), NewC);
   return TLO.CombineTo(Op, NewOp);
 }
 
-void X86TargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
-                                                      KnownBits &Known,
-                                                      const APInt &DemandedElts,
-                                                      const SelectionDAG &DAG,
-                                                      unsigned Depth) const {
+void X86TargetLowering::computeKnownBitsForTargetNode(
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
   unsigned BitWidth = Known.getBitWidth();
   unsigned Opc = Op.getOpcode();
   EVT VT = Op.getValueType();
   assert((Opc >= ISD::BUILTIN_OP_END ||
           Opc == ISD::INTRINSIC_WO_CHAIN ||
           Opc == ISD::INTRINSIC_W_CHAIN ||
           Opc == ISD::INTRINSIC_VOID) &&
          "Should use MaskedValueIsZero if you don't know whether Op"
case X86ISD::MOVMSK: {
     break;
   }
   case X86ISD::PEXTRB:
   case X86ISD::PEXTRW: {
     SDValue Src = Op.getOperand(0);
     EVT SrcVT = Src.getValueType();
     APInt DemandedElt = APInt::getOneBitSet(SrcVT.getVectorNumElements(),
                                             Op.getConstantOperandVal(1));
-    DAG.computeKnownBits(Src, Known, DemandedElt, Depth + 1);
+    DAG.computeKnownBits(Src, Known, DemandedElt, AnyToZeroExtLoads, Depth + 1);
     Known = Known.zextOrTrunc(BitWidth);
     Known.Zero.setBitsFrom(SrcVT.getScalarSizeInBits());
     break;
   }
   case X86ISD::VSHLI:
   case X86ISD::VSRLI: {
     if (auto *ShiftImm = dyn_cast<ConstantSDNode>(Op.getOperand(1))) {
       if (ShiftImm->getAPIntValue().uge(VT.getScalarSizeInBits())) {
         Known.setAllZero();
         break;
       }
 
-      DAG.computeKnownBits(Op.getOperand(0), Known, DemandedElts, Depth + 1);
+      DAG.computeKnownBits(Op.getOperand(0), Known, DemandedElts,
+                           AnyToZeroExtLoads, Depth + 1);
       unsigned ShAmt = ShiftImm->getZExtValue();
       if (Opc == X86ISD::VSHLI) {
         Known.Zero <<= ShAmt;
         Known.One <<= ShAmt;
         // Low bits are known zero.
         Known.Zero.setLowBits(ShAmt);
       } else {
         Known.Zero.lshrInPlace(ShAmt);
case X86ISD::VZEXT: {
 
     EVT SrcVT = N0.getValueType();
     unsigned InNumElts = SrcVT.getVectorNumElements();
     unsigned InBitWidth = SrcVT.getScalarSizeInBits();
     assert(InNumElts >= NumElts && "Illegal VZEXT input");
 
     Known = KnownBits(InBitWidth);
     APInt DemandedSrcElts = APInt::getLowBitsSet(InNumElts, NumElts);
-    DAG.computeKnownBits(N0, Known, DemandedSrcElts, Depth + 1);
+    DAG.computeKnownBits(N0, Known, DemandedSrcElts, AnyToZeroExtLoads,
+                         Depth + 1);
     Known = Known.zext(BitWidth);
     Known.Zero.setBitsFrom(InBitWidth);
     break;
   }
   case X86ISD::CMOV: {
-    DAG.computeKnownBits(Op.getOperand(1), Known, Depth+1);
+    DAG.computeKnownBits(Op.getOperand(1), Known, AnyToZeroExtLoads, Depth + 1);
     // If we don't know any bits, early out.
     if (Known.isUnknown())
       break;
     KnownBits Known2;
-    DAG.computeKnownBits(Op.getOperand(0), Known2, Depth+1);
+    DAG.computeKnownBits(Op.getOperand(0), Known2, AnyToZeroExtLoads,
+                         Depth + 1);
 
     // Only known if known in both the LHS and RHS.
     Known.One &= Known2.One;
     Known.Zero &= Known2.Zero;
     break;
   }
   case X86ISD::UDIVREM8_ZEXT_HREG:
     // TODO: Support more than just the zero extended bits?

lib/Target/X86/X86InstrCompiler.td

 // into "disjoint bits" pseudo ops.
 
 // Treat an 'or' node is as an 'add' if the or'ed bits are known to be zero.
 def or_is_add : PatFrag<(ops node:$lhs, node:$rhs), (or node:$lhs, node:$rhs),[{
   if (ConstantSDNode *CN = dyn_cast<ConstantSDNode>(N->getOperand(1)))
     return CurDAG->MaskedValueIsZero(N->getOperand(0), CN->getAPIntValue());
 
   KnownBits Known0;
-  CurDAG->computeKnownBits(N->getOperand(0), Known0, 0);
+  CurDAG->computeKnownBits(N->getOperand(0), Known0, nullptr, 0);
   KnownBits Known1;
-  CurDAG->computeKnownBits(N->getOperand(1), Known1, 0);
+  CurDAG->computeKnownBits(N->getOperand(1), Known1, nullptr, 0);
   return (~Known0.Zero & ~Known1.Zero) == 0;
 }]>;
 
 
 // (or x1, x2) -> (add x1, x2) if two operands are known not to share bits.
 // Try this before the selecting to OR.
 let AddedComplexity = 5, SchedRW = [WriteALU] in {
 

lib/Target/XCore/XCoreISelLowering.h

getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
                                  StringRef Constraint, MVT VT) const override;
 
     // Expand specifics
     SDValue TryExpandADDWithMul(SDNode *Op, SelectionDAG &DAG) const;
     SDValue ExpandADDSUB(SDNode *Op, SelectionDAG &DAG) const;
 
     SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
 
-    void computeKnownBitsForTargetNode(const SDValue Op,
-                                       KnownBits &Known,
-                                       const APInt &DemandedElts,
-                                       const SelectionDAG &DAG,
-                                       unsigned Depth = 0) const override;
+    void computeKnownBitsForTargetNode(
+        const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+        SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
 
     SDValue
     LowerFormalArguments(SDValue Chain, CallingConv::ID CallConv, bool isVarArg,
                          const SmallVectorImpl<ISD::InputArg> &Ins,
                          const SDLoc &dl, SelectionDAG &DAG,
                          SmallVectorImpl<SDValue> &InVals) const override;
 
     SDValue

lib/Target/XCore/XCoreISelLowering.cpp

if (LoadSDNode *LD = dyn_cast<LoadSDNode>(ST->getValue())) {
       }
     }
     break;
   }
   }
   return SDValue();
 }
 
-void XCoreTargetLowering::computeKnownBitsForTargetNode(const SDValue Op,
-                                                        KnownBits &Known,
-                                                        const APInt &DemandedElts,
-                                                        const SelectionDAG &DAG,
-                                                        unsigned Depth) const {
+void XCoreTargetLowering::computeKnownBitsForTargetNode(
+    const SDValue Op, KnownBits &Known, const APInt &DemandedElts,
+    const SelectionDAG &DAG, SmallPtrSetImpl<LoadSDNode *> *AnyToZeroExtLoads,
   Known.resetAll();
   switch (Op.getOpcode()) {
   default: break;
   case XCoreISD::LADD:
   case XCoreISD::LSUB:
     if (Op.getResNo() == 1) {
       // Top bits of carry / borrow are clear.
       Known.Zero = APInt::getHighBitsSet(Known.getBitWidth(),

test/CodeGen/Thumb/setcc_xor.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv6m-eabi -asm-verbose=false %s -o - | FileCheck %s --check-prefix=CHECK-V6M
 ; RUN: llc -mtriple=thumbv7m-eabi -asm-verbose=false %s -o - | FileCheck %s --check-prefix=CHECK-V7M
 
 define i8 @test1(i8 zeroext %x, i8 zeroext %y) {
 ; CHECK-V6M-LABEL: test1:
-; CHECK-V6M:         movs r2, #255
-; CHECK-V6M-NEXT:    mov r3, r2
-; CHECK-V6M-NEXT:    bics r3, r1
-; CHECK-V6M-NEXT:    bics r2, r0
-; CHECK-V6M-NEXT:    mvns r0, r0
+; CHECK-V6M:         mvns r0, r0
 ; CHECK-V6M-NEXT:    mvns r1, r1
-; CHECK-V6M-NEXT:    cmp r2, r3
+; CHECK-V6M-NEXT:    cmp r0, r1
 ; CHECK-V6M-NEXT:    bls .LBB0_2
 ; CHECK-V6M-NEXT:    mov r0, r1
 ; CHECK-V6M-NEXT:  .LBB0_2:
 ; CHECK-V6M-NEXT:    bx lr
 ;
 ; CHECK-V7M-LABEL: test1:
-; CHECK-V7M:         mvns r1, r1
-; CHECK-V7M-NEXT:    mvns r0, r0
-; CHECK-V7M-NEXT:    uxtb r2, r1
-; CHECK-V7M-NEXT:    uxtb r3, r0
-; CHECK-V7M-NEXT:    cmp r3, r2
+; CHECK-V7M:         mvns r2, r0
+; CHECK-V7M-NEXT:    mvns r0, r1
+; CHECK-V7M-NEXT:    cmp r2, r0
 ; CHECK-V7M-NEXT:    it ls
-; CHECK-V7M-NEXT:    movls r1, r0
-; CHECK-V7M-NEXT:    mov r0, r1
+; CHECK-V7M-NEXT:    movls r0, r2
 ; CHECK-V7M-NEXT:    bx lr
 entry:
   %nx = xor i8 %x, 255
   %ny = xor i8 %y, 255
   %cmp = icmp ule i8 %nx, %ny
   %sel = select i1 %cmp, i8 %nx, i8 %ny
   ret i8 %sel
 }
 
 define void @test2(i8* %X, i8* %Y) {
 ; CHECK-V6M-LABEL: test2:
-; CHECK-V6M:         .save {r4, r5, r7, lr}
-; CHECK-V6M-NEXT:    push {r4, r5, r7, lr}
-; CHECK-V6M-NEXT:    ldrb r2, [r1]
-; CHECK-V6M-NEXT:    movs r4, #255
-; CHECK-V6M-NEXT:    mov r5, r4
-; CHECK-V6M-NEXT:    bics r5, r2
-; CHECK-V6M-NEXT:    ldrb r3, [r0]
-; CHECK-V6M-NEXT:    bics r4, r3
-; CHECK-V6M-NEXT:    mvns r3, r3
+; CHECK-V6M:         .save {r4, lr}
+; CHECK-V6M-NEXT:    push {r4, lr}
+; CHECK-V6M-NEXT:    ldrb r2, [r0]
 ; CHECK-V6M-NEXT:    mvns r2, r2
-; CHECK-V6M-NEXT:    cmp r4, r5
-; CHECK-V6M-NEXT:    mov r4, r3
-; CHECK-V6M-NEXT:    blo .LBB1_2
+; CHECK-V6M-NEXT:    ldrb r3, [r1]
+; CHECK-V6M-NEXT:    mvns r3, r3
+; CHECK-V6M-NEXT:    cmp r2, r3
 ; CHECK-V6M-NEXT:    mov r4, r2
+; CHECK-V6M-NEXT:    blo .LBB1_2
+; CHECK-V6M-NEXT:    mov r4, r3
 ; CHECK-V6M-NEXT:  .LBB1_2:
-; CHECK-V6M-NEXT:    subs r3, r3, r4
-; CHECK-V6M-NEXT:    strb r3, [r0]
-; CHECK-V6M-NEXT:    subs r0, r2, r4
+; CHECK-V6M-NEXT:    subs r2, r2, r4
+; CHECK-V6M-NEXT:    strb r2, [r0]
+; CHECK-V6M-NEXT:    subs r0, r3, r4
 ; CHECK-V6M-NEXT:    strb r0, [r1]
-; CHECK-V6M-NEXT:    pop {r4, r5, r7, pc}
+; CHECK-V6M-NEXT:    pop {r4, pc}
 ;
 ; CHECK-V7M-LABEL: test2:
 ; CHECK-V7M:         .save {r7, lr}
 ; CHECK-V7M-NEXT:    push {r7, lr}
-; CHECK-V7M-NEXT:    ldrb r3, [r0]
-; CHECK-V7M-NEXT:    ldrb r2, [r1]
-; CHECK-V7M-NEXT:    mvns r3, r3
-; CHECK-V7M-NEXT:    mvn.w lr, r2
-; CHECK-V7M-NEXT:    uxtb r2, r3
-; CHECK-V7M-NEXT:    uxtb.w r12, lr
-; CHECK-V7M-NEXT:    cmp r2, r12
-; CHECK-V7M-NEXT:    mov r2, lr
+; CHECK-V7M-NEXT:    ldrb r3, [r1]
+; CHECK-V7M-NEXT:    ldrb r2, [r0]
+; CHECK-V7M-NEXT:    mvn.w lr, r3
+; CHECK-V7M-NEXT:    mvn.w r12, r2
+; CHECK-V7M-NEXT:    cmp r12, lr
+; CHECK-V7M-NEXT:    mov r3, lr
 ; CHECK-V7M-NEXT:    it lo
-; CHECK-V7M-NEXT:    movlo r2, r3
-; CHECK-V7M-NEXT:    subs r3, r3, r2
-; CHECK-V7M-NEXT:    strb r3, [r0]
-; CHECK-V7M-NEXT:    sub.w r0, lr, r2
+; CHECK-V7M-NEXT:    movlo r3, r12
+; CHECK-V7M-NEXT:    sub.w r2, r12, r3
+; CHECK-V7M-NEXT:    strb r2, [r0]
+; CHECK-V7M-NEXT:    sub.w r0, lr, r3
 ; CHECK-V7M-NEXT:    strb r0, [r1]
 ; CHECK-V7M-NEXT:    pop {r7, pc}
 entry:
   %x = load i8, i8* %X
   %y = load i8, i8* %Y
   %nx = xor i8 %x, 255
   %ny = xor i8 %y, 255
   %cmp = icmp ult i8 %nx, %ny
   %sel = select i1 %cmp, i8 %nx, i8 %ny
   %xr = sub i8 %nx, %sel
   %yr = sub i8 %ny, %sel
   store i8 %xr, i8* %X
   store i8 %yr, i8* %Y
   ret void
 }
 
 
 define void @testloop(i32 %I, i8* nocapture readonly %A, i8* nocapture %B) {
 ; CHECK-V6M-LABEL: testloop:
-; CHECK-V6M:         .save {r4, r5, r6, r7, lr}
-; CHECK-V6M-NEXT:    push {r4, r5, r6, r7, lr}
-; CHECK-V6M-NEXT:    .pad #4
-; CHECK-V6M-NEXT:    sub sp, #4
+; CHECK-V6M:         .save {r4, r5, r6, lr}
+; CHECK-V6M-NEXT:    push {r4, r5, r6, lr}
 ; CHECK-V6M-NEXT:    cmp r0, #1
 ; CHECK-V6M-NEXT:    blt .LBB2_6
 ; CHECK-V6M-NEXT:  .LBB2_1:
+; CHECK-V6M-NEXT:    ldrb r3, [r1]
+; CHECK-V6M-NEXT:    mvns r4, r3
 ; CHECK-V6M-NEXT:    ldrb r3, [r1, #2]
-; CHECK-V6M-NEXT:    movs r6, #255
-; CHECK-V6M-NEXT:    mov r5, r6
-; CHECK-V6M-NEXT:    bics r5, r3
-; CHECK-V6M-NEXT:    ldrb r4, [r1]
-; CHECK-V6M-NEXT:    mov r7, r6
-; CHECK-V6M-NEXT:    bics r7, r4
-; CHECK-V6M-NEXT:    mvns r4, r4
 ; CHECK-V6M-NEXT:    mvns r3, r3
-; CHECK-V6M-NEXT:    cmp r7, r5
+; CHECK-V6M-NEXT:    cmp r4, r3
 ; CHECK-V6M-NEXT:    mov r5, r4
 ; CHECK-V6M-NEXT:    blo .LBB2_3
 ; CHECK-V6M-NEXT:    mov r5, r3
 ; CHECK-V6M-NEXT:  .LBB2_3:
-; CHECK-V6M-NEXT:    str r3, [sp]
-; CHECK-V6M-NEXT:    uxtb r3, r5
-; CHECK-V6M-NEXT:    ldrb r7, [r1, #1]
-; CHECK-V6M-NEXT:    bics r6, r7
-; CHECK-V6M-NEXT:    mvns r7, r7
-; CHECK-V6M-NEXT:    cmp r3, r6
+; CHECK-V6M-NEXT:    ldrb r6, [r1, #1]
+; CHECK-V6M-NEXT:    mvns r6, r6
+; CHECK-V6M-NEXT:    cmp r5, r6
 ; CHECK-V6M-NEXT:    blo .LBB2_5
-; CHECK-V6M-NEXT:    mov r5, r7
+; CHECK-V6M-NEXT:    mov r5, r6
 ; CHECK-V6M-NEXT:  .LBB2_5:
 ; CHECK-V6M-NEXT:    strb r5, [r2]
-; CHECK-V6M-NEXT:    subs r3, r4, r5
-; CHECK-V6M-NEXT:    strb r3, [r2, #1]
-; CHECK-V6M-NEXT:    subs r3, r7, r5
-; CHECK-V6M-NEXT:    strb r3, [r2, #2]
-; CHECK-V6M-NEXT:    ldr r3, [sp]
+; CHECK-V6M-NEXT:    subs r4, r4, r5
+; CHECK-V6M-NEXT:    strb r4, [r2, #1]
+; CHECK-V6M-NEXT:    subs r4, r6, r5
+; CHECK-V6M-NEXT:    strb r4, [r2, #2]
 ; CHECK-V6M-NEXT:    subs r3, r3, r5
 ; CHECK-V6M-NEXT:    strb r3, [r2, #3]
 ; CHECK-V6M-NEXT:    adds r2, r2, #4
 ; CHECK-V6M-NEXT:    adds r1, r1, #3
 ; CHECK-V6M-NEXT:    subs r0, r0, #1
 ; CHECK-V6M-NEXT:    bne .LBB2_1
 ; CHECK-V6M-NEXT:  .LBB2_6:
-; CHECK-V6M-NEXT:    add sp, #4
-; CHECK-V6M-NEXT:    pop {r4, r5, r6, r7, pc}
+; CHECK-V6M-NEXT:    pop {r4, r5, r6, pc}
 ;
 ; CHECK-V7M-LABEL: testloop:
-; CHECK-V7M:         .save {r4, r5, r6, r7, lr}
-; CHECK-V7M-NEXT:    push {r4, r5, r6, r7, lr}
+; CHECK-V7M:         .save {r4, r5, r7, lr}
+; CHECK-V7M-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-V7M-NEXT:    cmp r0, #1
 ; CHECK-V7M-NEXT:    blt .LBB2_2
 ; CHECK-V7M-NEXT:  .LBB2_1:
 ; CHECK-V7M-NEXT:    ldrb.w lr, [r1]
-; CHECK-V7M-NEXT:    ldrb r3, [r1, #2]
+; CHECK-V7M-NEXT:    ldrb r4, [r1, #2]
 ; CHECK-V7M-NEXT:    ldrb.w r12, [r1, #1]
 ; CHECK-V7M-NEXT:    adds r1, #3
-; CHECK-V7M-NEXT:    mvn.w r4, lr
-; CHECK-V7M-NEXT:    mvns r7, r3
-; CHECK-V7M-NEXT:    uxtb r5, r4
-; CHECK-V7M-NEXT:    uxtb r6, r7
-; CHECK-V7M-NEXT:    cmp r5, r6
-; CHECK-V7M-NEXT:    mov r3, r7
-; CHECK-V7M-NEXT:    mvn.w r5, r12
+; CHECK-V7M-NEXT:    mvn.w r5, lr
+; CHECK-V7M-NEXT:    mvn.w lr, r4
+; CHECK-V7M-NEXT:    cmp r5, lr
+; CHECK-V7M-NEXT:    mov r4, lr
+; CHECK-V7M-NEXT:    mvn.w r3, r12
 ; CHECK-V7M-NEXT:    it lo
-; CHECK-V7M-NEXT:    movlo r3, r4
-; CHECK-V7M-NEXT:    uxtb r6, r5
-; CHECK-V7M-NEXT:    uxtb.w lr, r3
-; CHECK-V7M-NEXT:    cmp lr, r6
+; CHECK-V7M-NEXT:    movlo r4, r5
+; CHECK-V7M-NEXT:    cmp r4, r3
 ; CHECK-V7M-NEXT:    it hs
-; CHECK-V7M-NEXT:    movhs r3, r5
+; CHECK-V7M-NEXT:    movhs r4, r3
 ; CHECK-V7M-NEXT:    subs r0, #1
-; CHECK-V7M-NEXT:    sub.w r6, r4, r3
-; CHECK-V7M-NEXT:    strb r3, [r2]
-; CHECK-V7M-NEXT:    strb r6, [r2, #1]
-; CHECK-V7M-NEXT:    sub.w r6, r5, r3
-; CHECK-V7M-NEXT:    strb r6, [r2, #2]
-; CHECK-V7M-NEXT:    sub.w r3, r7, r3
+; CHECK-V7M-NEXT:    sub.w r3, r3, r4
+; CHECK-V7M-NEXT:    strb r4, [r2]
+; CHECK-V7M-NEXT:    sub.w r5, r5, r4
+; CHECK-V7M-NEXT:    strb r5, [r2, #1]
+; CHECK-V7M-NEXT:    strb r3, [r2, #2]
+; CHECK-V7M-NEXT:    sub.w r3, lr, r4
 ; CHECK-V7M-NEXT:    strb r3, [r2, #3]
 ; CHECK-V7M-NEXT:    add.w r2, r2, #4
 ; CHECK-V7M-NEXT:    bne .LBB2_1
 ; CHECK-V7M-NEXT:  .LBB2_2:
-; CHECK-V7M-NEXT:    pop {r4, r5, r6, r7, pc}
+; CHECK-V7M-NEXT:    pop {r4, r5, r7, pc}
 entry:
   %cmp74 = icmp sgt i32 %I, 0
   br i1 %cmp74, label %for.body.preheader, label %for.cond.cleanup
 
 for.body.preheader:
   br label %for.body
 
 for.body: