[Intrinsics] Move dest arg functions of MemIntrinsicBase deeper in mem intrinsics hierarchy

[dmakogon]

Summary

Currently we can access destination argument of any memory intrinsic (MemCpyInst, MemSetInst, ...) through the functions of their common base class - MemIntrinsicBase.
But I think these functions must not be declared such early in the class hierarchy (which now for MemSetInst is
IntrinsicInst <-- MemIntrinsicBase<MemIntrinsic> <-- MemIntrinsic <-- MemSetBase<MemWriteIntrinsic> <-- MemSetInst)
We might want to add some new memory intrinsics in the future that do not have dest argument (i.e. sth like memcmp which would have LHS and RHS instead of src and dest).

So this patch introduces new classes in this hierarchy: MemWriteIntrinsicBase in which we define functions related to dest argument; AtomicMemWriteIntrinsic and
MemWriteIntrinsic which derive from MWIB and define classof functions; AnyMemWriteIntrinsic. If now we were to add a new intrinsic which doesn't have dest argument we would derive from MemIntrinsic (or AtomicMemIntrinsic) and wouldn't have that dest argument related logic.
Having introduced these classes, this patch rewrites all uses of MemIntrinsicBase's dest functions so that MemWriteIntrinsicBase is used.

[dmakogon]

Forgot to remove dest from MemTransferBase
Added/edited some comments

[dmakogon]

clang-format

[mkazantsev]

I guess it's an NFC. If so, please add [NFC] in the commit name.

Generally looks good. For sanity check, pls make sure clang builds with your changes.

[dmakogon]

Update uses of MemIntrinsic in polly

[nikic]

This looks okay if we actually want to add new non-write intrinsics in this family (and it makes sense to have them in the same hierarchy). Is there a pending proposal for that?

[mkazantsev]

In D109219#2984715, @nikic wrote:

This looks okay if we actually want to add new non-write intrinsics in this family (and it makes sense to have them in the same hierarchy). Is there a pending proposal for that?

I don't think it has been proposed yet, but we plan to introduce an [element-atomic] memcmp intrinsic. Hierarchy-wise, it is a MemIntrinsic but is not a MemWriteIntrinsic.

Generally, I think this change makes sense even without this notion, just because dest isn't placed where it belongs, and there is no legit way to express a read-only memory intrinsic. So we shouldn't be waiting for proposal details to just do it.

[dmakogon]

Fixed one more use of MemIntrinsic::getDest

[lebedev.ri]

What others said, if there is an upcoming patch to make use of this, then this LG, else NACK.

[dmakogon]

Fixed formatting problems and changed one more use of MemIntrinsic to MemWriteIntrinsic

[dmakogon]

We want to introduce a new memory intrinsic and we need this patch to do that. Please take a look - https://reviews.llvm.org/D109504

[mkazantsev]

Is the provided motivation OK for this refactoring? It's generally good, and I don't really see reasons to hold it away.

[nikic]

In D109219#3003344, @mkazantsev wrote:

Is the provided motivation OK for this refactoring? It's generally good, and I don't really see reasons to hold it away.

A remaining question I have is where the new MemIntrinsic would get actually used. The only commonality between current MemIntrinsics is that they have a destination -- accessing that destination is what MemIntrinsic is used for. After this change, it's not clear to me under what circumstances I could do something useful with MemIntrinsic that would not require a MemWriteIntrinsic (or a newly introduced MemCmpIntrinsic). The only thing MemIntrinsic tells me is that the intrinsic has "something to do with memory". Scrolling through this patch, there seems to be exactly one place where MemIntrinsic is being used (a volatile check) and that use could actually be replaced by MemWriteIntrinsic as well.

[mkazantsev]

In D109219#3003348, @nikic wrote:

In D109219#3003344, @mkazantsev wrote:

Is the provided motivation OK for this refactoring? It's generally good, and I don't really see reasons to hold it away.

A remaining question I have is where the new MemIntrinsic would get actually used. The only commonality between current MemIntrinsics is that they have a destination -- accessing that destination is what MemIntrinsic is used for. After this change, it's not clear to me under what circumstances I could do something useful with MemIntrinsic that would not require a MemWriteIntrinsic (or a newly introduced MemCmpIntrinsic). The only thing MemIntrinsic tells me is that the intrinsic has "something to do with memory". Scrolling through this patch, there seems to be exactly one place where MemIntrinsic is being used (a volatile check) and that use could actually be replaced by MemWriteIntrinsic as well.

For example, we can LICM read-only memory intrinsic with invariant arguments if loop doesn't write (or we've proven non-alias with all writes). We can also CSE them or do whatever else opts we can do to read-only stuff. The most important bit is to have read-only memory intrinsics, and MemoryIntrinsic is just left as their parent because it makes sense.

[dmakogon]

Ping

[xbolva00]

In D109219#3024748, @dmakogon wrote:

Ping

I think you want to “sell” the community new memcmp intrinsic so after that patch is accepted, this one will be basically too.

[mkazantsev]

Byt why can't it go ahead? It's getting hard to sustain it because of multiple changes in different places. Rebasing really becomes pain. This is a (supposedly) NFC, so why not?

[lebedev.ri]

This review may be stuck/dead, consider abandoning if no longer relevant.
Removing myself as reviewer in attempt to clean dashboard.

llvm/include/llvm/Analysis/MemoryLocation.h

 
 namespace llvm {
 
 class CallBase;
 class Instruction;
 class LoadInst;
 class StoreInst;
 class MemTransferInst;
-class MemIntrinsic;
+class MemWriteIntrinsic;
 class AtomicCmpXchgInst;
 class AtomicMemTransferInst;
 class AtomicMemIntrinsic;
+class AtomicMemWriteIntrinsic;
 class AtomicRMWInst;
 class AnyMemTransferInst;
 class AnyMemIntrinsic;
+class AnyMemWriteIntrinsic;
 class TargetLibraryInfo;
 class VAArgInst;
 
 // Represents the size of a MemoryLocation. Logically, it's an
 // Optional<uint63_t> that also carries a bit to represent whether the integer
 // it contains, N, is 'precise'. Precise, in this context, means that we know
 // that the area of storage referenced by the given MemoryLocation must be
 // precisely N bytes. An imprecise value is formed as the union of two or more
public:
 
   /// Return a location representing the source of a memory transfer.
   static MemoryLocation getForSource(const MemTransferInst *MTI);
   static MemoryLocation getForSource(const AtomicMemTransferInst *MTI);
   static MemoryLocation getForSource(const AnyMemTransferInst *MTI);
 
   /// Return a location representing the destination of a memory set or
   /// transfer.
-  static MemoryLocation getForDest(const MemIntrinsic *MI);
-  static MemoryLocation getForDest(const AtomicMemIntrinsic *MI);
-  static MemoryLocation getForDest(const AnyMemIntrinsic *MI);
+  static MemoryLocation getForDest(const MemWriteIntrinsic *MI);
+  static MemoryLocation getForDest(const AtomicMemWriteIntrinsic *MI);
+  static MemoryLocation getForDest(const AnyMemWriteIntrinsic *MI);
 
   /// Return a location representing a particular argument of a call.
   static MemoryLocation getForArgument(const CallBase *Call, unsigned ArgIdx,
                                        const TargetLibraryInfo *TLI);
   static MemoryLocation getForArgument(const CallBase *Call, unsigned ArgIdx,
                                        const TargetLibraryInfo &TLI) {
     return getForArgument(Call, ArgIdx, &TLI);
   }

llvm/include/llvm/IR/IntrinsicInst.h

 };
 
 /// Common base class for all memory intrinsics. Simply provides
 /// common methods.
 /// Written as CRTP to avoid a common base class amongst the
 /// three atomicity hierarchies.
 template <typename Derived> class MemIntrinsicBase : public IntrinsicInst {
 private:
-  enum { ARG_DEST = 0, ARG_LENGTH = 2 };
+  enum { ARG_LENGTH = 2 };
 
-public:
-  Value *getRawDest() const {
-    return const_cast<Value *>(getArgOperand(ARG_DEST));
+protected:
+  Value *getRawArg(unsigned ArgIndex) const {
+    return const_cast<Value *>(getArgOperand(ArgIndex));
   }
-  const Use &getRawDestUse() const { return getArgOperandUse(ARG_DEST); }
-  Use &getRawDestUse() { return getArgOperandUse(ARG_DEST); }
 
-  Value *getLength() const {
-    return const_cast<Value *>(getArgOperand(ARG_LENGTH));
+  const Use &getRawArgUse(unsigned ArgIndex) const {
+    return getArgOperandUse(ArgIndex);
   }
-  const Use &getLengthUse() const { return getArgOperandUse(ARG_LENGTH); }
-  Use &getLengthUse() { return getArgOperandUse(ARG_LENGTH); }
 
-  /// This is just like getRawDest, but it strips off any cast
-  /// instructions (including addrspacecast) that feed it, giving the
-  /// original input.  The returned value is guaranteed to be a pointer.
-  Value *getDest() const { return getRawDest()->stripPointerCasts(); }
+  Use &getRawArgUse(unsigned ArgIndex) { return getArgOperandUse(ArgIndex); }
 
-  unsigned getDestAddressSpace() const {
-    return cast<PointerType>(getRawDest()->getType())->getAddressSpace();
+  Value *getArg(unsigned ArgIndex) const {
+    return getRawArg(ArgIndex)->stripPointerCasts();
+  }
+
+  unsigned getArgAddressSpace(unsigned ArgIndex) const {
+    return cast<PointerType>(getRawArg(ArgIndex)->getType())->getAddressSpace();
   }
 
   /// FIXME: Remove this function once transition to Align is over.
-  /// Use getDestAlign() instead.
-  unsigned getDestAlignment() const {
-    if (auto MA = getParamAlign(ARG_DEST))
+  /// Use getArgAlign() instead.
+  unsigned getArgAlignment(unsigned ArgIndex) const {
+    if (auto MA = getParamAlign(ArgIndex))
       return MA->value();
     return 0;
   }
-  MaybeAlign getDestAlign() const { return getParamAlign(ARG_DEST); }
 
-  /// Set the specified arguments of the instruction.
-  void setDest(Value *Ptr) {
-    assert(getRawDest()->getType() == Ptr->getType() &&
-           "setDest called with pointer of wrong type!");
-    setArgOperand(ARG_DEST, Ptr);
+  MaybeAlign getArgAlign(unsigned ArgIndex) const {
+    return getParamAlign(ArgIndex);
+  }
+
+  void setArg(unsigned ArgIndex, Value *Ptr) {
+    assert(getRawArg(ArgIndex)->getType() == Ptr->getType() &&
+           "setArg called with pointer of wrong type!");
+    setArgOperand(ArgIndex, Ptr);
   }
 
   /// FIXME: Remove this function once transition to Align is over.
   /// Use the version that takes MaybeAlign instead of this one.
-  void setDestAlignment(unsigned Alignment) {
-    setDestAlignment(MaybeAlign(Alignment));
+  void setArgAlignment(unsigned ArgIndex, unsigned Alignment) {
+    setArgAlignment(ArgIndex, MaybeAlign(Alignment));
   }
-  void setDestAlignment(MaybeAlign Alignment) {
-    removeParamAttr(ARG_DEST, Attribute::Alignment);
+
+  void setArgAlignment(unsigned ArgIndex, MaybeAlign Alignment) {
+    removeParamAttr(ArgIndex, Attribute::Alignment);
     if (Alignment)
-      addParamAttr(ARG_DEST,
+      addParamAttr(ArgIndex,
                    Attribute::getWithAlignment(getContext(), *Alignment));
   }
-  void setDestAlignment(Align Alignment) {
-    removeParamAttr(ARG_DEST, Attribute::Alignment);
-    addParamAttr(ARG_DEST,
+
+  void setArgAlignment(unsigned ArgIndex, Align Alignment) {
+    removeParamAttr(ArgIndex, Attribute::Alignment);
+    addParamAttr(ArgIndex,
                  Attribute::getWithAlignment(getContext(), Alignment));
   }
 
-  void setLength(Value *L) {
-    assert(getLength()->getType() == L->getType() &&
-           "setLength called with value of wrong type!");
-    setArgOperand(ARG_LENGTH, L);
-  }
+public:
+  Value *getLength() const { return getRawArg(ARG_LENGTH); }
+
+  const Use &getLengthUse() const { return getRawArgUse(ARG_LENGTH); }
+
+  Use &getLengthUse() { return getRawArgUse(ARG_LENGTH); }
+
+  void setLength(Value *L) { setArg(ARG_LENGTH, L); }
 };
 
 /// Common base class for all memory transfer intrinsics. Simply provides
 /// common methods.
 template <class BaseCL> class MemTransferBase : public BaseCL {
 private:
   enum { ARG_SOURCE = 1 };
 
 public:
   /// Return the arguments to the instruction.
-  Value *getRawSource() const {
-    return const_cast<Value *>(BaseCL::getArgOperand(ARG_SOURCE));
-  }
+  Value *getRawSource() const { return BaseCL::getRawArg(ARG_SOURCE); }
+
   const Use &getRawSourceUse() const {
-    return BaseCL::getArgOperandUse(ARG_SOURCE);
+    return BaseCL::getRawArgUse(ARG_SOURCE);
   }
-  Use &getRawSourceUse() { return BaseCL::getArgOperandUse(ARG_SOURCE); }
+
+  Use &getRawSourceUse() { return BaseCL::getRawArgUse(ARG_SOURCE); }
 
   /// This is just like getRawSource, but it strips off any cast
   /// instructions that feed it, giving the original input.  The returned
   /// value is guaranteed to be a pointer.
-  Value *getSource() const { return getRawSource()->stripPointerCasts(); }
+  Value *getSource() const { return BaseCL::getArg(ARG_SOURCE); }
 
   unsigned getSourceAddressSpace() const {
-    return cast<PointerType>(getRawSource()->getType())->getAddressSpace();
+    return BaseCL::getArgAddressSpace(ARG_SOURCE);
   }
 
   /// FIXME: Remove this function once transition to Align is over.
   /// Use getSourceAlign() instead.
   unsigned getSourceAlignment() const {
-    if (auto MA = BaseCL::getParamAlign(ARG_SOURCE))
-      return MA->value();
-    return 0;
+    return BaseCL::getArgAlignment(ARG_SOURCE);
   }
 
-  MaybeAlign getSourceAlign() const {
-    return BaseCL::getParamAlign(ARG_SOURCE);
-  }
+  MaybeAlign getSourceAlign() const { return BaseCL::getArgAlign(ARG_SOURCE); }
 
-  void setSource(Value *Ptr) {
-    assert(getRawSource()->getType() == Ptr->getType() &&
-           "setSource called with pointer of wrong type!");
-    BaseCL::setArgOperand(ARG_SOURCE, Ptr);
-  }
+  void setSource(Value *Ptr) { BaseCL::setArg(ARG_SOURCE, Ptr); }
 
   /// FIXME: Remove this function once transition to Align is over.
   /// Use the version that takes MaybeAlign instead of this one.
   void setSourceAlignment(unsigned Alignment) {
-    setSourceAlignment(MaybeAlign(Alignment));
+    BaseCL::setArgAlignment(ARG_SOURCE, Alignment);
   }
+
   void setSourceAlignment(MaybeAlign Alignment) {
-    BaseCL::removeParamAttr(ARG_SOURCE, Attribute::Alignment);
-    if (Alignment)
-      BaseCL::addParamAttr(ARG_SOURCE, Attribute::getWithAlignment(
-                                           BaseCL::getContext(), *Alignment));
+    BaseCL::setArgAlignment(ARG_SOURCE, Alignment);
   }
+
   void setSourceAlignment(Align Alignment) {
-    BaseCL::removeParamAttr(ARG_SOURCE, Attribute::Alignment);
-    BaseCL::addParamAttr(ARG_SOURCE, Attribute::getWithAlignment(
-                                         BaseCL::getContext(), Alignment));
+    BaseCL::setArgAlignment(ARG_SOURCE, Alignment);
   }
 };
 
 /// Common base class for all memset intrinsics. Simply provides
 /// common methods.
 template <class BaseCL> class MemSetBase : public BaseCL {
 private:
   enum { ARG_VALUE = 1 };
 
 public:
-  Value *getValue() const {
-    return const_cast<Value *>(BaseCL::getArgOperand(ARG_VALUE));
-  }
-  const Use &getValueUse() const { return BaseCL::getArgOperandUse(ARG_VALUE); }
-  Use &getValueUse() { return BaseCL::getArgOperandUse(ARG_VALUE); }
+  Value *getValue() const { return BaseCL::getRawArg(ARG_VALUE); }
 
-  void setValue(Value *Val) {
-    assert(getValue()->getType() == Val->getType() &&
-           "setValue called with value of wrong type!");
-    BaseCL::setArgOperand(ARG_VALUE, Val);
-  }
+  const Use &getValueUse() const { return BaseCL::getArgUse(ARG_VALUE); }
+
+  Use &getValueUse() { return BaseCL::getArgUse(ARG_VALUE); }
+
+  void setValue(Value *Val) { BaseCL::setArg(ARG_VALUE, Val); }
 };
 
 // The common base class for the atomic memset/memmove/memcpy intrinsics
 // i.e. llvm.element.unordered.atomic.memset/memcpy/memmove
 class AtomicMemIntrinsic : public MemIntrinsicBase<AtomicMemIntrinsic> {
 private:
   enum { ARG_ELEMENTSIZE = 3 };
 
 public:
-  Value *getRawElementSizeInBytes() const {
-    return const_cast<Value *>(getArgOperand(ARG_ELEMENTSIZE));
-  }
+  Value *getRawElementSizeInBytes() const { return getRawArg(ARG_ELEMENTSIZE); }
 
   ConstantInt *getElementSizeInBytesCst() const {
     return cast<ConstantInt>(getRawElementSizeInBytes());
   }
 
   uint32_t getElementSizeInBytes() const {
     return getElementSizeInBytesCst()->getZExtValue();
   }
 
-  void setElementSizeInBytes(Constant *V) {
-    assert(V->getType() == Type::getInt8Ty(getContext()) &&
-           "setElementSizeInBytes called with value of wrong type!");
-    setArgOperand(ARG_ELEMENTSIZE, V);
-  }
+  void setElementSizeInBytes(Constant *V) { setArg(ARG_ELEMENTSIZE, V); }
 
   static bool classof(const IntrinsicInst *I) {
     switch (I->getIntrinsicID()) {
     case Intrinsic::memcpy_element_unordered_atomic:
     case Intrinsic::memmove_element_unordered_atomic:
     case Intrinsic::memset_element_unordered_atomic:
       return true;
     default:
       return false;
     }
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 };
 
+// The common base class for all memory write intrinsics (memcpy/memset/memmove)
+template <class BaseCL> class MemWriteIntrinsicBase : public BaseCL {
+private:
+  enum { ARG_DEST = 0 };
+
+public:
+  /// This is just like getRawDest, but it strips off any cast
+  /// instructions (including addrspacecast) that feed it, giving the
+  /// original input.  The returned value is guaranteed to be a pointer.
+  Value *getDest() const { return BaseCL::getArg(ARG_DEST); }
+
+  unsigned getDestAddressSpace() const {
+    return BaseCL::getArgAddressSpace(ARG_DEST);
+  }
+
+  /// FIXME: Remove this function once transition to Align is over.
+  /// Use getDestAlign() instead.
+  unsigned getDestAlignment() const {
+    return BaseCL::getArgAlignment(ARG_DEST);
+  }
+
+  MaybeAlign getDestAlign() const { return BaseCL::getArgAlign(ARG_DEST); }
+
+  /// Set the specified arguments of the instruction.
+  void setDest(Value *Ptr) { BaseCL::setArg(ARG_DEST, Ptr); }
+
+  /// FIXME: Remove this function once transition to Align is over.
+  /// Use the version that takes MaybeAlign instead of this one.
+  void setDestAlignment(unsigned Alignment) {
+    BaseCL::setArgAlignment(ARG_DEST, Alignment);
+  }
+
+  void setDestAlignment(MaybeAlign Alignment) {
+    BaseCL::setArgAlignment(ARG_DEST, Alignment);
+  }
+
+  void setDestAlignment(Align Alignment) {
+    BaseCL::setArgAlignment(ARG_DEST, Alignment);
+  }
+
+  Value *getRawDest() const { return BaseCL::getRawArg(ARG_DEST); }
+
+  const Use &getRawDestUse() const { return BaseCL::getRawArgUse(ARG_DEST); }
+
+  Use &getRawDestUse() { return BaseCL::getRawArgUse(ARG_DEST); }
+};
+
+class AtomicMemWriteIntrinsic
+    : public MemWriteIntrinsicBase<AtomicMemIntrinsic> {
+public:
+  static bool classof(const IntrinsicInst *I) {
+    switch (I->getIntrinsicID()) {
+    case Intrinsic::memcpy_element_unordered_atomic:
+    case Intrinsic::memset_element_unordered_atomic:
+    case Intrinsic::memmove_element_unordered_atomic:
+      return true;
+    default:
+      return false;
+    }
+  }
+  static bool classof(const Value *V) {
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
+  }
+};
+
 /// This class represents atomic memset intrinsic
 // i.e. llvm.element.unordered.atomic.memset
-class AtomicMemSetInst : public MemSetBase<AtomicMemIntrinsic> {
+class AtomicMemSetInst : public MemSetBase<AtomicMemWriteIntrinsic> {
 public:
   static bool classof(const IntrinsicInst *I) {
     return I->getIntrinsicID() == Intrinsic::memset_element_unordered_atomic;
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 };
 
 // This class wraps the atomic memcpy/memmove intrinsics
 // i.e. llvm.element.unordered.atomic.memcpy/memmove
-class AtomicMemTransferInst : public MemTransferBase<AtomicMemIntrinsic> {
+class AtomicMemTransferInst : public MemTransferBase<AtomicMemWriteIntrinsic> {
 public:
   static bool classof(const IntrinsicInst *I) {
     switch (I->getIntrinsicID()) {
     case Intrinsic::memcpy_element_unordered_atomic:
     case Intrinsic::memmove_element_unordered_atomic:
       return true;
     default:
       return false;
default:
       return false;
     }
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 };
 
+class MemWriteIntrinsic : public MemWriteIntrinsicBase<MemIntrinsic> {
+public:
+  // Methods for support type inquiry through isa, cast, and dyn_cast:
+  static bool classof(const IntrinsicInst *I) {
+    switch (I->getIntrinsicID()) {
+    case Intrinsic::memcpy:
+    case Intrinsic::memmove:
+    case Intrinsic::memset:
+    case Intrinsic::memcpy_inline:
+      return true;
+    default:
+      return false;
+    }
+  }
+  static bool classof(const Value *V) {
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
+  }
+};
+
 /// This class wraps the llvm.memset intrinsic.
-class MemSetInst : public MemSetBase<MemIntrinsic> {
+class MemSetInst : public MemSetBase<MemWriteIntrinsic> {
 public:
   // Methods for support type inquiry through isa, cast, and dyn_cast:
   static bool classof(const IntrinsicInst *I) {
     return I->getIntrinsicID() == Intrinsic::memset;
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 };
 
 /// This class wraps the llvm.memcpy/memmove intrinsics.
-class MemTransferInst : public MemTransferBase<MemIntrinsic> {
+class MemTransferInst : public MemTransferBase<MemWriteIntrinsic> {
 public:
   // Methods for support type inquiry through isa, cast, and dyn_cast:
   static bool classof(const IntrinsicInst *I) {
     switch (I->getIntrinsicID()) {
     case Intrinsic::memcpy:
     case Intrinsic::memmove:
     case Intrinsic::memcpy_inline:
       return true;
default:
       return false;
     }
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 };
 
+class AnyMemWriteIntrinsic : public MemWriteIntrinsicBase<AnyMemIntrinsic> {
+public:
+  static bool classof(const IntrinsicInst *I) {
+    switch (I->getIntrinsicID()) {
+    case Intrinsic::memcpy:
+    case Intrinsic::memcpy_inline:
+    case Intrinsic::memmove:
+    case Intrinsic::memset:
+    case Intrinsic::memcpy_element_unordered_atomic:
+    case Intrinsic::memmove_element_unordered_atomic:
+    case Intrinsic::memset_element_unordered_atomic:
+      return true;
+    default:
+      return false;
+    }
+  }
+  static bool classof(const Value *V) {
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
+  }
+};
+
 /// This class represents any memset intrinsic
 // i.e. llvm.element.unordered.atomic.memset
 // and  llvm.memset
-class AnyMemSetInst : public MemSetBase<AnyMemIntrinsic> {
+class AnyMemSetInst : public MemSetBase<AnyMemWriteIntrinsic> {
 public:
   static bool classof(const IntrinsicInst *I) {
     switch (I->getIntrinsicID()) {
     case Intrinsic::memset:
     case Intrinsic::memset_element_unordered_atomic:
       return true;
     default:
       return false;
     }
   }
   static bool classof(const Value *V) {
     return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 };
 
 // This class wraps any memcpy/memmove intrinsics
 // i.e. llvm.element.unordered.atomic.memcpy/memmove
 // and  llvm.memcpy/memmove
-class AnyMemTransferInst : public MemTransferBase<AnyMemIntrinsic> {
+class AnyMemTransferInst : public MemTransferBase<AnyMemWriteIntrinsic> {
 public:
   static bool classof(const IntrinsicInst *I) {
     switch (I->getIntrinsicID()) {
     case Intrinsic::memcpy:
     case Intrinsic::memcpy_inline:
     case Intrinsic::memmove:
     case Intrinsic::memcpy_element_unordered_atomic:
     case Intrinsic::memmove_element_unordered_atomic:

llvm/include/llvm/Transforms/Utils/VNCoercion.h

 
 #ifndef LLVM_TRANSFORMS_UTILS_VNCOERCION_H
 #define LLVM_TRANSFORMS_UTILS_VNCOERCION_H
 
 namespace llvm {
 class Constant;
 class StoreInst;
 class LoadInst;
-class MemIntrinsic;
+class MemWriteIntrinsic;
 class Instruction;
 class IRBuilderBase;
 class Value;
 class Type;
 class DataLayout;
 namespace VNCoercion {
 /// Return true if CoerceAvailableValueToLoadType would succeed if it was
 /// called.
int analyzeLoadFromClobberingLoad(Type *LoadTy, Value *LoadPtr, LoadInst *DepLI,
                                   const DataLayout &DL);
 
 /// This function determines whether a value for the pointer LoadPtr can be
 /// extracted from the memory intrinsic at DepMI.
 ///
 /// On success, it returns the offset into DepMI that extraction would start.
 /// On failure, it returns -1.
 int analyzeLoadFromClobberingMemInst(Type *LoadTy, Value *LoadPtr,
-                                     MemIntrinsic *DepMI, const DataLayout &DL);
+                                     MemWriteIntrinsic *DepMI,
+                                     const DataLayout &DL);
 
 /// If analyzeLoadFromClobberingStore returned an offset, this function can be
 /// used to actually perform the extraction of the bits from the store. It
 /// inserts instructions to do so at InsertPt, and returns the extracted value.
 Value *getStoreValueForLoad(Value *SrcVal, unsigned Offset, Type *LoadTy,
                             Instruction *InsertPt, const DataLayout &DL);
 // This is the same as getStoreValueForLoad, except it performs no insertion
 // It only allows constant inputs.

 // a constant. It returns nullptr if it would require widening the load.
 Constant *getConstantLoadValueForLoad(Constant *SrcVal, unsigned Offset,
                                       Type *LoadTy, const DataLayout &DL);
 
 /// If analyzeLoadFromClobberingMemInst returned an offset, this function can be
 /// used to actually perform the extraction of the bits from the memory
 /// intrinsic.  It inserts instructions to do so at InsertPt, and returns the
 /// extracted value.
-Value *getMemInstValueForLoad(MemIntrinsic *SrcInst, unsigned Offset,
+Value *getMemInstValueForLoad(MemWriteIntrinsic *SrcInst, unsigned Offset,
                               Type *LoadTy, Instruction *InsertPt,
                               const DataLayout &DL);
 // This is the same as getStoreValueForLoad, except it performs no insertion.
 // It returns nullptr if it cannot produce a constant.
-Constant *getConstantMemInstValueForLoad(MemIntrinsic *SrcInst, unsigned Offset,
-                                         Type *LoadTy, const DataLayout &DL);
+Constant *getConstantMemInstValueForLoad(MemWriteIntrinsic *SrcInst,
+                                         unsigned Offset, Type *LoadTy,
+                                         const DataLayout &DL);
 }
 }
 #endif

llvm/lib/Analysis/LazyValueInfo.cpp

 }
 
 static void AddNonNullPointer(Value *Ptr, NonNullPointerSet &PtrSet) {
   // TODO: Use NullPointerIsDefined instead.
   if (Ptr->getType()->getPointerAddressSpace() == 0)
     PtrSet.insert(getUnderlyingObject(Ptr));
 }
 
-static void AddNonNullPointersByInstruction(
-    Instruction *I, NonNullPointerSet &PtrSet) {
+static void AddNonNullPointersByInstruction(Instruction *I,
+                                            NonNullPointerSet &PtrSet) {
   if (LoadInst *L = dyn_cast<LoadInst>(I)) {
     AddNonNullPointer(L->getPointerOperand(), PtrSet);
   } else if (StoreInst *S = dyn_cast<StoreInst>(I)) {
     AddNonNullPointer(S->getPointerOperand(), PtrSet);
-  } else if (MemIntrinsic *MI = dyn_cast<MemIntrinsic>(I)) {
-    if (MI->isVolatile()) return;
+  } else if (MemWriteIntrinsic *MWI = dyn_cast<MemWriteIntrinsic>(I)) {
+    if (MWI->isVolatile())
+      return;
 
     // FIXME: check whether it has a valuerange that excludes zero?
-    ConstantInt *Len = dyn_cast<ConstantInt>(MI->getLength());
-    if (!Len || Len->isZero()) return;
+    ConstantInt *Len = dyn_cast<ConstantInt>(MWI->getLength());
+    if (!Len || Len->isZero())
+      return;
 
-    AddNonNullPointer(MI->getRawDest(), PtrSet);
-    if (MemTransferInst *MTI = dyn_cast<MemTransferInst>(MI))
+    AddNonNullPointer(MWI->getRawDest(), PtrSet);
+    if (MemTransferInst *MTI = dyn_cast<MemTransferInst>(MWI))
       AddNonNullPointer(MTI->getRawSource(), PtrSet);
   }
 }
 
 bool LazyValueInfoImpl::isNonNullAtEndOfBlock(Value *Val, BasicBlock *BB) {
   if (NullPointerIsDefined(BB->getParent(),
                            Val->getType()->getPointerAddressSpace()))
     return false;

llvm/lib/Analysis/MemoryLocation.cpp

MemoryLocation MemoryLocation::getForSource(const AnyMemTransferInst *MTI) {
   // memcpy/memmove can have AA tags. For memcpy, they apply
   // to both the source and the destination.
   AAMDNodes AATags;
   MTI->getAAMetadata(AATags);
 
   return MemoryLocation(MTI->getRawSource(), Size, AATags);
 }
 
-MemoryLocation MemoryLocation::getForDest(const MemIntrinsic *MI) {
-  return getForDest(cast<AnyMemIntrinsic>(MI));
+MemoryLocation MemoryLocation::getForDest(const MemWriteIntrinsic *MI) {
+  return getForDest(cast<AnyMemWriteIntrinsic>(MI));
 }
 
-MemoryLocation MemoryLocation::getForDest(const AtomicMemIntrinsic *MI) {
-  return getForDest(cast<AnyMemIntrinsic>(MI));
+MemoryLocation MemoryLocation::getForDest(const AtomicMemWriteIntrinsic *AMI) {
+  return getForDest(cast<AnyMemWriteIntrinsic>(AMI));
 }
 
-MemoryLocation MemoryLocation::getForDest(const AnyMemIntrinsic *MI) {
+MemoryLocation MemoryLocation::getForDest(const AnyMemWriteIntrinsic *MWI) {
   auto Size = LocationSize::afterPointer();
-  if (ConstantInt *C = dyn_cast<ConstantInt>(MI->getLength()))
+  if (ConstantInt *C = dyn_cast<ConstantInt>(MWI->getLength()))
     Size = LocationSize::precise(C->getValue().getZExtValue());
 
   // memcpy/memmove can have AA tags. For memcpy, they apply
   // to both the source and the destination.
   AAMDNodes AATags;
-  MI->getAAMetadata(AATags);
+  MWI->getAAMetadata(AATags);
 
-  return MemoryLocation(MI->getRawDest(), Size, AATags);
+  return MemoryLocation(MWI->getRawDest(), Size, AATags);
 }
 
 MemoryLocation MemoryLocation::getForArgument(const CallBase *Call,
                                               unsigned ArgIdx,
                                               const TargetLibraryInfo *TLI) {
   AAMDNodes AATags;
   Call->getAAMetadata(AATags);
   const Value *Arg = Call->getArgOperand(ArgIdx);

llvm/lib/Analysis/StackSafetyAnalysis.cpp

return getAccessRange(
       Addr, Base, ConstantRange(APInt::getNullValue(PointerSize), APSize));
 }
 
 ConstantRange StackSafetyLocalAnalysis::getMemIntrinsicAccessRange(
     const MemIntrinsic *MI, const Use &U, Value *Base) {
   if (const auto *MTI = dyn_cast<MemTransferInst>(MI)) {
     if (MTI->getRawSource() != U && MTI->getRawDest() != U)
       return ConstantRange::getEmpty(PointerSize);
-  } else {
-    if (MI->getRawDest() != U)
+  } else if (const auto *MWI = dyn_cast<MemWriteIntrinsic>(MI)) {
+    if (MWI->getRawDest() != U)
       return ConstantRange::getEmpty(PointerSize);
   }
 
   auto *CalculationTy = IntegerType::getIntNTy(SE.getContext(), PointerSize);
   if (!SE.isSCEVable(MI->getLength()->getType()))
     return UnknownRange;
 
   const SCEV *Expr =

llvm/lib/CodeGen/CodeGenPrepare.cpp

for (auto &Arg : CI->arg_operands()) {
       if ((GV = dyn_cast<GlobalVariable>(Val)) && GV->canIncreaseAlignment() &&
           GV->getPointerAlignment(*DL) < PrefAlign &&
           DL->getTypeAllocSize(GV->getValueType()) >=
               MinSize + Offset2)
         GV->setAlignment(MaybeAlign(PrefAlign));
     }
     // If this is a memcpy (or similar) then we may be able to improve the
     // alignment
-    if (MemIntrinsic *MI = dyn_cast<MemIntrinsic>(CI)) {
-      Align DestAlign = getKnownAlignment(MI->getDest(), *DL);
-      MaybeAlign MIDestAlign = MI->getDestAlign();
-      if (!MIDestAlign || DestAlign > *MIDestAlign)
-        MI->setDestAlignment(DestAlign);
-      if (MemTransferInst *MTI = dyn_cast<MemTransferInst>(MI)) {
+    if (MemWriteIntrinsic *MWI = dyn_cast<MemWriteIntrinsic>(CI)) {
+      Align DestAlign = getKnownAlignment(MWI->getDest(), *DL);
+      MaybeAlign MWIDestAlign = MWI->getDestAlign();
+      if (!MWIDestAlign || DestAlign > *MWIDestAlign)
+        MWI->setDestAlignment(DestAlign);
+      if (MemTransferInst *MTI = dyn_cast<MemTransferInst>(MWI)) {
         MaybeAlign MTISrcAlign = MTI->getSourceAlign();
         Align SrcAlign = getKnownAlignment(MTI->getSource(), *DL);
         if (!MTISrcAlign || SrcAlign > *MTISrcAlign)
           MTI->setSourceAlignment(SrcAlign);
       }
     }
   }
 

llvm/lib/CodeGen/SafeStack.cpp

LLVM_DEBUG(
              << "            " << (Safe ? "safe" : "unsafe") << "\n");
 
   return Safe;
 }
 
 bool SafeStack::IsMemIntrinsicSafe(const MemIntrinsic *MI, const Use &U,
                                    const Value *AllocaPtr,
                                    uint64_t AllocaSize) {
-  if (auto MTI = dyn_cast<MemTransferInst>(MI)) {
+  if (const auto *MTI = dyn_cast<MemTransferInst>(MI)) {
     if (MTI->getRawSource() != U && MTI->getRawDest() != U)
       return true;
-  } else {
-    if (MI->getRawDest() != U)
+  } else if (const auto *MWI = dyn_cast<MemWriteIntrinsic>(MI)) {
+    if (MWI->getRawDest() != U)
       return true;
   }
 
   const auto *Len = dyn_cast<ConstantInt>(MI->getLength());
   // Non-constant size => unsafe. FIXME: try SCEV getRange.
-  if (!Len) return false;
+  if (!Len)
+    return false;
   return IsAccessSafe(U, Len->getZExtValue(), AllocaPtr, AllocaSize);
 }
 
 /// Check whether a given allocation must be put on the safe
 /// stack or not. The function analyzes all uses of AI and checks whether it is
 /// only accessed in a memory safe way (as decided statically).
 bool SafeStack::IsSafeStackAlloca(const Value *AllocaPtr, uint64_t AllocaSize) {
   // Go through all uses of this alloca and check whether all accesses to the

llvm/lib/IR/AutoUpgrade.cpp

if (CI->getNumArgOperands() != 5) {
       DefaultCase();
       return;
     }
     // Remove alignment argument (3), and add alignment attributes to the
     // dest/src pointers.
     Value *Args[4] = {CI->getArgOperand(0), CI->getArgOperand(1),
                       CI->getArgOperand(2), CI->getArgOperand(4)};
     NewCall = Builder.CreateCall(NewFn, Args);
-    auto *MemCI = cast<MemIntrinsic>(NewCall);
+    auto *MemWriteIntr = cast<MemWriteIntrinsic>(NewCall);
     // All mem intrinsics support dest alignment.
     const ConstantInt *Align = cast<ConstantInt>(CI->getArgOperand(3));
-    MemCI->setDestAlignment(Align->getMaybeAlignValue());
+    MemWriteIntr->setDestAlignment(Align->getMaybeAlignValue());
     // Memcpy/Memmove also support source alignment.
-    if (auto *MTI = dyn_cast<MemTransferInst>(MemCI))
+    if (auto *MTI = dyn_cast<MemTransferInst>(MemWriteIntr))
       MTI->setSourceAlignment(Align->getMaybeAlignValue());
     break;
   }
   }
   assert(NewCall && "Should have either set this variable or returned through "
                     "the default case");
   NewCall->takeName(CI);
   CI->replaceAllUsesWith(NewCall);

llvm/lib/IR/Verifier.cpp

case Intrinsic::dbg_value: // llvm.dbg.value
     break;
   case Intrinsic::dbg_label: // llvm.dbg.label
     visitDbgLabelIntrinsic("label", cast<DbgLabelInst>(Call));
     break;
   case Intrinsic::memcpy:
   case Intrinsic::memcpy_inline:
   case Intrinsic::memmove:
   case Intrinsic::memset: {
-    const auto *MI = cast<MemIntrinsic>(&Call);
+    const auto *MWI = cast<MemWriteIntrinsic>(&Call);
     auto IsValidAlignment = [&](unsigned Alignment) -> bool {
       return Alignment == 0 || isPowerOf2_32(Alignment);
     };
-    Assert(IsValidAlignment(MI->getDestAlignment()),
+    Assert(IsValidAlignment(MWI->getDestAlignment()),
            "alignment of arg 0 of memory intrinsic must be 0 or a power of 2",
            Call);
-    if (const auto *MTI = dyn_cast<MemTransferInst>(MI)) {
+    if (const auto *MTI = dyn_cast<MemTransferInst>(MWI)) {
       Assert(IsValidAlignment(MTI->getSourceAlignment()),
              "alignment of arg 1 of memory intrinsic must be 0 or a power of 2",
              Call);
     }
-
     break;
   }
   case Intrinsic::memcpy_element_unordered_atomic:
   case Intrinsic::memmove_element_unordered_atomic:
   case Intrinsic::memset_element_unordered_atomic: {
-    const auto *AMI = cast<AtomicMemIntrinsic>(&Call);
+    const auto *AMWI = cast<AtomicMemWriteIntrinsic>(&Call);
 
     ConstantInt *ElementSizeCI =
-        cast<ConstantInt>(AMI->getRawElementSizeInBytes());
+        cast<ConstantInt>(AMWI->getRawElementSizeInBytes());
     const APInt &ElementSizeVal = ElementSizeCI->getValue();
     Assert(ElementSizeVal.isPowerOf2(),
            "element size of the element-wise atomic memory intrinsic "
            "must be a power of 2",
            Call);
 
     auto IsValidAlignment = [&](uint64_t Alignment) {
       return isPowerOf2_64(Alignment) && ElementSizeVal.ule(Alignment);
     };
-    uint64_t DstAlignment = AMI->getDestAlignment();
+    uint64_t DstAlignment = AMWI->getDestAlignment();
     Assert(IsValidAlignment(DstAlignment),
            "incorrect alignment of the destination argument", Call);
-    if (const auto *AMT = dyn_cast<AtomicMemTransferInst>(AMI)) {
+    if (const auto *AMT = dyn_cast<AtomicMemTransferInst>(AMWI)) {
       uint64_t SrcAlignment = AMT->getSourceAlignment();
       Assert(IsValidAlignment(SrcAlignment),
              "incorrect alignment of the source argument", Call);
     }
     break;
   }
   case Intrinsic::call_preallocated_setup: {
     auto *NumArgs = dyn_cast<ConstantInt>(Call.getArgOperand(0));

llvm/lib/Target/AMDGPU/AMDGPUPerfHintAnalysis.cpp

#endif
   bool isLargeStride(const Instruction *Inst);
 
   bool isGlobalAddr(const Value *V) const;
   bool isLocalAddr(const Value *V) const;
   bool isConstantAddr(const Value *V) const;
 };
 
 static const Value *getMemoryInstrPtr(const Instruction *Inst) {
-  if (auto LI = dyn_cast<LoadInst>(Inst)) {
+  if (const auto *LI = dyn_cast<LoadInst>(Inst)) {
     return LI->getPointerOperand();
   }
-  if (auto SI = dyn_cast<StoreInst>(Inst)) {
+  if (const auto *SI = dyn_cast<StoreInst>(Inst)) {
     return SI->getPointerOperand();
   }
-  if (auto AI = dyn_cast<AtomicCmpXchgInst>(Inst)) {
+  if (const auto *AI = dyn_cast<AtomicCmpXchgInst>(Inst)) {
     return AI->getPointerOperand();
   }
-  if (auto AI = dyn_cast<AtomicRMWInst>(Inst)) {
+  if (const auto *AI = dyn_cast<AtomicRMWInst>(Inst)) {
     return AI->getPointerOperand();
   }
-  if (auto MI = dyn_cast<AnyMemIntrinsic>(Inst)) {
-    return MI->getRawDest();
+  if (const auto *MWI = dyn_cast<AnyMemWriteIntrinsic>(Inst)) {
+    return MWI->getRawDest();
   }
 
   return nullptr;
 }
 
 bool AMDGPUPerfHint::isIndirectAccess(const Instruction *Inst) const {
   LLVM_DEBUG(dbgs() << "[isIndirectAccess] " << *Inst << '\n');
   SmallSet<const Value *, 32> WorkSet;

llvm/lib/Transforms/IPO/GlobalOpt.cpp

if (LoadInst *LI = dyn_cast<LoadInst>(U)) {
           SubInit = Constant::getNullValue(GEP->getResultElementType());
       }
       Changed |= CleanupConstantGlobalUsers(GEP, SubInit, DL, GetTLI);
 
       if (GEP->use_empty()) {
         GEP->eraseFromParent();
         Changed = true;
       }
-    } else if (MemIntrinsic *MI = dyn_cast<MemIntrinsic>(U)) { // memset/cpy/mv
-      if (MI->getRawDest() == V) {
-        MI->eraseFromParent();
+    } else if (MemWriteIntrinsic *MWI = dyn_cast<MemWriteIntrinsic>(U)) {
+      // memset/cpy/mv
+      if (MWI->getRawDest() == V) {
+        MWI->eraseFromParent();
         Changed = true;
       }
 
     } else if (Constant *C = dyn_cast<Constant>(U)) {
       // If we have a chain of dead constantexprs or other things dangling from
       // us, and if they are all dead, nuke them without remorse.
       if (isSafeToDestroyConstant(C)) {
         C->destroyConstant();

llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp

if (ConstantInt *NumBytes = dyn_cast<ConstantInt>(AMI->getLength()))
         CreateNonTerminatorUnreachable(AMI);
         assert(AMI->getType()->isVoidTy() &&
                "non void atomic unordered mem intrinsic");
         return eraseInstFromFunction(*AMI);
       }
 
   // Intrinsics cannot occur in an invoke or a callbr, so handle them here
   // instead of in visitCallBase.
-  if (auto *MI = dyn_cast<AnyMemIntrinsic>(II)) {
+  if (auto *MWI = dyn_cast<AnyMemWriteIntrinsic>(II)) {
     bool Changed = false;
 
     // memmove/cpy/set of zero bytes is a noop.
-    if (Constant *NumBytes = dyn_cast<Constant>(MI->getLength())) {
+    if (Constant *NumBytes = dyn_cast<Constant>(MWI->getLength())) {
       if (NumBytes->isNullValue())
         return eraseInstFromFunction(CI);
 
       if (ConstantInt *CI = dyn_cast<ConstantInt>(NumBytes))
         if (CI->getZExtValue() == 1) {
           // Replace the instruction with just byte operations.  We would
           // transform other cases to loads/stores, but we don't know if
           // alignment is sufficient.
         }
     }
 
     // No other transformations apply to volatile transfers.
-    if (auto *M = dyn_cast<MemIntrinsic>(MI))
+    if (auto *M = dyn_cast<MemIntrinsic>(MWI))
       if (M->isVolatile())
         return nullptr;
 
     // If we have a memmove and the source operation is a constant global,
     // then the source and dest pointers can't alias, so we can change this
     // into a call to memcpy.
-    if (auto *MMI = dyn_cast<AnyMemMoveInst>(MI)) {
+    if (auto *MMI = dyn_cast<AnyMemMoveInst>(MWI)) {
       if (GlobalVariable *GVSrc = dyn_cast<GlobalVariable>(MMI->getSource()))
         if (GVSrc->isConstant()) {
           Module *M = CI.getModule();
           Intrinsic::ID MemCpyID =
               isa<AtomicMemMoveInst>(MMI)
                   ? Intrinsic::memcpy_element_unordered_atomic
                   : Intrinsic::memcpy;
           Type *Tys[3] = { CI.getArgOperand(0)->getType(),
                            CI.getArgOperand(1)->getType(),
                            CI.getArgOperand(2)->getType() };
           CI.setCalledFunction(Intrinsic::getDeclaration(M, MemCpyID, Tys));
           Changed = true;
         }
     }
 
-    if (AnyMemTransferInst *MTI = dyn_cast<AnyMemTransferInst>(MI)) {
+    if (AnyMemTransferInst *MTI = dyn_cast<AnyMemTransferInst>(MWI)) {
       // memmove(x,x,size) -> noop.
       if (MTI->getSource() == MTI->getDest())
         return eraseInstFromFunction(CI);
     }
 
     // If we can determine a pointer alignment that is bigger than currently
     // set, update the alignment.
-    if (auto *MTI = dyn_cast<AnyMemTransferInst>(MI)) {
+    if (auto *MTI = dyn_cast<AnyMemTransferInst>(MWI)) {
       if (Instruction *I = SimplifyAnyMemTransfer(MTI))
         return I;
-    } else if (auto *MSI = dyn_cast<AnyMemSetInst>(MI)) {
+    } else if (auto *MSI = dyn_cast<AnyMemSetInst>(MWI)) {
       if (Instruction *I = SimplifyAnyMemSet(MSI))
         return I;
     }
 
     if (Changed) return II;
   }
 
   // For fixed width vector result intrinsics, use the generic demanded vector

llvm/lib/Transforms/InstCombine/InstructionCombining.cpp

for (User *U : PI->users()) {
         if (IntrinsicInst *II = dyn_cast<IntrinsicInst>(I)) {
           switch (II->getIntrinsicID()) {
           default:
             return false;
 
           case Intrinsic::memmove:
           case Intrinsic::memcpy:
           case Intrinsic::memset: {
-            MemIntrinsic *MI = cast<MemIntrinsic>(II);
-            if (MI->isVolatile() || MI->getRawDest() != PI)
+            MemWriteIntrinsic *MWI = cast<MemWriteIntrinsic>(II);
+            if (MWI->isVolatile() || MWI->getRawDest() != PI)
               return false;
             LLVM_FALLTHROUGH;
           }
           case Intrinsic::assume:
           case Intrinsic::invariant_start:
           case Intrinsic::invariant_end:
           case Intrinsic::lifetime_start:
           case Intrinsic::lifetime_end:

llvm/lib/Transforms/Scalar/AlignmentFromAssumptions.cpp

if (LoadInst *LI = dyn_cast<LoadInst>(J)) {
       if (!isValidAssumeForContext(ACall, J, DT))
         continue;
       Align NewAlignment = getNewAlignment(AASCEV, AlignSCEV, OffSCEV,
                                            SI->getPointerOperand(), SE);
       if (NewAlignment > SI->getAlign()) {
         SI->setAlignment(NewAlignment);
         ++NumStoreAlignChanged;
       }
-    } else if (MemIntrinsic *MI = dyn_cast<MemIntrinsic>(J)) {
+    } else if (MemWriteIntrinsic *MWI = dyn_cast<MemWriteIntrinsic>(J)) {
       if (!isValidAssumeForContext(ACall, J, DT))
         continue;
       Align NewDestAlignment =
-          getNewAlignment(AASCEV, AlignSCEV, OffSCEV, MI->getDest(), SE);
+          getNewAlignment(AASCEV, AlignSCEV, OffSCEV, MWI->getDest(), SE);
 
       LLVM_DEBUG(dbgs() << "\tmem inst: " << DebugStr(NewDestAlignment)
                         << "\n";);
-      if (NewDestAlignment > *MI->getDestAlign()) {
-        MI->setDestAlignment(NewDestAlignment);
+      if (NewDestAlignment > *MWI->getDestAlign()) {
+        MWI->setDestAlignment(NewDestAlignment);
         ++NumMemIntAlignChanged;
       }
 
       // For memory transfers, there is also a source alignment that
       // can be set.
-      if (MemTransferInst *MTI = dyn_cast<MemTransferInst>(MI)) {
+      if (MemTransferInst *MTI = dyn_cast<MemTransferInst>(MWI)) {
         Align NewSrcAlignment =
             getNewAlignment(AASCEV, AlignSCEV, OffSCEV, MTI->getSource(), SE);
 
         LLVM_DEBUG(dbgs() << "\tmem trans: " << DebugStr(NewSrcAlignment)
                           << "\n";);
 
         if (NewSrcAlignment > *MTI->getSourceAlign()) {
           MTI->setSourceAlignment(NewSrcAlignment);

llvm/lib/Transforms/Scalar/DeadStoreElimination.cpp

 /// returns true, this function and getLocForRead completely describe the memory
 /// operations for this instruction.
 static MemoryLocation getLocForWrite(Instruction *Inst,
                                      const TargetLibraryInfo &TLI) {
   if (StoreInst *SI = dyn_cast<StoreInst>(Inst))
     return MemoryLocation::get(SI);
 
   // memcpy/memmove/memset.
-  if (auto *MI = dyn_cast<AnyMemIntrinsic>(Inst))
-    return MemoryLocation::getForDest(MI);
+  if (auto *MWI = dyn_cast<AnyMemWriteIntrinsic>(Inst))
+    return MemoryLocation::getForDest(MWI);
 
   if (IntrinsicInst *II = dyn_cast<IntrinsicInst>(Inst)) {
     switch (II->getIntrinsicID()) {
     default:
       return MemoryLocation(); // Unhandled intrinsic.
     case Intrinsic::init_trampoline:
       return MemoryLocation::getAfter(II->getArgOperand(0));
     case Intrinsic::masked_store:
while (!WorkList.empty()) {
     }
   }
   return true;
 }
 
 static bool tryToShorten(Instruction *EarlierWrite, int64_t &EarlierStart,
                          uint64_t &EarlierSize, int64_t LaterStart,
                          uint64_t LaterSize, bool IsOverwriteEnd) {
-  auto *EarlierIntrinsic = cast<AnyMemIntrinsic>(EarlierWrite);
+  auto *EarlierIntrinsic = cast<AnyMemWriteIntrinsic>(EarlierWrite);
   Align PrefAlign = EarlierIntrinsic->getDestAlign().valueOrOne();
 
   // We assume that memet/memcpy operates in chunks of the "largest" native
   // type size and aligned on the same value. That means optimal start and size
   // of memset/memcpy should be modulo of preferred alignment of that type. That
   // is it there is no any sense in trying to reduce store size any further
   // since any "extra" stores comes for free anyway.
   // On the other hand, maximum alignment we can achieve is limited by alignment
bool isInvisibleToCallerBeforeRet(const Value *V) {
     }
     return I.first->second;
   }
 
   Optional<MemoryLocation> getLocForWriteEx(Instruction *I) const {
     if (!I->mayWriteToMemory())
       return None;
 
-    if (auto *MTI = dyn_cast<AnyMemIntrinsic>(I))
-      return {MemoryLocation::getForDest(MTI)};
+    if (auto *MWI = dyn_cast<AnyMemWriteIntrinsic>(I))
+      return {MemoryLocation::getForDest(MWI)};
 
     if (auto *CB = dyn_cast<CallBase>(I)) {
       // If the functions may write to memory we do not know about, bail out.
       if (!CB->onlyAccessesArgMemory() &&
           !CB->onlyAccessesInaccessibleMemOrArgMem())
         return None;
 
       LibFunc LF;

llvm/lib/Transforms/Scalar/GVN.cpp

struct llvm::gvn::AvailableValue {
   static AvailableValue get(Value *V, unsigned Offset = 0) {
     AvailableValue Res;
     Res.Val.setPointer(V);
     Res.Val.setInt(SimpleVal);
     Res.Offset = Offset;
     return Res;
   }
 
-  static AvailableValue getMI(MemIntrinsic *MI, unsigned Offset = 0) {
+  static AvailableValue getMI(MemWriteIntrinsic *MWI, unsigned Offset = 0) {
     AvailableValue Res;
-    Res.Val.setPointer(MI);
+    Res.Val.setPointer(MWI);
     Res.Val.setInt(MemIntrin);
     Res.Offset = Offset;
     return Res;
   }
 
   static AvailableValue getLoad(LoadInst *Load, unsigned Offset = 0) {
     AvailableValue Res;
     Res.Val.setPointer(Load);
Value *getSimpleValue() const {
     return Val.getPointer();
   }
 
   LoadInst *getCoercedLoadValue() const {
     assert(isCoercedLoadValue() && "Wrong accessor");
     return cast<LoadInst>(Val.getPointer());
   }
 
-  MemIntrinsic *getMemIntrinValue() const {
+  MemWriteIntrinsic *getMemIntrinValue() const {
     assert(isMemIntrinValue() && "Wrong accessor");
-    return cast<MemIntrinsic>(Val.getPointer());
+    return cast<MemWriteIntrinsic>(Val.getPointer());
   }
 
   /// Emit code at the specified insertion point to adjust the value defined
   /// here to the specified type. This handles various coercion cases.
   Value *MaterializeAdjustedValue(LoadInst *Load, Instruction *InsertPt,
                                   GVN &gvn) const;
 };
 
if (LoadInst *DepLoad = dyn_cast<LoadInst>(DepInst)) {
           Res = AvailableValue::getLoad(DepLoad, Offset);
           return true;
         }
       }
     }
 
     // If the clobbering value is a memset/memcpy/memmove, see if we can
     // forward a value on from it.
-    if (MemIntrinsic *DepMI = dyn_cast<MemIntrinsic>(DepInst)) {
+    if (MemWriteIntrinsic *DepMI = dyn_cast<MemWriteIntrinsic>(DepInst)) {
       if (Address && !Load->isAtomic()) {
         int Offset = analyzeLoadFromClobberingMemInst(Load->getType(), Address,
                                                       DepMI, DL);
         if (Offset != -1) {
           Res = AvailableValue::getMI(DepMI, Offset);
           return true;
         }
       }

llvm/lib/Transforms/Scalar/InferAddressSpaces.cpp

for (Instruction &I : instructions(F)) {
     } else if (auto *LI = dyn_cast<LoadInst>(&I))
       PushPtrOperand(LI->getPointerOperand());
     else if (auto *SI = dyn_cast<StoreInst>(&I))
       PushPtrOperand(SI->getPointerOperand());
     else if (auto *RMW = dyn_cast<AtomicRMWInst>(&I))
       PushPtrOperand(RMW->getPointerOperand());
     else if (auto *CmpX = dyn_cast<AtomicCmpXchgInst>(&I))
       PushPtrOperand(CmpX->getPointerOperand());
-    else if (auto *MI = dyn_cast<MemIntrinsic>(&I)) {
+    else if (auto *MWI = dyn_cast<MemWriteIntrinsic>(&I)) {
       // For memset/memcpy/memmove, any pointer operand can be replaced.
-      PushPtrOperand(MI->getRawDest());
+      PushPtrOperand(MWI->getRawDest());
 
       // Handle 2nd operand for memcpy/memmove.
-      if (auto *MTI = dyn_cast<MemTransferInst>(MI))
+      if (auto *MTI = dyn_cast<MemTransferInst>(MWI))
         PushPtrOperand(MTI->getRawSource());
     } else if (auto *II = dyn_cast<IntrinsicInst>(&I))
       collectRewritableIntrinsicOperands(II, PostorderStack, Visited);
     else if (ICmpInst *Cmp = dyn_cast<ICmpInst>(&I)) {
       // FIXME: Handle vectors of pointers
       if (Cmp->getOperand(0)->getType()->isPointerTy()) {
         PushPtrOperand(Cmp->getOperand(0));
         PushPtrOperand(Cmp->getOperand(1));

llvm/lib/Transforms/Scalar/NewGVN.cpp

if (Offset >= 0) {
       if (auto *C = dyn_cast<Constant>(lookupOperandLeader(DepLI)))
         if (auto *PossibleConstant =
                 getConstantLoadValueForLoad(C, Offset, LoadType, DL)) {
           LLVM_DEBUG(dbgs() << "Coercing load from load " << *LI
                             << " to constant " << *PossibleConstant << "\n");
           return createConstantExpression(PossibleConstant);
         }
     }
-  } else if (auto *DepMI = dyn_cast<MemIntrinsic>(DepInst)) {
+  } else if (auto *DepMI = dyn_cast<MemWriteIntrinsic>(DepInst)) {
     int Offset = analyzeLoadFromClobberingMemInst(LoadType, LoadPtr, DepMI, DL);
     if (Offset >= 0) {
       if (auto *PossibleConstant =
               getConstantMemInstValueForLoad(DepMI, Offset, LoadType, DL)) {
         LLVM_DEBUG(dbgs() << "Coercing load from meminst " << *DepMI
                           << " to constant " << *PossibleConstant << "\n");
         return createConstantExpression(PossibleConstant);
       }

llvm/lib/Transforms/Utils/VNCoercion.cpp

int analyzeLoadFromClobberingLoad(Type *LoadTy, Value *LoadPtr, LoadInst *DepLI,
   // able to materialize this potentially available value
   assert(DepLI->isSimple() && "Cannot widen volatile/atomic load!");
   assert(DepLI->getType()->isIntegerTy() && "Can't widen non-integer load");
 
   return analyzeLoadFromClobberingWrite(LoadTy, LoadPtr, DepPtr, Size * 8, DL);
 }
 
 int analyzeLoadFromClobberingMemInst(Type *LoadTy, Value *LoadPtr,
-                                     MemIntrinsic *MI, const DataLayout &DL) {
+                                     MemWriteIntrinsic *MWI,
+                                     const DataLayout &DL) {
   // If the mem operation is a non-constant size, we can't handle it.
-  ConstantInt *SizeCst = dyn_cast<ConstantInt>(MI->getLength());
+  ConstantInt *SizeCst = dyn_cast<ConstantInt>(MWI->getLength());
   if (!SizeCst)
     return -1;
   uint64_t MemSizeInBits = SizeCst->getZExtValue() * 8;
 
   // If this is memset, we just need to see if the offset is valid in the size
   // of the memset..
-  if (MI->getIntrinsicID() == Intrinsic::memset) {
+  if (MWI->getIntrinsicID() == Intrinsic::memset) {
     if (DL.isNonIntegralPointerType(LoadTy->getScalarType())) {
-      auto *CI = dyn_cast<ConstantInt>(cast<MemSetInst>(MI)->getValue());
+      auto *CI = dyn_cast<ConstantInt>(cast<MemSetInst>(MWI)->getValue());
       if (!CI || !CI->isZero())
         return -1;
     }
-    return analyzeLoadFromClobberingWrite(LoadTy, LoadPtr, MI->getDest(),
+    return analyzeLoadFromClobberingWrite(LoadTy, LoadPtr, MWI->getDest(),
                                           MemSizeInBits, DL);
   }
 
   // If we have a memcpy/memmove, the only case we can handle is if this is a
   // copy from constant memory.  In that case, we can read directly from the
   // constant memory.
-  MemTransferInst *MTI = cast<MemTransferInst>(MI);
+  MemTransferInst *MTI = cast<MemTransferInst>(MWI);
 
   Constant *Src = dyn_cast<Constant>(MTI->getSource());
   if (!Src)
     return -1;
 
   GlobalVariable *GV = dyn_cast<GlobalVariable>(getUnderlyingObject(Src));
   if (!GV || !GV->isConstant() || !GV->hasDefinitiveInitializer())
     return -1;
 
   // See if the access is within the bounds of the transfer.
-  int Offset = analyzeLoadFromClobberingWrite(LoadTy, LoadPtr, MI->getDest(),
+  int Offset = analyzeLoadFromClobberingWrite(LoadTy, LoadPtr, MWI->getDest(),
                                               MemSizeInBits, DL);
   if (Offset == -1)
     return Offset;
 
   unsigned AS = Src->getType()->getPointerAddressSpace();
   // Otherwise, see if we can constant fold a load from the constant with the
   // offset applied as appropriate.
   if (Offset) {
unsigned SrcValStoreSize =
       DL.getTypeStoreSize(SrcVal->getType()).getFixedSize();
   unsigned LoadSize = DL.getTypeStoreSize(LoadTy).getFixedSize();
   if (Offset + LoadSize > SrcValStoreSize)
     return nullptr;
   return getConstantStoreValueForLoad(SrcVal, Offset, LoadTy, DL);
 }
 
 template <class T, class HelperClass>
-T *getMemInstValueForLoadHelper(MemIntrinsic *SrcInst, unsigned Offset,
+T *getMemInstValueForLoadHelper(MemWriteIntrinsic *SrcInst, unsigned Offset,
                                 Type *LoadTy, HelperClass &Helper,
                                 const DataLayout &DL) {
   LLVMContext &Ctx = LoadTy->getContext();
   uint64_t LoadSize = DL.getTypeSizeInBits(LoadTy).getFixedSize() / 8;
 
   // We know that this method is only called when the mem transfer fully
   // provides the bits for the load.
   if (MemSetInst *MSI = dyn_cast<MemSetInst>(SrcInst)) {
Src = ConstantExpr::getGetElementPtr(Type::getInt8Ty(Src->getContext()),
                                          Src, OffsetCst);
   }
   Src = ConstantExpr::getBitCast(Src, PointerType::get(LoadTy, AS));
   return ConstantFoldLoadFromConstPtr(Src, LoadTy, DL);
 }
 
 /// This function is called when we have a
 /// memdep query of a load that ends up being a clobbering mem intrinsic.
-Value *getMemInstValueForLoad(MemIntrinsic *SrcInst, unsigned Offset,
+Value *getMemInstValueForLoad(MemWriteIntrinsic *SrcInst, unsigned Offset,
                               Type *LoadTy, Instruction *InsertPt,
                               const DataLayout &DL) {
   IRBuilder<> Builder(InsertPt);
   return getMemInstValueForLoadHelper<Value, IRBuilder<>>(SrcInst, Offset,
                                                           LoadTy, Builder, DL);
 }
 
-Constant *getConstantMemInstValueForLoad(MemIntrinsic *SrcInst, unsigned Offset,
-                                         Type *LoadTy, const DataLayout &DL) {
+Constant *getConstantMemInstValueForLoad(MemWriteIntrinsic *SrcInst,
+                                         unsigned Offset, Type *LoadTy,
+                                         const DataLayout &DL) {
   // The only case analyzeLoadFromClobberingMemInst cannot be converted to a
   // constant is when it's a memset of a non-constant.
   if (auto *MSI = dyn_cast<MemSetInst>(SrcInst))
     if (!isa<Constant>(MSI->getValue()))
       return nullptr;
   ConstantFolder F;
   return getMemInstValueForLoadHelper<Constant, ConstantFolder>(SrcInst, Offset,
                                                                 LoadTy, F, DL);
 }
 } // namespace VNCoercion
 } // namespace llvm

polly/include/polly/Support/ScopHelper.h

 using ParameterSetTy = llvm::SetVector<const llvm::SCEV *>;
 
 /// Set of loops (used to remember loops in non-affine subregions).
 using BoxedLoopsSetTy = llvm::SetVector<const llvm::Loop *>;
 
 /// Utility proxy to wrap the common members of LoadInst and StoreInst.
 ///
 /// This works like the LLVM utility class CallSite, ie. it forwards all calls
-/// to either a LoadInst, StoreInst, MemIntrinsic or MemTransferInst.
-/// It is similar to LLVM's utility classes IntrinsicInst, MemIntrinsic,
+/// to either a LoadInst, StoreInst, MemWriteIntrinsic or MemTransferInst.
+/// It is similar to LLVM's utility classes IntrinsicInst, MemWriteIntrinsic,
 /// MemTransferInst, etc. in that it offers a common interface, but does not act
 /// as a fake base class.
 /// It is similar to StringRef and ArrayRef in that it holds a pointer to the
 /// referenced object and should be passed by-value as it is small enough.
 ///
 /// This proxy can either represent a LoadInst instance, a StoreInst instance,
-/// a MemIntrinsic instance (memset, memmove, memcpy), a CallInst instance or a
-/// nullptr (only creatable using the default constructor); never an Instruction
-/// that is neither of the above mentioned. When representing a nullptr, only
-/// the following methods are defined:
-/// isNull(), isInstruction(), isLoad(), isStore(), ..., isMemTransferInst(),
-/// operator bool(), operator!()
+/// a MemWriteIntrinsic instance (memset, memmove, memcpy), a CallInst instance
+/// or a nullptr (only creatable using the default constructor); never an
+/// Instruction that is neither of the above mentioned. When representing a
+/// nullptr, only the following methods are defined: isNull(), isInstruction(),
+/// isLoad(), isStore(), ..., isMemTransferInst(), operator bool(), operator!()
 ///
 /// The functions isa, cast, cast_or_null, dyn_cast are modeled te resemble
 /// those from llvm/Support/Casting.h. Partial template function specialization
 /// is currently not supported in C++ such that those cannot be used directly.
 /// (llvm::isa could, but then llvm:cast etc. would not have the expected
 /// behavior)
 class MemAccInst {
 private:
   llvm::Instruction *I;
 
 public:
   MemAccInst() : I(nullptr) {}
   MemAccInst(const MemAccInst &Inst) : I(Inst.I) {}
   /* implicit */ MemAccInst(llvm::LoadInst &LI) : I(&LI) {}
   /* implicit */ MemAccInst(llvm::LoadInst *LI) : I(LI) {}
   /* implicit */ MemAccInst(llvm::StoreInst &SI) : I(&SI) {}
   /* implicit */ MemAccInst(llvm::StoreInst *SI) : I(SI) {}
-  /* implicit */ MemAccInst(llvm::MemIntrinsic *MI) : I(MI) {}
+  /* implicit */ MemAccInst(llvm::MemWriteIntrinsic *MWI) : I(MWI) {}
   /* implicit */ MemAccInst(llvm::CallInst *CI) : I(CI) {}
   explicit MemAccInst(llvm::Instruction &I) : I(&I) { assert(isa(I)); }
   explicit MemAccInst(llvm::Instruction *I) : I(I) { assert(isa(I)); }
 
   static bool isa(const llvm::Value &V) {
     return llvm::isa<llvm::LoadInst>(V) || llvm::isa<llvm::StoreInst>(V) ||
-           llvm::isa<llvm::CallInst>(V) || llvm::isa<llvm::MemIntrinsic>(V);
+           llvm::isa<llvm::CallInst>(V) ||
+           llvm::isa<llvm::MemWriteIntrinsic>(V);
   }
   static bool isa(const llvm::Value *V) {
     return llvm::isa<llvm::LoadInst>(V) || llvm::isa<llvm::StoreInst>(V) ||
-           llvm::isa<llvm::CallInst>(V) || llvm::isa<llvm::MemIntrinsic>(V);
+           llvm::isa<llvm::CallInst>(V) ||
+           llvm::isa<llvm::MemWriteIntrinsic>(V);
   }
   static MemAccInst cast(llvm::Value &V) {
     return MemAccInst(llvm::cast<llvm::Instruction>(V));
   }
   static MemAccInst cast(llvm::Value *V) {
     return MemAccInst(llvm::cast<llvm::Instruction>(V));
   }
   static MemAccInst cast_or_null(llvm::Value &V) {
public:
   MemAccInst &operator=(llvm::StoreInst &SI) {
     I = &SI;
     return *this;
   }
   MemAccInst &operator=(llvm::StoreInst *SI) {
     I = SI;
     return *this;
   }
-  MemAccInst &operator=(llvm::MemIntrinsic &MI) {
+  MemAccInst &operator=(llvm::MemWriteIntrinsic &MI) {
     I = &MI;
     return *this;
   }
-  MemAccInst &operator=(llvm::MemIntrinsic *MI) {
-    I = MI;
+  MemAccInst &operator=(llvm::MemWriteIntrinsic *MWI) {
+    I = MWI;
     return *this;
   }
   MemAccInst &operator=(llvm::CallInst &CI) {
     I = &CI;
     return *this;
   }
   MemAccInst &operator=(llvm::CallInst *CI) {
     I = CI;
public:
   explicit operator bool() const { return isInstruction(); }
   bool operator!() const { return isNull(); }
 
   llvm::Value *getValueOperand() const {
     if (isLoad())
       return asLoad();
     if (isStore())
       return asStore()->getValueOperand();
-    if (isMemIntrinsic())
+    if (isMemWriteIntrinsic())
       return nullptr;
     if (isCallInst())
       return nullptr;
     llvm_unreachable("Operation not supported on nullptr");
   }
   llvm::Value *getPointerOperand() const {
     if (isLoad())
       return asLoad()->getPointerOperand();
     if (isStore())
       return asStore()->getPointerOperand();
-    if (isMemIntrinsic())
-      return asMemIntrinsic()->getRawDest();
+    if (isMemWriteIntrinsic())
+      return asMemWriteIntrinsic()->getRawDest();
     if (isCallInst())
       return nullptr;
     llvm_unreachable("Operation not supported on nullptr");
   }
 
   unsigned getAlignment() const {
     if (isLoad())
       return asLoad()->getAlignment();
     if (isStore())
       return asStore()->getAlignment();
     if (isMemTransferInst())
       return std::min(asMemTransferInst()->getDestAlignment(),
                       asMemTransferInst()->getSourceAlignment());
-    if (isMemIntrinsic())
-      return asMemIntrinsic()->getDestAlignment();
+    if (isMemWriteIntrinsic())
+      return asMemWriteIntrinsic()->getDestAlignment();
     if (isCallInst())
       return 0;
     llvm_unreachable("Operation not supported on nullptr");
   }
   bool isVolatile() const {
     if (isLoad())
       return asLoad()->isVolatile();
     if (isStore())
       return asStore()->isVolatile();
-    if (isMemIntrinsic())
-      return asMemIntrinsic()->isVolatile();
+    if (isMemWriteIntrinsic())
+      return asMemWriteIntrinsic()->isVolatile();
     if (isCallInst())
       return false;
     llvm_unreachable("Operation not supported on nullptr");
   }
   bool isSimple() const {
     if (isLoad())
       return asLoad()->isSimple();
     if (isStore())
       return asStore()->isSimple();
-    if (isMemIntrinsic())
-      return !asMemIntrinsic()->isVolatile();
+    if (isMemWriteIntrinsic())
+      return !asMemWriteIntrinsic()->isVolatile();
     if (isCallInst())
       return true;
     llvm_unreachable("Operation not supported on nullptr");
   }
   llvm::AtomicOrdering getOrdering() const {
     if (isLoad())
       return asLoad()->getOrdering();
     if (isStore())
       return asStore()->getOrdering();
-    if (isMemIntrinsic())
+    if (isMemWriteIntrinsic())
       return llvm::AtomicOrdering::NotAtomic;
     if (isCallInst())
       return llvm::AtomicOrdering::NotAtomic;
     llvm_unreachable("Operation not supported on nullptr");
   }
   bool isUnordered() const {
     if (isLoad())
       return asLoad()->isUnordered();
     if (isStore())
       return asStore()->isUnordered();
     // Copied from the Load/Store implementation of isUnordered:
-    if (isMemIntrinsic())
-      return !asMemIntrinsic()->isVolatile();
+    if (isMemWriteIntrinsic())
+      return !asMemWriteIntrinsic()->isVolatile();
     if (isCallInst())
       return true;
     llvm_unreachable("Operation not supported on nullptr");
   }
 
   bool isNull() const { return !I; }
   bool isInstruction() const { return I; }
 
   llvm::Instruction *asInstruction() const { return I; }
 
 private:
   bool isLoad() const { return I && llvm::isa<llvm::LoadInst>(I); }
   bool isStore() const { return I && llvm::isa<llvm::StoreInst>(I); }
   bool isCallInst() const { return I && llvm::isa<llvm::CallInst>(I); }
-  bool isMemIntrinsic() const { return I && llvm::isa<llvm::MemIntrinsic>(I); }
+  bool isMemWriteIntrinsic() const {
+    return I && llvm::isa<llvm::MemWriteIntrinsic>(I);
+  }
   bool isMemSetInst() const { return I && llvm::isa<llvm::MemSetInst>(I); }
   bool isMemTransferInst() const {
     return I && llvm::isa<llvm::MemTransferInst>(I);
   }
 
   llvm::LoadInst *asLoad() const { return llvm::cast<llvm::LoadInst>(I); }
   llvm::StoreInst *asStore() const { return llvm::cast<llvm::StoreInst>(I); }
   llvm::CallInst *asCallInst() const { return llvm::cast<llvm::CallInst>(I); }
-  llvm::MemIntrinsic *asMemIntrinsic() const {
-    return llvm::cast<llvm::MemIntrinsic>(I);
+  llvm::MemWriteIntrinsic *asMemWriteIntrinsic() const {
+    return llvm::cast<llvm::MemWriteIntrinsic>(I);
   }
   llvm::MemSetInst *asMemSetInst() const {
     return llvm::cast<llvm::MemSetInst>(I);
   }
   llvm::MemTransferInst *asMemTransferInst() const {
     return llvm::cast<llvm::MemTransferInst>(I);
   }
 };

polly/lib/Analysis/ScopBuilder.cpp

if (ElementSize != DelinearizedSize)
     scop->invalidate(DELINEARIZATION, Inst->getDebugLoc(), Inst->getParent());
 
   addArrayAccess(Stmt, Inst, AccType, BasePointer->getValue(), ElementType,
                  true, AccItr->second.DelinearizedSubscripts, Sizes, Val);
   return true;
 }
 
 bool ScopBuilder::buildAccessMemIntrinsic(MemAccInst Inst, ScopStmt *Stmt) {
-  auto *MemIntr = dyn_cast_or_null<MemIntrinsic>(Inst);
+  auto *MemWriteIntr = dyn_cast_or_null<MemWriteIntrinsic>(Inst);
 
-  if (MemIntr == nullptr)
+  if (MemWriteIntr == nullptr)
     return false;
 
   auto *L = LI.getLoopFor(Inst->getParent());
-  auto *LengthVal = SE.getSCEVAtScope(MemIntr->getLength(), L);
+  auto *LengthVal = SE.getSCEVAtScope(MemWriteIntr->getLength(), L);
   assert(LengthVal);
 
   // Check if the length val is actually affine or if we overapproximate it
   InvariantLoadsSetTy AccessILS;
   const InvariantLoadsSetTy &ScopRIL = scop->getRequiredInvariantLoads();
 
   Loop *SurroundingLoop = Stmt->getSurroundingLoop();
   bool LengthIsAffine = isAffineExpr(&scop->getRegion(), SurroundingLoop,
                                      LengthVal, SE, &AccessILS);
   for (LoadInst *LInst : AccessILS)
     if (!ScopRIL.count(LInst))
       LengthIsAffine = false;
   if (!LengthIsAffine)
     LengthVal = nullptr;
 
-  auto *DestPtrVal = MemIntr->getDest();
+  auto *DestPtrVal = MemWriteIntr->getDest();
   assert(DestPtrVal);
 
   auto *DestAccFunc = SE.getSCEVAtScope(DestPtrVal, L);
   assert(DestAccFunc);
   // Ignore accesses to "NULL".
   // TODO: We could use this to optimize the region further, e.g., intersect
   //       the context with
   //          isl_set_complement(isl_set_params(getDomain()))
bool ScopBuilder::buildAccessMemIntrinsic(MemAccInst Inst, ScopStmt *Stmt) {
   auto *DestPtrSCEV = dyn_cast<SCEVUnknown>(SE.getPointerBase(DestAccFunc));
   assert(DestPtrSCEV);
   DestAccFunc = SE.getMinusSCEV(DestAccFunc, DestPtrSCEV);
   addArrayAccess(Stmt, Inst, MemoryAccess::MUST_WRITE, DestPtrSCEV->getValue(),
                  IntegerType::getInt8Ty(DestPtrVal->getContext()),
                  LengthIsAffine, {DestAccFunc, LengthVal}, {nullptr},
                  Inst.getValueOperand());
 
-  auto *MemTrans = dyn_cast<MemTransferInst>(MemIntr);
+  auto *MemTrans = dyn_cast<MemTransferInst>(MemWriteIntr);
   if (!MemTrans)
     return true;
 
   auto *SrcPtrVal = MemTrans->getSource();
   assert(SrcPtrVal);
 
   auto *SrcAccFunc = SE.getSCEVAtScope(SrcPtrVal, L);
   assert(SrcAccFunc);

polly/lib/Analysis/ScopDetection.cpp

case Intrinsic::memcpy:
     if (!AF->isZero()) {
       BP = dyn_cast<SCEVUnknown>(SE.getPointerBase(AF));
       // Bail if the source pointer is not valid.
       if (!isValidAccess(&II, AF, BP, Context))
         return false;
     }
     LLVM_FALLTHROUGH;
   case Intrinsic::memset:
-    AF = SE.getSCEVAtScope(cast<MemIntrinsic>(II).getDest(), L);
+    AF = SE.getSCEVAtScope(cast<MemWriteIntrinsic>(II).getDest(), L);
     if (!AF->isZero()) {
       BP = dyn_cast<SCEVUnknown>(SE.getPointerBase(AF));
       // Bail if the destination pointer is not valid.
       if (!isValidAccess(&II, AF, BP, Context))
         return false;
     }
 
     // Bail if the length is not affine.