[AliasAnalysis] Introduce getModRefInfoMask() as a generalization of pointsToConstantMemory().

[pcwalton]

Summary

The pointsToConstantMemory() method returns true only if the memory pointed to
by the memory location is globally invariant. However, the LLVM memory model
also has the semantic notion of *locally-invariant*: memory that is known to be
invariant for the life of the SSA value representing that pointer. The most
common example of this is a pointer argument that is marked readonly noalias,
which the Rust compiler frequently emits.

It'd be desirable for LLVM to treat locally-invariant memory the same way as
globally-invariant memory when it's safe to do so. This patch implements that,
by introducing the concept of a *ModRefInfo mask*. A ModRefInfo mask is a bound
on the Mod/Ref behavior of an instruction that writes to a memory location,
based on the knowledge that the memory is globally-constant memory (in which
case the mask is NoModRef) or locally-constant memory (in which case the mask
is Ref). ModRefInfo values for an instruction can be combined with the
ModRefInfo mask by simply using the & operator. Where appropriate, this patch
has modified uses of pointsToConstantMemory() to instead examine the mask.

The most notable optimization change I noticed with this patch is that now
redundant loads from readonly noalias pointers can be eliminated across calls,
even when the pointer is captured. Internally, before this patch,
AliasAnalysis was assigning Ref to reads from constant memory; now AA can
assign NoModRef, which is a tighter bound.

[pcwalton]

Marking as non-draft.

[nikic]

llvm/lib/Analysis/AliasAnalysis.cpp

567

Should use getModRefInfoMask() here as well.

586

And here.

605

And here.

llvm/lib/Analysis/BasicAliasAnalysis.cpp

810

Rather than making this a separate method, this should replace pointsToConstantMemory() instead, at least as far as AA providers are concerned. (At the top-level interface, we can keep pointsToConstantMemory() as a thin wrapper like getModRefInfoMask() == NoModRef.)

The problem with your current implementation is that it does not handle cases where there are multiple underlying objects (or a single one that requires looking through phis) which all ultimately end in noalias readonly memory.

llvm/lib/Transforms/IPO/FunctionAttrs.cpp

149

This should be something like MR &= AAR.getModRefInfoMask(Loc, /*IgnoreLocal*/true), after merging with pointsToConstantMemory().

llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp

136

This one is non-trivial and could use a test.

llvm/test/Analysis/BasicAA/pr18573.ll

11 ↗

(On Diff #470365)

Did you mean to drop the readonly rather than the noalias here?

llvm/test/Analysis/BasicAA/readonly-noalias.ll

8 ↗

(On Diff #470365)

This test already returns NoModRef prior to your changes. You need to pass ptr %p as an argument to @foo(), otherwise it can't access it due to noalias.

[pcwalton]

llvm/lib/Analysis/BasicAliasAnalysis.cpp

810

Sure, in fact that was what I originally tried to do, but I got stuck on what to do with the OrLocal parameter. How should getModRefInfoMask() deal with the case in which OrLocal is true and the memory location resolves to a local? Should it return NoModRef? Should it return Ref? Or should I split out the logic of getModRefInfoMask() and pointsToConstantMemory() into a shared helper function that both of them call into?

[pcwalton]

llvm/lib/Analysis/BasicAliasAnalysis.cpp

810

Oh never mind, I saw the IgnoreLocal part of your other comment. OK, I'll return NoModRef in that case.

[pcwalton]

Merge pointsToConstantMemory() into getModRefMask().

[pcwalton]

Should be ready for review again. I'll look at the failing test cases in the meantime.

[pcwalton]

Fix AMDGPU alias analysis.

[pcwalton]

Fix AMDGPU alias analysis and new test failure.

[nikic]

llvm/include/llvm/Analysis/AliasAnalysis.h

769–770

This method should be dropped (together with the ones in AAResults::Model and AAResultBase).

llvm/lib/Analysis/AliasAnalysis.cpp

567

This should be return getModRefInfoMask(Loc, AAQI) (we may have to preserve the Ref).

586

Same here, just return the mask.

605

Same here.

llvm/lib/Analysis/BasicAliasAnalysis.cpp

839

We should probably also check for ReadNone here to be thorough, though it's likely not terribly useful in practice.

840

Result |=

llvm/test/Transforms/InstCombine/copied-from-readonly-noalias.ll

1 ↗

(On Diff #470946)

Use update_test_checks.py here.

3 ↗

(On Diff #470946)

The -aa-pipeline and 2>&1 are not necessary here.

17 ↗

(On Diff #470946)

I'd replace readnone -> readonly here, I think that would be a bit clearer (wouldn't be able to read the argument otherwise anyway...)

llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll

583 ↗

(On Diff #470946)

Can you please commit these fixes for readonly in tests ahead of time?

[pcwalton]

Updated to address review comments.

[pcwalton]

llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll

583 ↗

(On Diff #470946)

You mean as a separate Phabricator diff? Sure, I can do that.

[nikic]

This looks basically good to me. I'd suggest precommitting the test changes with baseline checks, and then rebasing this patch.

llvm/docs/AliasAnalysis.rst

166

This doesn't line up with the implementation anymore. (Feel free to just delete this section entirely.)

llvm/lib/Analysis/BasicAliasAnalysis.cpp

809

Keeping the previous location of the function would make it a bit clearer what changed here.

llvm/test/Analysis/BasicAA/readonly-noalias.ll

1 ↗

(On Diff #471381)

This test can be merged into constant-memory.ll.

13 ↗

(On Diff #471381)

We should also have a negative test for the case where noalias on the argument is missing.

llvm/test/Transforms/InstCombine/copied-from-readonly-noalias.ll

1 ↗

(On Diff #471381)

This test can be merged into memcpy-from-global.ll.

[pcwalton]

Addressed comments. Note that D136993 contains the diff this is now based on top of.

[pcwalton]

Rebased on top of updated D136993.

[pcwalton]

OK, I pushed the baseline test changes. @nikic Let me know if this looks good and I can push this.

[pcwalton]

Rebased. This should be ready for sign off now.

[nikic]

llvm/lib/Analysis/BasicAliasAnalysis.cpp

683

I'd prefer omitting this check here and instead keeping it in the fence implementation. We should try to move away from locations without pointers rather than pushing them deeper into the API.

llvm/lib/Analysis/ObjCARCAliasAnalysis.cpp

93

Huh, this whole function is a very complicated way to do nothing at all. The function above it as well. Not related to your patch though...

llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp

131

Same here.

140–141

Can't we drop this part of the code entirely? This should be subsumed by the generic handling in BasicAA now.

[pcwalton]

Address review comments.

[nikic]

LGTM. The patch description needs an update though, in particular the test changes are no longer part of this patch.

[xry111]

We observed a crash in Thunderbird built with Rustc using LLVM 16.0.0 (happens with Rustc 1.68.x built with a system LLVM 16, and Rustc nightly 2023-03-25). See https://bugzilla.mozilla.org/show_bug.cgi?id=1824544.

Bisect suggests this change is the first bad commit.

Any pointer for diagnose?

[nikic]

In D136659#4233968, @xry111 wrote:

We observed a crash in Thunderbird built with Rustc using LLVM 16.0.0 (happens with Rustc 1.68.x built with a system LLVM 16, and Rustc nightly 2023-03-25). See https://bugzilla.mozilla.org/show_bug.cgi?id=1824544.

Bisect suggests this change is the first bad commit.

Any pointer for diagnose?

Probably the same issue as https://github.com/llvm/llvm-project/issues/58776. It needs a smaller reproducer than "build all of firefox".

[xry111]

In D136659#4233990, @nikic wrote:

In D136659#4233968, @xry111 wrote:

We observed a crash in Thunderbird built with Rustc using LLVM 16.0.0 (happens with Rustc 1.68.x built with a system LLVM 16, and Rustc nightly 2023-03-25). See https://bugzilla.mozilla.org/show_bug.cgi?id=1824544.

Bisect suggests this change is the first bad commit.

Any pointer for diagnose?

Probably the same issue as https://github.com/llvm/llvm-project/issues/58776. It needs a smaller reproducer than "build all of firefox".

Well this is annoying. When we reported this to Mozilla guys they just believe we are building Thunderbird in a wrong way. And it seems they have no interest to use a latest Rustc/LLVM in their CI. We are not Mozilla developers so it seems we cannot review the large code base to find what's wrong...

Are there any tools (test case reducer or UBsan) may help? Or is it possible to disable this optimization via some RUSTFLAGS as a work around?

llvm/docs/AliasAnalysis.rst

 relation is not commutative.
 
 Other useful ``AliasAnalysis`` methods
 --------------------------------------
 
 Several other tidbits of information are often collected by various alias
 analysis implementations and can be put to good use by various clients.
 
-The ``pointsToConstantMemory`` method
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+The ``getModRefInfoMask`` method
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The ``pointsToConstantMemory`` method returns true if and only if the analysis
-can prove that the pointer only points to unchanging memory locations
-(functions, constant global variables, and the null pointer).  This information
-can be used to refine mod/ref information: it is impossible for an unchanging
-memory location to be modified.
+The ``getModRefInfoMask`` method returns a bound on Mod/Ref information for
+the supplied pointer, based on knowledge about whether the pointer points to
+globally-constant memory (for which it returns ``NoModRef``) or
+locally-invariant memory (for which it returns ``Ref``). Globally-constant
+memory includes functions, constant global variables, and the null pointer.
+Locally-invariant memory is memory that we know is invariant for the lifetime
+of its SSA value, but not necessarily for the life of the program: for example,
+the memory pointed to by ``readonly`` ``noalias`` parameters is known-invariant
+for the duration of the corresponding function call. Given Mod/Ref information
+``MRI`` for a memory location ``Loc``, ``MRI`` can be refined with a statement
+like ``MRI &= AA.getModRefInfoMask(Loc);``. Another useful idiom is
+``isModSet(AA.getModRefInfoMask(Loc))``; this checks to see if the given
+location can be modified at all. For convenience, there is also a method
+``pointsToConstantMemory(Loc)``; this is synonymous with
+``isNoModRef(AA.getModRefInfoMask(Loc))``.
 
 .. _never access memory or only read memory:
 
 The ``doesNotAccessMemory`` and  ``onlyReadsMemory`` methods
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 These methods are used to provide very simple mod/ref information for function
 calls.  The ``doesNotAccessMemory`` method returns true for a function if the

llvm/include/llvm/Analysis/AliasAnalysis.h

public:
   /// A convenience wrapper around the \c isMustAlias helper interface.
   bool isMustAlias(const Value *V1, const Value *V2) {
     return alias(V1, LocationSize::precise(1), V2, LocationSize::precise(1)) ==
            AliasResult::MustAlias;
   }
 
   /// Checks whether the given location points to constant memory, or if
   /// \p OrLocal is true whether it points to a local alloca.
-  bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false);
+  bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
+    return isNoModRef(getModRefInfoMask(Loc, OrLocal));
+  }
 
   /// A convenience wrapper around the primary \c pointsToConstantMemory
   /// interface.
   bool pointsToConstantMemory(const Value *P, bool OrLocal = false) {
     return pointsToConstantMemory(MemoryLocation::getBeforeOrAfter(P), OrLocal);
   }
 
   /// @}
   //===--------------------------------------------------------------------===//
   /// \name Simple mod/ref information
   /// @{
 
+  /// Returns a bitmask that should be unconditionally applied to the ModRef
+  /// info of a memory location. This allows us to eliminate Mod and/or Ref
+  /// from the ModRef info based on the knowledge that the memory location
+  /// points to constant and/or locally-invariant memory.
+  ///
+  /// If IgnoreLocals is true, then this method returns NoModRef for memory
+  /// that points to a local alloca.
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+                               bool IgnoreLocals = false);
+
+  /// A convenience wrapper around the primary \c getModRefInfoMask
+  /// interface.
+  ModRefInfo getModRefInfoMask(const Value *P, bool IgnoreLocals = false) {
+    return getModRefInfoMask(MemoryLocation::getBeforeOrAfter(P), IgnoreLocals);
+  }
+
   /// Get the ModRef info associated with a pointer argument of a call. The
   /// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
   /// that these bits do not necessarily account for the overall behavior of
   /// the function, but rather only provide additional per-argument
   /// information.
   ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx);
 
   /// Return the behavior of the given call site.
bool canInstructionRangeModRef(const Instruction &I1, const Instruction &I2,
                                  const ModRefInfo Mode) {
     return canInstructionRangeModRef(I1, I2, MemoryLocation(Ptr, Size), Mode);
   }
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI);
   bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
                               bool OrLocal = false);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals = false);
   ModRefInfo getModRefInfo(Instruction *I, const CallBase *Call2,
                            AAQueryInfo &AAQIP);
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
                            AAQueryInfo &AAQI);
   ModRefInfo getModRefInfo(const CallBase *Call1, const CallBase *Call2,
                            AAQueryInfo &AAQI);
   ModRefInfo getModRefInfo(const VAArgInst *V, const MemoryLocation &Loc,
                            AAQueryInfo &AAQI);
public:
   BatchAAResults(AAResults &AAR, CaptureInfo *CI) : AA(AAR), AAQI(AAR, CI) {}
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB) {
     return AA.alias(LocA, LocB, AAQI);
   }
   bool pointsToConstantMemory(const MemoryLocation &Loc, bool OrLocal = false) {
     return AA.pointsToConstantMemory(Loc, AAQI, OrLocal);
   }
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+                               bool IgnoreLocals = false) {
+    return AA.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+  }
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc) {
     return AA.getModRefInfo(Call, Loc, AAQI);
   }
   ModRefInfo getModRefInfo(const CallBase *Call1, const CallBase *Call2) {
     return AA.getModRefInfo(Call1, Call2, AAQI);
   }
   ModRefInfo getModRefInfo(const Instruction *I,
                            const Optional<MemoryLocation> &OptLoc) {
public:
   /// @{
 
   /// The main low level interface to the alias analysis implementation.
   /// Returns an AliasResult indicating whether the two pointers are aliased to
   /// each other. This is the interface that must be implemented by specific
   /// alias analysis implementations.
   virtual AliasResult alias(const MemoryLocation &LocA,
                             const MemoryLocation &LocB, AAQueryInfo &AAQI) = 0;
 
-  /// Checks whether the given location points to constant memory, or if
-  /// \p OrLocal is true whether it points to a local alloca.
-  virtual bool pointsToConstantMemory(const MemoryLocation &Loc,
-                                      AAQueryInfo &AAQI, bool OrLocal) = 0;
-
   /// @}
   //===--------------------------------------------------------------------===//
   /// \name Simple mod/ref information
   /// @{
 
+  /// Returns a bitmask that should be unconditionally applied to the ModRef
+  /// info of a memory location. This allows us to eliminate Mod and/or Ref from
+  /// the ModRef info based on the knowledge that the memory location points to
+  /// constant and/or locally-invariant memory.
+  virtual ModRefInfo getModRefInfoMask(const MemoryLocation &Loc,
+                                       AAQueryInfo &AAQI,
+                                       bool IgnoreLocals) = 0;
+
   /// Get the ModRef info associated with a pointer argument of a callsite. The
   /// result's bits are set to indicate the allowed aliasing ModRef kinds. Note
   /// that these bits do not necessarily account for the overall behavior of
   /// the function, but rather only provide additional per-argument
   /// information.
   virtual ModRefInfo getArgModRefInfo(const CallBase *Call,
                                       unsigned ArgIdx) = 0;
 
public:
   explicit Model(AAResultT &Result, AAResults &AAR) : Result(Result) {}
   ~Model() override = default;
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI) override {
     return Result.alias(LocA, LocB, AAQI);
   }
 
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal) override {
-    return Result.pointsToConstantMemory(Loc, AAQI, OrLocal);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals) override {
+    return Result.getModRefInfoMask(Loc, AAQI, IgnoreLocals);
   }
 
   ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) override {
     return Result.getArgModRefInfo(Call, ArgIdx);
   }
 
   MemoryEffects getMemoryEffects(const CallBase *Call,
                                  AAQueryInfo &AAQI) override {
protected:
   AAResultBase(AAResultBase &&Arg) {}
 
 public:
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI) {
     return AliasResult::MayAlias;
   }
 
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal) {
-    return false;
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals) {
+    return ModRefInfo::ModRef;
   }
 
   ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {
     return ModRefInfo::ModRef;
   }
 
   MemoryEffects getMemoryEffects(const CallBase *Call, AAQueryInfo &AAQI) {
     return MemoryEffects::unknown();

llvm/include/llvm/Analysis/BasicAliasAnalysis.h

AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI);
 
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
                            AAQueryInfo &AAQI);
 
   ModRefInfo getModRefInfo(const CallBase *Call1, const CallBase *Call2,
                            AAQueryInfo &AAQI);
 
-  /// Chases pointers until we find a (constant global) or not.
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal);
+  /// Returns a bitmask that should be unconditionally applied to the ModRef
+  /// info of a memory location. This allows us to eliminate Mod and/or Ref
+  /// from the ModRef info based on the knowledge that the memory location
+  /// points to constant and/or locally-invariant memory.
+  ///
+  /// If IgnoreLocals is true, then this method returns NoModRef for memory
+  /// that points to a local alloca.
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals = false);
 
   /// Get the location associated with a pointer argument of a callsite.
   ModRefInfo getArgModRefInfo(const CallBase *Call, unsigned ArgIdx);
 
   /// Returns the behavior when calling the given call site.
   MemoryEffects getMemoryEffects(const CallBase *Call, AAQueryInfo &AAQI);
 
   /// Returns the behavior when calling the given function. For use when the

llvm/include/llvm/Analysis/ObjCARCAliasAnalysis.h

public:
   /// By definition, this result is stateless and so remains valid.
   bool invalidate(Function &, const PreservedAnalyses &,
                   FunctionAnalysisManager::Invalidator &) {
     return false;
   }
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI);
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals);
 
   using AAResultBase::getMemoryEffects;
   MemoryEffects getMemoryEffects(const Function *F);
 
   using AAResultBase::getModRefInfo;
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
                            AAQueryInfo &AAQI);
 };

llvm/include/llvm/Analysis/TypeBasedAliasAnalysis.h

public:
   /// By definition, this result is stateless and so remains valid.
   bool invalidate(Function &, const PreservedAnalyses &,
                   FunctionAnalysisManager::Invalidator &) {
     return false;
   }
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI);
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals);
   MemoryEffects getMemoryEffects(const CallBase *Call, AAQueryInfo &AAQI);
   MemoryEffects getMemoryEffects(const Function *F);
   ModRefInfo getModRefInfo(const CallBase *Call, const MemoryLocation &Loc,
                            AAQueryInfo &AAQI);
   ModRefInfo getModRefInfo(const CallBase *Call1, const CallBase *Call2,
                            AAQueryInfo &AAQI);
 
 private:

llvm/lib/Analysis/AliasAnalysis.cpp

if (AAQI.Depth == 0) {
     else if (Result == AliasResult::MustAlias)
       ++NumMustAlias;
     else
       ++NumMayAlias;
   }
   return Result;
 }
 
-bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
-                                       bool OrLocal) {
+ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
+                                        bool IgnoreLocals) {
   SimpleAAQueryInfo AAQIP(*this);
-  return pointsToConstantMemory(Loc, AAQIP, OrLocal);
+  return getModRefInfoMask(Loc, AAQIP, IgnoreLocals);
 }
 
-bool AAResults::pointsToConstantMemory(const MemoryLocation &Loc,
-                                       AAQueryInfo &AAQI, bool OrLocal) {
-  for (const auto &AA : AAs)
-    if (AA->pointsToConstantMemory(Loc, AAQI, OrLocal))
-      return true;
+ModRefInfo AAResults::getModRefInfoMask(const MemoryLocation &Loc,
+                                        AAQueryInfo &AAQI, bool IgnoreLocals) {
+  ModRefInfo Result = ModRefInfo::ModRef;
 
-  return false;
+  for (const auto &AA : AAs) {
+    Result &= AA->getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+
+    // Early-exit the moment we reach the bottom of the lattice.
+    if (isNoModRef(Result))
+      return ModRefInfo::NoModRef;
+  }
+
+  return Result;
 }
 
 ModRefInfo AAResults::getArgModRefInfo(const CallBase *Call, unsigned ArgIdx) {
   ModRefInfo Result = ModRefInfo::ModRef;
 
   for (const auto &AA : AAs) {
     Result &= AA->getArgModRefInfo(Call, ArgIdx);
 
for (const auto &I : llvm::enumerate(Call->args())) {
       if (ArgAlias != AliasResult::NoAlias)
         AllArgsMask |= getArgModRefInfo(Call, ArgIdx);
     }
     ArgMR &= AllArgsMask;
   }
 
   Result &= ArgMR | OtherMR;
 
-  // If Loc is a constant memory location, the call definitely could not
+  // Apply the ModRef mask. This ensures that if Loc is a constant memory
+  // location, we take into account the fact that the call definitely could not
   // modify the memory location.
-  if (isModSet(Result) && pointsToConstantMemory(Loc, AAQI, /*OrLocal*/ false))
-    Result &= ModRefInfo::Ref;
+  if (!isNoModRef(Result))
+    Result &= getModRefInfoMask(Loc);
 
   return Result;
 }
 
 ModRefInfo AAResults::getModRefInfo(const CallBase *Call1,
                                     const CallBase *Call2) {
   SimpleAAQueryInfo AAQIP(*this);
   return getModRefInfo(Call1, Call2, AAQIP);
ModRefInfo AAResults::getModRefInfo(const StoreInst *S,
 
   if (Loc.Ptr) {
     AliasResult AR = alias(MemoryLocation::get(S), Loc, AAQI);
     // If the store address cannot alias the pointer in question, then the
     // specified memory cannot be modified by the store.
     if (AR == AliasResult::NoAlias)
       return ModRefInfo::NoModRef;
 
-    // If the pointer is a pointer to constant memory, then it could not have
-    // been modified by this store.
-    if (pointsToConstantMemory(Loc, AAQI))
+    // Examine the ModRef mask. If Mod isn't present, then return NoModRef.
+    // This ensures that if Loc is a constant memory location, we take into
+    // account the fact that the store definitely could not modify the memory
+    // location.
+    if (!isModSet(getModRefInfoMask(Loc)))
       return ModRefInfo::NoModRef;
   }
 
   // Otherwise, a store just writes.
   return ModRefInfo::Mod;
 }
 
 ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
                                     const MemoryLocation &Loc) {
   SimpleAAQueryInfo AAQIP(*this);
   return getModRefInfo(S, Loc, AAQIP);
 }
 
 ModRefInfo AAResults::getModRefInfo(const FenceInst *S,
                                     const MemoryLocation &Loc,
                                     AAQueryInfo &AAQI) {
-  // If we know that the location is a constant memory location, the fence
-  // cannot modify this location.
-  if (Loc.Ptr && pointsToConstantMemory(Loc, AAQI))
-    return ModRefInfo::Ref;
+  // All we know about a fence instruction is what we get from the ModRef
+  // mask: if Loc is a constant memory location, the fence definitely could
+  // not modify it.
+  if (Loc.Ptr)
+    return getModRefInfoMask(Loc);
   return ModRefInfo::ModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(const VAArgInst *V,
                                     const MemoryLocation &Loc) {
   SimpleAAQueryInfo AAQIP(*this);
   return getModRefInfo(V, Loc, AAQIP);
 }
 
 ModRefInfo AAResults::getModRefInfo(const VAArgInst *V,
                                     const MemoryLocation &Loc,
                                     AAQueryInfo &AAQI) {
   if (Loc.Ptr) {
     AliasResult AR = alias(MemoryLocation::get(V), Loc, AAQI);
     // If the va_arg address cannot alias the pointer in question, then the
     // specified memory cannot be accessed by the va_arg.
     if (AR == AliasResult::NoAlias)
       return ModRefInfo::NoModRef;
 
-    // If the pointer is a pointer to constant memory, then it could not have
+    // If the pointer is a pointer to invariant memory, then it could not have
     // been modified by this va_arg.
-    if (pointsToConstantMemory(Loc, AAQI))
-      return ModRefInfo::NoModRef;
+    return getModRefInfoMask(Loc, AAQI);
   }
 
   // Otherwise, a va_arg reads and writes.
   return ModRefInfo::ModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(const CatchPadInst *CatchPad,
                                     const MemoryLocation &Loc) {
   SimpleAAQueryInfo AAQIP(*this);
   return getModRefInfo(CatchPad, Loc, AAQIP);
 }
 
 ModRefInfo AAResults::getModRefInfo(const CatchPadInst *CatchPad,
                                     const MemoryLocation &Loc,
                                     AAQueryInfo &AAQI) {
   if (Loc.Ptr) {
-    // If the pointer is a pointer to constant memory,
+    // If the pointer is a pointer to invariant memory,
     // then it could not have been modified by this catchpad.
-    if (pointsToConstantMemory(Loc, AAQI))
-      return ModRefInfo::NoModRef;
+    return getModRefInfoMask(Loc, AAQI);
   }
 
   // Otherwise, a catchpad reads and writes.
   return ModRefInfo::ModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(const CatchReturnInst *CatchRet,
                                     const MemoryLocation &Loc) {
   SimpleAAQueryInfo AAQIP(*this);
   return getModRefInfo(CatchRet, Loc, AAQIP);
 }
 
 ModRefInfo AAResults::getModRefInfo(const CatchReturnInst *CatchRet,
                                     const MemoryLocation &Loc,
                                     AAQueryInfo &AAQI) {
   if (Loc.Ptr) {
-    // If the pointer is a pointer to constant memory,
+    // If the pointer is a pointer to invariant memory,
     // then it could not have been modified by this catchpad.
-    if (pointsToConstantMemory(Loc, AAQI))
-      return ModRefInfo::NoModRef;
+    return getModRefInfoMask(Loc, AAQI);
   }
 
   // Otherwise, a catchret reads and writes.
   return ModRefInfo::ModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(const AtomicCmpXchgInst *CX,
                                     const MemoryLocation &Loc) {

llvm/lib/Analysis/BasicAliasAnalysis.cpp

BasicAAResult::DecomposeGEPExpression(const Value *V, const DataLayout &DL,
   } while (--MaxLookup);
 
   // If the chain of expressions is too deep, just return early.
   Decomposed.Base = V;
   SearchLimitReached++;
   return Decomposed;
 }
 
-/// Returns whether the given pointer value points to memory that is local to
-/// the function, with global constants being considered local to all
-/// functions.
-bool BasicAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
-                                           AAQueryInfo &AAQI, bool OrLocal) {
+ModRefInfo BasicAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+                                            AAQueryInfo &AAQI,
+                                            bool IgnoreLocals) {
   assert(Visited.empty() && "Visited must be cleared after use!");
-  auto _ = make_scope_exit([&]{ Visited.clear(); });
+  auto _ = make_scope_exit([&] { Visited.clear(); });
 
   unsigned MaxLookup = 8;
   SmallVector<const Value *, 16> Worklist;
   Worklist.push_back(Loc.Ptr);
+  ModRefInfo Result = ModRefInfo::NoModRef;
+
   do {
     const Value *V = getUnderlyingObject(Worklist.pop_back_val());
     if (!Visited.insert(V).second)
       continue;
 
-    // An alloca instruction defines local memory.
-    if (OrLocal && isa<AllocaInst>(V))
+    // Ignore allocas if we were instructed to do so.
+    if (IgnoreLocals && isa<AllocaInst>(V))
+      continue;
+
+    // If the location points to memory that is known to be invariant for
+    // the life of the underlying SSA value, then we can exclude Mod from
+    // the set of valid memory effects.
+    //
+    // An argument that is marked readonly and noalias is known to be
+    // invariant while that function is executing.
+    if (const Argument *Arg = dyn_cast<Argument>(V)) {
+      if (Arg->hasNoAliasAttr() && Arg->onlyReadsMemory()) {
+        Result |= ModRefInfo::Ref;
         continue;
+      }
+    }
 
-    // A global constant counts as local memory for our purposes.
+    // A global constant can't be mutated.
     if (const GlobalVariable *GV = dyn_cast<GlobalVariable>(V)) {
       // Note: this doesn't require GV to be "ODR" because it isn't legal for a
       // global to be marked constant in some modules and non-constant in
       // others.  GV may even be a declaration, not a definition.
       if (!GV->isConstant())
-        return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+        return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
       continue;
     }
 
     // If both select values point to local memory, then so does the select.
     if (const SelectInst *SI = dyn_cast<SelectInst>(V)) {
       Worklist.push_back(SI->getTrueValue());
       Worklist.push_back(SI->getFalseValue());
       continue;
     }
 
     // If all values incoming to a phi node point to local memory, then so does
     // the phi.
     if (const PHINode *PN = dyn_cast<PHINode>(V)) {
       // Don't bother inspecting phi nodes with many operands.
       if (PN->getNumIncomingValues() > MaxLookup)
-        return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+        return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
       append_range(Worklist, PN->incoming_values());
       continue;
     }
 
     // Otherwise be conservative.
-    return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+    return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
   } while (!Worklist.empty() && --MaxLookup);
 
-  return Worklist.empty();
+  // If we hit the maximum number of instructions to examine, be conservative.
+  if (!Worklist.empty())
+    return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
+
+  return Result;
 }
 
 static bool isIntrinsicCall(const CallBase *Call, Intrinsic::ID IID) {
   const IntrinsicInst *II = dyn_cast<IntrinsicInst>(Call);
   return II && II->getIntrinsicID() == IID;
 }
 
 static MemoryEffects getMemoryEffectsFromAttrs(AttributeSet Attrs) {
case Intrinsic::experimental_deoptimize:
     // inaccessible memory to model control dependence.
     return MemoryEffects::readOnly() |
            MemoryEffects::inaccessibleMemOnly(ModRefInfo::ModRef);
   }
 
   return getMemoryEffectsFromAttrs(F->getAttributes().getFnAttrs());
 }
 
 ModRefInfo BasicAAResult::getArgModRefInfo(const CallBase *Call,
                                            unsigned ArgIdx) {
   if (Call->paramHasAttr(ArgIdx, Attribute::WriteOnly))
     return ModRefInfo::Mod;
 
   if (Call->paramHasAttr(ArgIdx, Attribute::ReadOnly))
     return ModRefInfo::Ref;
 
   if (Call->paramHasAttr(ArgIdx, Attribute::ReadNone))
     return ModRefInfo::NoModRef;
static const Function *getParent(const Value *V) {
   if (const Argument *arg = dyn_cast<Argument>(V))
     return arg->getParent();
 
   return nullptr;
 }
 
 static bool notDifferentParent(const Value *O1, const Value *O2) {
 
   const Function *F1 = getParent(O1);
   const Function *F2 = getParent(O2);
 
   return !F1 || !F2 || F1 == F2;
 }
 #endif
 
 AliasResult BasicAAResult::alias(const MemoryLocation &LocA,
                                  const MemoryLocation &LocB,
                                  AAQueryInfo &AAQI) {

llvm/lib/Analysis/MemoryDependenceAnalysis.cpp

if (LoadInst *LI = dyn_cast<LoadInst>(Inst)) {
         }
 
         // Random may-alias loads don't depend on each other without a
         // dependence.
         continue;
       }
 
       // Stores don't alias loads from read-only memory.
-      if (BatchAA.pointsToConstantMemory(LoadLoc))
+      if (!isModSet(BatchAA.getModRefInfoMask(LoadLoc)))
         continue;
 
       // Stores depend on may/must aliased loads.
       return MemDepResult::getDef(Inst);
     }
 
     if (StoreInst *SI = dyn_cast<StoreInst>(Inst)) {
       // Atomic stores have complications involved.

llvm/lib/Analysis/MemorySSA.cpp

 
 } // end anonymous namespace
 
 template <typename AliasAnalysisType>
 static bool isUseTriviallyOptimizableToLiveOnEntry(AliasAnalysisType &AA,
                                                    const Instruction *I) {
   // If the memory can't be changed, then loads of the memory can't be
   // clobbered.
-  if (auto *LI = dyn_cast<LoadInst>(I))
+  if (auto *LI = dyn_cast<LoadInst>(I)) {
     return I->hasMetadata(LLVMContext::MD_invariant_load) ||
-           AA.pointsToConstantMemory(MemoryLocation::get(LI));
+           !isModSet(AA.getModRefInfoMask(MemoryLocation::get(LI)));
+  }
   return false;
 }
 
 /// Verifies that `Start` is clobbered by `ClobberAt`, and that nothing
 /// inbetween `Start` and `ClobberAt` can clobbers `Start`.
 ///
 /// This is meant to be as simple and self-contained as possible. Because it
 /// uses no cache, etc., it can be relatively expensive.

llvm/lib/Analysis/ObjCARCAliasAnalysis.cpp

if (Result == AliasResult::NoAlias)
       return AliasResult::NoAlias;
   }
 
   // If that failed, fail. We don't need to chain here, since that's covered
   // by the earlier precise query.
   return AliasResult::MayAlias;
 }
 
-bool ObjCARCAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
-                                             AAQueryInfo &AAQI, bool OrLocal) {
+ModRefInfo ObjCARCAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+                                              AAQueryInfo &AAQI,
+                                              bool IgnoreLocals) {
   if (!EnableARCOpts)
-    return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+    return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
 
   // First, strip off no-ops, including ObjC-specific no-ops, and try making
   // a precise alias query.
   const Value *S = GetRCIdentityRoot(Loc.Ptr);
-  if (AAResultBase::pointsToConstantMemory(
-          MemoryLocation(S, Loc.Size, Loc.AATags), AAQI, OrLocal))
-    return true;
+  if (isNoModRef(AAResultBase::getModRefInfoMask(
+          MemoryLocation(S, Loc.Size, Loc.AATags), AAQI, IgnoreLocals)))
+    return ModRefInfo::NoModRef;
 
   // If that failed, climb to the underlying object, including climbing through
   // ObjC-specific no-ops, and try making an imprecise alias query.
   const Value *U = GetUnderlyingObjCPtr(S);
   if (U != S)
-    return AAResultBase::pointsToConstantMemory(
-        MemoryLocation::getBeforeOrAfter(U), AAQI, OrLocal);
+    return AAResultBase::getModRefInfoMask(MemoryLocation::getBeforeOrAfter(U),
+                                           AAQI, IgnoreLocals);
 
   // If that failed, fail. We don't need to chain here, since that's covered
   // by the earlier precise query.
-  return false;
+  return ModRefInfo::ModRef;
 }
 
 MemoryEffects ObjCARCAAResult::getMemoryEffects(const Function *F) {
   if (!EnableARCOpts)
     return AAResultBase::getMemoryEffects(F);
 
   switch (GetFunctionClass(F)) {
   case ARCInstKind::NoopCast:

llvm/lib/Analysis/TypeBasedAliasAnalysis.cpp

AliasResult TypeBasedAAResult::alias(const MemoryLocation &LocA,
   // If accesses may alias, chain to the next AliasAnalysis.
   if (Aliases(LocA.AATags.TBAA, LocB.AATags.TBAA))
     return AAResultBase::alias(LocA, LocB, AAQI);
 
   // Otherwise return a definitive result.
   return AliasResult::NoAlias;
 }
 
-bool TypeBasedAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
+ModRefInfo TypeBasedAAResult::getModRefInfoMask(const MemoryLocation &Loc,
                                                 AAQueryInfo &AAQI,
-                                               bool OrLocal) {
+                                                bool IgnoreLocals) {
   if (!EnableTBAA)
-    return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+    return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
 
   const MDNode *M = Loc.AATags.TBAA;
   if (!M)
-    return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+    return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
 
   // If this is an "immutable" type, we can assume the pointer is pointing
   // to constant memory.
   if ((!isStructPathTBAA(M) && TBAANode(M).isTypeImmutable()) ||
       (isStructPathTBAA(M) && TBAAStructTagNode(M).isTypeImmutable()))
-    return true;
+    return ModRefInfo::NoModRef;
 
-  return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+  return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
 }
 
 MemoryEffects TypeBasedAAResult::getMemoryEffects(const CallBase *Call,
                                                   AAQueryInfo &AAQI) {
   if (!EnableTBAA)
     return AAResultBase::getMemoryEffects(Call, AAQI);
 
   // If this is an "immutable" type, the access is not observable.

llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.h

public:
   /// By definition, this result is stateless and so remains valid.
   bool invalidate(Function &, const PreservedAnalyses &,
                   FunctionAnalysisManager::Invalidator &Inv) {
     return false;
   }
 
   AliasResult alias(const MemoryLocation &LocA, const MemoryLocation &LocB,
                     AAQueryInfo &AAQI);
-  bool pointsToConstantMemory(const MemoryLocation &Loc, AAQueryInfo &AAQI,
-                              bool OrLocal);
+  ModRefInfo getModRefInfoMask(const MemoryLocation &Loc, AAQueryInfo &AAQI,
+                               bool IgnoreLocals);
 };
 
 /// Analysis pass providing a never-invalidated alias analysis result.
 class AMDGPUAA : public AnalysisInfoMixin<AMDGPUAA> {
   friend AnalysisInfoMixin<AMDGPUAA>;
 
   static AnalysisKey Key;
 

llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp

if (const LoadInst *LI = dyn_cast<LoadInst>(ObjA)) {
       }
     }
   }
 
   // Forward the query to the next alias analysis.
   return AAResultBase::alias(LocA, LocB, AAQI);
 }
 
-bool AMDGPUAAResult::pointsToConstantMemory(const MemoryLocation &Loc,
-                                            AAQueryInfo &AAQI, bool OrLocal) {
+ModRefInfo AMDGPUAAResult::getModRefInfoMask(const MemoryLocation &Loc,
+                                             AAQueryInfo &AAQI,
+                                             bool IgnoreLocals) {
   unsigned AS = Loc.Ptr->getType()->getPointerAddressSpace();
   if (AS == AMDGPUAS::CONSTANT_ADDRESS ||
       AS == AMDGPUAS::CONSTANT_ADDRESS_32BIT)
-    return true;
+    return ModRefInfo::NoModRef;
 
   const Value *Base = getUnderlyingObject(Loc.Ptr);
   AS = Base->getType()->getPointerAddressSpace();
   if (AS == AMDGPUAS::CONSTANT_ADDRESS ||
       AS == AMDGPUAS::CONSTANT_ADDRESS_32BIT)
-    return true;
+    return ModRefInfo::NoModRef;
 
-  if (const GlobalVariable *GV = dyn_cast<GlobalVariable>(Base)) {
-    if (GV->isConstant())
-      return true;
-  } else if (const Argument *Arg = dyn_cast<Argument>(Base)) {
-    const Function *F = Arg->getParent();
-
-    // Only assume constant memory for arguments on kernels.
-    switch (F->getCallingConv()) {
-    default:
-      return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
-    case CallingConv::AMDGPU_LS:
-    case CallingConv::AMDGPU_HS:
-    case CallingConv::AMDGPU_ES:
-    case CallingConv::AMDGPU_GS:
-    case CallingConv::AMDGPU_VS:
-    case CallingConv::AMDGPU_PS:
-    case CallingConv::AMDGPU_CS:
-    case CallingConv::AMDGPU_KERNEL:
-    case CallingConv::SPIR_KERNEL:
-      break;
-    }
-
-    unsigned ArgNo = Arg->getArgNo();
-    /* On an argument, ReadOnly attribute indicates that the function does
-       not write through this pointer argument, even though it may write
-       to the memory that the pointer points to.
-       On an argument, ReadNone attribute indicates that the function does
-       not dereference that pointer argument, even though it may read or write
-       the memory that the pointer points to if accessed through other pointers.
-     */
-    if (F->hasParamAttribute(ArgNo, Attribute::NoAlias) &&
-        (F->hasParamAttribute(ArgNo, Attribute::ReadNone) ||
-         F->hasParamAttribute(ArgNo, Attribute::ReadOnly))) {
-      return true;
-    }
-  }
-  return AAResultBase::pointsToConstantMemory(Loc, AAQI, OrLocal);
+  return AAResultBase::getModRefInfoMask(Loc, AAQI, IgnoreLocals);
 }

llvm/lib/Transforms/IPO/FunctionAttrs.cpp

static MemoryEffects checkFunctionMemoryAccess(Function &F, bool ThisBody,
 
   MemoryEffects ME = MemoryEffects::none();
   // Inalloca and preallocated arguments are always clobbered by the call.
   if (F.getAttributes().hasAttrSomewhere(Attribute::InAlloca) ||
       F.getAttributes().hasAttrSomewhere(Attribute::Preallocated))
     ME |= MemoryEffects::argMemOnly(ModRefInfo::ModRef);
 
   auto AddLocAccess = [&](const MemoryLocation &Loc, ModRefInfo MR) {
-    // Ignore accesses to local memory.
-    if (AAR.pointsToConstantMemory(Loc, /*OrLocal=*/true))
+    // Ignore accesses to known-invariant or local memory.
+    MR &= AAR.getModRefInfoMask(Loc, /*IgnoreLocal=*/true);
+    if (isNoModRef(MR))
       return;
 
     const Value *UO = getUnderlyingObject(Loc.Ptr);
     assert(!isa<AllocaInst>(UO) &&
-           "Should have been handled by pointsToConstantMemory()");
+           "Should have been handled by getModRefInfoMask()");
     if (isa<Argument>(UO)) {
       ME |= MemoryEffects::argMemOnly(MR);
       return;
     }
 
     // If it's not an identified object, it might be an argument.
     if (!isIdentifiedObject(UO))
       ME |= MemoryEffects::argMemOnly(MR);

llvm/lib/Transforms/InstCombine/InstCombineCalls.cpp

Instruction *InstCombinerImpl::SimplifyAnyMemTransfer(AnyMemTransferInst *MI) {
   if (!CopySrcAlign || *CopySrcAlign < SrcAlign) {
     MI->setSourceAlignment(SrcAlign);
     return MI;
   }
 
   // If we have a store to a location which is known constant, we can conclude
   // that the store must be storing the constant value (else the memory
   // wouldn't be constant), and this must be a noop.
-  if (AA->pointsToConstantMemory(MI->getDest())) {
+  if (!isModSet(AA->getModRefInfoMask(MI->getDest()))) {
     // Set the size of the copy to 0, it will be deleted on the next iteration.
     MI->setLength(Constant::getNullValue(MI->getLength()->getType()));
     return MI;
   }
 
   // If the source is provably undef, the memcpy/memmove doesn't do anything
   // (unless the transfer is volatile).
   if (hasUndefSource(MI) && !MI->isVolatile()) {
Instruction *InstCombinerImpl::SimplifyAnyMemSet(AnyMemSetInst *MI) {
   if (!MemSetAlign || *MemSetAlign < KnownAlignment) {
     MI->setDestAlignment(KnownAlignment);
     return MI;
   }
 
   // If we have a store to a location which is known constant, we can conclude
   // that the store must be storing the constant value (else the memory
   // wouldn't be constant), and this must be a noop.
-  if (AA->pointsToConstantMemory(MI->getDest())) {
+  if (!isModSet(AA->getModRefInfoMask(MI->getDest()))) {
     // Set the size of the copy to 0, it will be deleted on the next iteration.
     MI->setLength(Constant::getNullValue(MI->getLength()->getType()));
     return MI;
   }
 
   // Remove memset with an undef value.
   // FIXME: This is technically incorrect because it might overwrite a poison
   // value. Change to PoisonValue once #52930 is resolved.

llvm/lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp

 using namespace llvm;
 using namespace PatternMatch;
 
 #define DEBUG_TYPE "instcombine"
 
 STATISTIC(NumDeadStore,    "Number of dead stores eliminated");
 STATISTIC(NumGlobalCopies, "Number of allocas copied from constant global");
 
-/// isOnlyCopiedFromConstantGlobal - Recursively walk the uses of a (derived)
+/// isOnlyCopiedFromConstantMemory - Recursively walk the uses of a (derived)
 /// pointer to an alloca.  Ignore any reads of the pointer, return false if we
 /// see any stores or other unknown uses.  If we see pointer arithmetic, keep
 /// track of whether it moves the pointer (with IsOffset) but otherwise traverse
 /// the uses.  If we see a memcpy/memmove that targets an unoffseted pointer to
-/// the alloca, and if the source pointer is a pointer to a constant global, we
-/// can optimize this.
+/// the alloca, and if the source pointer is a pointer to a constant memory
+/// location, we can optimize this.
 static bool
 isOnlyCopiedFromConstantMemory(AAResults *AA, AllocaInst *V,
                                MemTransferInst *&TheCopy,
                                SmallVectorImpl<Instruction *> &ToDelete) {
   // We track lifetime intrinsics as we encounter them.  If we decide to go
-  // ahead and replace the value with the global, this lets the caller quickly
-  // eliminate the markers.
+  // ahead and replace the value with the memory location, this lets the caller
+  // quickly eliminate the markers.
 
   SmallVector<std::pair<Value *, bool>, 35> ValuesToInspect;
   ValuesToInspect.emplace_back(V, false);
   while (!ValuesToInspect.empty()) {
     auto ValuePair = ValuesToInspect.pop_back_val();
     const bool IsOffset = ValuePair.second;
     for (auto &U : ValuePair.first->uses()) {
       auto *I = cast<Instruction>(U.getUser());
for (auto &U : ValuePair.first->uses()) {
 
       // If the pointer has been offset from the start of the alloca, we can't
       // safely handle this.
       if (IsOffset) return false;
 
       // If the memintrinsic isn't using the alloca as the dest, reject it.
       if (U.getOperandNo() != 0) return false;
 
-      // If the source of the memcpy/move is not a constant global, reject it.
-      if (!AA->pointsToConstantMemory(MI->getSource()))
+      // If the source of the memcpy/move is not constant, reject it.
+      if (isModSet(AA->getModRefInfoMask(MI->getSource())))
         return false;
 
       // Otherwise, the transform is safe.  Remember the copy instruction.
       TheCopy = MI;
     }
   }
   return true;
 }
 
-/// isOnlyCopiedFromConstantGlobal - Return true if the specified alloca is only
-/// modified by a copy from a constant global.  If we can prove this, we can
-/// replace any uses of the alloca with uses of the global directly.
+/// isOnlyCopiedFromConstantMemory - Return true if the specified alloca is only
+/// modified by a copy from a constant memory location. If we can prove this, we
+/// can replace any uses of the alloca with uses of the memory location
+/// directly.
 static MemTransferInst *
 isOnlyCopiedFromConstantMemory(AAResults *AA,
                                AllocaInst *AI,
                                SmallVectorImpl<Instruction *> &ToDelete) {
   MemTransferInst *TheCopy = nullptr;
   if (isOnlyCopiedFromConstantMemory(AA, AI, TheCopy, ToDelete))
     return TheCopy;
   return nullptr;
if (DL.getTypeAllocSize(AI.getAllocatedType()).getKnownMinSize() == 0) {
         if (AI.getType() != EntryAI->getType())
           return new BitCastInst(EntryAI, AI.getType());
         return replaceInstUsesWith(AI, EntryAI);
       }
     }
   }
 
   // Check to see if this allocation is only modified by a memcpy/memmove from
-  // a constant whose alignment is equal to or exceeds that of the allocation.
-  // If this is the case, we can change all users to use the constant global
-  // instead.  This is commonly produced by the CFE by constructs like "void
-  // foo() { int A[] = {1,2,3,4,5,6,7,8,9...}; }" if 'A' is only subsequently
-  // read.
+  // a memory location whose alignment is equal to or exceeds that of the
+  // allocation. If this is the case, we can change all users to use the
+  // constant memory location instead.  This is commonly produced by the CFE by
+  // constructs like "void foo() { int A[] = {1,2,3,4,5,6,7,8,9...}; }" if 'A'
+  // is only subsequently read.
   SmallVector<Instruction *, 4> ToDelete;
   if (MemTransferInst *Copy = isOnlyCopiedFromConstantMemory(AA, &AI, ToDelete)) {
     Value *TheSrc = Copy->getSource();
     Align AllocaAlign = AI.getAlign();
     Align SourceAlign = getOrEnforceKnownAlignment(
       TheSrc, AllocaAlign, DL, &AI, &AC, &DT);
     if (AllocaAlign <= SourceAlign &&
         isDereferenceableForAllocaSize(TheSrc, &AI, DL) &&
if (GetElementPtrInst *GEP = dyn_cast<GetElementPtrInst>(Ptr)) {
           return eraseInstFromFunction(SI);
       }
     }
   }
 
   // If we have a store to a location which is known constant, we can conclude
   // that the store must be storing the constant value (else the memory
   // wouldn't be constant), and this must be a noop.
-  if (AA->pointsToConstantMemory(Ptr))
+  if (!isModSet(AA->getModRefInfoMask(Ptr)))
     return eraseInstFromFunction(SI);
 
   // Do really simple DSE, to catch cases where there are several consecutive
   // stores to the same location, separated by a few arithmetic operations. This
   // situation often occurs with bitfield accesses.
   BasicBlock::iterator BBI(SI);
   for (unsigned ScanInsts = 6; BBI != SI.getParent()->begin() && ScanInsts;
        --ScanInsts) {

llvm/lib/Transforms/Scalar/LICM.cpp

bool llvm::canSinkOrHoistInst(Instruction &I, AAResults *AA, DominatorTree *DT,
   MemorySSA *MSSA = MSSAU.getMemorySSA();
   // Loads have extra constraints we have to verify before we can hoist them.
   if (LoadInst *LI = dyn_cast<LoadInst>(&I)) {
     if (!LI->isUnordered())
       return false; // Don't sink/hoist volatile or ordered atomic loads!
 
     // Loads from constant memory are always safe to move, even if they end up
     // in the same alias set as something that ends up being modified.
-    if (AA->pointsToConstantMemory(LI->getOperand(0)))
+    if (!isModSet(AA->getModRefInfoMask(LI->getOperand(0))))
       return true;
     if (LI->hasMetadata(LLVMContext::MD_invariant_load))
       return true;
 
     if (LI->isAtomic() && !TargetExecutesOncePerLoop)
       return false; // Don't risk duplicating unordered loads
 
     // This checks for an invariant.start dominating the load.

llvm/lib/Transforms/Scalar/LoopPredication.cpp

if (SE->isLoopInvariant(S, L))
     return true;
 
   // Handle a particular important case which SCEV doesn't yet know about which
   // shows up in range checks on arrays with immutable lengths.
   // TODO: This should be sunk inside SCEV.
   if (const SCEVUnknown *U = dyn_cast<SCEVUnknown>(S))
     if (const auto *LI = dyn_cast<LoadInst>(U->getValue()))
       if (LI->isUnordered() && L->hasLoopInvariantOperands(LI))
-        if (AA->pointsToConstantMemory(LI->getOperand(0)) ||
+        if (!isModSet(AA->getModRefInfoMask(LI->getOperand(0))) ||
             LI->hasMetadata(LLVMContext::MD_invariant_load))
           return true;
   return false;
 }
 
 Optional<Value *> LoopPredication::widenICmpRangeCheckIncrementingLoop(
     LoopICmp LatchCheck, LoopICmp RangeCheck,
     SCEVExpander &Expander, Instruction *Guard) {

llvm/test/Analysis/BasicAA/constant-memory.ll

 ; RUN: opt < %s -passes=aa-eval -print-all-alias-modref-info 2>&1 | FileCheck %s
 
 @c = constant [8 x i32] zeroinitializer
 
 declare void @dummy()
 
 declare void @foo(ptr)
 
-; FIXME: This could be NoModRef
 ; CHECK-LABEL: Function: basic
-; CHECK: Just Ref: Ptr: i32* @c <->  call void @dummy()
+; CHECK: NoModRef: Ptr: i32* @c <->  call void @dummy()
 define void @basic(ptr %p) {
   call void @dummy()
   load i32, ptr @c
   ret void
 }
 
-; FIXME: This could be NoModRef
 ; CHECK-LABEL: Function: recphi
-; CHECK: Just Ref: Ptr: i32* %p <->  call void @dummy()
+; CHECK: NoModRef: Ptr: i32* %p <->  call void @dummy()
 define void @recphi() {
 entry:
   br label %loop
 
 loop:
   %p = phi ptr [ @c, %entry ], [ %p.next, %loop ]
   call void @dummy()
   load i32, ptr %p
   %p.next = getelementptr i32, ptr %p, i64 1
   %c = icmp ne ptr %p.next, getelementptr (i32, ptr @c, i64 8)
   br i1 %c, label %loop, label %exit
 
 exit:
   ret void
 }
 
-; Tests that readonly noalias doesn't imply !Mod yet.
+; Tests that readonly noalias implies !Mod.
 ;
 ; CHECK-LABEL: Function: readonly_noalias
-; CHECK: Both ModRef: Ptr: i32* %p <->  call void @foo(ptr %p)
+; CHECK: Just Ref: Ptr: i32* %p <->  call void @foo(ptr %p)
 define void @readonly_noalias(ptr readonly noalias %p) {
     call void @foo(ptr %p)
     load i32, ptr %p
     ret void
 }
 
-; Tests that readnone noalias doesn't imply !Mod yet.
+; Tests that readnone noalias implies !Mod.
 ;
 ; CHECK-LABEL: Function: readnone_noalias
-; CHECK: Both ModRef: Ptr: i32* %p <->  call void @foo(ptr %p)
+; CHECK: Just Ref: Ptr: i32* %p <->  call void @foo(ptr %p)
 define void @readnone_noalias(ptr readnone noalias %p) {
     call void @foo(ptr %p)
     load i32, ptr %p
     ret void
 }
 
 ; Tests that writeonly noalias doesn't imply !Ref (since it's still possible
 ; to read from the object through other pointers if the pointer wasn't

llvm/test/Transforms/InstCombine/memcpy-from-global.ll

entry:
   %a = alloca [4 x float], align 4
   call void @llvm.lifetime.start.p0(i64 16, ptr %a)
   call void @llvm.memcpy.p0.p1.i64(ptr align 4 %a, ptr addrspace(1) align 4 @I, i64 16, i1 true)
   %g = getelementptr inbounds [4 x float], ptr %a, i64 0, i64 %i
   %r = load float, ptr %g, align 4
   ret float %r
 }
 
-; Tests that we can't eliminate allocas copied from readonly noalias pointers yet.
+; Tests that we can eliminate allocas copied from readonly noalias pointers.
 define void @memcpy_from_readonly_noalias(ptr readonly noalias align 8 dereferenceable(124) %arg) {
 ; CHECK-LABEL: @memcpy_from_readonly_noalias(
-; CHECK-NEXT:    [[ALLOCA:%.*]] = alloca [[T:%.*]], align 8
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i64(ptr noundef nonnull align 8 dereferenceable(124) [[ALLOCA]], ptr noundef nonnull align 8 dereferenceable(124) [[ARG:%.*]], i64 124, i1 false)
-; CHECK-NEXT:    call void @bar(ptr nonnull [[ALLOCA]]) #[[ATTR3]]
+; CHECK-NEXT:    call void @bar(ptr nonnull [[ARG:%.*]]) #[[ATTR3]]
 ; CHECK-NEXT:    ret void
 ;
   %alloca = alloca %T, align 8
   call void @llvm.memcpy.p0.p0.i64(ptr %alloca, ptr %arg, i64 124, i1 false)
   call void @bar(ptr %alloca) readonly
   ret void
 }
 

llvm/test/Transforms/InstCombine/store.ll

 define void @store_to_constant() {
 ; CHECK-LABEL: @store_to_constant(
 ; CHECK-NEXT:    ret void
 ;
   store i32 0, ptr @Unknown
   ret void
 }
 
-; We can't delete stores to readonly noalias pointers yet.
+; Delete stores to readonly noalias pointers.
 define void @store_to_readonly_noalias(ptr readonly noalias %0) {
 ; CHECK-LABEL: @store_to_readonly_noalias(
-; CHECK-NEXT:    store i32 3, ptr [[TMP0:%.*]], align 4
 ; CHECK-NEXT:    ret void
-;
   store i32 3, ptr %0, align 4
   ret void
 }
 
 !0 = !{!4, !4, i64 0}
 !1 = !{!"omnipotent char", !2}
 !2 = !{!"Simple C/C++ TBAA"}
 !3 = !{!"float", !1}
 !4 = !{!"int", !1}