Add must alias info to ModRefInfo.

[asbirlea]

Summary

Add an additional bit to ModRefInfo, MRI_Must, to be set for known must aliases.
Conservatively this bit will not be set (equivalent to MayAlias or NoAlias).

Notes:

• MRI_Must is almost entirely set in the AAResults methods, the remaining changes are trying to preserve it.
• Only some small changes to make custom AA passes set MRI_Must (BasicAA).
• GlobalsModRef already declares a bit of the same value as MRI_Must (MayReadAnyGlobal). No changes to shift the value of MayReadAnyGlobal (see AlignedMap). FunctionInfo.getModRef() masks out everything except MRI_ModRef, so correctness is preserved, but the MRI_Must info is not set/used.
• There are cases where the MRI_Must is not set, e.g. 2 calls that only read will return MRI_NoModRef, though they may read from exactly the same location.

[george.burgess.iv]

lib/Analysis/AliasAnalysis.cpp

128

Probably-silly question: what's the difference between Result == MRI_NoModRef and Result == MRI_Must?

(If the answer is "nothing", should we ever really see MRI_Must on its own?)

[asbirlea]

lib/Analysis/AliasAnalysis.cpp

128

Not silly at all. In theory Result == MRI_Must can happen when checking ModRefInfo for 2 locations that alias each other, and both only read.
I updated the check for safety. I am not sure we should see MRI_Must on its own, but there is nothing preventing it right now.

[asbirlea]

Gentle ping.

[hfinkel]

This certainly needs some test cases.

include/llvm/Analysis/AliasAnalysis.h

109

We should explicitly document here whether this is only for complete overlap, or whether it includes partial overlaps.

lib/Analysis/AliasAnalysis.cpp

128

Right, we currently don't consider read/read aliasing. With this change, we do. Why? (And this should be documented explicitly).

151

How is including MR's Must bit useful? What if the call has multiple pointer-typed arguments?

529

This still needs to be fixed?

lib/Analysis/AliasAnalysisEvaluator.cpp

248

We'll need to print the information for testing.

lib/Analysis/BasicAliasAnalysis.cpp

780

Why is this useful? If we don't know which operand it is, I don't see how we use the information that it must alias one of the operands.

lib/Analysis/GlobalsModRef.cpp

87

I think that, if passes want to define their own "local" bits, we should create a well-defined way to make this happen. Define some enum, something like MRI_FirstUserBit, or similar, and use that to define the bit number here.

[• dberlin]

lib/Analysis/AliasAnalysis.cpp

151

See below.
If you are trying to move a call, knowing that another call mustaliases everything you do, means that you don't have to give up and not move it. you can instead try to move it as well.

lib/Analysis/BasicAliasAnalysis.cpp

780

So, i gave an example earlier of how must information can be useful in the llvm-dev thread that i'll copy herre.
It's true you don't know what operand it is, but you can ask for the location for each operand.
Because it doesn't matter what the values are, only what they alias:

*a = something
foo(a)
b = *a

If foo mustalias a, not only can you possibly move foo with a, you can actually possibly clone foo here, change it to be pass-by-value, and promote the argument inside of it (if you wanted to).
IE it's just as useful as known noalias for the memorylocation, and you have to do the same thing with the data.

If you make it

*a = something
foo (a, c)
b = *a

You can only do anything if you know they are all must-alias or no-alias anyway.

So to me, this API is as bad/good for noalias as it is for mustalias.

It costs nothing to give MRI_Must, and while we don't currrently take advantage of such info, it *is* actually useful IMHO.

[hfinkel]

lib/Analysis/AliasAnalysis.cpp

151

That makes sense. MRI_Must on a call will mean that all operands must alias. We should make that clear in the documentation.

lib/Analysis/BasicAliasAnalysis.cpp

780

I can certainly see this being useful if it means that all arguments must alias the given location. What's the useful thing to do here for call vs. call queries? Do we add MRI_Must only if all arguments must alias all other arguments, or is it enough if at least one argument on the LHS must alias an argument on the RHS?

806

Or PartialAlias?

[asbirlea]

Documented purpose of MRI_Must in enum declaration.
Updated when MRI_Must is set on getModRef(CS, MLoc) and getModRef(CS, CS).
Updated testing accordingly.

[asbirlea]

Minor updates.

[asbirlea]

While updating the tests, I cannot see an obvious test scenario I missed.

include/llvm/Analysis/AliasAnalysis.h

109

Please let me know if the info I added here is reasonable.

lib/Analysis/AliasAnalysis.cpp

529

I want this to be the intended behavior, replaced with "Note"

lib/Analysis/BasicAliasAnalysis.cpp

780

I'm sorry if this is a stupid question, I don't know what you mean by call queries.

lib/Analysis/GlobalsModRef.cpp

87

I'd be happy with that approach, but not sure how to fix this particular case, since it appears to rely on that particular value, i.e. it looks like it needs that additional bit for alignment correctness.
I didn't get this to work with updated values, perhaps I'm missing something obvious.

[asbirlea]

Add testcase for Call-Loc to test Must alias is set when no May alias is found, only MustAlias or NoAlias.
@dberlin: could you please confirm this is the right sematics?

Remaining issue (that I don't know whether to address in this patch): other passes adding their own bits (GlobalsModRef).

[asbirlea]

Gentle ping.

[sanjoy]

include/llvm/Analysis/AliasAnalysis.h

113

*accesses

119

These last two sentences probably also belong on getArgModRefInfo and callCapturesBefore ?

I'd also wordsmith this a bit -- something like:

We usually do not try our best to infer MRI_Must, instead it is merely another piece of "free' information that is presented when available.  For instance:

 - (Your RAR example)
 - (Other examples)

lib/Analysis/AliasAnalysis.cpp

151

Can just be MR | MRI_ModRef.

226

This pattern (here and later) looks a little weird to me -- either downstream passes don't care about distinguishing between MRI_NoModRef and MRI_Must or they do. If the former is true then we should stop iterating early only when we see a MRI_Must and if the latter is true we should be returning Result & MRI_ModRef. Otherwise I suspect we'll see hard to explain performance bugs where (e.g.):

• Transform X optimizes better if AA returns MRI_Must
• We know that alias analysis P returns MRI_Must for the case we're trying to optimize
• But alias analysis Q prevents P's MRI_Must from being propagated by returning MRI_NoModRef (which is correct but not optimal).

What do you think?

279

is *not* found

308

"ArgMask never has MRI_Must set." -> is that incidental, or a fundamental property? In any case, if you're relying on it, then please add an assert.

418

Here and elsewhere - please end sentences with a period.

503

Unrelated to your change, but the name of this function is odd.

528

I think this is cleaner to track as bool MustAlias.

lib/Analysis/BasicAliasAnalysis.cpp

780

Perhaps this is cleaner to track as a bool IsMustAlias?

lib/Analysis/GlobalsModRef.cpp

87

How many of these do we have in the worst case (in a clang LTO bootstrap, say)? I'd be tempted to just have two words here.

136

I think it is better to Info.setInt(Info.getInt() | (NewMRI & MRI_ModRef));. That way we won't get surprising behaviors where making AA smarter to infer MRI_Must regresses because that bit aliases (pun intended!) with MayReadAnyGlobal.

lib/Analysis/MemoryDependenceAnalysis.cpp

696

Nit: how about switching on MR & MRI_ModRef?

[asbirlea]

Address comments.

Some comments and renaming in getModRef(ImmutableCallSite, ImmutableCallSite) for easier understanding of the code.

lib/Analysis/AliasAnalysis.cpp

226

Downstream passes shouldn't rely on having MRI_Must set, since this bit is best effort. Yes, you could have different optimizations if that particular optimization relies on AA setting MRI_Must. What you should *not* have, is a performance penalty to set MRI_Must.

We stop iterating regardless of MRI_Must, because we do not want to incur penalty from calling other AA passes that may set MRI_Must.
And conversely, we do not return Result & MRI_ModRef, because if MRI_Must is set, the additional 'free' info should be kept.

I also updated comments in getModRef(2 callsites) hoping to clarify this.

If later on we see that MRI_Must is so useful that we want to fully rely on it for optimizations, then I'm happy with changing this check. Right now it's not used, so the cost of stopping only after MRI_Must is set is not justified.

Please let me know if this makes sense.

308

Right now getArgModRefInfo never sets MRI_Must.
I tried to document this in ModRef enum and before getArgModRefInfo method declaration. Do you think I should also add an assert here (and above) ?

503

Ack, not planning to change in this patch though.

528

I find it easier to read having the logical OR on the final result: ModRefInfo(R | M), and I think if we track it as bool the check MustAlias?MRI_Must:MRI_NoModRef either needs another variable or a more complex return statement.
I'm inclined to keep as is, but both are ok.
What do you think?

lib/Analysis/BasicAliasAnalysis.cpp

780

Same as above. Whatever we decide on that, I will apply to this one too.

lib/Analysis/GlobalsModRef.cpp

87

I'm confused. If MayReadAnyGlobal = 2, it will trigger the static assert below for overlapping with MRI_ModRef.

[nlopes]

lib/Analysis/AliasAnalysis.cpp

468

AtomicCmpXchgInst must read, but may not write (depending on the value read).
I think the only safe value here is MRI_ModRef, since the current lattice doesn't have a more precise value for MustRead & MayReadWithMustAlias.

[nlopes]

lib/Analysis/AliasAnalysis.cpp

468

gah, I meant MustRef & MayModWithMustAlias.

[sanjoy]

(We've discussed the meat of this patch offline, but I had some stray comments from earlier)

lib/Analysis/AliasAnalysis.cpp

308

If your code will be incorrect when someone changes getArgModRefInfo to return results with MRI_Must set then you should definitely assert this.

528

Here's a reason for why I think a bool is better here -- right now looking at just the return statement in isolation, ModRefInfo(R | M), it isn't obvious that we're at most adding the MRI_Must bit to R (since M could be MRI_Mod, for instance). But if we have ModRefInfo(R | (MustAlias ? MRI_Must : MRI_NoModeRef)) then just by looking at the expression and nothing else we know that we're at most setting the MRI_Must bit.

[asbirlea]

Rebase and address comments.

[asbirlea]

Updated, PTAL.

lib/Analysis/AliasAnalysis.cpp

308

Should not be an issue anymore. If getArgModRefInfo returns results with MRI_Must, result should still be correct.

468

I updated the comments above for the MRI_Must* bits. MustRef & MayModWithMustAlias should have Must bit set.

528

Ack, updated.

[asbirlea]

Nit: periods.

[sanjoy]

Some minor nits inline, but I thought we were going with making bitwise MRI_Must be 0 (and MRI_NoModRef be 4) so that & works as expected?

include/llvm/Analysis/AliasAnalysis.h

109

(Minor) I'd remove "conservative" -- it is understood that these things are conservatively correct.

113

This is for calls that are known to be argmemonly, right?

120

I think this comment can be improved.

• Instead of starting with details on how we infer MRI_Must let's first describe what it means, followed by this note.
• I don't think we need a laundry list of examples here. I think it is better to just mention that we don't always return the most precise possible and add a "c.f. getArgModRefInfo, callCapturesBefore etc.".

125

"default" is a bit confusing here. How about just leaving out the second part, and commenting "MRI_Must is provided for completeness, but no routines will return only MRI_Must today".

583

The comma is unnecessary.

lib/Analysis/AliasAnalysisEvaluator.cpp

264

Unnecessary move? (Not a big deal, but avoid this will make the diff slightly smaller)

[asbirlea]

Address comments and rebase on ToT.

[sanjoy]

Unless you have a specific reason not to, I think we should merge https://reviews.llvm.org/D41091 into this one -- that'll make the patch easier to review (usually these kinds of things go the other way -- splitting the patch makes it easier to review :) ).

Meanwhile, I have some minor comments inline.

include/llvm/Analysis/AliasAnalysis.h

116

Nit s/MRI_Must/ModRefInfo::Must/ here and elsewhere

lib/Analysis/BasicAliasAnalysis.cpp

820

Add a comment here on why tracking MustAlias this way is correct despite the early exit (IIUC this is because if Result = ModRefInfo::ModRef the we don't use MustAlias)?

lib/Analysis/GlobalsModRef.cpp

133

This isn't clearing the must bit though, right? It is clearing the MayReadAnyGlobal bit?

Also the inline here is redundant -- member functions defined within their class are implicitly inline.

[asbirlea]

Address comments and merge in D41091.

[asbirlea]

Remove remaining MRI_ instances.

Issue (?): rephrase comments to clarify Must is cleared not set when MustAlias exists.
Possible reverse clearMust si setMust?

[asbirlea]

Added comment to clarify what setting Must means.

Ready for review, PTAL.

[sanjoy]

I think we can break this patch up further -- how about this breakdown:

1. Remove the changes related to calls from this patch and do something conservative there. That'll reduce the complexity of this patch a bit. I also have a comment about our treatment of calls inline, which we can discuss on the subsequent patch that proper support for calls. (Let me know if splitting this up will be annoying, and we can think of some other strategy.)
2. Make the refactoring I suggested inline in GlobalsModRef.cpp and rebase on top of it.

With both (1) and (2) done, this patch should be ready to go in modulo minor nits.

include/llvm/Analysis/AliasAnalysis.h

111

How about moving this meta-comment and the definition of Must to the end of the enum?

lib/Analysis/AliasAnalysis.cpp

197

I'm not sure about this treatment of calls. For a query between instructions like stores and loads that only touch one location the meaning of Must is pretty clear -- MustRef (say) means the store does not write to any location other than the location the load reads from. However, generalizing this to calls seems to mean that a MustRef between a call and a load would imply that the call does not write to any location other than what the load reads. This means we can't ignore NoAlias arguments -- a NoAlias (as opposed to Must) result should also set MayAlias to false.

499

It may be better to instead add this as a comment on the return instruction, as "Not returning MustModRef since we've not seen all the arguments" etc.

529

"incomplete" is a bit vague -- how about just leaving a comment on the return statement as "Not returning MustModRef since we've not seen all the arguments" or something like that?

lib/Analysis/GlobalsModRef.cpp

137

I wouldn't break the abstraction here this way. Instead, I think it is better to have:

int ModRefInfoToInteger(ModRefInfo mri) {
  switch (clearMust(mri)) {
  case NoModRef:
    return 0;
  case Mod:
    return 1;
  case Ref:
    return 2;
  case ModRef:
    return 3;
  }
}

int IntegerToModRef(ModRefInfo mri) {
  // Inverse of the above
}

and use these to "encode" and "decode" an opaque ModRefInfo instance to a 2 bit integer and back.

(You may even want to do this as a separate NFC change and land it independently.)

[asbirlea]

Address comments.

[asbirlea]

PTAL.

lib/Analysis/AliasAnalysis.cpp

197

Having looked at this again, it appears safe to set Must in this case, but I'm also quite ok with separating this into a separate patch.
I added the path to set MayAlias here and below getMRI(CS, CS), and these can be removed in a subsequent patch.

lib/Analysis/GlobalsModRef.cpp

137

I tried this and ended up with a more confusing code, and harder to maintain.
The general idea of encoding 2 bits and using that in GlobalsModRef is tempting for me, but in practice it looks like it complicates the code unnecessarily. I can be convinced otherwise, and this can be an add-on clean-up patch in that case.

[sanjoy]

Mostly minor stuff. The main non-nit comment I have is that the various places we're handling calls don't seem consistent with each other.

lib/Analysis/AliasAnalysis.cpp

202

How about:

bool MustAlias = true;
AliasResult ArgAlias = ...
MustAlias &= ArgAlias == MustAlias;

?

275–277

Unrelated to your change, but this is the second time I've been confused by this phrase. The variable Result sounds like it is the value we're going to return. But it isn't -- it is just the ModRef information we've computed till before this algorithm on arguments.

What do you think about (in a separate change) splitting out this loop (and the one after) into a separate function, in which Result will be passed as an argument with a more mnemonic name? Of course, this is totally unrelated to your change, so I'll understand if you don't think it is a good use of your time. :)

279

Given that you're setting MustAliasFound even if !isMustSet(ModRefCS1) perhaps it should be called something else?

Secondly, how about structuring this as:

bool MustAlias = true;
MustAias &= isMustSet(ModRefCS1);

?

Finally, I don't think MustAliasFound is necessary anymore -- if there were no MustAlias results and there was at least one argument, then we'd have cleared out MustAlias, so no need to track MustAliasFound I think?

281

I think we can simplify this a bit -- just keep track of the number of arguments you've seen to be MustAlias, and check that it is equal to the number of actual function args. That way we're not using MayAlias to track "have seen a may-alias OR have early exited".

317

The comments I made on the previous loop also apply here.

547–549

Should you be clearing MustAlias even if AR == NoAlias like you're doing in the other cases?

lib/Analysis/BasicAliasAnalysis.cpp

806

Same comment here w.r.t. setting MustAlias when AR == NoAlias.

858

Going by our current definition ("Must means no other location is Ref'ed or Mod'ed") this can't return Must right?

[asbirlea]

Address comments.

lib/Analysis/AliasAnalysis.cpp

275–277

Feedback noted. I may do this in a future patch :).

279

Also renamed bool MustAlias to bool IsMustAlias to avoid confusion with AliasResult MustAlias.

281

I think that actually complicates the code. Either keeping track of the number of arguments seen so far, or of all must alias arguments, or setting a different variable on early exit, they add more baggage than using a bool to track "have seen a may-alias OR have early exited".
The comment should make this choice clear.

[sanjoy]

lgtm!

include/llvm/Analysis/AliasAnalysisEvaluator.h

41 ↗

(On Diff #127914)

Another cleanup that might be worth doing as a separate change -- change these fields to use the C++11 member initializer like int64_6 MustCount = 0;.

[thakis]

A slightly late review comment:

llvm/trunk/lib/Analysis/AliasAnalysis.cpp

551 ↗

(On Diff #127936)

You need IsMustAlias (or AllOperandsMustAlias or what have you) here too; as is this shadows the MustAlias AliasResult, and AR != MustAlias below on line 568 now compares with this bool instead of with the enum value as intended. (Are we missing test coverage here?)

(found by msvc warning 4805 which I had enabled locally for a bit.)

[asbirlea]

llvm/trunk/lib/Analysis/AliasAnalysis.cpp

551 ↗

(On Diff #127936)

Should be fixed in r331222, thanks for the catch!
I'll look into expanding test coverage when starting to use the Must info.

include/llvm/Analysis/AliasAnalysis.h

enum AliasResult {
   /// The two locations precisely alias each other.
   MustAlias,
 };
 
 /// Flags indicating whether a memory access modifies or references memory.
 ///
 /// This is no access at all, a modification, a reference, or both
 /// a modification and a reference. These are specifically structured such that
-/// they form a two bit matrix and bit-tests for 'mod' or 'ref' work with any
-/// of the possible values.
+/// they form a three bit matrix and bit-tests for 'mod' or 'ref' or 'must'
+/// work with any of the possible values.
 enum ModRefInfo {
   /// The access neither references nor modifies the value stored in memory.
   MRI_NoModRef = 0,
-  /// The access references the value stored in memory.
+  /// The access may reference the value stored in memory.
   MRI_Ref = 1,
-  /// The access modifies the value stored in memory.
+  /// The access may modify the value stored in memory.
   MRI_Mod = 2,
-  /// The access both references and modifies the value stored in memory.
-  MRI_ModRef = MRI_Ref | MRI_Mod
+  /// The access may reference and may modify the value stored in memory.
+  MRI_ModRef = MRI_Ref | MRI_Mod,
+  /// The access must alias the value stored in memory.
+  MRI_Must = 4,
+  /// The access must reference the value stored in memory.
+  MRI_MustRef = MRI_Ref | MRI_Must,
+  /// The access must modify the value stored in memory.
+  MRI_MustMod = MRI_Mod | MRI_Must,
+  /// The access must reference and must modify the value stored in memory.
+  MRI_MustModRef = MRI_ModRef | MRI_Must
 };
 
 /// The locations at which a function might access memory.
 ///
 /// These are primarily used in conjunction with the \c AccessKind bits to
 /// describe both the nature of access and the locations of access for a
 /// function call.
 enum FunctionModRefLocation {
   /// Base case is no access to memory.
   FMRL_Nowhere = 0,
   /// Access to memory via argument pointers.
-  FMRL_ArgumentPointees = 4,
+  FMRL_ArgumentPointees = 8,
   /// Memory that is inaccessible via LLVM IR.
-  FMRL_InaccessibleMem = 8,
+  FMRL_InaccessibleMem = 16,
   /// Access to any memory.
-  FMRL_Anywhere = 16 | FMRL_InaccessibleMem | FMRL_ArgumentPointees
+  FMRL_Anywhere = 32 | FMRL_InaccessibleMem | FMRL_ArgumentPointees
 };
 
 /// Summary of how a function affects memory in the program.
 ///
 /// Loads from constant globals are not considered memory accesses for this
 /// interface. Also, functions may freely modify stack space local to their
 /// invocation without having to report it through these interfaces.
 enum FunctionModRefBehavior {
public:
   /// of memory locations. See the AA documentation for details:
   ///   http://llvm.org/docs/AliasAnalysis.html#ModRefInfo
   ModRefInfo getModRefInfo(ImmutableCallSite CS1, ImmutableCallSite CS2);
 
   /// \brief Return information about whether a particular call site modifies
   /// or reads the specified memory location \p MemLoc before instruction \p I
   /// in a BasicBlock. A ordered basic block \p OBB can be used to speed up
   /// instruction ordering queries inside the BasicBlock containing \p I.
   ModRefInfo callCapturesBefore(const Instruction *I,
                                 const MemoryLocation &MemLoc, DominatorTree *DT,
                                 OrderedBasicBlock *OBB = nullptr);
 
   /// \brief A convenience wrapper to synthesize a memory location.
   ModRefInfo callCapturesBefore(const Instruction *I, const Value *P,
                                 uint64_t Size, DominatorTree *DT,
                                 OrderedBasicBlock *OBB = nullptr) {
     return callCapturesBefore(I, MemoryLocation(P, Size), DT, OBB);

lib/Analysis/AliasAnalysis.cpp

 
 ModRefInfo AAResults::getArgModRefInfo(ImmutableCallSite CS, unsigned ArgIdx) {
   ModRefInfo Result = MRI_ModRef;
 
   for (const auto &AA : AAs) {
     Result = ModRefInfo(Result & AA->getArgModRefInfo(CS, ArgIdx));
 
     // Early-exit the moment we reach the bottom of the lattice.
-    if (Result == MRI_NoModRef)
+    if ((Result & MRI_ModRef) == MRI_NoModRef)
       return Result;
   }
 
   return Result;
 }
 
 ModRefInfo AAResults::getModRefInfo(Instruction *I, ImmutableCallSite Call) {
   // We may have two calls
   if (auto CS = ImmutableCallSite(I)) {
     // Check if the two calls modify the same memory
     return getModRefInfo(CS, Call);
   } else if (I->isFenceLike()) {
     // If this is a fence, just return MRI_ModRef.
     return MRI_ModRef;
   } else {
     // Otherwise, check if the call modifies or references the
     // location this memory access defines.  The best we can say
     // is that if the call references what this instruction
     // defines, it must be clobbered by this location.
     const MemoryLocation DefLoc = MemoryLocation::get(I);
-    if (getModRefInfo(Call, DefLoc) != MRI_NoModRef)
-      return MRI_ModRef;
+    ModRefInfo MR = getModRefInfo(Call, DefLoc);
+    if (MR & MRI_ModRef)
+      return ModRefInfo(MRI_ModRef | (MR & MRI_Must));
   }
   return MRI_NoModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(ImmutableCallSite CS,
                                     const MemoryLocation &Loc) {
   ModRefInfo Result = MRI_ModRef;
 
   for (const auto &AA : AAs) {
     Result = ModRefInfo(Result & AA->getModRefInfo(CS, Loc));
 
     // Early-exit the moment we reach the bottom of the lattice.
-    if (Result == MRI_NoModRef)
+    if ((Result & MRI_ModRef) == MRI_NoModRef)
       return Result;
   }
 
   // Try to refine the mod-ref info further using other API entry points to the
   // aggregate set of AA results.
   auto MRB = getModRefBehavior(CS);
   if (MRB == FMRB_DoesNotAccessMemory ||
       MRB == FMRB_OnlyAccessesInaccessibleMem)
     return MRI_NoModRef;
 
   if (onlyReadsMemory(MRB))
-    Result = ModRefInfo(Result & MRI_Ref);
+    Result = ModRefInfo(Result & MRI_MustRef);
   else if (doesNotReadMemory(MRB))
-    Result = ModRefInfo(Result & MRI_Mod);
+    Result = ModRefInfo(Result & MRI_MustMod);
 
   if (onlyAccessesArgPointees(MRB) || onlyAccessesInaccessibleOrArgMem(MRB)) {
     bool DoesAlias = false;
+    bool MayAlias = false;
     ModRefInfo AllArgsMask = MRI_NoModRef;
     if (doesAccessArgPointees(MRB)) {
       for (auto AI = CS.arg_begin(), AE = CS.arg_end(); AI != AE; ++AI) {
         const Value *Arg = *AI;
+        // FIXME: Set MRI_Must for non-pointer types?
         if (!Arg->getType()->isPointerTy())
           continue;
         unsigned ArgIdx = std::distance(CS.arg_begin(), AI);
         MemoryLocation ArgLoc = MemoryLocation::getForArgument(CS, ArgIdx, TLI);
         AliasResult ArgAlias = alias(ArgLoc, Loc);
         if (ArgAlias != NoAlias) {
           ModRefInfo ArgMask = getArgModRefInfo(CS, ArgIdx);
           DoesAlias = true;
           AllArgsMask = ModRefInfo(AllArgsMask | ArgMask);
+          if (ArgAlias != MustAlias)
+            MayAlias = true;
         }
       }
     }
     if (!DoesAlias)
       return MRI_NoModRef;
+    else if (!MayAlias)
+      AllArgsMask = ModRefInfo(AllArgsMask | MRI_Must);
     Result = ModRefInfo(Result & AllArgsMask);
   }
 
   // If Loc is a constant memory location, the call definitely could not
   // modify the memory location.
   if ((Result & MRI_Mod) &&
       pointsToConstantMemory(Loc, /*OrLocal*/ false))
     Result = ModRefInfo(Result & ~MRI_Mod);
 
   return Result;
 }
 
 ModRefInfo AAResults::getModRefInfo(ImmutableCallSite CS1,
                                     ImmutableCallSite CS2) {
   ModRefInfo Result = MRI_ModRef;
 
   for (const auto &AA : AAs) {
     Result = ModRefInfo(Result & AA->getModRefInfo(CS1, CS2));
 
     // Early-exit the moment we reach the bottom of the lattice.
-    if (Result == MRI_NoModRef)
+    if ((Result & MRI_ModRef) == MRI_NoModRef)
       return Result;
   }
 
   // Try to refine the mod-ref info further using other API entry points to the
   // aggregate set of AA results.
 
   // If CS1 or CS2 are readnone, they don't interact.
   auto CS1B = getModRefBehavior(CS1);
   if (CS1B == FMRB_DoesNotAccessMemory)
     return MRI_NoModRef;
 
   auto CS2B = getModRefBehavior(CS2);
   if (CS2B == FMRB_DoesNotAccessMemory)
     return MRI_NoModRef;
 
   // If they both only read from memory, there is no dependence.
   if (onlyReadsMemory(CS1B) && onlyReadsMemory(CS2B))
     return MRI_NoModRef;
 
   // If CS1 only reads memory, the only dependence on CS2 can be
   // from CS1 reading memory written by CS2.
   if (onlyReadsMemory(CS1B))
-    Result = ModRefInfo(Result & MRI_Ref);
+    Result = ModRefInfo(Result & MRI_MustRef);
   else if (doesNotReadMemory(CS1B))
-    Result = ModRefInfo(Result & MRI_Mod);
+    Result = ModRefInfo(Result & MRI_MustMod);
 
   // If CS2 only access memory through arguments, accumulate the mod/ref
   // information from CS1's references to the memory referenced by
   // CS2's arguments.
   if (onlyAccessesArgPointees(CS2B)) {
     ModRefInfo R = MRI_NoModRef;
     if (doesAccessArgPointees(CS2B)) {
       for (auto I = CS2.arg_begin(), E = CS2.arg_end(); I != E; ++I) {
         const Value *Arg = *I;
+        // FIXME: Set MRI_Must for non-pointer types?
         if (!Arg->getType()->isPointerTy())
           continue;
         unsigned CS2ArgIdx = std::distance(CS2.arg_begin(), I);
         auto CS2ArgLoc = MemoryLocation::getForArgument(CS2, CS2ArgIdx, TLI);
 
         // ArgMask indicates what CS2 might do to CS2ArgLoc, and the dependence
         // of CS1 on that location is the inverse.
         ModRefInfo ArgMask = getArgModRefInfo(CS2, CS2ArgIdx);
-        if (ArgMask == MRI_Mod)
-          ArgMask = MRI_ModRef;
-        else if (ArgMask == MRI_Ref)
-          ArgMask = MRI_Mod;
+        if (ArgMask & MRI_Mod)
+          ArgMask = ModRefInfo(MRI_ModRef | (ArgMask & MRI_Must));
+        else if (ArgMask & MRI_Ref)
+          ArgMask = ModRefInfo(MRI_Mod | (ArgMask & MRI_Must));
 
         ArgMask = ModRefInfo(ArgMask & getModRefInfo(CS1, CS2ArgLoc));
 
         R = ModRefInfo((R | ArgMask) & Result);
         if (R == Result)
           break;
       }
     }
     return R;
   }
 
   // If CS1 only accesses memory through arguments, check if CS2 references
   // any of the memory referenced by CS1's arguments. If not, return NoModRef.
   if (onlyAccessesArgPointees(CS1B)) {
     ModRefInfo R = MRI_NoModRef;
     if (doesAccessArgPointees(CS1B)) {
if (doesAccessArgPointees(CS1B)) {
         ModRefInfo ArgMask = getArgModRefInfo(CS1, CS1ArgIdx);
         ModRefInfo ArgR = getModRefInfo(CS2, CS1ArgLoc);
         if (((ArgMask & MRI_Mod) != MRI_NoModRef &&
              (ArgR & MRI_ModRef) != MRI_NoModRef) ||
             ((ArgMask & MRI_Ref) != MRI_NoModRef &&
              (ArgR & MRI_Mod) != MRI_NoModRef))
           R = ModRefInfo((R | ArgMask) & Result);
 
         if (R == Result)
           break;
       }
     }
     return R;
   }
 
   return Result;
 }
 
 FunctionModRefBehavior AAResults::getModRefBehavior(ImmutableCallSite CS) {
   FunctionModRefBehavior Result = FMRB_UnknownModRefBehavior;
 
   for (const auto &AA : AAs) {
     Result = FunctionModRefBehavior(Result & AA->getModRefBehavior(CS));
 
     // Early-exit the moment we reach the bottom of the lattice.
     if (Result == FMRB_DoesNotAccessMemory)

 ModRefInfo AAResults::getModRefInfo(const LoadInst *L,
                                     const MemoryLocation &Loc) {
   // Be conservative in the face of atomic.
   if (isStrongerThan(L->getOrdering(), AtomicOrdering::Unordered))
     return MRI_ModRef;
 
   // If the load address doesn't alias the given address, it doesn't read
   // or write the specified memory.
-  if (Loc.Ptr && !alias(MemoryLocation::get(L), Loc))
+  if (Loc.Ptr) {
+    AliasResult AR = alias(MemoryLocation::get(L), Loc);
+    if (AR == NoAlias)
       return MRI_NoModRef;
-
+    if (AR == MustAlias)
+      return MRI_MustRef;
+  }
   // Otherwise, a load just reads.
   return MRI_Ref;
 }
 
 ModRefInfo AAResults::getModRefInfo(const StoreInst *S,
                                     const MemoryLocation &Loc) {
   // Be conservative in the face of atomic.
   if (isStrongerThan(S->getOrdering(), AtomicOrdering::Unordered))
     return MRI_ModRef;
 
   if (Loc.Ptr) {
+    AliasResult AR = alias(MemoryLocation::get(S), Loc);
     // If the store address cannot alias the pointer in question, then the
     // specified memory cannot be modified by the store.
-    if (!alias(MemoryLocation::get(S), Loc))
+    if (AR == NoAlias)
       return MRI_NoModRef;
 
     // If the pointer is a pointer to constant memory, then it could not have
     // been modified by this store.
     if (pointsToConstantMemory(Loc))
       return MRI_NoModRef;
+
+    // If the store address aliases the pointer as must alias, set MRI_Must
+    if (AR == MustAlias)
+      return MRI_MustMod;
   }
 
   // Otherwise, a store just writes.
   return MRI_Mod;
 }
 
 ModRefInfo AAResults::getModRefInfo(const FenceInst *S, const MemoryLocation &Loc) {
   // If we know that the location is a constant memory location, the fence
   // cannot modify this location.
   if (Loc.Ptr && pointsToConstantMemory(Loc))
     return MRI_Ref;
   return MRI_ModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(const VAArgInst *V,
                                     const MemoryLocation &Loc) {
   if (Loc.Ptr) {
+    AliasResult AR = alias(MemoryLocation::get(V), Loc);
     // If the va_arg address cannot alias the pointer in question, then the
     // specified memory cannot be accessed by the va_arg.
-    if (!alias(MemoryLocation::get(V), Loc))
+    if (AR == NoAlias)
       return MRI_NoModRef;
 
     // If the pointer is a pointer to constant memory, then it could not have
     // been modified by this va_arg.
     if (pointsToConstantMemory(Loc))
       return MRI_NoModRef;
+
+    // If the va_arg aliases the pointer as must alias, set MRI_Must
+    if (AR == MustAlias)
+      return MRI_MustModRef;
   }
 
   // Otherwise, a va_arg reads and writes.
   return MRI_ModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(const CatchPadInst *CatchPad,
                                     const MemoryLocation &Loc) {

 }
 
 ModRefInfo AAResults::getModRefInfo(const AtomicCmpXchgInst *CX,
                                     const MemoryLocation &Loc) {
   // Acquire/Release cmpxchg has properties that matter for arbitrary addresses.
   if (isStrongerThanMonotonic(CX->getSuccessOrdering()))
     return MRI_ModRef;
 
-  // If the cmpxchg address does not alias the location, it does not access it.
-  if (Loc.Ptr && !alias(MemoryLocation::get(CX), Loc))
+  if (Loc.Ptr) {
+    AliasResult AR = alias(MemoryLocation::get(CX), Loc);
+    // If the cmpxchg address does not alias the location, it does not access
+    // it.
+    if (AR == NoAlias)
       return MRI_NoModRef;
 
+    // If the cmpxchg address aliases the pointer as must alias, set MRI_Must
+    if (AR == MustAlias)
+      return MRI_MustModRef;
+  }
+
   return MRI_ModRef;
 }
 
 ModRefInfo AAResults::getModRefInfo(const AtomicRMWInst *RMW,
                                     const MemoryLocation &Loc) {
   // Acquire/Release atomicrmw has properties that matter for arbitrary addresses.
   if (isStrongerThanMonotonic(RMW->getOrdering()))
     return MRI_ModRef;
 
-  // If the atomicrmw address does not alias the location, it does not access it.
-  if (Loc.Ptr && !alias(MemoryLocation::get(RMW), Loc))
+  if (Loc.Ptr) {
+    AliasResult AR = alias(MemoryLocation::get(RMW), Loc);
+    // If the atomicrmw address does not alias the location, it does not access
+    // it.
+    if (AR == NoAlias)
       return MRI_NoModRef;
 
+    // If the atomicrmw address aliases the pointer as must alias, set MRI_Must
+    if (AR == MustAlias)
+      return MRI_MustModRef;
+  }
+
   return MRI_ModRef;
 }
 
 /// \brief Return information about whether a particular call site modifies
 /// or reads the specified memory location \p MemLoc before instruction \p I
 /// in a BasicBlock. A ordered basic block \p OBB can be used to speed up
 /// instruction-ordering queries inside the BasicBlock containing \p I.
 /// FIXME: this is really just shoring-up a deficiency in alias analysis.
 /// BasicAA isn't willing to spend linear time determining whether an alloca
 /// was captured before or after this particular call, while we are. However,
 /// with a smarter AA in place, this test is just wasting compile time.
 ModRefInfo AAResults::callCapturesBefore(const Instruction *I,
                                          const MemoryLocation &MemLoc,
                                          DominatorTree *DT,
                                          OrderedBasicBlock *OBB) {
   if (!DT)
     return MRI_ModRef;
 
   const Value *Object =
       GetUnderlyingObject(MemLoc.Ptr, I->getModule()->getDataLayout());
   if (!isIdentifiedObject(Object) || isa<GlobalValue>(Object) ||
       isa<Constant>(Object))
     return MRI_ModRef;
 
   ImmutableCallSite CS(I);
   if (!CS.getInstruction() || CS.getInstruction() == Object)
     return MRI_ModRef;
 
   if (PointerMayBeCapturedBefore(Object, /* ReturnCaptures */ true,
                                  /* StoreCaptures */ true, I, DT,
                                  /* include Object */ true,
                                  /* OrderedBasicBlock */ OBB))
     return MRI_ModRef;
 
   unsigned ArgNo = 0;
   ModRefInfo R = MRI_NoModRef;
+  ModRefInfo M = MRI_Must;
+  // FIXME: MRI_Must info is incomplete due to early return.
+  // Set flag only if no May found and all operands processed.
   for (auto CI = CS.data_operands_begin(), CE = CS.data_operands_end();
        CI != CE; ++CI, ++ArgNo) {
     // Only look at the no-capture or byval pointer arguments.  If this
     // pointer were passed to arguments that were neither of these, then it
     // couldn't be no-capture.
     if (!(*CI)->getType()->isPointerTy() ||
         (!CS.doesNotCapture(ArgNo) &&
          ArgNo < CS.getNumArgOperands() && !CS.isByValArgument(ArgNo)))
       continue;
 
+    AliasResult AR = alias(MemoryLocation(*CI), MemoryLocation(Object));
     // If this is a no-capture pointer argument, see if we can tell that it
     // is impossible to alias the pointer we're checking.  If not, we have to
     // assume that the call could touch the pointer, even though it doesn't
     // escape.
-    if (isNoAlias(MemoryLocation(*CI), MemoryLocation(Object)))
+    if (AR == NoAlias)
       continue;
+    if (AR == MayAlias)
+      M = MRI_NoModRef;
     if (CS.doesNotAccessMemory(ArgNo))
       continue;
     if (CS.onlyReadsMemory(ArgNo)) {
       R = MRI_Ref;
       continue;
     }
     return MRI_ModRef;
   }
-  return R;
+  return ModRefInfo(R | M);
 }
 
 /// canBasicBlockModify - Return true if it is possible for execution of the
 /// specified basic block to modify the location Loc.
 ///
 bool AAResults::canBasicBlockModify(const BasicBlock &BB,
                                     const MemoryLocation &Loc) {
   return canInstructionRangeModRef(BB.front(), BB.back(), Loc, MRI_Mod);

lib/Analysis/AliasAnalysisEvaluator.cpp

for (CallSite C : CallSites) {
 
     for (auto Pointer : Pointers) {
       uint64_t Size = MemoryLocation::UnknownSize;
       Type *ElTy = cast<PointerType>(Pointer->getType())->getElementType();
       if (ElTy->isSized()) Size = DL.getTypeStoreSize(ElTy);
 
       switch (AA.getModRefInfo(C, Pointer, Size)) {
       case MRI_NoModRef:
+      case MRI_Must:
         PrintModRefResults("NoModRef", PrintNoModRef, I, Pointer,
                            F.getParent());
         ++NoModRefCount;
         break;
       case MRI_Mod:
+      case MRI_MustMod:
         PrintModRefResults("Just Mod", PrintMod, I, Pointer, F.getParent());
         ++ModCount;
         break;
       case MRI_Ref:
+      case MRI_MustRef:
         PrintModRefResults("Just Ref", PrintRef, I, Pointer, F.getParent());
         ++RefCount;
         break;
       case MRI_ModRef:
+      case MRI_MustModRef:
         PrintModRefResults("Both ModRef", PrintModRef, I, Pointer,
                            F.getParent());
         ++ModRefCount;
         break;
       }
     }
   }
 
   // Mod/ref alias analysis: compare all pairs of calls
   for (auto C = CallSites.begin(), Ce = CallSites.end(); C != Ce; ++C) {
     for (auto D = CallSites.begin(); D != Ce; ++D) {
       if (D == C)
         continue;
       switch (AA.getModRefInfo(*C, *D)) {
       case MRI_NoModRef:
+      case MRI_Must:
         PrintModRefResults("NoModRef", PrintNoModRef, *C, *D, F.getParent());
         ++NoModRefCount;
         break;
       case MRI_Mod:
+      case MRI_MustMod:
         PrintModRefResults("Just Mod", PrintMod, *C, *D, F.getParent());
         ++ModCount;
         break;
       case MRI_Ref:
+      case MRI_MustRef:
         PrintModRefResults("Just Ref", PrintRef, *C, *D, F.getParent());
         ++RefCount;
         break;
       case MRI_ModRef:
+      case MRI_MustModRef:
         PrintModRefResults("Both ModRef", PrintModRef, *C, *D, F.getParent());
         ++ModRefCount;
         break;
       }
     }
   }
 }
 

lib/Analysis/AliasSetTracker.cpp

for (iterator I = begin(), E = end(); I != E; ++I)
     if (AA.alias(MemoryLocation(Ptr, Size, AAInfo),
                  MemoryLocation(I.getPointer(), I.getSize(), I.getAAInfo())))
       return true;
 
   // Check the unknown instructions...
   if (!UnknownInsts.empty()) {
     for (unsigned i = 0, e = UnknownInsts.size(); i != e; ++i)
       if (auto *Inst = getUnknownInst(i))
-        if (AA.getModRefInfo(Inst, MemoryLocation(Ptr, Size, AAInfo)) !=
-            MRI_NoModRef)
+        if (AA.getModRefInfo(Inst, MemoryLocation(Ptr, Size, AAInfo)) &
+            MRI_ModRef)
           return true;
   }
 
   return false;
 }
 
 bool AliasSet::aliasesUnknownInst(const Instruction *Inst,
                                   AliasAnalysis &AA) const {
 
   if (AliasAny)
     return true;
 
   if (!Inst->mayReadOrWriteMemory())
     return false;
 
   for (unsigned i = 0, e = UnknownInsts.size(); i != e; ++i) {
     if (auto *UnknownInst = getUnknownInst(i)) {
       ImmutableCallSite C1(UnknownInst), C2(Inst);
-      if (!C1 || !C2 || AA.getModRefInfo(C1, C2) != MRI_NoModRef ||
-          AA.getModRefInfo(C2, C1) != MRI_NoModRef)
+      if (!C1 || !C2 || AA.getModRefInfo(C1, C2) & MRI_ModRef ||
+          AA.getModRefInfo(C2, C1) & MRI_ModRef)
         return true;
     }
   }
 
   for (iterator I = begin(), E = end(); I != E; ++I)
-    if (AA.getModRefInfo(Inst, MemoryLocation(I.getPointer(), I.getSize(),
-                                              I.getAAInfo())) != MRI_NoModRef)
+    if (AA.getModRefInfo(
+            Inst, MemoryLocation(I.getPointer(), I.getSize(), I.getAAInfo())) &
+        MRI_ModRef)
       return true;
 
   return false;
 }
 
 void AliasSetTracker::clear() {
   // Delete all the PointerRec entries.
   for (PointerMapType::iterator I = PointerMap.begin(), E = PointerMap.end();

lib/Analysis/BasicAliasAnalysis.cpp

ModRefInfo BasicAAResult::getModRefInfo(ImmutableCallSite CS,
   // then the call can not mod/ref the pointer unless the call takes the pointer
   // as an argument, and itself doesn't capture it.
   if (!isa<Constant>(Object) && CS.getInstruction() != Object &&
       isNonEscapingLocalObject(Object)) {
 
     // Optimistically assume that call doesn't touch Object and check this
     // assumption in the following loop.
     ModRefInfo Result = MRI_NoModRef;
+    ModRefInfo MustResult = MRI_Must;
 
     unsigned OperandNo = 0;
     for (auto CI = CS.data_operands_begin(), CE = CS.data_operands_end();
          CI != CE; ++CI, ++OperandNo) {
       // Only look at the no-capture or byval pointer arguments.  If this
       // pointer were passed to arguments that were neither of these, then it
       // couldn't be no-capture.
       if (!(*CI)->getType()->isPointerTy() ||
for (auto CI = CS.data_operands_begin(), CE = CS.data_operands_end();
       // If this is a no-capture pointer argument, see if we can tell that it
       // is impossible to alias the pointer we're checking.
       AliasResult AR =
           getBestAAResults().alias(MemoryLocation(*CI), MemoryLocation(Object));
 
       // Operand doesnt alias 'Object', continue looking for other aliases
       if (AR == NoAlias)
         continue;
+      if (AR == MayAlias)
+        MustResult = MRI_NoModRef;
       // Operand aliases 'Object', but call doesn't modify it. Strengthen
       // initial assumption and keep looking in case if there are more aliases.
       if (CS.onlyReadsMemory(OperandNo)) {
         Result = static_cast<ModRefInfo>(Result | MRI_Ref);
         continue;
       }
       // Operand aliases 'Object' but call only writes into it.
       if (CS.doesNotReadMemory(OperandNo)) {
         Result = static_cast<ModRefInfo>(Result | MRI_Mod);
         continue;
       }
       // This operand aliases 'Object' and call reads and writes into it.
       Result = MRI_ModRef;
       break;
     }
 
+    // No operand aliases, reset Must bit. Add below if at least one aliases
+    // and all aliases found are MustAlias.
+    if (Result == MRI_NoModRef)
+      MustResult = MRI_NoModRef;
+
     // Early return if we improved mod ref information
     if (Result != MRI_ModRef)
-      return Result;
+      return ModRefInfo(Result | MustResult);
   }
 
   // If the CallSite is to malloc or calloc, we can assume that it doesn't
   // modify any IR visible value.  This is only valid because we assume these
   // routines do not read values visible in the IR.  TODO: Consider special
   // casing realloc and strdup routines which access only their arguments as
   // well.  Or alternatively, replace all of this with inaccessiblememonly once
   // that's implemented fully. 
ModRefInfo BasicAAResult::getModRefInfo(ImmutableCallSite CS,
   // If Loc must-aliases either one of these two locations, then it necessarily
   // no-aliases the other.
   if (auto *Inst = dyn_cast<MemCpyInst>(CS.getInstruction())) {
     AliasResult SrcAA, DestAA;
 
     if ((SrcAA = getBestAAResults().alias(MemoryLocation::getForSource(Inst),
                                           Loc)) == MustAlias)
       // Loc is exactly the memcpy source thus disjoint from memcpy dest.
-      return MRI_Ref;
+      return MRI_MustRef;
     if ((DestAA = getBestAAResults().alias(MemoryLocation::getForDest(Inst),
                                            Loc)) == MustAlias)
       // The converse case.
-      return MRI_Mod;
+      return MRI_MustMod;
 
     // It's also possible for Loc to alias both src and dest, or neither.
     ModRefInfo rv = MRI_NoModRef;
     if (SrcAA != NoAlias)
       rv = static_cast<ModRefInfo>(rv | MRI_Ref);
     if (DestAA != NoAlias)
       rv = static_cast<ModRefInfo>(rv | MRI_Mod);
     return rv;

lib/Analysis/GlobalsModRef.cpp

struct AlignedMapPointerTraits {
     }
     enum { NumLowBitsAvailable = 3 };
     static_assert(alignof(AlignedMap) >= (1 << NumLowBitsAvailable),
                   "AlignedMap insufficiently aligned to have enough low bits.");
   };
 
   /// The bit that flags that this function may read any global. This is
   /// chosen to mix together with ModRefInfo bits.
+  /// FIXME: Overlaps with MRI_Must bit!
+  /// FunctionInfo.getModRefInfo() masks out everything except MRI_ModRef so
+  /// this remains correct, but the MRI_Must info is lost.
   enum { MayReadAnyGlobal = 4 };
 
   /// Checks to document the invariants of the bit packing here.
   static_assert((MayReadAnyGlobal & MRI_ModRef) == 0,
                 "ModRef and the MayReadAnyGlobal flag bits overlap.");
   static_assert(((MayReadAnyGlobal | MRI_ModRef) >>
                  AlignedMapPointerTraits::NumLowBitsAvailable) == 0,
                 "Insufficient low bits to store our flag and ModRef info.");
FunctionInfo &operator=(FunctionInfo &&RHS) {
     Info.setPointerAndInt(RHS.Info.getPointer(), RHS.Info.getInt());
     RHS.Info.setPointerAndInt(nullptr, 0);
     return *this;
   }
 
   /// Returns the \c ModRefInfo info for this function.
   ModRefInfo getModRefInfo() const {
     return ModRefInfo(Info.getInt() & MRI_ModRef);
   }
 
   /// Adds new \c ModRefInfo for this function to its state.
   void addModRefInfo(ModRefInfo NewMRI) {
     Info.setInt(Info.getInt() | NewMRI);
   }
 
   /// Returns whether this function may read any global variable, and we don't
   /// know which global.
   bool mayReadAnyGlobal() const { return Info.getInt() & MayReadAnyGlobal; }
 
   /// Sets this function as potentially reading from any global.
   void setMayReadAnyGlobal() { Info.setInt(Info.getInt() | MayReadAnyGlobal); }
void GlobalsAAResult::DeletionCallbackHandle::deleted() {
   GAR->Handles.erase(I);
   // This object is now destroyed!
 }
 
 FunctionModRefBehavior GlobalsAAResult::getModRefBehavior(const Function *F) {
   FunctionModRefBehavior Min = FMRB_UnknownModRefBehavior;
 
   if (FunctionInfo *FI = getFunctionInfo(F)) {
-    if (FI->getModRefInfo() == MRI_NoModRef)
+    if ((FI->getModRefInfo() & MRI_ModRef) == MRI_NoModRef)
       Min = FMRB_DoesNotAccessMemory;
     else if ((FI->getModRefInfo() & MRI_Mod) == 0)
       Min = FMRB_OnlyReadsMemory;
   }
 
   return FunctionModRefBehavior(AAResultBase::getModRefBehavior(F) & Min);
 }
 
 FunctionModRefBehavior
 GlobalsAAResult::getModRefBehavior(ImmutableCallSite CS) {
   FunctionModRefBehavior Min = FMRB_UnknownModRefBehavior;
 
   if (!CS.hasOperandBundles())
     if (const Function *F = CS.getCalledFunction())
       if (FunctionInfo *FI = getFunctionInfo(F)) {
-        if (FI->getModRefInfo() == MRI_NoModRef)
+        if ((FI->getModRefInfo() & MRI_ModRef) == MRI_NoModRef)
           Min = FMRB_DoesNotAccessMemory;
         else if ((FI->getModRefInfo() & MRI_Mod) == 0)
           Min = FMRB_OnlyReadsMemory;
       }
 
   return FunctionModRefBehavior(AAResultBase::getModRefBehavior(CS) & Min);
 }
 
for (auto *Node : SCC) {
           FI.addModRefInfo(MRI_Ref);
         if (I.mayWriteToMemory())
           FI.addModRefInfo(MRI_Mod);
       }
     }
 
     if ((FI.getModRefInfo() & MRI_Mod) == 0)
       ++NumReadMemFunctions;
-    if (FI.getModRefInfo() == MRI_NoModRef)
+    if ((FI.getModRefInfo() & MRI_ModRef) == MRI_NoModRef)
       ++NumNoMemFunctions;
 
     // Finally, now that we know the full effect on this SCC, clone the
     // information to each function in the SCC.
     // FI is a reference into FunctionInfos, so copy it now so that it doesn't
     // get invalidated if DenseMap decides to re-hash.
     FunctionInfo CachedFI = FI;
     for (unsigned i = 1, e = SCC.size(); i != e; ++i)
if (const GlobalValue *GV =
           dyn_cast<GlobalValue>(GetUnderlyingObject(Loc.Ptr, DL)))
     if (GV->hasLocalLinkage())
       if (const Function *F = CS.getCalledFunction())
         if (NonAddressTakenGlobals.count(GV))
           if (const FunctionInfo *FI = getFunctionInfo(F))
             Known = FI->getModRefInfoForGlobal(*GV) |
               getModRefInfoForArgument(CS, GV);
 
-  if (Known == MRI_NoModRef)
+  if ((Known & MRI_ModRef) == MRI_NoModRef)
     return MRI_NoModRef; // No need to query other mod/ref analyses
   return ModRefInfo(Known & AAResultBase::getModRefInfo(CS, Loc));
 }
 
 GlobalsAAResult::GlobalsAAResult(const DataLayout &DL,
                                  const TargetLibraryInfo &TLI)
     : AAResultBase(), DL(DL), TLI(TLI) {}
 

lib/Analysis/MemoryDependenceAnalysis.cpp

while (ScanIt != BB->begin()) {
 
     Instruction *Inst = &*--ScanIt;
 
     // If this inst is a memory op, get the pointer it accessed
     MemoryLocation Loc;
     ModRefInfo MR = GetLocation(Inst, Loc, TLI);
     if (Loc.Ptr) {
       // A simple instruction.
-      if (AA.getModRefInfo(CS, Loc) != MRI_NoModRef)
+      if (AA.getModRefInfo(CS, Loc) & MRI_ModRef)
         return MemDepResult::getClobber(Inst);
       continue;
     }
 
     if (auto InstCS = CallSite(Inst)) {
       // Debug intrinsics don't cause dependences.
       if (isa<DbgInfoIntrinsic>(Inst))
         continue;
       // If these two calls do not interfere, look past it.
       switch (AA.getModRefInfo(CS, InstCS)) {
       case MRI_NoModRef:
+      case MRI_Must:
         // If the two calls are the same, return InstCS as a Def, so that
         // CS can be found redundant and eliminated.
         if (isReadOnlyCall && !(MR & MRI_Mod) &&
             CS.getInstruction()->isIdenticalToWhenDefined(Inst))
           return MemDepResult::getDef(Inst);
 
         // Otherwise if the two calls don't interact (e.g. InstCS is readnone)
         // keep scanning.
         continue;
       default:
         return MemDepResult::getClobber(Inst);
       }
     }
 
     // If we could not obtain a pointer for the instruction and the instruction
     // touches memory then assume that this is a dependency.
-    if (MR != MRI_NoModRef)
+    if (MR & MRI_ModRef)
       return MemDepResult::getClobber(Inst);
   }
 
   // No dependence found.  If this is the entry block of the function, it is
   // unknown, otherwise it is non-local.
   if (BB != &BB->getParent()->getEntryBlock())
     return MemDepResult::getNonLocal();
   return MemDepResult::getNonFuncLocal();
if (StoreInst *SI = dyn_cast<StoreInst>(Inst)) {
       if (SI->isVolatile())
         if (!QueryInst || isNonSimpleLoadOrStore(QueryInst) ||
             isOtherMemAccess(QueryInst))
           return MemDepResult::getClobber(SI);
 
       // If alias analysis can tell that this store is guaranteed to not modify
       // the query pointer, ignore it.  Use getModRefInfo to handle cases where
       // the query pointer points to constant memory etc.
-      if (AA.getModRefInfo(SI, MemLoc) == MRI_NoModRef)
+      if ((AA.getModRefInfo(SI, MemLoc) & MRI_ModRef) == MRI_NoModRef)
         continue;
 
       // Ok, this store might clobber the query pointer.  Check to see if it is
       // a must alias: in this case, we want to return this as a def.
+      // FIXME: Use MRI_Must bit from getModRefInfo call above.
       MemoryLocation StoreLoc = MemoryLocation::get(SI);
 
       // If we found a pointer, check if it could be the same as our pointer.
       AliasResult R = AA.alias(StoreLoc, MemLoc);
 
       if (R == NoAlias)
         continue;
       if (R == MustAlias)
#endif
     // can't bypass the fence if the query instruction is a store.
     if (FenceInst *FI = dyn_cast<FenceInst>(Inst))
       if (isLoad && FI->getOrdering() == AtomicOrdering::Release)
         continue;
 
     // See if this instruction (e.g. a call or vaarg) mod/ref's the pointer.
     ModRefInfo MR = AA.getModRefInfo(Inst, MemLoc);
     // If necessary, perform additional analysis.
-    if (MR == MRI_ModRef)
-      MR = AA.callCapturesBefore(Inst, MemLoc, &DT, &OBB);
+    if ((MR & MRI_ModRef) == MRI_ModRef)
+      MR = ModRefInfo(AA.callCapturesBefore(Inst, MemLoc, &DT, &OBB) |
+                      (MR & MRI_Must));
     switch (MR) {
     case MRI_NoModRef:
+    case MRI_Must:
       // If the call has no effect on the queried pointer, just ignore it.
       continue;
     case MRI_Mod:
+    case MRI_MustMod:
       return MemDepResult::getClobber(Inst);
     case MRI_Ref:
+    case MRI_MustRef:
       // If the call is known to never store to the pointer, and if this is a
       // load query, we can safely ignore it (scan past it).
       if (isLoad)
         continue;
       LLVM_FALLTHROUGH;
     default:
       // Otherwise, there is a potential dependence.  Return a clobber.
       return MemDepResult::getClobber(Inst);

lib/Analysis/MemorySSA.cpp

case Intrinsic::assume:
       return false;
     default:
       break;
     }
   }
 
   if (UseCS) {
     ModRefInfo I = AA.getModRefInfo(DefInst, UseCS);
-    return I != MRI_NoModRef;
+    return I & MRI_ModRef;
   }
 
   if (auto *DefLoad = dyn_cast<LoadInst>(DefInst)) {
     if (auto *UseLoad = dyn_cast<LoadInst>(UseInst)) {
       switch (getLoadReorderability(UseLoad, DefLoad)) {
       case Reorderability::Always:
         return false;
       case Reorderability::Never:

lib/Transforms/Scalar/DeadStoreElimination.cpp

if (auto CS = CallSite(&*BBI)) {
         continue;
 
       // If the call might load from any of our allocas, then any store above
       // the call is live.
       DeadStackObjects.remove_if([&](Value *I) {
         // See if the call site touches the value.
         ModRefInfo A = AA->getModRefInfo(CS, I, getPointerSize(I, DL, *TLI));
 
-        return A == MRI_ModRef || A == MRI_Ref;
+        return A & MRI_Ref;
       });
 
       // If all of the allocas were clobbered by the call then we're not going
       // to find anything else to process.
       if (DeadStackObjects.empty())
         break;
 
       continue;

lib/Transforms/Scalar/MemCpyOptimizer.cpp

 // This method try to lift a store instruction before position P.
 // It will lift the store and its argument + that anything that
 // may alias with these.
 // The method returns true if it was successful.
 static bool moveUp(AliasAnalysis &AA, StoreInst *SI, Instruction *P,
                    const LoadInst *LI) {
   // If the store alias this position, early bail out.
   MemoryLocation StoreLoc = MemoryLocation::get(SI);
-  if (AA.getModRefInfo(P, StoreLoc) != MRI_NoModRef)
+  if (AA.getModRefInfo(P, StoreLoc) & MRI_ModRef)
     return false;
 
   // Keep track of the arguments of all instruction we plan to lift
   // so we can make sure to lift them as well if apropriate.
   DenseSet<Instruction*> Args;
   if (auto *Ptr = dyn_cast<Instruction>(SI->getPointerOperand()))
     if (Ptr->getParent() == SI->getParent())
       Args.insert(Ptr);
 
   // Instruction to lift before P.
   SmallVector<Instruction*, 8> ToLift;
 
   // Memory locations of lifted instructions.
   SmallVector<MemoryLocation, 8> MemLocs{StoreLoc};
 
   // Lifted callsites.
   SmallVector<ImmutableCallSite, 8> CallSites;
 
   const MemoryLocation LoadLoc = MemoryLocation::get(LI);
 
   for (auto I = --SI->getIterator(), E = P->getIterator(); I != E; --I) {
     auto *C = &*I;
 
-    bool MayAlias = AA.getModRefInfo(C, None) != MRI_NoModRef;
+    bool MayAlias = AA.getModRefInfo(C, None) & MRI_ModRef;
 
     bool NeedLift = false;
     if (Args.erase(C))
       NeedLift = true;
     else if (MayAlias) {
       NeedLift = llvm::any_of(MemLocs, [C, &AA](const MemoryLocation &ML) {
-        return AA.getModRefInfo(C, ML);
+        return AA.getModRefInfo(C, ML) & MRI_ModRef;
       });
 
       if (!NeedLift)
         NeedLift =
             llvm::any_of(CallSites, [C, &AA](const ImmutableCallSite &CS) {
-              return AA.getModRefInfo(C, CS);
+              return AA.getModRefInfo(C, CS) & MRI_ModRef;
             });
     }
 
     if (!NeedLift)
       continue;
 
     if (MayAlias) {
       // Since LI is implicitly moved downwards past the lifted instructions,
       // none of them may modify its source.
       if (AA.getModRefInfo(C, LoadLoc) & MRI_Mod)
         return false;
       else if (auto CS = ImmutableCallSite(C)) {
         // If we can't lift this before P, it's game over.
-        if (AA.getModRefInfo(P, CS) != MRI_NoModRef)
+        if (AA.getModRefInfo(P, CS) & MRI_ModRef)
           return false;
 
         CallSites.push_back(CS);
       } else if (isa<LoadInst>(C) || isa<StoreInst>(C) || isa<VAArgInst>(C)) {
         // If we can't lift this before P, it's game over.
         auto ML = MemoryLocation::get(C);
-        if (AA.getModRefInfo(P, ML) != MRI_NoModRef)
+        if (AA.getModRefInfo(P, ML) & MRI_ModRef)
           return false;
 
         MemLocs.push_back(ML);
       } else
         // We don't know how to lift this instruction.
         return false;
     }
 
if (LI->isSimple() && LI->hasOneUse() &&
         // Check that nothing touches the dest of the "copy" between
         // the call and the store.
         Value *CpyDest = SI->getPointerOperand()->stripPointerCasts();
         bool CpyDestIsLocal = isa<AllocaInst>(CpyDest);
         AliasAnalysis &AA = LookupAliasAnalysis();
         MemoryLocation StoreLoc = MemoryLocation::get(SI);
         for (BasicBlock::iterator I = --SI->getIterator(), E = C->getIterator();
              I != E; --I) {
-          if (AA.getModRefInfo(&*I, StoreLoc) != MRI_NoModRef) {
+          if (AA.getModRefInfo(&*I, StoreLoc) & MRI_ModRef) {
             C = nullptr;
             break;
           }
           // The store to dest may never happen if an exception can be thrown
           // between the load and the store.
           if (I->mayThrow() && !CpyDestIsLocal) {
             C = nullptr;
             break;
bool MemCpyOptPass::performCallSlotOptzn(Instruction *cpy, Value *cpyDest,
 
   // In addition to knowing that the call does not access src in some
   // unexpected manner, for example via a global, which we deduce from
   // the use analysis, we also need to know that it does not sneakily
   // access dest.  We rely on AA to figure this out for us.
   AliasAnalysis &AA = LookupAliasAnalysis();
   ModRefInfo MR = AA.getModRefInfo(C, cpyDest, srcSize);
   // If necessary, perform additional analysis.
-  if (MR != MRI_NoModRef)
+  if (MR & MRI_ModRef)
     MR = AA.callCapturesBefore(C, cpyDest, srcSize, &DT);
-  if (MR != MRI_NoModRef)
+  if (MR & MRI_ModRef)
     return false;
 
   // We can't create address space casts here because we don't know if they're
   // safe for the target.
   if (cpySrc->getType()->getPointerAddressSpace() !=
       cpyDest->getType()->getPointerAddressSpace())
     return false;
   for (unsigned i = 0; i < CS.arg_size(); ++i)