llvm: Add support for "-fno-delete-null-pointer-checks"

[manojgupta]

Summary

Support for this option is needed for building Linux kernel.
This is a very frequently requested feature by kernel developers.

More details : https://lkml.org/lkml/2018/4/4/601

GCC option description for -fdelete-null-pointer-checks:
This Assume that programs cannot safely dereference null pointers,
and that no code or data element resides at address zero.

-fno-delete-null-pointer-checks is the inverse of this implying that
null pointer dereferencing is not undefined.

This feature is implemented in LLVM IR in this CL as the function attribute
"null-pointer-is-valid"="true" in IR (Under review at D47894).
The CL updates several passes that assumed null pointer dereferencing is
undefined to not optimize when the "null-pointer-is-valid"="true"
attribute is present.

[xbolva00]

lib/Transforms/InstCombine/InstCombineCalls.cpp

3921

getParent()->getParent()
to
getFunction()

Same below

[efriedma]

Missing LangRef changes.

How did you come up with the set of checks which need to be updated? Quickly grepping through the source, I found you missed at least lib/Analysis/ValueTracking.cpp, include/llvm/IR/CallSite.h, lib/Analysis/LazyValueInfo.cpp, and lib/Analysis/ConstantFolding.cpp .

include/llvm/Analysis/Utils/Local.h

93

This needs to be one function which both the address-space check and the function attribute check; that's both easier to understand, and easier to extend in the future.

Maybe also put helpers on LoadInst and StoreInst to extract the appropriate address space and call this function.

[manojgupta]

In D47895#1125388, @efriedma wrote:

Missing LangRef changes.

How did you come up with the set of checks which need to be updated? Quickly grepping through the source, I found you missed at least lib/Analysis/ValueTracking.cpp, include/llvm/IR/CallSite.h, lib/Analysis/LazyValueInfo.cpp, and lib/Analysis/ConstantFolding.cpp .

Thanks Eli,

My interpretation (based on GCC description) is only the analysis/optimizations that assume pointer dereference => non-null and null dereference => undefined need to be fixed. If an analysis can prove non-nullablility based on any criteria except dereference, it should be fine to remove null checks.

I am going to recheck the list of files you suggested and also recheck LLVM overall.
I believe that there are probably more places that I might have missed. Appreciate a lot if you and other reviewers can point me to more optimizations/analysis that should be updated.

[efriedma]

If an analysis can prove non-nullablility based on any criteria except dereference, it should be fine to remove null checks.

Yes, that's right. At least, I think that's the only reasonable way to define the attribute.

I was basically just grepping for constructs which look like "getPointerAddressSpace() == 0"; anything that's doing that probably needs to be fixed because address space 0 doesn't have any other special properties. (Well, actually, that's not completely right: there are special rules for widening loads and stores in address-space 0. But IIRC almost nothing depends on that.)

[manojgupta]

Updated some passes and added more test cases.

[manojgupta]

In D47895#1125388, @efriedma wrote:

Missing LangRef changes.

Eli, Is this the place you are referring to?
https://llvm.org/docs/LangRef.html#attribute-groups

It does not cover much about the internal IR attributes, though I'll be happy to add "null-pointer-is-valid"="true" to the examples here.

[efriedma]

Tests look good, at first glance.

For LangRef, I was referring to https://llvm.org/docs/LangRef.html#function-attributes, yes. (There might be some gaps in the documentation, but ideally that should cover all the function attributes recognized by LLVM transforms and code generation.)

It looks like you still didn't look at all the files I mentioned earlier?

lib/Transforms/IPO/GlobalOpt.cpp

642

This doesn't handle address spaces correctly... at least, not at first glance.

[efriedma]

include/llvm/Analysis/Utils/Local.h

93

Not sure I like the default argument here.

[manojgupta]

In D47895#1131374, @efriedma wrote:

Tests look good, at first glance.

For LangRef, I was referring to https://llvm.org/docs/LangRef.html#function-attributes, yes. (There might be some gaps in the documentation, but ideally that should cover all the function attributes recognized by LLVM transforms and code generation.)

Thanks, Will add it there.

It looks like you still didn't look at all the files I mentioned earlier?

I went through those files but many of these addressSpace() == 0 (and != 0) checks are not for null pointers but for disabling/skipping some memory optimizations. And it is not clear to me if moving all of those checks to the new NullPointerIsDefined function makes sense there.

lib/Transforms/IPO/GlobalOpt.cpp

1931

This pass already skips non-zero address space values. I didn't want to bail out here since all of the accesses to a GV might be in functions without null-pointer-is-valid attribute.

[efriedma]

lib/Transforms/IPO/GlobalOpt.cpp

1931

Fine; please add a comment to AllUsesOfValueWillTrapIfNull.

[manojgupta]

Added more test cases, updated LangRef, added a check for address space 0
in GlobalOpt.

[efriedma]

I went through those files but many of these addressSpace() == 0 (and != 0) checks are not for null pointers but for disabling/skipping some memory optimizations. And it is not clear to me if moving all of those checks to the new NullPointerIsDefined function makes sense there.

We should probably add a section to LangRef on address spaces, and what transforms are specifically allowed only in address space 0.

I'm sure we need to handle null-pointer-is-valid in CallSite.h, at least.

docs/LangRef.rst

1467 ↗

(On Diff #151323)

Stating the ways to spell null pointer in C isn't helpful; this is the LLVM IR reference. A null pointer in IR is just "i8* null". (And the C/C++ notion of a "null pointer" can actually be translated to something which is not i8* null in some situations.)

[manojgupta]

Added checks to CallSite.h and LoopAccessAnalysis.cpp.
Added another test case for atomic loads.
Moved The NullPointerIsDefined function to Function.h.

[manojgupta]

Update LangRef.

[xbolva00]

include/llvm/IR/Function.h

788 ↗

(On Diff #151772)

Confusing..

nullPointerIsDefined or NullPointerIsDefined?

[manojgupta]

In D47895#1133937, @efriedma wrote:

I went through those files but many of these addressSpace() == 0 (and != 0) checks are not for null pointers but for disabling/skipping some memory optimizations. And it is not clear to me if moving all of those checks to the new NullPointerIsDefined function makes sense there.

We should probably add a section to LangRef on address spaces, and what transforms are specifically allowed only in address space 0.

I'm sure we need to handle null-pointer-is-valid in CallSite.h, at least.

Thanks! I updated CallSite.h and also LoopAccessAnalysis. Had to find another home for NullPointerIsDefined function so moved it to Function.h.
I am in split mind about adding theses checks to BasicAliasAnalysis so need your opinion.

[manojgupta]

include/llvm/IR/Function.h

788 ↗

(On Diff #151772)

This is part of Function Object so used lowercase to match the current code.

[manojgupta]

Added a comment in GlobalOpt.cpp

[manojgupta]

@efriedma I think I have addressed the changes you requested, can you ptal?

[efriedma]

Please fix lib/Analysis/BasicAliasAnalysis.cpp to allow null pointers to alias other objects. Please fix lib/Analysis/LoopAccessAnalysis.cpp to allow a loop to dereference null. Please fix isGEPKnownNonNull in lib/Analysis/ValueTracking.cpp to allow objects located at null. Please fix ConstantFoldScalarCall in lib/Analysis/ConstantFolding.cpp to allow objects located at null. Please fix Argument::hasNonNullAttr in lib/IR/Function.cpp to allow objects at null.

There are probably other places which need to be fixed, but there's no good way to find them all; we'll just have to stumble over them later.

LangRef needs some work to clarify exactly what properties we assume about address-space zero besides the special properties of null, but I should probably write that myself.

docs/LangRef.rst

1465 ↗

(On Diff #152122)

Please specifically say "in address-space 0".

1468 ↗

(On Diff #152122)

undefined isn't a keyword.

[sanjoy]

Drive by comments

You also need to change the inliner to not inline null-pointer-is-dereferenceable into functions that don't have that attribute.

docs/LangRef.rst

1464 ↗

(On Diff #152122)

This should be called null-pointer-is-dereferenceable or something like that.

1466 ↗

(On Diff #152122)

Might be worth clarifying if this is just loads or both loads and stores.

1468 ↗

(On Diff #152122)

What about functions called from this function?

lib/Transforms/Utils/Local.cpp

1813–1815

I don't think we can fold calls to undef either -- the only reason why we can normally do this is because Callee could be null.

[manojgupta]

Made requested changes and added more tests.

[manojgupta]

In D47895#1139800, @sanjoy wrote:

Drive by comments

You also need to change the inliner to not inline null-pointer-is-dereferenceable into functions that don't have that attribute.

Done.
Note that, I am expecting that functions with alwaysinline attribute should still get inlined. If that happens and caller does not have this attribute, then optimizer is free to remove the checks.

In D47895#1139757, @efriedma wrote:

Please fix lib/Analysis/BasicAliasAnalysis.cpp to allow null pointers to alias other objects. Please fix lib/Analysis/LoopAccessAnalysis.cpp to allow a loop to dereference null. Please fix isGEPKnownNonNull in lib/Analysis/ValueTracking.cpp to allow objects located at null. Please fix ConstantFoldScalarCall in lib/Analysis/ConstantFolding.cpp to allow objects located at null. Please fix Argument::hasNonNullAttr in lib/IR/Function.cpp to allow objects at null.

Done.

docs/LangRef.rst

1464 ↗

(On Diff #152122)

I actually had a typo here. The IR is using "null-pointer-is-valid" so changed to the same name.

1468 ↗

(On Diff #152122)

If the functions are not inlined, it shouldn't normally matter.
For functions that get inlined, I believe caller convention should be kept i.e. not propagating this attribute from callee to caller when inlining.

lib/IR/ConstantFold.cpp

1504 ↗

(On Diff #152954)

There does not seem to be a way to find the Function and thereby the attribute here. Any suggestions?

[manojgupta]

Fixed typo in inlining unit test.

[manojgupta]

@efriedma : can you take a look again?
Thanks!

[efriedma]

lib/IR/ConstantFold.cpp

1504 ↗

(On Diff #152954)

That's challenging... constants are, in general, outside any function, so we don't have any context for this. (More generally, I think ConstantExprs are a terrible idea, but we're basically stuck with them, at least for now.)

I guess in this particular case, we could just ignore the attribute and still get reasonable behavior; even if there's memory mapped at address 0, it probably isn't a GlobalValue. But please add a note here, and in LangRef.

lib/Transforms/IPO/GlobalOpt.cpp

1594

GV->getType() is the wrong type.

[manojgupta]

Fix an issue pointed by efriedma. All tests still pass.

[manojgupta]

lib/Transforms/IPO/GlobalOpt.cpp

1594

Done, Thanks for catching this.

[manojgupta]

Updated LangRef and added comment in ConstantFolding about inability
to query the attribute.

[efriedma]

Looking good. I think I want to look at it one more time when I'm fresh on Monday, and give a bit of time for anyone else to look at it.

docs/LangRef.rst

1469 ↗

(On Diff #153600)

global variable, not "GlobalVariable" (that's what it's called in other places in the reference).

[manojgupta]

Replace GlobalVariable by global variable.

[manojgupta]

Rebase to master + handle constant folding of intrinsic strip_invariant_group.

[george.burgess.iv]

Swooping in on this admittedly quite late: should the objectsize bits in MemoryBuiltins.cpp be made aware of this?

In particular, the ObjectSizeOffsetVisitor will happily hand you "0 bytes are accessible here" if you hand it null (...though not if you ask it via __builtin_object_size, since that has a gcc-compat hack around NULL, as well). We should probably make the NullIsUnknownSize field we pass into that factor in whether the current Function has this new null-is-ok attribute on it.

...Since it's possible for us to just hand it a ConstantPointerNull without a Function attached, we probably need to plumb this in from callers of llvm::getObjectSize/llvm::lowerObjectSizeCall/...

[manojgupta]

In D47895#1150325, @george.burgess.iv wrote:

Swooping in on this admittedly quite late: should the objectsize bits in MemoryBuiltins.cpp be made aware of this?

In particular, the ObjectSizeOffsetVisitor will happily hand you "0 bytes are accessible here" if you hand it null (...though not if you ask it via __builtin_object_size, since that has a gcc-compat hack around NULL, as well). We should probably make the NullIsUnknownSize field we pass into that factor in whether the current Function has this new null-is-ok attribute on it.

...Since it's possible for us to just hand it a ConstantPointerNull without a Function attached, we probably need to plumb this in from callers of llvm::getObjectSize/llvm::lowerObjectSizeCall/...

I don't know much about __builtin_object_size but I think GCC compatibility is the key here. Does GCC returns different results for __builtin_object_size when "-fno-delete-null-pointer-checks" is specified?

[efriedma]

In D47895#1150952, @manojgupta wrote:

I don't know much about __builtin_object_size but I think GCC compatibility is the key here. Does GCC returns different results for __builtin_object_size when "-fno-delete-null-pointer-checks" is specified?

__builtin_object_size lowers to llvm.objectsize with nullunknown = true, so we essentially treat it like an object anyway; it's not affected. The issue is for other users of llvm::getObjectSize (like alias analysis)

[george.burgess.iv]

What Eli said :)

This can probably be fixed by setting NullIsUnknownSize appropriately in callers of llvm::getObjectSize.

(We'll likely also want to ensure that the objectsize sanitizer in clang sets the nullunknown bit when it lowers to calls of @llvm.objectsize if clang's been handed this flag, but that's a different patch)

[manojgupta]

Handle getObjectSize by passing the ObjectSizeOpts in callers.

[chandlerc]

Adding George explicitly as a reviewer as I think this is now waiting on him.

[george.burgess.iv]

Ah, sorry. I thought I hit submit, but apparently I didn't. :)

Objectsize-related changes look good to me, modulo a few tiny nits. Thanks!

lib/Analysis/InstructionSimplify.cpp

2125 ↗

(On Diff #153985)

s/dyn_cast/cast/, please

(Or

if (auto *Foo = dyn_cast<AllocaInst>(LHS))
  if (isa<AllocaInst>(RHS) || isa<GlobalVariable>(RHS)) {
    // ...

; I'm impartial.)

lib/Transforms/Scalar/DeadStoreElimination.cpp

351 ↗

(On Diff #153985)

const Function *F?

718 ↗

(On Diff #153985)

const Function *F?

[manojgupta]

Made suggested changes.

[george.burgess.iv]

Nit fixes LGTM. Please wait for an official stamp from Eli before committing, since I can’t tell if he got the second look he wanted. (I’m inclined to believe he did, but don’t want to jump the gun)

Thanks for the patch!

[efriedma]

LGTM

[manojgupta]

Thanks everyone, specially Eli for the reviews.

[a.elovikov]

Note that, I am expecting that functions with alwaysinline attribute should still get inlined. If that happens and caller does not have this attribute, then optimizer is free to remove the checks.

Not that I actually hit that, but it was questionable when reading the code - why that way? From LangRef: alwaysinline "indicates that the inliner should attempt to inline this function into callers whenever possible, ignoring any active inlining size threshold for this caller." So there is some freedom by not doing the inlining, even in the presence of the attribute.

Wouldn't it be better not to inline or to add the attribute to the caller?

[manojgupta]

In D47895#1175164, @a.elovikov wrote:

Note that, I am expecting that functions with alwaysinline attribute should still get inlined. If that happens and caller does not have this attribute, then optimizer is free to remove the checks.

Not that I actually hit that, but it was questionable when reading the code - why that way? From LangRef: alwaysinline "indicates that the inliner should attempt to inline this function into callers whenever possible, ignoring any active inlining size threshold for this caller." So there is some freedom by not doing the inlining, even in the presence of the attribute.

Wouldn't it be better not to inline or to add the attribute to the caller?

Thanks for the comments.

Generally clang should add the attribute to all the functions if "-fno-delete-null-pointer-checks" is used so the mismatch should not happen in practice.

As per my understanding, If the "null-pointer-is-valid" is missing on a caller (but present on callee), this probably would happen in LTO where caller was intentionally compiled without "-fno-delete-null-pointer-checks". So adding the attribute to caller is probably not desired.
It was not clear to me if "-fno-delete-null-pointer-checks" mismatch should override always_inline but my inclination was to honor the user/system specified attribute over the compiler option.

I would be happy to change this logic if there is a consensus on how inlining should be handled in case of "null-pointer-isvalid" attribute mismatches.

[lebedev.ri]

In D47895#1175667, @manojgupta wrote:

In D47895#1175164, @a.elovikov wrote:

Note that, I am expecting that functions with alwaysinline attribute should still get inlined. If that happens and caller does not have this attribute, then optimizer is free to remove the checks.

Not that I actually hit that, but it was questionable when reading the code - why that way? From LangRef: alwaysinline "indicates that the inliner should attempt to inline this function into callers whenever possible, ignoring any active inlining size threshold for this caller." So there is some freedom by not doing the inlining, even in the presence of the attribute.

Wouldn't it be better not to inline or to add the attribute to the caller?

Thanks for the comments.

Generally clang should add the attribute to all the functions if "-fno-delete-null-pointer-checks" is used so the mismatch should not happen in practice.

Random uneducated thought - how does it work with multiple translation units, where not all of them have/don't have that flag set?

As per my understanding, If the "null-pointer-is-valid" is missing on a caller (but present on callee), this probably would happen in LTO where caller was intentionally compiled without "-fno-delete-null-pointer-checks". So adding the attribute to caller is probably not desired.
It was not clear to me if "-fno-delete-null-pointer-checks" mismatch should override always_inline but my inclination was to honor the user/system specified attribute over the compiler option.

I would be happy to change this logic if there is a consensus on how inlining should be handled in case of "null-pointer-isvalid" attribute mismatches.

[a.elovikov]

to change this logic if there is a consensus on how inlining should be handled in case of "null-pointer-isvalid" attribute mismatches.

I don't (and probably can't :) insist on changing the logic, but we should probably update the LangRef either in the entry for alwaysinline or in the entry of "null-pointer-isvalid". Otherwise, current transformation, IMO, breaks our own LangRef. Or do I miss something in its wording?

[manojgupta]

Random uneducated thought - how does it work with multiple translation units, where not all of them have/don't have that flag set?

Currently the logic disables inlining of functions with "null-pointer-is-valid" into functions that does not have this attribute.
However, if the callee has always_inline, inlining would still happen.

Here is an example of what GCC does in case of mismatched "fno-delete-null-pointer-checks".
https://godbolt.org/g/78Z2S4

There are 4 cases:
Case 1 : Attribute is on caller
Case 2 : Attribute is on callee
Case 3 : Attribute is on caller, callee has always_inline
Case 4 : Attribute is on callee, callee has always_inline

Inlining happens in all cases except (2). I think clang behavior matches GCC here.

Not sure of Langref changes, @efriedma What do you suggest?

[efriedma]

If the user forces inlining with always_inline, we should probably be conservative and propagate the attribute. If we do that, I don't think we need to say anything about it in LangRef.

[xbolva00]

llvm/trunk/lib/Analysis/InlineCost.cpp

1999 ↗

(On Diff #154716)

Why not put it to functionsHaveCompatibleAttributes?

[manojgupta]

llvm/trunk/lib/Analysis/InlineCost.cpp

1999 ↗

(On Diff #154716)

I can not not recall why (this change is > 3 years old ) but it does sound like the right thing to do.

include/llvm/Analysis/Utils/Local.h

for (User::op_iterator i = GEP->op_begin() + 1, e = GEP->op_end(); i != e;
     }
 
     // Emit an add instruction.
     Result = Builder->CreateAdd(Op, Result, GEP->getName()+".offs");
   }
   return Result;
 }
 
+/// Check whether null pointer dereferencing is considered undefined behavior
+/// for the function.
+/// Return value: false => null pointer dereference is undefined.
+/// Return value: true =>  null pointer dereference is not undefined.
+bool NullPointerIsDefined(const Function *F, unsigned AS = 0);
 }
 
 #endif // LLVM_TRANSFORMS_UTILS_LOCAL_H

lib/Analysis/LazyValueInfo.cpp

 #include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/ConstantFolding.h"
 #include "llvm/Analysis/InstructionSimplify.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/ValueTracking.h"
 #include "llvm/Analysis/ValueLattice.h"
+#include "llvm/Analysis/Utils/Local.h"
 #include "llvm/IR/AssemblyAnnotationWriter.h"
 #include "llvm/IR/CFG.h"
 #include "llvm/IR/ConstantRange.h"
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
bool LazyValueInfoImpl::solveBlockValueNonLocal(ValueLatticeElement &BBLV,
   ValueLatticeElement Result;  // Start Undefined.
 
   // If this is the entry block, we must be asking about an argument.  The
   // value is overdefined.
   if (BB == &BB->getParent()->getEntryBlock()) {
     assert(isa<Argument>(Val) && "Unknown live-in to the entry block");
     // Before giving up, see if we can prove the pointer non-null local to
     // this particular block.
-    if (Val->getType()->isPointerTy() &&
-        (isKnownNonZero(Val, DL) || isObjectDereferencedInBlock(Val, BB))) {
-      PointerType *PTy = cast<PointerType>(Val->getType());
+    PointerType *PTy = dyn_cast<PointerType>(Val->getType());
+    if (PTy &&
+        (isKnownNonZero(Val, DL) ||
+          (isObjectDereferencedInBlock(Val, BB) &&
+           !NullPointerIsDefined(BB->getParent(), PTy->getAddressSpace())))) {
       Result = ValueLatticeElement::getNot(ConstantPointerNull::get(PTy));
     } else {
       Result = ValueLatticeElement::getOverdefined();
     }
     BBLV = Result;
     return true;
   }
 
for (pred_iterator PI = pred_begin(BB), E = pred_end(BB); PI != E; ++PI) {
 
     // If we hit overdefined, exit early.  The BlockVals entry is already set
     // to overdefined.
     if (Result.isOverdefined()) {
       LLVM_DEBUG(dbgs() << " compute BB '" << BB->getName()
                         << "' - overdefined because of pred (non local).\n");
       // Before giving up, see if we can prove the pointer non-null local to
       // this particular block.
-      if (Val->getType()->isPointerTy() &&
-          isObjectDereferencedInBlock(Val, BB)) {
-        PointerType *PTy = cast<PointerType>(Val->getType());
+      PointerType *PTy = dyn_cast<PointerType>(Val->getType());
+      if (PTy && isObjectDereferencedInBlock(Val, BB) &&
+          !NullPointerIsDefined(BB->getParent(), PTy->getAddressSpace())) {
         Result = ValueLatticeElement::getNot(ConstantPointerNull::get(PTy));
       }
 
       BBLV = Result;
       return true;
     }
   }
 

lib/Transforms/IPO/GlobalOpt.cpp

static GlobalVariable *SRAGlobal(GlobalVariable *GV, const DataLayout &DL) {
   return FirstGlobal != NewGlobals.size() ? NewGlobals[FirstGlobal] : nullptr;
 }
 
 /// Return true if all users of the specified value will trap if the value is
 /// dynamically null.  PHIs keeps track of any phi nodes we've seen to avoid
 /// reprocessing them.
 static bool AllUsesOfValueWillTrapIfNull(const Value *V,
                                         SmallPtrSetImpl<const PHINode*> &PHIs) {
-  for (const User *U : V->users())
+  for (const User *U : V->users()) {
+    if (const Instruction *I = dyn_cast<Instruction>(U)) {
+      if (NullPointerIsDefined(I->getFunction()))
+        return false;
+    }
     if (isa<LoadInst>(U)) {
       // Will trap.
     } else if (const StoreInst *SI = dyn_cast<StoreInst>(U)) {
       if (SI->getOperand(0) == V) {
         //cerr << "NONTRAPPING USE: " << *U;
         return false;  // Storing the value.
       }
     } else if (const CallInst *CI = dyn_cast<CallInst>(U)) {
if (isa<LoadInst>(U)) {
         return false;
     } else if (isa<ICmpInst>(U) &&
                isa<ConstantPointerNull>(U->getOperand(1))) {
       // Ignore icmp X, null
     } else {
       //cerr << "NONTRAPPING USE: " << *U;
       return false;
     }
-
+  }
   return true;
 }
 
 /// Return true if all uses of any loads from GV will trap if the loaded value
 /// is null.  Note that this also permits comparisons of the loaded value
 /// against null, as a special case.
 static bool AllUsesOfLoadedValueWillTrapIfNull(const GlobalVariable *GV) {
   for (const User *U : GV->users())
static bool optimizeOnceStoredGlobal(GlobalVariable *GV, Value *StoredOnceVal,
 
   // If we are dealing with a pointer global that is initialized to null and
   // only has one (non-null) value stored into it, then we can optimize any
   // users of the loaded value (often calls and loads) that would trap if the
   // value was null.
   if (GV->getInitializer()->getType()->isPointerTy() &&
       GV->getInitializer()->isNullValue()) {
     if (Constant *SOVC = dyn_cast<Constant>(StoredOnceVal)) {
       if (GV->getInitializer()->getType() != SOVC->getType())
         SOVC = ConstantExpr::getBitCast(SOVC, GV->getInitializer()->getType());
 
       // Optimize away any trapping uses of the loaded value.
       if (OptimizeAwayTrappingUsesOfLoads(GV, SOVC, DL, TLI))
         return true;
     } else if (CallInst *CI = extractMallocCall(StoredOnceVal, TLI)) {
       Type *MallocType = getMallocAllocatedType(CI, TLI);
       if (MallocType && tryToOptimizeStoreOfMallocToGlobal(GV, CI, MallocType,
static bool processInternalGlobal(
   //
   // NOTE: It doesn't make sense to promote non-single-value types since we
   // are just replacing static memory to stack memory.
   //
   // If the global is in different address space, don't bring it to stack.
   if (!GS.HasMultipleAccessingFunctions &&
       GS.AccessingFunction &&
       GV->getValueType()->isSingleValueType() &&
       GV->getType()->getAddressSpace() == 0 &&
       !GV->isExternallyInitialized() &&
       allNonInstructionUsersCanBeMadeInstructions(GV) &&
       GS.AccessingFunction->doesNotRecurse() &&
       isPointerValueDeadOnEntryToFunction(GS.AccessingFunction, GV,
                                           LookupDomTree)) {
     const DataLayout &DL = GV->getParent()->getDataLayout();
 
     LLVM_DEBUG(dbgs() << "LOCALIZING GLOBAL: " << *GV << "\n");

lib/Transforms/InstCombine/InstCombineCalls.cpp

if (CalleeF->getCallingConv() != CS.getCallingConv() &&
       // We cannot remove an invoke, because it would change the CFG, just
       // change the callee to a null pointer.
       cast<InvokeInst>(OldCall)->setCalledFunction(
                                     Constant::getNullValue(CalleeF->getType()));
       return nullptr;
     }
   }
 
-  if (isa<ConstantPointerNull>(Callee) || isa<UndefValue>(Callee)) {
+  if ((isa<ConstantPointerNull>(Callee) &&
+       !NullPointerIsDefined(CS.getInstruction()->getFunction())) ||
+      isa<UndefValue>(Callee)) {
     // If CS does not return void then replaceAllUsesWith undef.
     // This allows ValueHandlers and custom metadata to adjust itself.
     if (!CS.getInstruction()->getType()->isVoidTy())
       replaceInstUsesWith(*CS.getInstruction(),
                           UndefValue::get(CS.getInstruction()->getType()));
 
     if (isa<InvokeInst>(CS.getInstruction())) {
       // Can't remove an invoke because we cannot change the CFG.

lib/Transforms/InstCombine/InstCombineLoadStoreAlloca.cpp

 
 static bool canSimplifyNullStoreOrGEP(StoreInst &SI) {
   if (SI.getPointerAddressSpace() != 0)
     return false;
 
   auto *Ptr = SI.getPointerOperand();
   if (GetElementPtrInst *GEPI = dyn_cast<GetElementPtrInst>(Ptr))
     Ptr = GEPI->getOperand(0);
-  return isa<ConstantPointerNull>(Ptr);
+  return (isa<ConstantPointerNull>(Ptr) &&
+          !NullPointerIsDefined(SI.getFunction(), SI.getPointerAddressSpace()));
 }
 
 static bool canSimplifyNullLoadOrGEP(LoadInst &LI, Value *Op) {
   if (GetElementPtrInst *GEPI = dyn_cast<GetElementPtrInst>(Op)) {
     const Value *GEPI0 = GEPI->getOperand(0);
-    if (isa<ConstantPointerNull>(GEPI0) && GEPI->getPointerAddressSpace() == 0)
+    if (isa<ConstantPointerNull>(GEPI0) &&
+        !NullPointerIsDefined(LI.getFunction(), GEPI->getPointerAddressSpace()))
       return true;
   }
   if (isa<UndefValue>(Op) ||
-      (isa<ConstantPointerNull>(Op) && LI.getPointerAddressSpace() == 0))
+      (isa<ConstantPointerNull>(Op) &&
+       !NullPointerIsDefined(LI.getFunction(), LI.getPointerAddressSpace())))
     return true;
   return false;
 }
 
 Instruction *InstCombiner::visitLoadInst(LoadInst &LI) {
   Value *Op = LI.getOperand(0);
 
   // Try to canonicalize the loaded type.
if (SelectInst *SI = dyn_cast<SelectInst>(Op)) {
         V1->setAtomic(LI.getOrdering(), LI.getSyncScopeID());
         V2->setAlignment(Align);
         V2->setAtomic(LI.getOrdering(), LI.getSyncScopeID());
         return SelectInst::Create(SI->getCondition(), V1, V2);
       }
 
       // load (select (cond, null, P)) -> load P
       if (isa<ConstantPointerNull>(SI->getOperand(1)) &&
-          LI.getPointerAddressSpace() == 0) {
+          !NullPointerIsDefined(SI->getFunction(),
+                                LI.getPointerAddressSpace())) {
         LI.setOperand(0, SI->getOperand(2));
         return &LI;
       }
 
       // load (select (cond, P, null)) -> load P
       if (isa<ConstantPointerNull>(SI->getOperand(2)) &&
-          LI.getPointerAddressSpace() == 0) {
+          !NullPointerIsDefined(SI->getFunction(),
+                                LI.getPointerAddressSpace())) {
         LI.setOperand(0, SI->getOperand(1));
         return &LI;
       }
     }
   }
   return nullptr;
 }
 

lib/Transforms/Scalar/SCCP.cpp

void SCCPSolver::visitLoadInst(LoadInst &I) {
   if (IV.isOverdefined()) return;
 
   if (!PtrVal.isConstant() || I.isVolatile())
     return markOverdefined(IV, &I);
 
   Constant *Ptr = PtrVal.getConstant();
 
   // load null is undefined.
-  if (isa<ConstantPointerNull>(Ptr) && I.getPointerAddressSpace() == 0)
+  if (isa<ConstantPointerNull>(Ptr)) {
+    if (NullPointerIsDefined(I.getFunction(), I.getPointerAddressSpace()))
+      return markOverdefined(IV, &I);
+    else
       return;
+  }
 
   // Transform load (constant global) into the value loaded.
   if (auto *GV = dyn_cast<GlobalVariable>(Ptr)) {
     if (!TrackedGlobals.empty()) {
       // If we are tracking this global, merge in the known value for it.
       DenseMap<GlobalVariable*, LatticeVal>::iterator It =
         TrackedGlobals.find(GV);
       if (It != TrackedGlobals.end()) {

lib/Transforms/Utils/Local.cpp

for (Instruction &I : *BB) {
               Changed = true;
               break;
             }
         }
       }
 
       if (auto *CI = dyn_cast<CallInst>(&I)) {
         Value *Callee = CI->getCalledValue();
-        if (isa<ConstantPointerNull>(Callee) || isa<UndefValue>(Callee)) {
+        if ((isa<ConstantPointerNull>(Callee) &&
+             !NullPointerIsDefined(CI->getFunction())) ||
+            isa<UndefValue>(Callee)) {
           changeToUnreachable(CI, /*UseLLVMTrap=*/false, false, DDT);
           Changed = true;
           break;
         }
         if (CI->doesNotReturn()) {
           // If we found a call to a no-return function, insert an unreachable
           // instruction after it.  Make sure there isn't *already* one there
           // though.
for (Instruction &I : *BB) {
         }
       }
     }
 
     TerminatorInst *Terminator = BB->getTerminator();
     if (auto *II = dyn_cast<InvokeInst>(Terminator)) {
       // Turn invokes that call 'nounwind' functions into ordinary calls.
       Value *Callee = II->getCalledValue();
-      if (isa<ConstantPointerNull>(Callee) || isa<UndefValue>(Callee)) {
+      if ((isa<ConstantPointerNull>(Callee) &&
+           !NullPointerIsDefined(BB->getParent())) ||
+          isa<UndefValue>(Callee)) {
         changeToUnreachable(II, true, false, DDT);
         Changed = true;
       } else if (II->doesNotThrow() && canSimplifyInvokeNoUnwind(&F)) {
         if (II->use_empty() && II->onlyReadsMemory()) {
           // jump to the normal destination branch.
           BasicBlock *NormalDestBB = II->getNormalDest();
           BasicBlock *UnwindDestBB = II->getUnwindDest();
           BranchInst::Create(NormalDestBB, II);
if (OpIdx == 0)
       return true;
     gep_type_iterator It = gep_type_begin(I);
     for (auto E = std::next(It, OpIdx); It != E; ++It)
       if (It.isStruct())
         return false;
     return true;
   }
 }
+
+bool llvm::NullPointerIsDefined(const Function *F, unsigned AS) {
+  if (F && F->getFnAttribute("null-pointer-is-valid")
+               .getValueAsString()
+               .equals("true"))
+    return true;
+
+  if (AS != 0)
+    return true;
+
+  return false;
+}

lib/Transforms/Utils/SimplifyCFG.cpp

if (C->isNullValue() || isa<UndefValue>(C)) {
 
     // Look through bitcasts.
     if (BitCastInst *BC = dyn_cast<BitCastInst>(Use))
       return passingValueIsAlwaysUndefined(V, BC);
 
     // Load from null is undefined.
     if (LoadInst *LI = dyn_cast<LoadInst>(Use))
       if (!LI->isVolatile())
-        return LI->getPointerAddressSpace() == 0;
+        return !NullPointerIsDefined(LI->getFunction(),
+                                     LI->getPointerAddressSpace());
 
     // Store to null is undefined.
     if (StoreInst *SI = dyn_cast<StoreInst>(Use))
       if (!SI->isVolatile())
-        return SI->getPointerAddressSpace() == 0 &&
+        return (!NullPointerIsDefined(SI->getFunction(),
+                                      SI->getPointerAddressSpace())) &&
                SI->getPointerOperand() == I;
 
     // A call to null is undefined.
     if (auto CS = CallSite(Use))
-      return CS.getCalledValue() == I;
+      return !NullPointerIsDefined(CS->getFunction()) &&
+             CS.getCalledValue() == I;
   }
   return false;
 }
 
 /// If BB has an incoming value that will always trigger undefined behavior
 /// (eg. null pointer dereference), remove the branch leading here.
 static bool removeUndefIntroducingPredecessor(BasicBlock *BB) {
   for (PHINode &PHI : BB->phis())

test/Transforms/CorrelatedValuePropagation/non-null.ll

 ; RUN: opt < %s -correlated-propagation -S | FileCheck %s
 
 define void @test1(i8* %ptr) {
 ; CHECK: test1
   %A = load i8, i8* %ptr
   br label %bb
 bb:
   icmp ne i8* %ptr, null
 ; CHECK-NOT: icmp
   ret void
 }
 
+define void @test1_no_null_opt(i8* %ptr) #0 {
+; CHECK: test1_no_null_opt
+  %A = load i8, i8* %ptr
+  br label %bb
+bb:
+  icmp ne i8* %ptr, null
+; CHECK: icmp ne i8* %ptr, null
+  ret void
+}
+
 define void @test2(i8* %ptr) {
 ; CHECK: test2
   store i8 0, i8* %ptr
   br label %bb
 bb:
   icmp ne i8* %ptr, null
 ; CHECK-NOT: icmp
   ret void
 }
 
+define void @test2_no_null_opt(i8* %ptr) #0 {
+; CHECK: test2_no_null_opt
+  store i8 0, i8* %ptr
+  br label %bb
+bb:
+  icmp ne i8* %ptr, null
+; CHECK: icmp ne i8* %ptr, null
+  ret void
+}
+
 define void @test3() {
 ; CHECK: test3
   %ptr = alloca i8
   br label %bb
 bb:
   icmp ne i8* %ptr, null
 ; CHECK-NOT: icmp
   ret void
 }
 
+;; OK to remove icmp here since ptr is coming from alloca.
+define void @test3_no_null_opt() #0 {
+; CHECK: test3
+  %ptr = alloca i8
+  br label %bb
+bb:
+  icmp ne i8* %ptr, null
+; CHECK-NOT: icmp
+  ret void
+}
+
 declare void @llvm.memcpy.p0i8.p0i8.i32(i8*, i8*, i32, i1)
 define void @test4(i8* %dest, i8* %src) {
 ; CHECK: test4
   call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 1, i1 false)
   br label %bb
 bb:
   icmp ne i8* %dest, null
   icmp ne i8* %src, null
 ; CHECK-NOT: icmp
   ret void
 }
 
+define void @test4_no_null_opt(i8* %dest, i8* %src) #0 {
+; CHECK: test4_no_null_opt
+  call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 1, i1 false)
+  br label %bb
+bb:
+  icmp ne i8* %dest, null
+  icmp ne i8* %src, null
+; CHECK: icmp ne i8* %dest, null
+; CHECK: icmp ne i8* %src, null
+  ret void
+}
+
 declare void @llvm.memmove.p0i8.p0i8.i32(i8*, i8*, i32, i1)
 define void @test5(i8* %dest, i8* %src) {
 ; CHECK: test5
   call void @llvm.memmove.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 1, i1 false)
   br label %bb
 bb:
   icmp ne i8* %dest, null
   icmp ne i8* %src, null
 ; CHECK-NOT: icmp
   ret void
 }
 
+define void @test5_no_null_opt(i8* %dest, i8* %src) #0 {
+; CHECK: test5_no_null_opt
+  call void @llvm.memmove.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 1, i1 false)
+  br label %bb
+bb:
+  icmp ne i8* %dest, null
+  icmp ne i8* %src, null
+; CHECK: icmp ne i8* %dest, null
+; CHECK: icmp ne i8* %src, null
+  ret void
+}
+
 declare void @llvm.memset.p0i8.i32(i8*, i8, i32, i1)
 define void @test6(i8* %dest) {
 ; CHECK: test6
   call void @llvm.memset.p0i8.i32(i8* %dest, i8 255, i32 1, i1 false)
   br label %bb
 bb:
   icmp ne i8* %dest, null
 ; CHECK-NOT: icmp
   ret void
 }
 
+define void @test6_no_null_opt(i8* %dest) #0 {
+; CHECK: test6_no_null_opt
+  call void @llvm.memset.p0i8.i32(i8* %dest, i8 255, i32 1, i1 false)
+  br label %bb
+bb:
+  icmp ne i8* %dest, null
+; CHECK: icmp ne i8* %dest, null
+  ret void
+}
+
 define void @test7(i8* %dest, i8* %src, i32 %len) {
 ; CHECK: test7
   call void @llvm.memcpy.p0i8.p0i8.i32(i8* %dest, i8* %src, i32 %len, i1 false)
   br label %bb
 bb:
   %KEEP1 = icmp ne i8* %dest, null
 ; CHECK: KEEP1
   %KEEP2 = icmp ne i8* %src, null
null:
   br label %merge
 
 merge:
   %merged_arg = phi i8* [%another_arg, %null], [%arg1, %non_null]
   call void @test12_helper(i8* %merged_arg)
   ; CHECK: call void @test12_helper(i8* nonnull %merged_arg)
   ret void
 }
+
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GVN/PRE/2018-06-08-pre-load-dbgloc-no-null-opt.ll

+; This test checks if debug loc is propagated to load/store created by GVN/Instcombine.
+; RUN: opt < %s -gvn -S | FileCheck %s --check-prefixes=ALL
+; RUN: opt < %s -gvn -instcombine -S | FileCheck %s --check-prefixes=ALL
+
+; struct node {
+;  int  *v;
+; struct desc *descs;
+; };
+
+; struct desc {
+;  struct node *node;
+; };
+
+; extern int bar(void *v, void* n);
+
+; int test(struct desc *desc)
+; {
+;  void *v, *n;
+;  v = !desc ? ((void *)0) : desc->node->v;  // Line 15
+;  n = &desc->node->descs[0];                // Line 16
+;  return bar(v, n);
+; }
+
+; Line 16, Column 13:
+;   n = &desc->node->descs[0];
+;              ^
+
+target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+target triple = "aarch64--linux-gnu"
+
+%struct.desc = type { %struct.node* }
+%struct.node = type { i32*, %struct.desc* }
+
+define i32 @test_no_null_opt(%struct.desc* readonly %desc) local_unnamed_addr #0 !dbg !4 {
+entry:
+  %tobool = icmp eq %struct.desc* %desc, null
+  br i1 %tobool, label %cond.end, label %cond.false, !dbg !9
+; ALL: br i1 %tobool, label %entry.cond.end_crit_edge, label %cond.false, !dbg [[LOC_15_6:![0-9]+]]
+; ALL: entry.cond.end_crit_edge:
+; ALL: load %struct.node*, %struct.node** null, align {{[0-9]+}}, !dbg [[LOC_16_13:![0-9]+]]
+
+cond.false:
+  %0 = bitcast %struct.desc* %desc to i8***, !dbg !11
+  %1 = load i8**, i8*** %0, align 8, !dbg !11
+  %2 = load i8*, i8** %1, align 8
+  br label %cond.end, !dbg !9
+
+cond.end:
+; ALL: phi %struct.node* [ %3, %cond.false ], [ %.pre, %entry.cond.end_crit_edge ]
+; ALL: phi i8* [ %2, %cond.false ], [ null, %entry.cond.end_crit_edge ]
+
+  %3 = phi i8* [ %2, %cond.false ], [ null, %entry ], !dbg !9
+  %node2 = getelementptr inbounds %struct.desc, %struct.desc* %desc, i64 0, i32 0
+  %4 = load %struct.node*, %struct.node** %node2, align 8, !dbg !10
+  %descs = getelementptr inbounds %struct.node, %struct.node* %4, i64 0, i32 1
+  %5 = bitcast %struct.desc** %descs to i8**
+  %6 = load i8*, i8** %5, align 8
+  %call = tail call i32 @bar(i8* %3, i8* %6)
+  ret i32 %call
+}
+attributes #0 = { "null-pointer-is-valid"="true" }
+
+declare i32 @bar(i8*, i8*) local_unnamed_addr #1
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!2, !3}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, emissionKind: FullDebug)
+!1 = !DIFile(filename: "test.c", directory: ".")
+!2 = !{i32 2, !"Dwarf Version", i32 4}
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = distinct !DISubprogram(name: "test_no_null_opt", scope: !1, file: !1, line: 12, type: !5, isLocal: false, isDefinition: true, scopeLine: 13, flags: DIFlagPrototyped, isOptimized: true, unit: !0, retainedNodes: !8)
+!5 = !DISubroutineType(types: !6)
+!6 = !{!7}
+!7 = !DIBasicType(name: "int", size: 32, encoding: DW_ATE_signed)
+!8 = !{}
+!9 = !DILocation(line: 15, column: 6, scope: !4)
+!10 = !DILocation(line: 16, column: 13, scope: !4)
+!11 = !DILocation(line: 15, column: 34, scope: !4)
+
+;ALL: [[SCOPE:![0-9]+]] = distinct  !DISubprogram(name: "test_no_null_opt",{{.*}}
+;ALL: [[LOC_15_6]] = !DILocation(line: 15, column: 6, scope: [[SCOPE]])
+;ALL: [[LOC_16_13]] = !DILocation(line: 16, column: 13, scope: [[SCOPE]])

test/Transforms/GlobalOpt/heap-sra-1-no-null-opt.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
-  %struct.foo = type { i32, i32 }
 @X = internal global %struct.foo* null
-; CHECK: @X.f0
-; CHECK: @X.f1
+; CHECK: @X
+; CHECK-NOT: @X.f0
 
-define void @bar(i64 %Size) nounwind noinline {
+define void @bar(i64 %Size) nounwind noinline #0 {
 entry:
   %mallocsize = mul i64 %Size, 8                  ; <i64> [#uses=1]
   %malloccall = tail call i8* @malloc(i64 %mallocsize) ; <i8*> [#uses=1]
   %.sub = bitcast i8* %malloccall to %struct.foo* ; <%struct.foo*> [#uses=1]
   store %struct.foo* %.sub, %struct.foo** @X, align 4
   ret void
 }
 
 declare noalias i8* @malloc(i64)
 
-define i32 @baz() nounwind readonly noinline {
+define i32 @baz() nounwind readonly noinline #0 {
 bb1.thread:
-  %0 = load %struct.foo*, %struct.foo** @X, align 4    
+  %0 = load %struct.foo*, %struct.foo** @X, align 4
   br label %bb1
 
 bb1:    ; preds = %bb1, %bb1.thread
   %i.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %indvar.next, %bb1 ]
   %sum.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %3, %bb1 ]
   %1 = getelementptr %struct.foo, %struct.foo* %0, i32 %i.0.reg2mem.0, i32 0
   %2 = load i32, i32* %1, align 4
-  %3 = add i32 %2, %sum.0.reg2mem.0  
-  %indvar.next = add i32 %i.0.reg2mem.0, 1  
-  %exitcond = icmp eq i32 %indvar.next, 1200    
+  %3 = add i32 %2, %sum.0.reg2mem.0
+  %indvar.next = add i32 %i.0.reg2mem.0, 1
+  %exitcond = icmp eq i32 %indvar.next, 1200
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+attributes #0 = { "null-pointer-is-valid"="true" }
+

test/Transforms/GlobalOpt/heap-sra-1.ll

bb1:    ; preds = %bb1, %bb1.thread
   %indvar.next = add i32 %i.0.reg2mem.0, 1  
   %exitcond = icmp eq i32 %indvar.next, 1200    
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+define void @bam(i64 %Size) nounwind noinline #0 {
+entry:
+  %0 = load %struct.foo*, %struct.foo** @X, align 4
+        ret void
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/heap-sra-2-no-null-opt.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
-  %struct.foo = type { i32, i32 }
 @X = internal global %struct.foo* null    ; <%struct.foo**> [#uses=2]
-; CHECK: @X.f0
-; CHECK: @X.f1
+; CHECK: @X
+; CHECK-NOT: @X.f0
 
-define void @bar(i32 %Size) nounwind noinline {
+define void @bar(i32 %Size) nounwind noinline #0 {
 entry:
   %malloccall = tail call i8* @malloc(i64 8000000) ; <i8*> [#uses=1]
   %0 = bitcast i8* %malloccall to [1000000 x %struct.foo]* ; <[1000000 x %struct.foo]*> [#uses=1]
   %.sub = getelementptr [1000000 x %struct.foo], [1000000 x %struct.foo]* %0, i32 0, i32 0    ; <%struct.foo*> [#uses=1]
   store %struct.foo* %.sub, %struct.foo** @X, align 4
   ret void
 }
 
 declare noalias i8* @malloc(i64)
 
-define i32 @baz() nounwind readonly noinline {
+define i32 @baz() nounwind readonly noinline #0 {
 bb1.thread:
   %0 = load %struct.foo*, %struct.foo** @X, align 4    ; <%struct.foo*> [#uses=1]
   br label %bb1
 
 bb1:    ; preds = %bb1, %bb1.thread
   %i.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %indvar.next, %bb1 ]    ; <i32> [#uses=2]
   %sum.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %3, %bb1 ]    ; <i32> [#uses=1]
   %1 = getelementptr %struct.foo, %struct.foo* %0, i32 %i.0.reg2mem.0, i32 0    ; <i32*> [#uses=1]
   %2 = load i32, i32* %1, align 4    ; <i32> [#uses=1]
   %3 = add i32 %2, %sum.0.reg2mem.0    ; <i32> [#uses=2]
   %indvar.next = add i32 %i.0.reg2mem.0, 1    ; <i32> [#uses=2]
   %exitcond = icmp eq i32 %indvar.next, 1200    ; <i1> [#uses=1]
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/heap-sra-2.ll

bb1:    ; preds = %bb1, %bb1.thread
   %indvar.next = add i32 %i.0.reg2mem.0, 1    ; <i32> [#uses=2]
   %exitcond = icmp eq i32 %indvar.next, 1200    ; <i1> [#uses=1]
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+define void @bam(i64 %Size) nounwind noinline #0 {
+entry:
+        %0 = load %struct.foo*, %struct.foo** @X, align 4
+        ret void
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/heap-sra-3-no-null-opt.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
-  %struct.foo = type { i32, i32 }
 @X = internal global %struct.foo* null
-; CHECK: @X.f0
-; CHECK: @X.f1
+; CHECK: @X
+; CHECK-NOT: @X.f0
 
-define void @bar(i64 %Size) nounwind noinline {
+define void @bar(i64 %Size) nounwind noinline #0 {
 entry:
   %mallocsize = mul i64 8, %Size ; <i64> [#uses=1]
-; CHECK: mul i64 %Size, 4
+; CHECK: mul i64 8, %Size
   %malloccall = tail call i8* @malloc(i64 %mallocsize) ; <i8*> [#uses=1]
   %.sub = bitcast i8* %malloccall to %struct.foo* ; <%struct.foo*> [#uses=1]
   store %struct.foo* %.sub, %struct.foo** @X, align 4
   ret void
 }
 
 declare noalias i8* @malloc(i64)
 
-define i32 @baz() nounwind readonly noinline {
+define i32 @baz() nounwind readonly noinline #0 {
 bb1.thread:
-  %0 = load %struct.foo*, %struct.foo** @X, align 4    
+; CHECK: load %struct.foo*, %struct.foo** @X, align 4
+  %0 = load %struct.foo*, %struct.foo** @X, align 4
   br label %bb1
 
 bb1:    ; preds = %bb1, %bb1.thread
   %i.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %indvar.next, %bb1 ]
   %sum.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %3, %bb1 ]
   %1 = getelementptr %struct.foo, %struct.foo* %0, i32 %i.0.reg2mem.0, i32 0
   %2 = load i32, i32* %1, align 4
-  %3 = add i32 %2, %sum.0.reg2mem.0  
-  %indvar.next = add i32 %i.0.reg2mem.0, 1  
-  %exitcond = icmp eq i32 %indvar.next, 1200    
+  %3 = add i32 %2, %sum.0.reg2mem.0
+  %indvar.next = add i32 %i.0.reg2mem.0, 1
+  %exitcond = icmp eq i32 %indvar.next, 1200
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/heap-sra-3.ll

bb1:    ; preds = %bb1, %bb1.thread
   %indvar.next = add i32 %i.0.reg2mem.0, 1  
   %exitcond = icmp eq i32 %indvar.next, 1200    
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+define void @bam(i64 %Size) nounwind noinline #0 {
+entry:
+        %0 = load %struct.foo*, %struct.foo** @X, align 4
+        ret void
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/heap-sra-4-no-null-opt.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
-  %struct.foo = type { i32, i32 }
+
 @X = internal global %struct.foo* null
-; CHECK: @X.f0
-; CHECK: @X.f1
+; CHECK: @X
+; CHECK-NOT: @X.f0
 
-define void @bar(i64 %Size) nounwind noinline {
+define void @bar(i64 %Size) nounwind noinline #0 {
 entry:
   %mallocsize = shl i64 %Size, 3                  ; <i64> [#uses=1]
   %malloccall = tail call i8* @malloc(i64 %mallocsize) ; <i8*> [#uses=1]
-; CHECK: mul i64 %Size, 4
+; CHECK: shl i64 %Size, 3
   %.sub = bitcast i8* %malloccall to %struct.foo* ; <%struct.foo*> [#uses=1]
   store %struct.foo* %.sub, %struct.foo** @X, align 4
   ret void
 }
 
 declare noalias i8* @malloc(i64)
 
-define i32 @baz() nounwind readonly noinline {
+define i32 @baz() nounwind readonly noinline #0 {
+; CHECK-LABEL: @baz(
 bb1.thread:
-  %0 = load %struct.foo*, %struct.foo** @X, align 4    
+; CHECK: load %struct.foo*, %struct.foo** @X, align 4
+  %0 = load %struct.foo*, %struct.foo** @X, align 4
   br label %bb1
 
 bb1:    ; preds = %bb1, %bb1.thread
   %i.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %indvar.next, %bb1 ]
   %sum.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %3, %bb1 ]
   %1 = getelementptr %struct.foo, %struct.foo* %0, i32 %i.0.reg2mem.0, i32 0
   %2 = load i32, i32* %1, align 4
-  %3 = add i32 %2, %sum.0.reg2mem.0  
-  %indvar.next = add i32 %i.0.reg2mem.0, 1  
-  %exitcond = icmp eq i32 %indvar.next, 1200    
+; CHECK: load i32, i32* %1, align 4
+  %3 = add i32 %2, %sum.0.reg2mem.0
+  %indvar.next = add i32 %i.0.reg2mem.0, 1
+  %exitcond = icmp eq i32 %indvar.next, 1200
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/heap-sra-4.ll

bb1:    ; preds = %bb1, %bb1.thread
   %indvar.next = add i32 %i.0.reg2mem.0, 1  
   %exitcond = icmp eq i32 %indvar.next, 1200    
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %3
 }
 
+define void @bam(i64 %Size) nounwind noinline #0 {
+entry:
+        %0 = load %struct.foo*, %struct.foo** @X, align 4
+        ret void
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }
+

test/Transforms/GlobalOpt/heap-sra-phi-no-null-opt.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
-; CHECK: tmp.f1 = phi i32*
-; CHECK: tmp.f0 = phi i32*
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
-  %struct.foo = type { i32, i32 }
+
 @X = internal global %struct.foo* null    ; <%struct.foo**> [#uses=2]
+; CHECK: @X
+; CHECK-NOT: @X.f0
 
-define void @bar(i32 %Size) nounwind noinline {
+define void @bar(i32 %Size) nounwind noinline #0 {
+;  CHECK-LABEL: @bar(
 entry:
   %malloccall = tail call i8* @malloc(i64 8000000) ; <i8*> [#uses=1]
   %tmp = bitcast i8* %malloccall to [1000000 x %struct.foo]* ; <[1000000 x %struct.foo]*> [#uses=1]
   %.sub = getelementptr [1000000 x %struct.foo], [1000000 x %struct.foo]* %tmp, i32 0, i32 0    ; <%struct.foo*> [#uses=1]
   store %struct.foo* %.sub, %struct.foo** @X, align 4
   ret void
 }
 
 declare noalias i8* @malloc(i64)
 
-define i32 @baz() nounwind readonly noinline {
+define i32 @baz() nounwind readonly noinline #0 {
+; CHECK-LABEL: @baz(
 bb1.thread:
   %tmpLD1 = load %struct.foo*, %struct.foo** @X, align 4    ; <%struct.foo*> [#uses=1]
+; CHECK: load %struct.foo*, %struct.foo** @X, align 4
   br label %bb1
 
 bb1:    ; preds = %bb1, %bb1.thread
         %tmp = phi %struct.foo* [%tmpLD1, %bb1.thread ], [ %tmpLD2, %bb1 ]    ; <i32> [#uses=2]
+; CHECK: %tmp = phi %struct.foo* [ %tmpLD1, %bb1.thread ], [ %tmpLD2, %bb1 ]
   %i.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %indvar.next, %bb1 ]    ; <i32> [#uses=2]
   %sum.0.reg2mem.0 = phi i32 [ 0, %bb1.thread ], [ %tmp3, %bb1 ]    ; <i32> [#uses=1]
   %tmp1 = getelementptr %struct.foo, %struct.foo* %tmp, i32 %i.0.reg2mem.0, i32 0    ; <i32*> [#uses=1]
   %tmp2 = load i32, i32* %tmp1, align 4    ; <i32> [#uses=1]
+; CHECK: load i32, i32* %tmp1, align 4
   %tmp6 = add i32 %tmp2, %sum.0.reg2mem.0   ; <i32> [#uses=2]
   %tmp4 = getelementptr %struct.foo, %struct.foo* %tmp, i32 %i.0.reg2mem.0, i32 1    ; <i32*> [#uses=1]
         %tmp5 = load i32 , i32 * %tmp4
+;  CHECK: load i32, i32* %tmp4
         %tmp3 = add i32 %tmp5, %tmp6
   %indvar.next = add i32 %i.0.reg2mem.0, 1    ; <i32> [#uses=2]
-        
         %tmpLD2 = load %struct.foo*, %struct.foo** @X, align 4    ; <%struct.foo*> [#uses=1]
+; CHECK: load %struct.foo*, %struct.foo** @X, align 4
 
   %exitcond = icmp eq i32 %indvar.next, 1200    ; <i1> [#uses=1]
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %tmp3
 }
+
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/heap-sra-phi.ll

%indvar.next = add i32 %i.0.reg2mem.0, 1    ; <i32> [#uses=2]
         %tmpLD2 = load %struct.foo*, %struct.foo** @X, align 4    ; <%struct.foo*> [#uses=1]
 
   %exitcond = icmp eq i32 %indvar.next, 1200    ; <i1> [#uses=1]
   br i1 %exitcond, label %bb2, label %bb1
 
 bb2:    ; preds = %bb1
   ret i32 %tmp3
 }
+
+define void @bam(i64 %Size) nounwind noinline #0 {
+entry:
+        %0 = load %struct.foo*, %struct.foo** @X, align 4
+        ret void
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/malloc-promote-1-no-null-opt.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
 @G = internal global i32* null          ; <i32**> [#uses=3]
-; CHECK-NOT: global
+; CHECK: global
 
-define void @init() {
+define void @init() #0 {
+; CHECK-LABEL: @init(
+; CHECK: store
+; CHECK: load
         %malloccall = tail call i8* @malloc(i64 4)      ; <i8*> [#uses=1]
         %P = bitcast i8* %malloccall to i32*            ; <i32*> [#uses=1]
         store i32* %P, i32** @G
         %GV = load i32*, i32** @G             ; <i32*> [#uses=1]
         store i32 0, i32* %GV
         ret void
 }
 
 declare noalias i8* @malloc(i64)
 
-define i32 @get() {
+define i32 @get() #0 {
+; CHECK-LABEL: @get(
+; CHECK: load i32*, i32** @G
+; CHECK-NEXT: load i32, i32* %GV
         %GV = load i32*, i32** @G             ; <i32*> [#uses=1]
         %V = load i32, i32* %GV              ; <i32> [#uses=1]
         ret i32 %V
-; CHECK: ret i32 0
+; CHECK: ret i32 %V
 }
 
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/malloc-promote-1.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
-@G = internal global i32* null          ; <i32**> [#uses=3]
+@G = internal global i32* null          ; <i32**> [#uses=4]
 ; CHECK-NOT: global
 
 define void @init() {
         %malloccall = tail call i8* @malloc(i64 4)      ; <i8*> [#uses=1]
         %P = bitcast i8* %malloccall to i32*            ; <i32*> [#uses=1]
         store i32* %P, i32** @G
         %GV = load i32*, i32** @G             ; <i32*> [#uses=1]
         store i32 0, i32* %GV
         ret void
 }
 
 declare noalias i8* @malloc(i64)
 
 define i32 @get() {
         %GV = load i32*, i32** @G             ; <i32*> [#uses=1]
         %V = load i32, i32* %GV              ; <i32> [#uses=1]
         ret i32 %V
 ; CHECK: ret i32 0
 }
 
+define void @foo(i64 %Size) nounwind noinline #0 {
+entry:
+        %0 = load i32*, i32** @G, align 4
+        ret void
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }
+

test/Transforms/GlobalOpt/malloc-promote-2-no-null-opt.ll

 ; RUN: opt < %s -globalopt -S | FileCheck %s
 target datalayout = "E-p:64:64:64-a0:0:8-f32:32:32-f64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-v64:64:64-v128:128:128"
 
 @G = internal global i32* null
 
-define void @t() {
+define void @t() #0 {
 ; CHECK: @t()
-; CHECK-NOT: call i8* @malloc
-; CHECK-NEXT: ret void
+; CHECK: call i8* @malloc
+; CHECK: bitcast
+; CHECK: store
+; CHECK: load
+; CHECK: getelementptr
+; CHECK: store
   %malloccall = tail call i8* @malloc(i64 mul (i64 100, i64 4))
   %P = bitcast i8* %malloccall to i32*
   store i32* %P, i32** @G
   %GV = load i32*, i32** @G
   %GVe = getelementptr i32, i32* %GV, i32 40
   store i32 20, i32* %GVe
   ret void
 }
 
 declare noalias i8* @malloc(i64)
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/GlobalOpt/malloc-promote-2.ll

; CHECK-NEXT: ret void
   store i32* %P, i32** @G
   %GV = load i32*, i32** @G
   %GVe = getelementptr i32, i32* %GV, i32 40
   store i32 20, i32* %GVe
   ret void
 }
 
 declare noalias i8* @malloc(i64)
+
+define void @foo(i64 %Size) nounwind noinline #0 {
+entry:
+        %0 = load i32*, i32** @G, align 4
+        ret void
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }
+

test/Transforms/IPConstantProp/PR26044.ll

 
 define internal i32 @fn1(i32 %p1) {
 entry:
   %tobool = icmp ne i32 %p1, 0
   %cond = select i1 %tobool, i32 %p1, i32 %p1
   ret i32 %cond
 }
 
+define void @fn_no_null_opt() #0 {
+entry:
+  br label %if.end
+
+for.cond1:                                        ; preds = %if.end, %for.end
+  br i1 undef, label %if.end, label %if.end
+
+if.end:                                           ; preds = %lbl, %for.cond1
+  %e.2 = phi i32* [ undef, %entry ], [ null, %for.cond1 ], [ null, %for.cond1 ]
+  %0 = load i32, i32* %e.2, align 4
+  %call = call i32 @fn0(i32 %0)
+  br label %for.cond1
+}
+
+define internal i32 @fn0(i32 %p1) {
+entry:
+  %tobool = icmp ne i32 %p1, 0
+  %cond = select i1 %tobool, i32 %p1, i32 %p1
+  ret i32 %cond
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }
+
 ; CHECK-LABEL: define void @fn2(
 ; CHECK: call i32 @fn1(i32 undef)
 
 ; CHECK-LABEL: define internal i32 @fn1(
 ; CHECK:%[[COND:.*]] = select i1 undef, i32 undef, i32 undef
 ; CHECK: ret i32 %[[COND]]
+
+; CHECK-LABEL: define void @fn_no_null_opt(
+; CHECK: call i32 @fn0(i32 %0)
+
+; CHECK-LABEL: define internal i32 @fn0(
+; CHECK:%[[TOBOOL:.*]] = icmp ne i32 %p1, 0
+; CHECK:%[[COND:.*]] = select i1 %[[TOBOOL]], i32 %p1, i32 %p1
+; CHECK: ret i32 %[[COND]]

test/Transforms/InstCombine/invoke.ll

 lpad:
   %1 = landingpad { i8*, i32 }
           filter [0 x i8*] zeroinitializer
   %2 = extractvalue { i8*, i32 } %1, 0
   tail call void @__cxa_call_unexpected(i8* %2) noreturn nounwind
   unreachable
 }
 
+; CHECK-LABEL: @f2_no_null_opt(
+define i64 @f2_no_null_opt() nounwind uwtable ssp #0 personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
+entry:
+; CHECK: invoke noalias i8* null()
+  %call = invoke noalias i8* null()
+          to label %invoke.cont unwind label %lpad
+
+invoke.cont:
+; CHECK: call i64 @llvm.objectsize.i64.p0i8(i8* %call, i1 false, i1 false)
+  %0 = tail call i64 @llvm.objectsize.i64(i8* %call, i1 false)
+  ret i64 %0
+
+lpad:
+  %1 = landingpad { i8*, i32 }
+          filter [0 x i8*] zeroinitializer
+  %2 = extractvalue { i8*, i32 } %1, 0
+  tail call void @__cxa_call_unexpected(i8* %2) noreturn nounwind
+  unreachable
+}
+attributes #0 = { "null-pointer-is-valid"="true" }
+
 ; CHECK-LABEL: @f3(
 define void @f3() nounwind uwtable ssp personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
 ; CHECK: invoke void @llvm.donothing()
   %call = invoke noalias i8* @_Znwm(i64 13)
           to label %invoke.cont unwind label %lpad
 
 invoke.cont:
   ret void
 
 lpad:
   %1 = landingpad { i8*, i32 }
           filter [0 x i8*] zeroinitializer
   %2 = extractvalue { i8*, i32 } %1, 0
   tail call void @__cxa_call_unexpected(i8* %2) noreturn nounwind
   unreachable
 }

test/Transforms/InstCombine/load.ll

 ; CHECK-LABEL: @test7(
 ; CHECK-NOT: load
 define i32 @test7(i32 %X) {
   %V = getelementptr i32, i32* null, i32 %X    ; <i32*> [#uses=1]
   %R = load i32, i32* %V    ; <i32> [#uses=1]
   ret i32 %R
 }
 
+; CHECK-LABEL: @test7_no_null_opt(
+; CHECK: %V = getelementptr i32, i32* null
+; CHECK: %R = load i32, i32* %V
+define i32 @test7_no_null_opt(i32 %X) #0 {
+        %V = getelementptr i32, i32* null, i32 %X               ; <i32*> [#uses=1]
+        %R = load i32, i32* %V          ; <i32> [#uses=1]
+        ret i32 %R
+}
+attributes #0 = { "null-pointer-is-valid"="true" }
+
 ; CHECK-LABEL: @test8(
 ; CHECK-NOT: load
 define i32 @test8(i32* %P) {
   store i32 1, i32* %P
   %X = load i32, i32* %P    ; <i32> [#uses=1]
   ret i32 %X
 }
 

test/Transforms/InstCombine/select.ll

 ; CHECK-NEXT:    [[V:%.*]] = load i32, i32 addrspace(1)* [[P2]], align 4
 ; CHECK-NEXT:    ret i32 [[V]]
 ;
   %P2 = select i1 %C, i32 addrspace(1)* null, i32 addrspace(1)* %P
   %V = load i32, i32 addrspace(1)* %P2
   ret i32 %V
 }
 
+;; It may be legal to load from a null address with null pointer valid attribute.
+define i32 @test16_no_null_opt(i1 %C, i32* %P) #0 {
+; CHECK-LABEL: @test16_no_null_opt(
+; CHECK-NEXT:    [[P2:%.*]] = select i1 [[C:%.*]], i32* [[P:%.*]], i32* null
+; CHECK-NEXT:    [[V:%.*]] = load i32, i32* [[P2]], align 4
+; CHECK-NEXT:    ret i32 [[V]]
+;
+  %P2 = select i1 %C, i32* %P, i32* null
+  %V = load i32, i32* %P2
+  ret i32 %V
+}
+
+define i32 @test16_no_null_opt_2(i1 %C, i32* %P) #0 {
+; CHECK-LABEL: @test16_no_null_opt_2(
+; CHECK-NEXT:    [[P2:%.*]] = select i1 [[C:%.*]], i32* null, i32* [[P:%.*]]
+; CHECK-NEXT:    [[V:%.*]] = load i32, i32* [[P2]], align 4
+; CHECK-NEXT:    ret i32 [[V]]
+;
+  %P2 = select i1 %C, i32* null, i32* %P
+  %V = load i32, i32* %P2
+  ret i32 %V
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }
+
 define i1 @test17(i32* %X, i1 %C) {
 ; CHECK-LABEL: @test17(
 ; CHECK-NEXT:    [[RV1:%.*]] = icmp eq i32* [[X:%.*]], null
 ; CHECK-NEXT:    [[NOT_C:%.*]] = xor i1 [[C:%.*]], true
 ; CHECK-NEXT:    [[RV:%.*]] = or i1 [[RV1]], [[NOT_C]]
 ; CHECK-NEXT:    ret i1 [[RV]]
 ;
   %R = select i1 %C, i32* %X, i32* null

test/Transforms/InstCombine/store.ll

 define void @store_at_gep_off_null(i64 %offset) {
 ; CHECK-LABEL: @store_at_gep_off_null
 ; CHECK: store i32 undef, i32* %ptr
    %ptr = getelementptr i32, i32 *null, i64 %offset
    store i32 24, i32* %ptr
    ret void
 }
 
+define void @store_at_gep_off_no_null_opt(i64 %offset) #0 {
+   %ptr = getelementptr i32, i32 *null, i64 %offset
+   store i32 24, i32* %ptr
+   ret void
+; CHECK-LABEL: @store_at_gep_off_no_null_opt(i64 %offset)
+; CHECK-NEXT: %ptr = getelementptr i32, i32* null, i64 %offset
+; CHECK-NEXT: store i32 24, i32* %ptr
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }
+
 ;; Simple sinking tests
 
 ; "if then else"
 define i32 @test3(i1 %C) {
   %A = alloca i32
         br i1 %C, label %Cond, label %Cond2
 
 Cond:

test/Transforms/SimplifyCFG/UnreachableEliminate.ll

  br label %bb2
 
 bb2:
   %ptr.2 = phi i8* [ %ptr, %bb3 ], [ null, %bb1 ]
   store i8 2, i8* %ptr.2, align 8
   ret void
 }
 
+define void @test5_no_null_opt(i1 %cond, i8* %ptr) #0 {
+
+; CHECK-LABEL: test5_no_null_opt
+; CHECK: entry:
+; CHECK: %[[SEL:.*]] = select i1 %cond, i8* null, i8* %ptr
+; CHECK: store i8 2, i8* %[[SEL]]
+
+entry:
+  br i1 %cond, label %bb1, label %bb3
+
+bb3:
+ br label %bb2
+
+bb1:
+ br label %bb2
+
+bb2:
+  %ptr.2 = phi i8* [ %ptr, %bb3 ], [ null, %bb1 ]
+  store i8 2, i8* %ptr.2, align 8
+  ret void
+}
+
 ; CHECK-LABEL: test6
 ; CHECK: entry:
 ; CHECK-NOT: select
 ; CHECK:  store i8 2, i8* %ptr
 ; CHECK:  ret
 
 define void @test6(i1 %cond, i8* %ptr) {
 entry:
   br i1 %cond, label %bb1, label %bb2
 
 bb1:
   br label %bb2
 
 bb2:
   %ptr.2 = phi i8* [ %ptr, %entry ], [ null, %bb1 ]
   store i8 2, i8* %ptr.2, align 8
   ret void
 }
 
+; CHECK-LABEL: test6_no_null_opt
+; CHECK: entry:
+; CHECK: %[[SEL:.*]] = select i1 %cond, i8* null, i8* %ptr
+; CHECK: store i8 2, i8* %[[SEL]]
+
+define void @test6_no_null_opt(i1 %cond, i8* %ptr) #0 {
+entry:
+  br i1 %cond, label %bb1, label %bb2
+
+bb1:
+  br label %bb2
+
+bb2:
+  %ptr.2 = phi i8* [ %ptr, %entry ], [ null, %bb1 ]
+  store i8 2, i8* %ptr.2, align 8
+  ret void
+}
+
+
 define i32 @test7(i1 %X) {
 entry:
   br i1 %X, label %if, label %else
 
 if:
   call void undef()
   br label %else
 

 
 else:
   %phi = phi void ()* [ %Y, %entry ], [ null, %if ]
   call void %phi()
   ret void
 }
 ; CHECK-LABEL: define void @test8(
 ; CHECK: call void %Y(
+
+define void @test8_no_null_opt(i1 %X, void ()* %Y) #0 {
+entry:
+  br i1 %X, label %if, label %else
+
+if:
+  br label %else
+
+else:
+  %phi = phi void ()* [ %Y, %entry ], [ null, %if ]
+  call void %phi()
+  ret void
+}
+attributes #0 = { "null-pointer-is-valid"="true" }
+
+; CHECK-LABEL: define void @test8_no_null_opt(
+; CHECK: %[[SEL:.*]] = select i1 %X, void ()* null, void ()* %Y
+; CHECK: call void %[[SEL]]

test/Transforms/SimplifyCFG/invoke.ll

 lpad:
   %0 = landingpad { i8*, i32 }
           filter [0 x i8*] zeroinitializer
   %1 = extractvalue { i8*, i32 } %0, 0
   tail call void @__cxa_call_unexpected(i8* %1) noreturn nounwind
   unreachable
 }
 
+; CHECK-LABEL: @f2_no_null_opt(
+define i8* @f2_no_null_opt() nounwind uwtable ssp #0 personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
+entry:
+; CHECK: invoke noalias i8* null()
+  %call = invoke noalias i8* null()
+          to label %invoke.cont unwind label %lpad
+
+; CHECK: invoke.cont:
+; CHECK: ret i8* %call
+invoke.cont:
+  ret i8* %call
+
+lpad:
+  %0 = landingpad { i8*, i32 }
+          filter [0 x i8*] zeroinitializer
+  %1 = extractvalue { i8*, i32 } %0, 0
+  tail call void @__cxa_call_unexpected(i8* %1) noreturn nounwind
+; CHECK: unreachable
+  unreachable
+}
+
 ; CHECK-LABEL: @f3(
 define i32 @f3() nounwind uwtable ssp personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
 ; CHECK-NEXT: entry
 entry:
 ; CHECK-NEXT: ret i32 3
   %call = invoke i32 @read_only()
           to label %invoke.cont unwind label %lpad
 

 
 lpad:
 ; CHECK-NOT: phi
   %tmp = phi i8* [ null, %invoke.cont1 ], [ null, %entry ]
   landingpad { i8*, i32 }
           cleanup
   ret void
 }
+
+attributes #0 = { "null-pointer-is-valid"="true" }

test/Transforms/SimplifyCFG/phi-undef-loadstore.ll

 
 ; CHECK-LABEL: @test1(
 ; CHECK: if.else:
 ; CHECK: br label %if.end7
 
 ; CHECK: phi i32* [ %a, %if.then ], [ %c, %if.else ]
 }
 
+define i32 @test1_no_null_opt(i32* %a, i32 %b, i32* %c, i32 %d) nounwind #0 {
+entry:
+  %tobool = icmp eq i32 %b, 0
+  br i1 %tobool, label %if.else, label %if.then
+
+if.then:                                          ; preds = %entry
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.else:                                          ; preds = %entry
+  %tobool3 = icmp eq i32 %d, 0
+  br i1 %tobool3, label %if.end7, label %if.then4
+
+if.then4:                                         ; preds = %if.else
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.end7:                                          ; preds = %if.else, %if.then4, %if.then
+  %x.0 = phi i32* [ %a, %if.then ], [ %c, %if.then4 ], [ null, %if.else ]
+  %tmp9 = load i32, i32* %x.0
+  ret i32 %tmp9
+
+; CHECK-LABEL: @test1_no_null_opt(
+; CHECK: if.then:
+; CHECK: if.else:
+; CHECK: if.then4:
+; CHECK: br label %if.end7
+; CHECK: if.end7:
+; CHECK-NEXT: phi i32* [ %a, %if.then ], [ %c, %if.then4 ], [ null, %if.else ]
+}
+
 define i32 @test2(i32* %a, i32 %b, i32* %c, i32 %d) nounwind {
 entry:
   %tobool = icmp eq i32 %b, 0
   br i1 %tobool, label %if.else, label %if.then
 
 if.then:                                          ; preds = %entry
   tail call void @bar() nounwind
   br label %if.end7
if.end7:                                          ; preds = %if.else, %if.then4, %if.then
   ret i32 %tmp9
 ; CHECK-LABEL: @test2(
 ; CHECK: if.else:
 ; CHECK: unreachable
 
 ; CHECK-NOT: phi
 }
 
+define i32 @test2_no_null_opt(i32* %a, i32 %b, i32* %c, i32 %d) nounwind #0 {
+entry:
+  %tobool = icmp eq i32 %b, 0
+  br i1 %tobool, label %if.else, label %if.then
+
+if.then:                                          ; preds = %entry
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.else:                                          ; preds = %entry
+  %tobool3 = icmp eq i32 %d, 0
+  br i1 %tobool3, label %if.end7, label %if.then4
+
+if.then4:                                         ; preds = %if.else
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.end7:                                          ; preds = %if.else, %if.then4, %if.then
+  %x.0 = phi i32* [ %a, %if.then ], [ null, %if.then4 ], [ null, %if.else ]
+  %tmp9 = load i32, i32* %x.0
+  ret i32 %tmp9
+; CHECK-LABEL: @test2_no_null_opt(
+; CHECK: if.then:
+; CHECK: if.else:
+; CHECK: if.then4:
+; CHECK: if.end7:
+; CHECK-NEXT: phi i32* [ %a, %if.then ], [ null, %if.then4 ], [ null, %if.else ]
+}
+
 define i32 @test3(i32* %a, i32 %b, i32* %c, i32 %d) nounwind {
 entry:
   %tobool = icmp eq i32 %b, 0
   br i1 %tobool, label %if.else, label %if.then
 
 if.then:                                          ; preds = %entry
   tail call void @bar() nounwind
   br label %if.end7
if.end7:                                          ; preds = %if.else, %if.then4, %if.then
   tail call void @bar() nounwind
   %tmp9 = load i32, i32* %x.0
   ret i32 %tmp9
 ; CHECK-LABEL: @test3(
 ; CHECK: if.end7:
 ; CHECK: phi i32* [ %a, %if.then ], [ null, %if.then4 ], [ null, %if.else ]
 }
 
+define i32 @test3_no_null_opt(i32* %a, i32 %b, i32* %c, i32 %d) nounwind #0 {
+entry:
+  %tobool = icmp eq i32 %b, 0
+  br i1 %tobool, label %if.else, label %if.then
+
+if.then:                                          ; preds = %entry
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.else:                                          ; preds = %entry
+  %tobool3 = icmp eq i32 %d, 0
+  br i1 %tobool3, label %if.end7, label %if.then4
+
+if.then4:                                         ; preds = %if.else
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.end7:                                          ; preds = %if.else, %if.then4, %if.then
+  %x.0 = phi i32* [ %a, %if.then ], [ null, %if.then4 ], [ null, %if.else ]
+  tail call void @bar() nounwind
+  %tmp9 = load i32, i32* %x.0
+  ret i32 %tmp9
+; CHECK-LABEL: @test3_no_null_opt(
+; CHECK: if.then:
+; CHECK: if.else:
+; CHECK: if.then4:
+; CHECK: if.end7:
+; CHECK-NEXT: phi i32* [ %a, %if.then ], [ null, %if.then4 ], [ null, %if.else ]
+}
+
 define i32 @test4(i32* %a, i32 %b, i32* %c, i32 %d) nounwind {
 entry:
   %tobool = icmp eq i32 %b, 0
   br i1 %tobool, label %if.else, label %if.then
 
 if.then:                                          ; preds = %entry
   tail call void @bar() nounwind
   br label %if.end7
if.end7:                                          ; preds = %if.else, %if.then4, %if.then
   %gep = getelementptr i32, i32* %x.0, i32 10
   %tmp9 = load i32, i32* %gep
   %tmp10 = or i32 %tmp9, 1
   store i32 %tmp10, i32* %gep
   ret i32 %tmp9
 ; CHECK-LABEL: @test4(
 ; CHECK-NOT: phi
 }
+
+define i32 @test4_no_null_opt(i32* %a, i32 %b, i32* %c, i32 %d) nounwind #0 {
+entry:
+  %tobool = icmp eq i32 %b, 0
+  br i1 %tobool, label %if.else, label %if.then
+
+if.then:                                          ; preds = %entry
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.else:                                          ; preds = %entry
+  %tobool3 = icmp eq i32 %d, 0
+  br i1 %tobool3, label %if.end7, label %if.then4
+
+if.then4:                                         ; preds = %if.else
+  tail call void @bar() nounwind
+  br label %if.end7
+
+if.end7:                                          ; preds = %if.else, %if.then4, %if.then
+  %x.0 = phi i32* [ %a, %if.then ], [ null, %if.then4 ], [ null, %if.else ]
+  %gep = getelementptr i32, i32* %x.0, i32 10
+  %tmp9 = load i32, i32* %gep
+  %tmp10 = or i32 %tmp9, 1
+  store i32 %tmp10, i32* %gep
+  ret i32 %tmp9
+; CHECK-LABEL: @test4_no_null_opt(
+; CHECK: if.then:
+; CHECK: if.else:
+; CHECK: if.then4:
+; CHECK: if.end7:
+; CHECK-NEXT: phi i32* [ %a, %if.then ], [ null, %if.then4 ], [ null, %if.else ]
+}
+
+attributes #0 = { "null-pointer-is-valid"="true" }