[CodeGen] Add a pass to fold null checks into nearby memory operations.

[sanjoy]

Summary

This change adds an "ImplicitNullChecks" target dependent pass. This
pass folds null checks into memory operation using the FAULTING_LOAD
pseudo-op introduced in previous patches.

Depends on D10197
Depends on D10199
Depends on D10200

[JosephTremoulet]

lib/CodeGen/ImplicitNullChecks.cpp

127–130 ↗

(On Diff #27010)

Just curious: do you still intend to eventually have this check branch weights and only do the transformation when the null path is very cold? And/or to add some other heuristics here? Or is the plan now to be this aggressive here and rely entirely on patching to handle the cases where the null path is not very cold?

[sanjoy]

Just curious: do you still intend to eventually have this check branch weights and only do the transformation when the null path is very cold? And/or to add some other heuristics here? Or is the plan now to be this aggressive here and rely entirely on patching to handle the cases where the null path is not very cold?

I plan to do the former. I kept that logic out of this patch to keep
the initial review simple.

• Sanjoy

http://reviews.llvm.org/D10201

EMAIL PREFERENCES

http://reviews.llvm.org/settings/panel/emailpreferences/

[sanjoy]

ping

[ab]

include/llvm/CodeGen/Passes.h

553–554 ↗

(On Diff #27010)

triple slash?

lib/CodeGen/ImplicitNullChecks.cpp

46 ↗

(On Diff #27010)

"t -> "T

52 ↗

(On Diff #27010)

Why not class? For that matter, why not private for the non-override members?

72 ↗

(On Diff #27010)

why not:

NullCheck() : MemOperation(), ...

?

Also, CheckOperation is missing.

115 ↗

(On Diff #27010)

How about the more common name, "Changed"? Also, this should be updated somewhere. Instead, perhaps simply return !WorkList.empty(), or fold return true into the rewriteNullChecks block?

127–129 ↗

(On Diff #27010)

Why not triple-slash? (for the other methods as well)

139 ↗

(On Diff #27010)

This is what I feared with the AnalyzeBranchPredicate patch: I'm not comfortable with mentioning CmpInst here. A couple equally-icky alternatives:

• 'using CmpInst::Predicate' in MachineBranchPredicate
• duplicating it for the cases we're interested in, just NE/EQ ?

217–219 ↗

(On Diff #27010)

Why not chaining BuildMI().addSym().addImm?

222 ↗

(On Diff #27010)

Except for the first operand, does this ever happen?
If not, can you change the for() to ignore it instead, and turn this into an assert on LoadMI->getOperand(0) ?

253 ↗

(On Diff #27010)

I don't like this SmallVector<>() construct, but I don't have a better idea ;)

265 ↗

(On Diff #27010)

Is this actually used?

lib/CodeGen/Passes.cpp

85–88 ↗

(On Diff #27010)

Can you move this elsewhere, say before PrintLSR? I feel like it doesn't belong here.

[sanjoy]

• address review
• make the pass work with updates in dependencies

[sanjoy]

ping

include/llvm/CodeGen/Passes.h

553–554 ↗

(On Diff #27010)

Fixed.

lib/CodeGen/ImplicitNullChecks.cpp

46 ↗

(On Diff #27010)

Fixed.

52 ↗

(On Diff #27010)

Fixed.

72 ↗

(On Diff #27010)

Fixed.

115 ↗

(On Diff #27010)

Fixed.

127–129 ↗

(On Diff #27010)

Fixed.

139 ↗

(On Diff #27010)

The second option seems reasonable, I'll go with that.

217–219 ↗

(On Diff #27010)

Fixed.

222 ↗

(On Diff #27010)

The first two operands are defs for ADD32rm like instructions.

265 ↗

(On Diff #27010)

Removed.

lib/CodeGen/Passes.cpp

85–88 ↗

(On Diff #27010)

Makes sense, this isn't a diagnostic option.

[ab]

lib/CodeGen/ImplicitNullChecks.cpp

100–102 ↗

(On Diff #27426)

Is this necessary?

134 ↗

(On Diff #27426)

How about 'using TargetInstrInfo::MachineBranchPredicate;'?

192–195 ↗

(On Diff #27426)

Would (MayLoad && !Predicable) be (conservatively) correct?

213 ↗

(On Diff #27426)

I'm not sure an assert is appropriate: I don't have an example but this sounds easy to trigger. Should this be checked before even adding the NullCheck to the worklist?

255 ↗

(On Diff #27426)

With r239553, you should be able to replace this with the IMO much more readable:

/*Cond=*/None, DL);

[sanjoy]

• address review

lib/CodeGen/ImplicitNullChecks.cpp

100–102 ↗

(On Diff #27426)

Nope, thanks for catching!

134 ↗

(On Diff #27426)

That does not seem to compile.

192–195 ↗

(On Diff #27426)

Should be, unless there are other ways MayLoad instructions can avoid actually doing the load.

213 ↗

(On Diff #27426)

Kept the assert for documentation, but added a check before adding to worklist.

255 ↗

(On Diff #27426)

Done.

[JosephTremoulet]

LGTM

[atrick]

Awesome. Nice job splitting the patches.

A few comments inline..

lib/CodeGen/ImplicitNullChecks.cpp

45–48 ↗

(On Diff #27547)

Huh... There's no platform API for this? It's probably fine since this pass will only be used by people who know what they're doing. I'm just surprised.

201–202 ↗

(On Diff #27547)

Do we need to introduce a new term "WrappedLoad"? It initially confused me a bit. Why not just stick with FaultingLoad?

217–218 ↗

(On Diff #27547)

MachineDesc.getNumDefs() does not include any implicit defs that may have been tacked onto the end of the load. However, I think MO.isDef() will still return true for those loads so you'll end up dropping them. Rather than checking for Defs you could just iterate over MachineInst.uses(). Then you'll naturally skip the defs that you want to skip. Yes, it's crazy that MachineOperand.uses() also includes implicit defs.

227–228 ↗

(On Diff #27547)

I think the name WorkList is misleading, because once you have analyzed all blocks, the list never grows. It's just a plain old list of null check candidates. To make it clear that the list is immutable, rewriteNullChecks can take ArrayRef<NullCheck>.

[sanjoy]

• address review

lib/CodeGen/ImplicitNullChecks.cpp

45–48 ↗

(On Diff #27547)

There is Process::getPageSize but we need the page size for the target, not the host. For a JIT they'll be the same, but I'd rather not bake that assumption in.

201–202 ↗

(On Diff #27547)

Fixed.

217–218 ↗

(On Diff #27547)

Fixed.

227–228 ↗

(On Diff #27547)

Fixed.

[atrick]

LGTM

llvm/trunk/include/llvm/CodeGen/Passes.h

/// MachineDominanaceFrontier - This pass is a machine dominators analysis pass.
 
   /// Creates a pass to print GC metadata.
   ///
   FunctionPass *createGCInfoPrinter(raw_ostream &OS);
 
   /// MachineCSE - This pass performs global CSE on machine instructions.
   extern char &MachineCSEID;
 
+  /// ImplicitNullChecks - This pass folds null pointer checks into nearby
+  /// memory operations.
+  extern char &ImplicitNullChecksID;
+
   /// MachineLICM - This pass performs LICM on machine instructions.
   extern char &MachineLICMID;
 
   /// MachineSinking - This pass performs sinking on machine instructions.
   extern char &MachineSinkingID;
 
   /// MachineCopyPropagation - This pass performs copy propagation on
   /// machine instructions.

llvm/trunk/include/llvm/InitializePasses.h

 void initializeLowerIntrinsicsPass(PassRegistry&);
 void initializeLowerInvokePass(PassRegistry&);
 void initializeLowerSwitchPass(PassRegistry&);
 void initializeMachineBlockFrequencyInfoPass(PassRegistry&);
 void initializeMachineBlockPlacementPass(PassRegistry&);
 void initializeMachineBlockPlacementStatsPass(PassRegistry&);
 void initializeMachineBranchProbabilityInfoPass(PassRegistry&);
 void initializeMachineCSEPass(PassRegistry&);
+void initializeImplicitNullChecksPass(PassRegistry&);
 void initializeMachineDominatorTreePass(PassRegistry&);
 void initializeMachineDominanceFrontierPass(PassRegistry&);
 void initializeMachinePostDominatorTreePass(PassRegistry&);
 void initializeMachineLICMPass(PassRegistry&);
 void initializeMachineLoopInfoPass(PassRegistry&);
 void initializeMachineModuleInfoPass(PassRegistry&);
 void initializeMachineRegionInfoPassPass(PassRegistry&);
 void initializeMachineSchedulerPass(PassRegistry&);

llvm/trunk/lib/CodeGen/CMakeLists.txt

add_llvm_library(LLVMCodeGen
   ExpandPostRAPseudos.cpp
   FaultMaps.cpp
   GCMetadata.cpp
   GCMetadataPrinter.cpp
   GCRootLowering.cpp
   GCStrategy.cpp
   GlobalMerge.cpp
   IfConversion.cpp
+  ImplicitNullChecks.cpp
   InlineSpiller.cpp
   InterferenceCache.cpp
   IntrinsicLowering.cpp
   LLVMTargetMachine.cpp
   LatencyPriorityQueue.cpp
   LexicalScopes.cpp
   LiveDebugVariables.cpp
   LiveInterval.cpp

llvm/trunk/lib/CodeGen/CodeGen.cpp

void llvm::initializeCodeGen(PassRegistry &Registry) {
   initializeLiveStacksPass(Registry);
   initializeLiveVariablesPass(Registry);
   initializeLocalStackSlotPassPass(Registry);
   initializeLowerIntrinsicsPass(Registry);
   initializeMachineBlockFrequencyInfoPass(Registry);
   initializeMachineBlockPlacementPass(Registry);
   initializeMachineBlockPlacementStatsPass(Registry);
   initializeMachineCSEPass(Registry);
+  initializeImplicitNullChecksPass(Registry);
   initializeMachineCombinerPass(Registry);
   initializeMachineCopyPropagationPass(Registry);
   initializeMachineDominatorTreePass(Registry);
   initializeMachineFunctionPrinterPassPass(Registry);
   initializeMachineLICMPass(Registry);
   initializeMachineLoopInfoPass(Registry);
   initializeMachineModuleInfoPass(Registry);
   initializeMachinePostDominatorTreePass(Registry);

llvm/trunk/lib/CodeGen/ImplicitNullChecks.cpp

+//===-- ImplicitNullChecks.cpp - Fold null checks into memory accesses ----===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass turns explicit null checks of the form
+//
+//   test %r10, %r10
+//   je throw_npe
+//   movl (%r10), %esi
+//   ...
+//
+// to
+//
+//   faulting_load_op("movl (%r10), %esi", throw_npe)
+//   ...
+//
+// With the help of a runtime that understands the .fault_maps section,
+// faulting_load_op branches to throw_npe if executing movl (%r10), %esi incurs
+// a page fault.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/CodeGen/MachineFunction.h"
+#include "llvm/CodeGen/MachineOperand.h"
+#include "llvm/CodeGen/MachineFunctionPass.h"
+#include "llvm/CodeGen/MachineInstrBuilder.h"
+#include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/CodeGen/MachineModuleInfo.h"
+#include "llvm/IR/BasicBlock.h"
+#include "llvm/IR/Instruction.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Target/TargetSubtargetInfo.h"
+#include "llvm/Target/TargetInstrInfo.h"
+
+using namespace llvm;
+
+static cl::opt<unsigned> PageSize("imp-null-check-page-size",
+                                  cl::desc("The page size of the target in "
+                                           "bytes"),
+                                  cl::init(4096));
+
+namespace {
+
+class ImplicitNullChecks : public MachineFunctionPass {
+  /// Represents one null check that can be made implicit.
+  struct NullCheck {
+    // The memory operation the null check can be folded into.
+    MachineInstr *MemOperation;
+
+    // The instruction actually doing the null check (Ptr != 0).
+    MachineInstr *CheckOperation;
+
+    // The block the check resides in.
+    MachineBasicBlock *CheckBlock;
+
+    // The block branched to if the the pointer is non-null.
+    MachineBasicBlock *NotNullSucc;
+
+    // The block branched to if the the pointer is null.
+    MachineBasicBlock *NullSucc;
+
+    NullCheck()
+        : MemOperation(), CheckOperation(), CheckBlock(), NotNullSucc(),
+          NullSucc() {}
+
+    explicit NullCheck(MachineInstr *memOperation, MachineInstr *checkOperation,
+                       MachineBasicBlock *checkBlock,
+                       MachineBasicBlock *notNullSucc,
+                       MachineBasicBlock *nullSucc)
+        : MemOperation(memOperation), CheckOperation(checkOperation),
+          CheckBlock(checkBlock), NotNullSucc(notNullSucc), NullSucc(nullSucc) {
+    }
+  };
+
+  const TargetInstrInfo *TII = nullptr;
+  const TargetRegisterInfo *TRI = nullptr;
+  MachineModuleInfo *MMI = nullptr;
+
+  bool analyzeBlockForNullChecks(MachineBasicBlock &MBB,
+                                 SmallVectorImpl<NullCheck> &NullCheckList);
+  MachineInstr *insertFaultingLoad(MachineInstr *LoadMI, MachineBasicBlock *MBB,
+                                   MCSymbol *HandlerLabel);
+  void rewriteNullChecks(ArrayRef<NullCheck> NullCheckList);
+
+public:
+  static char ID;
+
+  ImplicitNullChecks() : MachineFunctionPass(ID) {
+    initializeImplicitNullChecksPass(*PassRegistry::getPassRegistry());
+  }
+
+  bool runOnMachineFunction(MachineFunction &MF) override;
+};
+}
+
+bool ImplicitNullChecks::runOnMachineFunction(MachineFunction &MF) {
+  TII = MF.getSubtarget().getInstrInfo();
+  TRI = MF.getRegInfo().getTargetRegisterInfo();
+  MMI = &MF.getMMI();
+
+  SmallVector<NullCheck, 16> NullCheckList;
+
+  for (auto &MBB : MF)
+    analyzeBlockForNullChecks(MBB, NullCheckList);
+
+  if (!NullCheckList.empty())
+    rewriteNullChecks(NullCheckList);
+
+  return !NullCheckList.empty();
+}
+
+/// Analyze MBB to check if its terminating branch can be turned into an
+/// implicit null check.  If yes, append a description of the said null check to
+/// NullCheckList and return true, else return false.
+bool ImplicitNullChecks::analyzeBlockForNullChecks(
+    MachineBasicBlock &MBB, SmallVectorImpl<NullCheck> &NullCheckList) {
+  typedef TargetInstrInfo::MachineBranchPredicate MachineBranchPredicate;
+
+  MachineBranchPredicate MBP;
+
+  if (TII->AnalyzeBranchPredicate(MBB, MBP, true))
+    return false;
+
+  // Is the predicate comparing an integer to zero?
+  if (!(MBP.LHS.isReg() && MBP.RHS.isImm() && MBP.RHS.getImm() == 0 &&
+        (MBP.Predicate == MachineBranchPredicate::PRED_NE ||
+         MBP.Predicate == MachineBranchPredicate::PRED_EQ)))
+    return false;
+
+  // If we cannot erase the test instruction itself, then making the null check
+  // implicit does not buy us much.
+  if (!MBP.SingleUseCondition)
+    return false;
+
+  MachineBasicBlock *NotNullSucc, *NullSucc;
+
+  if (MBP.Predicate == MachineBranchPredicate::PRED_NE) {
+    NotNullSucc = MBP.TrueDest;
+    NullSucc = MBP.FalseDest;
+  } else {
+    NotNullSucc = MBP.FalseDest;
+    NullSucc = MBP.TrueDest;
+  }
+
+  // We handle the simplest case for now.  We can potentially do better by using
+  // the machine dominator tree.
+  if (NotNullSucc->pred_size() != 1)
+    return false;
+
+  // Starting with a code fragment like:
+  //
+  //   test %RAX, %RAX
+  //   jne LblNotNull
+  //
+  //  LblNull:
+  //   callq throw_NullPointerException
+  //
+  //  LblNotNull:
+  //   Def = Load (%RAX + <offset>)
+  //   ...
+  //
+  //
+  // we want to end up with
+  //
+  //   Def = TrappingLoad (%RAX + <offset>), LblNull
+  //   jmp LblNotNull ;; explicit or fallthrough
+  //
+  //  LblNotNull:
+  //   ...
+  //
+  //  LblNull:
+  //   callq throw_NullPointerException
+  //
+
+  unsigned PointerReg = MBP.LHS.getReg();
+  MachineInstr *MemOp = &*NotNullSucc->begin();
+  unsigned BaseReg, Offset;
+  if (TII->getMemOpBaseRegImmOfs(MemOp, BaseReg, Offset, TRI))
+    if (MemOp->mayLoad() && !MemOp->isPredicable() && BaseReg == PointerReg &&
+        Offset < PageSize && MemOp->getDesc().getNumDefs() == 1) {
+      NullCheckList.emplace_back(MemOp, MBP.ConditionDef, &MBB, NotNullSucc,
+                                 NullSucc);
+      return true;
+    }
+
+  return false;
+}
+
+/// Wrap a machine load instruction, LoadMI, into a FAULTING_LOAD_OP machine
+/// instruction.  The FAULTING_LOAD_OP instruction does the same load as LoadMI
+/// (defining the same register), and branches to HandlerLabel if the load
+/// faults.  The FAULTING_LOAD_OP instruction is inserted at the end of MBB.
+MachineInstr *ImplicitNullChecks::insertFaultingLoad(MachineInstr *LoadMI,
+                                                     MachineBasicBlock *MBB,
+                                                     MCSymbol *HandlerLabel) {
+  DebugLoc DL;
+  unsigned NumDefs = LoadMI->getDesc().getNumDefs();
+  assert(NumDefs == 1 && "other cases unhandled!");
+  (void)NumDefs;
+
+  unsigned DefReg = LoadMI->defs().begin()->getReg();
+  assert(std::distance(LoadMI->defs().begin(), LoadMI->defs().end()) == 1 &&
+         "expected exactly one def!");
+
+  auto MIB = BuildMI(MBB, DL, TII->get(TargetOpcode::FAULTING_LOAD_OP), DefReg)
+                 .addSym(HandlerLabel)
+                 .addImm(LoadMI->getOpcode());
+
+  for (auto &MO : LoadMI->uses())
+    MIB.addOperand(MO);
+
+  MIB.setMemRefs(LoadMI->memoperands_begin(), LoadMI->memoperands_end());
+
+  return MIB;
+}
+
+/// Rewrite the null checks in NullCheckList into implicit null checks.
+void ImplicitNullChecks::rewriteNullChecks(
+    ArrayRef<ImplicitNullChecks::NullCheck> NullCheckList) {
+  DebugLoc DL;
+
+  for (auto &NC : NullCheckList) {
+    MCSymbol *HandlerLabel = MMI->getContext().createTempSymbol();
+
+    // Remove the conditional branch dependent on the null check.
+    unsigned BranchesRemoved = TII->RemoveBranch(*NC.CheckBlock);
+    (void)BranchesRemoved;
+    assert(BranchesRemoved > 0 && "expected at least one branch!");
+
+    // Insert a faulting load where the conditional branch was originally.  We
+    // check earlier ensures that this bit of code motion is legal.  We do not
+    // touch the successors list for any basic block since we haven't changed
+    // control flow, we've just made it implicit.
+    insertFaultingLoad(NC.MemOperation, NC.CheckBlock, HandlerLabel);
+    NC.MemOperation->removeFromParent();
+    NC.CheckOperation->eraseFromParent();
+
+    // Insert an *unconditional* branch to not-null successor.
+    TII->InsertBranch(*NC.CheckBlock, NC.NotNullSucc, nullptr, /*Cond=*/None,
+                      DL);
+
+    // Emit the HandlerLabel as an EH_LABEL.
+    BuildMI(*NC.NullSucc, NC.NullSucc->begin(), DL,
+            TII->get(TargetOpcode::EH_LABEL)).addSym(HandlerLabel);
+  }
+}
+
+char ImplicitNullChecks::ID = 0;
+char &llvm::ImplicitNullChecksID = ImplicitNullChecks::ID;
+INITIALIZE_PASS_BEGIN(ImplicitNullChecks, "implicit-null-checks",
+                      "Implicit null checks", false, false)
+INITIALIZE_PASS_END(ImplicitNullChecks, "implicit-null-checks",
+                    "Implicit null checks", false, false)

llvm/trunk/lib/CodeGen/Passes.cpp

 static cl::opt<bool> DisableConstantHoisting("disable-constant-hoisting",
     cl::Hidden, cl::desc("Disable ConstantHoisting"));
 static cl::opt<bool> DisableCGP("disable-cgp", cl::Hidden,
     cl::desc("Disable Codegen Prepare"));
 static cl::opt<bool> DisableCopyProp("disable-copyprop", cl::Hidden,
     cl::desc("Disable Copy Propagation pass"));
 static cl::opt<bool> DisablePartialLibcallInlining("disable-partial-libcall-inlining",
     cl::Hidden, cl::desc("Disable Partial Libcall Inlining"));
+static cl::opt<bool> EnableImplicitNullChecks(
+    "enable-implicit-null-checks",
+    cl::desc("Fold null checks into faulting memory operations"),
+    cl::init(false));
 static cl::opt<bool> PrintLSR("print-lsr-output", cl::Hidden,
     cl::desc("Print LLVM IR produced by the loop-reduce pass"));
 static cl::opt<bool> PrintISelInput("print-isel-input", cl::Hidden,
     cl::desc("Print LLVM IR input to isel pass"));
 static cl::opt<bool> PrintGCInfo("print-gc", cl::Hidden,
     cl::desc("Dump garbage collector data"));
 static cl::opt<bool> VerifyMachineCode("verify-machineinstrs", cl::Hidden,
     cl::desc("Verify generated machine code"),
if (getOptLevel() != CodeGenOpt::None)
     addMachineLateOptimization();
 
   // Expand pseudo instructions before second scheduling pass.
   addPass(&ExpandPostRAPseudosID);
 
   // Run pre-sched2 passes.
   addPreSched2();
 
+  if (EnableImplicitNullChecks)
+    addPass(&ImplicitNullChecksID);
+
   // Second pass scheduler.
   if (getOptLevel() != CodeGenOpt::None) {
     if (MISchedPostRA)
       addPass(&PostMachineSchedulerID);
     else
       addPass(&PostRASchedulerID);
   }
 

llvm/trunk/test/CodeGen/X86/implicit-null-check-negative.ll

+; RUN: llc -mtriple=x86_64-apple-macosx -O3 -debug-only=faultmaps -enable-implicit-null-checks < %s | FileCheck %s
+
+; List cases where we should *not* be emitting implicit null checks.
+
+; CHECK-NOT: Fault Map Output
+
+define i32 @imp_null_check_load(i32* %x, i32* %y) {
+ entry:
+  %c = icmp eq i32* %x, null
+; It isn't legal to move the load from %x from "not_null" to here --
+; the store to %y could be aliasing it.
+  br i1 %c, label %is_null, label %not_null
+
+ is_null:
+  ret i32 42
+
+ not_null:
+  store i32 0, i32* %y
+  %t = load i32, i32* %x
+  ret i32 %t
+}
+
+define i32 @imp_null_check_gep_load(i32* %x) {
+ entry:
+  %c = icmp eq i32* %x, null
+  br i1 %c, label %is_null, label %not_null
+
+ is_null:
+  ret i32 42
+
+ not_null:
+; null + 5000 * sizeof(i32) lies outside the null page and hence the
+; load to %t cannot be assumed to be reliably faulting.
+  %x.gep = getelementptr i32, i32* %x, i32 5000
+  %t = load i32, i32* %x.gep
+  ret i32 %t
+}
+
+define i32 @imp_null_check_load_no_md(i32* %x) {
+; Everything is okay except that the !never.executed metadata is
+; missing.
+ entry:
+  %c = icmp eq i32* %x, null
+  br i1 %c, label %is_null, label %not_null
+
+ is_null:
+  ret i32 42
+
+ not_null:
+  %t = load i32, i32* %x
+  ret i32 %t
+}

llvm/trunk/test/CodeGen/X86/implicit-null-check.ll

+; RUN: llc -O3 -mtriple=x86_64-apple-macosx -enable-implicit-null-checks < %s | FileCheck %s
+
+define i32 @imp_null_check_load(i32* %x) {
+; CHECK-LABEL: _imp_null_check_load:
+; CHECK: Ltmp1:
+; CHECK: movl (%rdi), %eax
+; CHECK: retq
+; CHECK: Ltmp0:
+; CHECK: movl $42, %eax
+; CHECK: retq
+
+ entry:
+  %c = icmp eq i32* %x, null
+  br i1 %c, label %is_null, label %not_null
+
+ is_null:
+  ret i32 42
+
+ not_null:
+  %t = load i32, i32* %x
+  ret i32 %t
+}
+
+define i32 @imp_null_check_gep_load(i32* %x) {
+; CHECK-LABEL: _imp_null_check_gep_load:
+; CHECK: Ltmp3:
+; CHECK: movl 128(%rdi), %eax
+; CHECK: retq
+; CHECK: Ltmp2:
+; CHECK: movl $42, %eax
+; CHECK: retq
+
+ entry:
+  %c = icmp eq i32* %x, null
+  br i1 %c, label %is_null, label %not_null
+
+ is_null:
+  ret i32 42
+
+ not_null:
+  %x.gep = getelementptr i32, i32* %x, i32 32
+  %t = load i32, i32* %x.gep
+  ret i32 %t
+}
+
+define i32 @imp_null_check_add_result(i32* %x, i32 %p) {
+; CHECK-LABEL: _imp_null_check_add_result:
+; CHECK: Ltmp5:
+; CHECK: addl (%rdi), %esi
+; CHECK: movl %esi, %eax
+; CHECK: retq
+; CHECK: Ltmp4:
+; CHECK: movl $42, %eax
+; CHECK: retq
+
+ entry:
+  %c = icmp eq i32* %x, null
+  br i1 %c, label %is_null, label %not_null
+
+ is_null:
+  ret i32 42
+
+ not_null:
+  %t = load i32, i32* %x
+  %p1 = add i32 %t, %p
+  ret i32 %p1
+}
+
+; CHECK-LABEL: __LLVM_FaultMaps:
+
+; Version:
+; CHECK-NEXT: .byte 1
+
+; Reserved x2
+; CHECK-NEXT: .byte 0
+; CHECK-NEXT: .short 0
+
+; # functions:
+; CHECK-NEXT: .long 3
+
+; FunctionAddr:
+; CHECK-NEXT: .quad _imp_null_check_add_result
+; NumFaultingPCs
+; CHECK-NEXT: .long 1
+; Reserved:
+; CHECK-NEXT: .long 0
+; Fault[0].Type:
+; CHECK-NEXT: .long 1
+; Fault[0].FaultOffset:
+; CHECK-NEXT: .long Ltmp5-_imp_null_check_add_result
+; Fault[0].HandlerOffset:
+; CHECK-NEXT: .long Ltmp4-_imp_null_check_add_result
+
+; FunctionAddr:
+; CHECK-NEXT: .quad _imp_null_check_gep_load
+; NumFaultingPCs
+; CHECK-NEXT: .long 1
+; Reserved:
+; CHECK-NEXT: .long 0
+; Fault[0].Type:
+; CHECK-NEXT: .long 1
+; Fault[0].FaultOffset:
+; CHECK-NEXT: .long Ltmp3-_imp_null_check_gep_load
+; Fault[0].HandlerOffset:
+; CHECK-NEXT: .long Ltmp2-_imp_null_check_gep_load
+
+; FunctionAddr:
+; CHECK-NEXT: .quad _imp_null_check_load
+; NumFaultingPCs
+; CHECK-NEXT: .long 1
+; Reserved:
+; CHECK-NEXT: .long 0
+; Fault[0].Type:
+; CHECK-NEXT: .long 1
+; Fault[0].FaultOffset:
+; CHECK-NEXT: .long Ltmp1-_imp_null_check_load
+; Fault[0].HandlerOffset:
+; CHECK-NEXT: .long Ltmp0-_imp_null_check_load