[SSP, 2/2] Create llvm.stackguard() intrinsic and lower it to LOAD_STACK_GUARD

[timshen]

Summary

With this change, ideally IR pass can always generate llvm.stackguard
call to get the stack guard; but for now there are still IR form stack
guard customizations around (see getIRStackGuard()). Future SSP
customization should go through LOAD_STACK_GUARD.

There is a behavior change: stack guard values are not CSEed anymore,
since we should never reuse the value in case that it has been spilled (and
corrupted). See ssp-guard-spill.ll. This also cause the change of stack size and codegen in X86 and AArch64 test cases.

The solution above isn't ideal. Ideally we'd like to know if the guard created in llvm.stackprotector() gets
spilled or not. If the value is spilled, discard the value and reload
stack guard; otherwise reuse the value. This can be done by teaching
register allocator to know how to rematerialize LOAD_STACK_GUARD and
force a rematerialization (which seems hard), or check for spilling in
expandPostRAPseudo. It only makes sense when the stack guard is a global
variable, which requires more instructions to load. Anyway, this seems to go out
of the scope of the current patch.

[echristo]

Can you add in the patch description explanations of the changes in stack size and code generation?

A few inline comments as well.

Thanks!

-eric

lib/CodeGen/StackProtector.cpp

282–284

This looks weird, can you explain it?

test/CodeGen/AArch64/stack-guard-remat-bitcast.ll

9

This looks like what I'd expect, but can you explain why the test changed?

test/CodeGen/X86/ssp-guard-spill.ll

3

Add comments of what you're testing here.

[timshen]

Updated comments.

[echristo]

I'm going to go ahead and approve this. It's definitely going down the path we want to go, fixes some fairly problematic issues in a nice way, provides an extension point, fixes a bit of a bug, and points out the way forward.

Thanks for all the work Tim!

-eric

[timshen]

Thanks Eric for review!

[timshen]

This is committed as r266806.

docs/LangRef.rst

 This intrinsic causes the prologue/epilogue inserter to force the position of
 the ``AllocaInst`` stack slot to be before local variables on the stack. This is
 to ensure that if a local variable on the stack is overwritten, it will destroy
 the value of the guard. When the function exits, the guard on the stack is
 checked against the original guard by ``llvm.stackprotectorcheck``. If they are
 different, then ``llvm.stackprotectorcheck`` causes the program to abort by
 calling the ``__stack_chk_fail()`` function.
 
+'``llvm.stackguard``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare i8* @llvm.stackguard()
+
+Overview:
+"""""""""
+
+The ``llvm.stackguard`` intrinsic returns the system stack guard value.
+
+It should not be generated by frontends, since it is only for internal usage.
+The reason why we create this intrinsic is that we still support IR form Stack
+Protector in FastISel.
+
+Arguments:
+""""""""""
+
+None.
+
+Semantics:
+""""""""""
+
+On some platforms, the value returned by this intrinsic remains unchanged
+between loads in the same thread. On other platforms, it returns the same
+global variable value, if any, e.g. ``@__stack_chk_guard``.
+
+Currently some platforms have IR-level customized stack guard loading (e.g.
+X86 Linux) that is not handled by ``llvm.stackguard()``, while they should be
+in the future.
+
 '``llvm.objectsize``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 
 ::
 

include/llvm/IR/Intrinsics.td

 
 // The assume intrinsic is marked as arbitrarily writing so that proper
 // control dependencies will be maintained.
 def int_assume        : Intrinsic<[], [llvm_i1_ty], []>;
 
 // Stack Protector Intrinsic - The stackprotector intrinsic writes the stack
 // guard to the correct place on the stack frame.
 def int_stackprotector : Intrinsic<[], [llvm_ptr_ty, llvm_ptrptr_ty], []>;
+def int_stackguard : Intrinsic<[llvm_ptr_ty], [], []>;
 
 // A counter increment for instrumentation based profiling.
 def int_instrprof_increment : Intrinsic<[],
                                         [llvm_ptr_ty, llvm_i64_ty,
                                          llvm_i32_ty, llvm_i32_ty],
                                         []>;
 
 // A call to profile runtime for value profiling of target expressions

include/llvm/Target/TargetLowering.h

public:
 
   /// Return the preferred loop alignment.
   virtual unsigned getPrefLoopAlignment(MachineLoop *ML = nullptr) const {
     return PrefLoopAlignment;
   }
 
   /// If the target has a standard location for the stack protector guard,
   /// returns the address of that location. Otherwise, returns nullptr.
+  /// DEPRECATED: please override useLoadStackGuardNode and customize
+  ///             LOAD_STACK_GUARD, or customize @llvm.stackguard().
   virtual Value *getIRStackGuard(IRBuilder<> &IRB) const;
 
   /// Inserts necessary declarations for SSP purpose. Should be used only when
   /// getIRStackGuard returns nullptr.
   virtual void insertSSPDeclarations(Module &M) const;
 
   /// Return the variable that's previously inserted by insertSSPDeclarations,
   /// if any, otherwise return nullptr. Should be used only when

lib/CodeGen/MachineCSE.cpp

if (MI->mayLoad()) {
     // to decide whether the loaded value is actually a constant. If so, we can
     // actually use it as a load.
     if (!MI->isInvariantLoad(AA))
       // FIXME: we should be able to hoist loads with no other side effects if
       // there are no other instructions which can change memory in this loop.
       // This is a trivial form of alias analysis.
       return false;
   }
+
+  // Ignore stack guard loads, otherwise the register that holds CSEed value may
+  // be spilled and get loaded back with corrupted data.
+  if (MI->getOpcode() == TargetOpcode::LOAD_STACK_GUARD)
+    return false;
+
   return true;
 }
 
 /// isProfitableToCSE - Return true if it's profitable to eliminate MI with a
 /// common expression that defines Reg.
 bool MachineCSE::isProfitableToCSE(unsigned CSReg, unsigned Reg,
                                    MachineInstr *CSMI, MachineInstr *MI) {
   // FIXME: Heuristics that works around the lack the live range splitting.

lib/CodeGen/SelectionDAG/SelectionDAGBuilder.h

private:
   ///        the parent basic block consisting of the two loads, the comparison,
   ///        and finally two branches to the success/failure basic blocks. We
   ///        conclude by code-gening the failure basic block if we have not
   ///        code-gened it already (all stack protector checks we generate in
   ///        the same function, use the same failure basic block).
   class StackProtectorDescriptor {
   public:
     StackProtectorDescriptor()
-        : ParentMBB(nullptr), SuccessMBB(nullptr), FailureMBB(nullptr),
-          GuardReg(0) {}
+        : ParentMBB(nullptr), SuccessMBB(nullptr), FailureMBB(nullptr) {}
 
     /// Returns true if all fields of the stack protector descriptor are
     /// initialized implying that we should/are ready to emit a stack protector.
     bool shouldEmitStackProtector() const {
       return ParentMBB && SuccessMBB && FailureMBB;
     }
 
     /// Initialize the stack protector descriptor structure for a new basic
public:
     ///
     /// 1. FailureMBB since we reuse the failure code path for all stack
     /// protector checks created in an individual function.
     ///
     /// 2.The guard variable since the guard variable we are checking against is
     /// always the same.
     void resetPerFunctionState() {
       FailureMBB = nullptr;
-      GuardReg = 0;
     }
 
     MachineBasicBlock *getParentMBB() { return ParentMBB; }
     MachineBasicBlock *getSuccessMBB() { return SuccessMBB; }
     MachineBasicBlock *getFailureMBB() { return FailureMBB; }
 
-    unsigned getGuardReg() const { return GuardReg; }
-    void setGuardReg(unsigned R) { GuardReg = R; }
-
   private:
     /// The basic block for which we are generating the stack protector.
     ///
     /// As a result of stack protector generation, we will splice the
     /// terminators of this basic block into the successor mbb SuccessMBB and
     /// replace it with a compare/branch to the successor mbbs
     /// SuccessMBB/FailureMBB depending on whether or not the stack protector
     /// was violated.
     MachineBasicBlock *ParentMBB;
 
     /// A basic block visited on stack protector check success that contains the
     /// terminators of ParentMBB.
     MachineBasicBlock *SuccessMBB;
 
     /// This basic block visited on stack protector check failure that will
     /// contain a call to __stack_chk_fail().
     MachineBasicBlock *FailureMBB;
 
-    /// The virtual register holding the stack guard value.
-    unsigned GuardReg;
-
     /// Add a successor machine basic block to ParentMBB. If the successor mbb
     /// has not been created yet (i.e. if SuccMBB = 0), then the machine basic
     /// block will be created. Assign a large weight if IsLikely is true.
     MachineBasicBlock *AddSuccessorMBB(const BasicBlock *BB,
                                        MachineBasicBlock *ParentMBB,
                                        bool IsLikely,
                                        MachineBasicBlock *SuccMBB = nullptr);
   };

lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

void SelectionDAGBuilder::visitJumpTableHeader(JumpTable &JT,
   // Avoid emitting unnecessary branches to the next block.
   if (JT.MBB != NextBlock(SwitchBB))
     BrCond = DAG.getNode(ISD::BR, dl, MVT::Other, BrCond,
                          DAG.getBasicBlock(JT.MBB));
 
   DAG.setRoot(BrCond);
 }
 
+/// Create a LOAD_STACK_GUARD node, and let it carry the target specific global
+/// variable if there exists one.
+static SDValue getLoadStackGuard(SelectionDAG &DAG, SDLoc DL, SDValue &Chain) {
+  const TargetLowering &TLI = DAG.getTargetLoweringInfo();
+  EVT PtrTy = TLI.getPointerTy(DAG.getDataLayout());
+  MachineFunction &MF = DAG.getMachineFunction();
+  Value *Global = TLI.getSDStackGuard(*MF.getFunction()->getParent());
+  MachineSDNode *Node =
+      DAG.getMachineNode(TargetOpcode::LOAD_STACK_GUARD, DL, PtrTy, Chain);
+  if (Global) {
+    MachinePointerInfo MPInfo(Global);
+    MachineInstr::mmo_iterator MemRefs = MF.allocateMemRefsArray(1);
+    unsigned Flags = MachineMemOperand::MOLoad | MachineMemOperand::MOInvariant;
+    *MemRefs = MF.getMachineMemOperand(MPInfo, Flags, PtrTy.getSizeInBits() / 8,
+                                       DAG.getEVTAlignment(PtrTy));
+    Node->setMemRefs(MemRefs, MemRefs + 1);
+  }
+  return SDValue(Node, 0);
+}
+
 /// Codegen a new tail for a stack protector check ParentMBB which has had its
 /// tail spliced into a stack protector check success bb.
 ///
 /// For a high level explanation of how this fits into the stack protector
 /// generation see the comment on the declaration of class
 /// StackProtectorDescriptor.
 void SelectionDAGBuilder::visitSPDescriptorParent(StackProtectorDescriptor &SPD,
                                                   MachineBasicBlock *ParentBB) {
void SelectionDAGBuilder::visitSPDescriptorParent(StackProtectorDescriptor &SPD,
   SDValue GuardPtr = getValue(IRGuard);
   SDValue StackSlotPtr = DAG.getFrameIndex(FI, PtrTy);
 
   unsigned Align = DL->getPrefTypeAlignment(IRGuard->getType());
 
   SDValue Guard;
   SDLoc dl = getCurSDLoc();
 
-  // If GuardReg is set and useLoadStackGuardNode returns true, retrieve the
-  // guard value from the virtual register holding the value. Otherwise, emit a
-  // volatile load to retrieve the stack guard value.
-  unsigned GuardReg = SPD.getGuardReg();
-
-  if (GuardReg && TLI.useLoadStackGuardNode())
-    Guard = DAG.getCopyFromReg(DAG.getEntryNode(), dl, GuardReg,
-                               PtrTy);
+  // If useLoadStackGuardNode returns true, generate LOAD_STACK_GUARD.
+  // Otherwise, emit a volatile load to retrieve the stack guard value.
+  SDValue Chain = DAG.getEntryNode();
+  if (TLI.useLoadStackGuardNode())
+    Guard = getLoadStackGuard(DAG, dl, Chain);
   else
-    Guard = DAG.getLoad(PtrTy, dl, DAG.getEntryNode(),
-                        GuardPtr, MachinePointerInfo(IRGuard, 0),
-                        true, false, false, Align);
+    Guard =
+        DAG.getLoad(PtrTy, dl, Chain, GuardPtr, MachinePointerInfo(IRGuard, 0),
 
   SDValue StackSlot = DAG.getLoad(
       PtrTy, dl, DAG.getEntryNode(), StackSlotPtr,
       MachinePointerInfo::getFixedStack(DAG.getMachineFunction(), FI), true,
       false, false, Align);
 
   // Perform the comparison via a subtract/getsetcc.
   EVT VT = Guard.getValueType();
if (PtrTy != ResTy)
       report_fatal_error("Wrong result type for @llvm.get.dynamic.area.offset"
                          " intrinsic!");
     Res = DAG.getNode(ISD::GET_DYNAMIC_AREA_OFFSET, sdl, DAG.getVTList(ResTy),
                       Op);
     DAG.setRoot(Op);
     setValue(&I, Res);
     return nullptr;
   }
+  case Intrinsic::stackguard: {
+    EVT PtrTy = TLI.getPointerTy(DAG.getDataLayout());
+    MachineFunction &MF = DAG.getMachineFunction();
+    const Module &M = *MF.getFunction()->getParent();
+    SDValue Chain = getRoot();
+    if (TLI.useLoadStackGuardNode()) {
+      Res = getLoadStackGuard(DAG, sdl, Chain);
+    } else {
+      const Value *Global = TLI.getSDStackGuard(M);
+      unsigned Align = DL->getPrefTypeAlignment(Global->getType());
+      Res =
+          DAG.getLoad(PtrTy, sdl, Chain, getValue(Global),
+                      MachinePointerInfo(Global, 0), true, false, false, Align);
+    }
+    DAG.setRoot(Chain);
+    setValue(&I, Res);
+    return nullptr;
+  }
   case Intrinsic::stackprotector: {
     // Emit code into the DAG to store the stack guard onto the stack.
     MachineFunction &MF = DAG.getMachineFunction();
     MachineFrameInfo *MFI = MF.getFrameInfo();
     EVT PtrTy = TLI.getPointerTy(DAG.getDataLayout());
     SDValue Src, Chain = getRoot();
-    const Value *Ptr = cast<LoadInst>(I.getArgOperand(0))->getPointerOperand();
-    const GlobalVariable *GV = dyn_cast<GlobalVariable>(Ptr);
 
-    // See if Ptr is a bitcast. If it is, look through it and see if we can get
-    // global variable __stack_chk_guard.
-    if (!GV)
-      if (const Operator *BC = dyn_cast<Operator>(Ptr))
-        if (BC->getOpcode() == Instruction::BitCast)
-          GV = dyn_cast<GlobalVariable>(BC->getOperand(0));
-
-    if (GV && TLI.useLoadStackGuardNode()) {
-      // Emit a LOAD_STACK_GUARD node.
-      MachineSDNode *Node = DAG.getMachineNode(TargetOpcode::LOAD_STACK_GUARD,
-                                               sdl, PtrTy, Chain);
-      MachinePointerInfo MPInfo(GV);
-      MachineInstr::mmo_iterator MemRefs = MF.allocateMemRefsArray(1);
-      unsigned Flags = MachineMemOperand::MOLoad |
-                       MachineMemOperand::MOInvariant;
-      *MemRefs = MF.getMachineMemOperand(MPInfo, Flags,
-                                         PtrTy.getSizeInBits() / 8,
-                                         DAG.getEVTAlignment(PtrTy));
-      Node->setMemRefs(MemRefs, MemRefs + 1);
-
-      // Copy the guard value to a virtual register so that it can be
-      // retrieved in the epilogue.
-      Src = SDValue(Node, 0);
-      const TargetRegisterClass *RC =
-          TLI.getRegClassFor(Src.getSimpleValueType());
-      unsigned Reg = MF.getRegInfo().createVirtualRegister(RC);
-
-      SPDescriptor.setGuardReg(Reg);
-      Chain = DAG.getCopyToReg(Chain, sdl, Reg, Src);
-    } else {
+    if (TLI.useLoadStackGuardNode())
+      Src = getLoadStackGuard(DAG, sdl, Chain);
+    else
       Src = getValue(I.getArgOperand(0));   // The guard's value.
-    }
 
     AllocaInst *Slot = cast<AllocaInst>(I.getArgOperand(1));
 
     int FI = FuncInfo.StaticAllocaMap[Slot];
     MFI->setStackProtectorIndex(FI);
 
     SDValue FIN = DAG.getFrameIndex(FI, PtrTy);
 

lib/CodeGen/StackProtector.cpp

for (const Instruction &I : BB) {
         }
       }
     }
   }
 
   return NeedsProtector;
 }
 
-/// Insert code into the entry block that stores the __stack_chk_guard
+/// Create a stack guard loading and populate whether SelectionDAG SSP is
+/// supported.
+static Value *getStackGuard(const TargetLoweringBase *TLI, Module *M,
+                            IRBuilder<> &B,
+                            bool *SupportsSelectionDAGSP = nullptr) {
+  if (Value *Guard = TLI->getIRStackGuard(B))
+    return B.CreateLoad(Guard, true, "StackGuard");
+
+  /// Use SelectionDAG SSP handling, since there isn't an IR guard.
+  if (SupportsSelectionDAGSP)
+    *SupportsSelectionDAGSP = true;
+  TLI->insertSSPDeclarations(*M);
+  return B.CreateCall(Intrinsic::getDeclaration(M, Intrinsic::stackguard));
+}
+
+/// Insert code into the entry block that stores the stack guard
 /// variable onto the stack:
 ///
 ///   entry:
 ///     StackGuardSlot = alloca i8*
-///     StackGuard = load __stack_chk_guard
-///     call void @llvm.stackprotect.create(StackGuard, StackGuardSlot)
+///     StackGuard = <stack guard>
+///     call void @llvm.stackprotector(StackGuard, StackGuardSlot)
 ///
 /// Returns true if the platform/triple supports the stackprotectorcreate pseudo
 /// node.
 static bool CreatePrologue(Function *F, Module *M, ReturnInst *RI,
-                           const TargetLoweringBase *TLI, AllocaInst *&AI,
-                           Value *&StackGuardVar) {
+                           const TargetLoweringBase *TLI, AllocaInst *&AI) {
   bool SupportsSelectionDAGSP = false;
   IRBuilder<> B(&F->getEntryBlock().front());
-
-  StackGuardVar = TLI->getIRStackGuard(B);
-  if (!StackGuardVar) {
-    /// Use SelectionDAG SSP handling, since there isn't an IR guard.
-    SupportsSelectionDAGSP = true;
-    TLI->insertSSPDeclarations(*M);
-    StackGuardVar = TLI->getSDStackGuard(*M);
-  }
-  assert(StackGuardVar && "Must have stack guard available");
-
   PointerType *PtrTy = Type::getInt8PtrTy(RI->getContext());
   AI = B.CreateAlloca(PtrTy, nullptr, "StackGuardSlot");
-  LoadInst *LI = B.CreateLoad(StackGuardVar, "StackGuard");
+
+  Value *Guard = getStackGuard(TLI, M, B, &SupportsSelectionDAGSP);
   B.CreateCall(Intrinsic::getDeclaration(M, Intrinsic::stackprotector),
-               {LI, AI});
+               {Guard, AI});
   return SupportsSelectionDAGSP;
 }
 
 /// InsertStackProtectors - Insert code into the prologue and epilogue of the
 /// function.
 ///
 ///  - The prologue code loads and stores the stack guard onto the stack.
 ///  - The epilogue checks the value stored in the prologue against the original
 ///    value. It calls __stack_chk_fail if they differ.
 bool StackProtector::InsertStackProtectors() {
   bool SupportsSelectionDAGSP =
       EnableSelectionDAGSP && !TM->Options.EnableFastISel;
   AllocaInst *AI = nullptr;       // Place on stack that stores the stack guard.
-  Value *StackGuardVar = nullptr; // The stack guard variable.
 
   for (Function::iterator I = F->begin(), E = F->end(); I != E;) {
     BasicBlock *BB = &*I++;
     ReturnInst *RI = dyn_cast<ReturnInst>(BB->getTerminator());
     if (!RI)
       continue;
 
     if (!HasPrologue) {
       HasPrologue = true;
-      SupportsSelectionDAGSP &=
-          CreatePrologue(F, M, RI, TLI, AI, StackGuardVar);
+      SupportsSelectionDAGSP &= CreatePrologue(F, M, RI, TLI, AI);
     }
 
     if (!SupportsSelectionDAGSP) {
       // If we do not support SelectionDAG based tail calls, generate IR level
       // tail calls.
       //
       // For each block with a return instruction, convert this:
       //
       //   return:
       //     ...
       //     ret ...
       //
       // into this:
       //
       //   return:
       //     ...
-      //     %1 = load __stack_chk_guard
+      //     %1 = <stack guard>
       //     %2 = load StackGuardSlot
       //     %3 = cmp i1 %1, %2
       //     br i1 %3, label %SP_return, label %CallStackCheckFailBlk
       //
       //   SP_return:
       //     ret ...
       //
       //   CallStackCheckFailBlk:
if (!SupportsSelectionDAGSP) {
       BB->getTerminator()->eraseFromParent();
 
       // Move the newly created basic block to the point right after the old
       // basic block so that it's in the "fall through" position.
       NewBB->moveAfter(BB);
 
       // Generate the stack protector instructions in the old basic block.
       IRBuilder<> B(BB);
-      LoadInst *LI1 = B.CreateLoad(StackGuardVar);
-      LoadInst *LI2 = B.CreateLoad(AI);
-      Value *Cmp = B.CreateICmpEQ(LI1, LI2);
+      Value *Guard = getStackGuard(TLI, M, B);
+      LoadInst *LI2 = B.CreateLoad(AI, true);
+      Value *Cmp = B.CreateICmpEQ(Guard, LI2);
       auto SuccessProb =
           BranchProbabilityInfo::getBranchProbStackProtector(true);
       auto FailureProb =
           BranchProbabilityInfo::getBranchProbStackProtector(false);
       MDNode *Weights = MDBuilder(F->getContext())
                             .createBranchWeights(SuccessProb.getNumerator(),
                                                  FailureProb.getNumerator());
       B.CreateCondBr(Cmp, NewBB, FailBB, Weights);

lib/Target/AArch64/AArch64ISelLowering.cpp

case ISD::FP_TO_SINT:
     return;
   case ISD::ATOMIC_CMP_SWAP:
     ReplaceCMP_SWAP_128Results(N, Results, DAG);
     return;
   }
 }
 
 bool AArch64TargetLowering::useLoadStackGuardNode() const {
+  if (!Subtarget->isTargetAndroid())
     return true;
+  return TargetLowering::useLoadStackGuardNode();
 }
 
 unsigned AArch64TargetLowering::combineRepeatedFPDivisors() const {
   // Combine multiple FDIVs with the same divisor into multiple FMULs by the
   // reciprocal if there are three or more FDIVs.
   return 3;
 }
 

lib/Target/X86/X86ISelLowering.cpp

Value *X86TargetLowering::getIRStackGuard(IRBuilder<> &IRB) const {
   return ConstantExpr::getIntToPtr(
       ConstantInt::get(Type::getInt32Ty(IRB.getContext()), Offset),
       Type::getInt8PtrTy(IRB.getContext())->getPointerTo(AddressSpace));
 }
 
 void X86TargetLowering::insertSSPDeclarations(Module &M) const {
   if (!Subtarget.isTargetLinux())
     TargetLowering::insertSSPDeclarations(M);
-  else
-    llvm_unreachable("X86 Linux supports customized IR stack guard load");
 }
 
 Value *X86TargetLowering::getSDStackGuard(const Module &M) const {
   if (!Subtarget.isTargetLinux())
     return TargetLowering::getSDStackGuard(M);
-  llvm_unreachable("X86 Linux supports customized IR stack guard load");
+  return nullptr;
 }
 
 Value *X86TargetLowering::getSafeStackPointerLocation(IRBuilder<> &IRB) const {
   if (!Subtarget.isTargetAndroid())
     return TargetLowering::getSafeStackPointerLocation(IRB);
 
   // Android provides a fixed TLS slot for the SafeStack pointer. See the
   // definition of TLS_SLOT_SAFESTACK in

test/CodeGen/AArch64/stack-guard-remat-bitcast.ll

 ; RUN: llc < %s -mtriple=arm64-apple-ios -relocation-model=pic -disable-fp-elim | FileCheck %s
 
 @__stack_chk_guard = external global i64*
 
 ; PR20558
 
 ; CHECK: adrp [[R0:x[0-9]+]], ___stack_chk_guard@GOTPAGE
 ; CHECK: ldr  [[R1:x[0-9]+]], {{\[}}[[R0]], ___stack_chk_guard@GOTPAGEOFF{{\]}}
+; CHECK: adrp [[GUARD_PAGE:x[0-9]+]], ___stack_chk_guard@GOTPAGE
 ; CHECK: ldr  [[R2:x[0-9]+]], {{\[}}[[R1]]{{\]}}
 ; CHECK: stur [[R2]], {{\[}}x29, [[SLOT0:[0-9#\-]+]]{{\]}}
 ; CHECK: ldur [[R3:x[0-9]+]], {{\[}}x29, [[SLOT0]]{{\]}}
-; CHECK: sub  [[R4:x[0-9]+]], [[R2]], [[R3]]
+; CHECK: ldr  [[GUARD_ADDR:x[0-9]+]], {{\[}}[[GUARD_PAGE]], ___stack_chk_guard@GOTPAGEOFF{{\]}}
+; CHECK: ldr  [[GUARD:x[0-9]+]], {{\[}}[[GUARD_ADDR]]{{\]}}
+; CHECK: sub  [[R4:x[0-9]+]], [[GUARD]], [[R3]]
 ; CHECK: cbnz [[R4]], LBB
 
 define i32 @test_stack_guard_remat2() {
 entry:
   %StackGuardSlot = alloca i8*
   %StackGuard = load i8*, i8** bitcast (i64** @__stack_chk_guard to i8**)
   call void @llvm.stackprotector(i8* %StackGuard, i8** %StackGuardSlot)
   %container = alloca [32 x i8], align 1
   call void @llvm.stackprotectorcheck(i8** bitcast (i64** @__stack_chk_guard to i8**))
   ret i32 -1
 }
 
 declare void @llvm.stackprotector(i8*, i8**)
 declare void @llvm.stackprotectorcheck(i8**)

test/CodeGen/ARM/warn-stack.ll

 ; RUN: llc -mtriple thumbv7-apple-ios3.0.0 -warn-stack-size=80 < %s 2>&1 >/dev/null | FileCheck %s
 ; Check the internal option that warns when the stack size exceeds the
 ; given amount.
 ; <rdar://13987214>
 
 ; CHECK-NOT: nowarn
 define void @nowarn() nounwind ssp {
 entry:
   %buffer = alloca [12 x i8], align 1
   %arraydecay = getelementptr inbounds [12 x i8], [12 x i8]* %buffer, i64 0, i64 0
   call void @doit(i8* %arraydecay) nounwind
   ret void
 }
 
-; CHECK: warning: stack size limit exceeded (96) in warn
+; CHECK: warning: stack size limit exceeded (92) in warn
 define void @warn() nounwind ssp {
 entry:
   %buffer = alloca [80 x i8], align 1
   %arraydecay = getelementptr inbounds [80 x i8], [80 x i8]* %buffer, i64 0, i64 0
   call void @doit(i8* %arraydecay) nounwind
   ret void
 }
 
 declare void @doit(i8*)

test/CodeGen/X86/2009-02-12-DebugInfoVLA.ll

 ; RUN: llc < %s
 ; RUN: llc < %s -stack-symbol-ordering=0 -march=x86-64 -verify-machineinstrs | FileCheck %s
 ; PR3538
 target datalayout = "e-p:32:32:32-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f32:32:32-f64:32:64-v64:64:64-v128:128:128-a0:0:64-f80:128:128"
 target triple = "i386-apple-darwin9"
 define signext i8 @foo(i8* %s1) nounwind ssp {
 
 ; Make sure we generate:
 ;  movq -40(%rbp), %rsp
 ; Instead of:
 ;  movq -40(%rbp), %rax
 ;  movq %rax, %rsp
 
 ; CHECK-LABEL: @foo
-; CHECK: movq -40(%rbp), %rsp
+; CHECK: movq -{{[0-9]+}}(%rbp), %rsp
 
 entry:
   %s1_addr = alloca i8*                           ; <i8**> [#uses=2]
   %retval = alloca i32                            ; <i32*> [#uses=2]
   %saved_stack.1 = alloca i8*                     ; <i8**> [#uses=2]
   %0 = alloca i32                                 ; <i32*> [#uses=2]
   %str.0 = alloca [0 x i8]*                       ; <[0 x i8]**> [#uses=3]
   %1 = alloca i64                                 ; <i64*> [#uses=2]

test/CodeGen/X86/StackColoring.ll

bb2:
   %t7 = add i32 %t5, %t6
   ret i32 %t7
 bb3:
   ret i32 0
 }
 
 ; Adopt the test from Transforms/Inline/array_merge.ll'
 ;CHECK-LABEL: array_merge:
-;YESCOLOR: subq  $816, %rsp
-;NOCOLOR: subq  $1616, %rsp
+;YESCOLOR: subq  $808, %rsp
+;NOCOLOR: subq  $1608, %rsp
 define void @array_merge() nounwind ssp {
 entry:
   %A.i1 = alloca [100 x i32], align 4
   %B.i2 = alloca [100 x i32], align 4
   %A.i = alloca [100 x i32], align 4
   %B.i = alloca [100 x i32], align 4
   %0 = bitcast [100 x i32]* %A.i to i8*
   call void @llvm.lifetime.start(i64 -1, i8* %0) nounwind

test/CodeGen/X86/dynamic-allocas-VLAs.ll

entry:
   call void @t3_helper(i32* %a, i32* %vla) nounwind
   %0 = load i32, i32* %a, align 4
   %add = add nsw i32 %0, 13
   ret i32 %add
 
 ; CHECK: _t3
 ; CHECK: pushq %rbp
 ; CHECK: movq %rsp, %rbp
-; CHECK: pushq %rbx
 ; CHECK-NOT: andq $-{{[0-9]+}}, %rsp
 ; CHECK: subq ${{[0-9]+}}, %rsp
 ;
-; CHECK: leaq -{{[0-9]+}}(%rbp), %rsp
-; CHECK: popq %rbx
+; CHECK: movq %rbp, %rsp
 ; CHECK: popq %rbp
 }
 
 declare void @t3_helper(i32*, i32*)
 
 ; VLAs + Dynamic realignment
 define i32 @t4(i64 %sz) nounwind uwtable ssp {
 entry:
   %a = alloca i32, align 4
   %v = alloca <8 x float>, align 32
   %vla = alloca i32, i64 %sz, align 16
   call void @t4_helper(i32* %a, i32* %vla, <8 x float>* %v) nounwind
   %0 = load i32, i32* %a, align 4
   %add = add nsw i32 %0, 13
   ret i32 %add
 
 ; CHECK: _t4
 ; CHECK: pushq %rbp
 ; CHECK: movq %rsp, %rbp
-; CHECK: pushq %r14
 ; CHECK: pushq %rbx
 ; CHECK: andq $-32, %rsp
 ; CHECK: subq ${{[0-9]+}}, %rsp
 ; CHECK: movq %rsp, %rbx
 ;
 ; CHECK: leaq {{[0-9]*}}(%rbx), %rdi
 ; CHECK: leaq {{[0-9]*}}(%rbx), %rdx
 ; CHECK: callq   _t4_helper
 ;
-; CHECK: leaq -16(%rbp), %rsp
+; CHECK: leaq -{{[0-9]+}}(%rbp), %rsp
 ; CHECK: popq %rbx
-; CHECK: popq %r14
 ; CHECK: popq %rbp
 }
 
 declare void @t4_helper(i32*, i32*, <8 x float>*)
 
 ; Spilling an AVX register shouldn't cause dynamic realignment
 define i32 @t5(float* nocapture %f) nounwind uwtable ssp {
 entry:

test/CodeGen/X86/fast-isel-stackcheck.ll

 ; RUN: llc -o - %s | FileCheck %s
 target datalayout = "e-m:o-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-apple-macosx"
 
 ; selectiondag stack protector uses a GuardReg which the fast-isel stack
 ; protection code did not but the state was not reset properly.
 ; The optnone attribute on @bar forces fast-isel.
 
 ; CHECK-LABEL: foo:
 ; CHECK: movq ___stack_chk_guard@GOTPCREL(%rip), %rax
-; CHECK-NOT: movq ___stack_chk_guard@GOTPCREL(%rip), %rax
+; CHECK: movq ___stack_chk_guard@GOTPCREL(%rip), %rax
 define void @foo() #0 {
 entry:
   %_tags = alloca [3 x i32], align 4
   ret void
 }
 
 ; CHECK-LABEL: bar:
-; CHECK: movq ___stack_chk_guard@GOTPCREL(%rip), %rax
-; CHECK: movq ___stack_chk_guard@GOTPCREL(%rip), %rax
+; CHECK: movq ___stack_chk_guard@GOTPCREL(%rip), %{{r.x}}
+; CHECK-DAG: movq ___stack_chk_guard@GOTPCREL(%rip), %[[GUARD:r.x]]
+; CHECK-DAG: movq {{[0-9]+}}(%rsp), %[[CANARY:r.x]]
+; CHECK: subq %[[CANARY]], %[[GUARD]]
 define void @bar() #1 {
 entry:
   %vt = alloca [2 x double], align 16
   br i1 undef, label %cleanup.4091, label %for.cond.3850
 
 unreachable:
   unreachable
 

test/CodeGen/X86/ssp-guard-spill.ll

+; RUN: llc < %s | FileCheck %s
+target triple = "x86_64-apple-macosx10.4.0"
+
+; bool Bar(int*);
+; bool Foo(int n) {
+;   int a[10];
+;   for (int i = 0; i < n; i++) {
+;     a[i] = 0;
+;   }
+;   return Bar(a);
+; }
+; CHECK: movq ___stack_chk_guard
+; CHECK: movq ___stack_chk_guard
+define zeroext i1 @_Z3Fooi(i32 %n) sspstrong {
+entry:
+  %n.addr = alloca i32, align 4
+  %a = alloca [10 x i32], align 16
+  %i = alloca i32, align 4
+  store i32 %n, i32* %n.addr, align 4
+  store i32 0, i32* %i, align 4
+  br label %for.cond
+
+for.cond:                                         ; preds = %for.inc, %entry
+  %tmp = load i32, i32* %i, align 4
+  %tmp1 = load i32, i32* %n.addr, align 4
+  %cmp = icmp slt i32 %tmp, %tmp1
+  br i1 %cmp, label %for.body, label %for.end
+
+for.body:                                         ; preds = %for.cond
+  %tmp2 = load i32, i32* %i, align 4
+  %idxprom = sext i32 %tmp2 to i64
+  %arrayidx = getelementptr inbounds [10 x i32], [10 x i32]* %a, i64 0, i64 %idxprom
+  store i32 0, i32* %arrayidx, align 4
+  br label %for.inc
+
+for.inc:                                          ; preds = %for.body
+  %tmp3 = load i32, i32* %i, align 4
+  %inc = add nsw i32 %tmp3, 1
+  store i32 %inc, i32* %i, align 4
+  br label %for.cond
+
+for.end:                                          ; preds = %for.cond
+  %arraydecay = getelementptr inbounds [10 x i32], [10 x i32]* %a, i32 0, i32 0
+  %call = call zeroext i1 @_Z3BarPi(i32* %arraydecay)
+  ret i1 %call
+}
+
+declare zeroext i1 @_Z3BarPi(i32*)

test/CodeGen/X86/warn-stack.ll

 ; RUN: llc -mtriple x86_64-apple-macosx10.8.0 -warn-stack-size=80 < %s 2>&1 >/dev/null | FileCheck %s
 ; Check the internal option that warns when the stack size exceeds the
 ; given amount.
 ; <rdar://13987214>
 
 ; CHECK-NOT: nowarn
 define void @nowarn() nounwind ssp {
 entry:
   %buffer = alloca [12 x i8], align 1
   %arraydecay = getelementptr inbounds [12 x i8], [12 x i8]* %buffer, i64 0, i64 0
   call void @doit(i8* %arraydecay) nounwind
   ret void
 }
 
-; CHECK: warning: stack size limit exceeded (104) in warn
+; CHECK: warning: stack size limit exceeded (88) in warn
 define void @warn() nounwind ssp {
 entry:
   %buffer = alloca [80 x i8], align 1
   %arraydecay = getelementptr inbounds [80 x i8], [80 x i8]* %buffer, i64 0, i64 0
   call void @doit(i8* %arraydecay) nounwind
   ret void
 }
 
 declare void @doit(i8*)

test/DebugInfo/X86/vla.ll

 ; RUN: llc -O0 -mtriple=x86_64-apple-darwin -filetype=asm %s -o - | FileCheck %s
 ; Ensure that we generate an indirect location for the variable length array a.
-; CHECK: ##DEBUG_VALUE: vla:a <- [%RDX+0]
-; CHECK: DW_OP_breg1
+; CHECK: ##DEBUG_VALUE: vla:a <- [%RCX+0]
+; CHECK: DW_OP_breg2
 ; rdar://problem/13658587
 ;
 ; generated from:
 ;
 ; int vla(int n) {
 ;   int a[n];
 ;   a[0] = 42;
 ;   return a[n-1];