Add target-independent backend modifications for pagerando

[rinon]

Summary

This patch adds several features to support codegen for pagerando. We add a new ISD opcode and pseudo-source value for the page offset table (analogous to the global offset table) which allows backends to emit references to the POT. The patch extends target-independent lowering slightly to store a POT base register for backends that support pagerando, and handles interactions between pagerando sections, unique sections, and jump tables. Finally, the bulk of the patch supports emission of the POT in AsmPrinter.

This patch set (D37580, D37581, D37582, D37583, D37584, D37585, D37586, D37587)
is a first draft of the pagerando implementation described in
http://lists.llvm.org/pipermail/llvm-dev/2017-June/113794.html.

[rinon]

This patch set (D37580, D37581, D37582, D37583, D37584, D37585, D37586, D37587) is a first draft of the pagerando implementation described in http://lists.llvm.org/pipermail/llvm-dev/2017-June/113794.html.

[rinon]

• Fix style nits

[rinon]

• Rebase

[rinon]

Rebase

[peter.smith]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

435

I think that this is worth doing now, when reviewing I had to double check that the duplication was computing the same thing.

1829–1869

Given that POT is not as widely documented or known as the GOT, I think it would be helpful to put a comment describing what it is. I mention it here as we need to know the structure of it in order to emit it. There could be better places though.

1847

It may be worth taking some care over using _GLOBAL_OFFSET_TABLE_. It is a linker convention to define _GLOBAL_OFFSET_TABLE_ to be the base of the .GOT but it might not always hold. For example earlier versions of lld set _GLOBAL_OFFSET_TABLE_ to 0 as the relocation types used by the compiler didn't look up the symbol value. Some backends (not Arm or AArch64) recognize the symbol and do something special with it as well.

I think what you have will work for gold on Arm and AArch64 but it may be worth letting targets override it with a "get me the got base" relocation.

[rinon]

• Refactor constantpool section lookup
• Document POT emission
• Allow targets to customize GOT symbol

[rinon]

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

1847

I added a function to TargetObjectFileLowering for ELF, since other formats don't use a GOT. Targets can override this if they need a special symbol. Sound alright?

[rinon]

Rebase

[rinon]

Rebase

[rinon]

Rebase

[rinon]

Rebase.

Fix constantpool section lookup to propagate section alignment to its callers. This fixes test/CodeGen/X86/win_cst_pool.ll.

[rinon]

Rebase

include/llvm/CodeGen/AsmPrinter.h

 class DwarfDebug;
 class GCMetadataPrinter;
 class GCStrategy;
 class GlobalIndirectSymbol;
 class GlobalObject;
 class GlobalValue;
 class GlobalVariable;
 class MachineBasicBlock;
+class MachineConstantPoolEntry;
 class MachineConstantPoolValue;
 class MachineDominatorTree;
 class MachineFunction;
 class MachineInstr;
 class MachineJumpTableInfo;
 class MachineLoopInfo;
 class MachineModuleInfo;
 class MachineOptimizationRemarkEmitter;
private:
   // The garbage collection metadata printer table.
   void *GCMetadataPrinters = nullptr; // Really a DenseMap.
 
   /// Emit comments in assembly output if this is true.
   bool VerboseAsm;
 
   static char ID;
 
+  /// Sections that need to be referenced in the POT
+  std::vector<const MCSection*> POT;
+
   struct HandlerInfo {
     AsmPrinterHandler *Handler;
     const char *TimerName;
     const char *TimerDescription;
     const char *TimerGroupName;
     const char *TimerGroupDescription;
 
     HandlerInfo(AsmPrinterHandler *Handler, const char *TimerName,
public:
   /// Return the current section we are emitting to.
   const MCSection *getCurrentSection() const;
 
   void getNameWithPrefix(SmallVectorImpl<char> &Name,
                          const GlobalValue *GV) const;
 
   MCSymbol *getSymbol(const GlobalValue *GV) const;
 
+  /// Return the section this constant pool entry should be emitted to.
+  MCSection *getSectionForCPE(const MachineConstantPoolEntry &CPE) const;
+
+  /// Lookup the constant pool entry corresponding to the given constant pool
+  /// index for the current function and return the section to emit this entry
+  /// to.
+  MCSection *getSectionForCPI(unsigned CPID) const;
+
   //===------------------------------------------------------------------===//
   // XRay instrumentation implementation.
   //===------------------------------------------------------------------===//
 public:
   // This describes the kind of sled we're storing in the XRay table.
   enum class SledKind : uint8_t {
     FUNCTION_ENTER = 0,
     FUNCTION_EXIT = 1,
public:
   /// FIXME: privatize to AsmPrinter.
   MCSymbol *GetJTSetSymbol(unsigned UID, unsigned MBBID) const;
 
   /// Return the MCSymbol used to satisfy BlockAddress uses of the specified
   /// basic block.
   MCSymbol *GetBlockAddressSymbol(const BlockAddress *BA) const;
   MCSymbol *GetBlockAddressSymbol(const BasicBlock *BB) const;
 
+  /// Return the MCSymbol for the start of the section containing this global
+  /// object, if available.
+  MCSymbol *GetSectionSymbol(const GlobalObject *GO) const;
+  MCSymbol *GetSectionSymbol(unsigned CPID) const;
+
+  /// Return the POT index of the section containing this global object.
+  unsigned GetPOTIndex(const GlobalObject *GO);
+  /// Return the POT index of the section containing this constant pool ID.
+  unsigned GetPOTIndex(unsigned CPID);
+
   //===------------------------------------------------------------------===//
   // Emission Helper Routines.
   //===------------------------------------------------------------------===//
 
   /// This is just convenient handler for printing offsets.
   void printOffset(int64_t Offset, raw_ostream &OS) const;
 
   /// Emit a byte directive and value.
private:
   /// the target.
   void EmitVisibility(MCSymbol *Sym, unsigned Visibility,
                       bool IsDefinition = true) const;
 
   void EmitLinkage(const GlobalValue *GV, MCSymbol *GVSym) const;
 
   void EmitJumpTableEntry(const MachineJumpTableInfo *MJTI,
                           const MachineBasicBlock *MBB, unsigned uid) const;
+  void EmitPOT();
   void EmitLLVMUsedList(const ConstantArray *InitList);
   /// Emit llvm.ident metadata in an '.ident' directive.
   void EmitModuleIdents(Module &M);
   void EmitXXStructorList(const DataLayout &DL, const Constant *List,
                           bool isCtor);
 
   GCMetadataPrinter *GetOrCreateGCPrinter(GCStrategy &C);
   /// Emit GlobalAlias or GlobalIFunc.
   void emitGlobalIndirectSymbol(Module &M, const GlobalIndirectSymbol &GIS);
   void setupCodePaddingContext(const MachineBasicBlock &MBB,
                                MCCodePaddingContext &Context) const;
+  /// Return the POT index of the section.
+  unsigned GetPOTIndex(const MCSection *Sec);
 };
 
 } // end namespace llvm
 
 #endif // LLVM_CODEGEN_ASMPRINTER_H

include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
     BasicBlock, VALUETYPE, CONDCODE, Register, RegisterMask,
     Constant, ConstantFP,
     GlobalAddress, GlobalTLSAddress, FrameIndex,
     JumpTable, ConstantPool, ExternalSymbol, BlockAddress,
 
     /// The address of the GOT
     GLOBAL_OFFSET_TABLE,
 
+    /// The address of the POT
+    PAGE_OFFSET_TABLE,
+
     /// FRAMEADDR, RETURNADDR - These nodes represent llvm.frameaddress and
     /// llvm.returnaddress on the DAG.  These nodes take one operand, the index
     /// of the frame or return address to return.  An index of zero corresponds
     /// to the current function's frame or return address, an index of one to
     /// the parent's frame or return address, and so on.
     FRAMEADDR, RETURNADDR, ADDROFRETURNADDR,
 
     /// LOCAL_RECOVER - Represents the llvm.localrecover intrinsic.

include/llvm/CodeGen/MachineMemOperand.h

static MachinePointerInfo getFixedStack(MachineFunction &MF, int FI,
                                           int64_t Offset = 0);
 
   /// Return a MachinePointerInfo record that refers to a jump table entry.
   static MachinePointerInfo getJumpTable(MachineFunction &MF);
 
   /// Return a MachinePointerInfo record that refers to a GOT entry.
   static MachinePointerInfo getGOT(MachineFunction &MF);
 
+  /// Return a MachinePointerInfo record that refers to a Page Offset Table
+  /// (POT) entry.
+  static MachinePointerInfo getPOT(MachineFunction &MF);
+
   /// Stack pointer relative access.
   static MachinePointerInfo getStack(MachineFunction &MF, int64_t Offset,
                                      uint8_t ID = 0);
 
   /// Stack memory without other information.
   static MachinePointerInfo getUnknownStack(MachineFunction &MF);
 };
 

include/llvm/CodeGen/PseudoSourceValue.h

 /// Special value supplied for machine level alias analysis. It indicates that
 /// a memory access references the functions stack frame (e.g., a spill slot),
 /// below the stack frame (e.g., argument space), or constant pool.
 class PseudoSourceValue {
 public:
   enum PSVKind {
     Stack,
     GOT,
+    POT,
     JumpTable,
     ConstantPool,
     FixedStack,
     GlobalValueCallEntry,
     ExternalSymbolCallEntry,
     TargetCustom
   };
 
public:
   explicit PseudoSourceValue(PSVKind Kind, const TargetInstrInfo &TII);
 
   virtual ~PseudoSourceValue();
 
   PSVKind kind() const { return Kind; }
 
   bool isStack() const { return Kind == Stack; }
   bool isGOT() const { return Kind == GOT; }
+  bool isPOT() const { return Kind == POT; }
   bool isConstantPool() const { return Kind == ConstantPool; }
   bool isJumpTable() const { return Kind == JumpTable; }
 
   unsigned getAddressSpace() const { return AddressSpace; }
 
   unsigned getTargetCustom() const {
     return (Kind >= TargetCustom) ? ((Kind+1) - TargetCustom) : 0;
   }
public:
   }
 
   const char *getSymbol() const { return ES; }
 };
 
 /// Manages creation of pseudo source values.
 class PseudoSourceValueManager {
   const TargetInstrInfo &TII;
-  const PseudoSourceValue StackPSV, GOTPSV, JumpTablePSV, ConstantPoolPSV;
+  const PseudoSourceValue StackPSV, GOTPSV, POTPSV, JumpTablePSV, ConstantPoolPSV;
   std::map<int, std::unique_ptr<FixedStackPseudoSourceValue>> FSValues;
   StringMap<std::unique_ptr<const ExternalSymbolPseudoSourceValue>>
       ExternalCallEntries;
   ValueMap<const GlobalValue *,
            std::unique_ptr<const GlobalValuePseudoSourceValue>>
       GlobalCallEntries;
 
 public:
   PseudoSourceValueManager(const TargetInstrInfo &TII);
 
   /// Return a pseudo source value referencing the area below the stack frame of
   /// a function, e.g., the argument space.
   const PseudoSourceValue *getStack();
 
   /// Return a pseudo source value referencing the global offset table
   /// (or something the like).
   const PseudoSourceValue *getGOT();
 
+  /// Return a pseudo source value referencing the POT.
+  const PseudoSourceValue *getPOT();
+
   /// Return a pseudo source value referencing the constant pool. Since constant
   /// pools are constant, this doesn't need to identify a specific constant
   /// pool entry.
   const PseudoSourceValue *getConstantPool();
 
   /// Return a pseudo source value referencing a jump table. Since jump tables
   /// are constant, this doesn't need to identify a specific jump table.
   const PseudoSourceValue *getJumpTable();

include/llvm/CodeGen/TargetLowering.h

getExceptionSelectorRegister(const Constant *PersonalityFn) const {
     // 0 is guaranteed to be the NoRegister value on all targets
     return 0;
   }
 
   virtual bool needsFixedCatchObjects() const {
     report_fatal_error("Funclet EH is not implemented for this target");
   }
 
+  /// If a physical register, this returns the register that stores the base
+  /// address of the POT.
+  unsigned getPOTBaseRegister() const {
+    return POTBaseRegister;
+  }
+
   /// Returns the target's jmp_buf size in bytes (if never set, the default is
   /// 200)
   unsigned getJumpBufSize() const {
     return JumpBufSize;
   }
 
   /// Returns the target's jmp_buf alignment in bytes (if never set, the default
   /// is 0)
protected:
   void setMaximumJumpTableSize(unsigned);
 
   /// If set to a physical register, this specifies the register that
   /// llvm.savestack/llvm.restorestack should save and restore.
   void setStackPointerRegisterToSaveRestore(unsigned R) {
     StackPointerRegisterToSaveRestore = R;
   }
 
+  /// If set to a physical register, this sets the register that stores the POT
+  /// base address.
+  void setPOTBaseRegister(unsigned R) {
+    POTBaseRegister = R;
+  }
+
   /// Tells the code generator that the target has multiple (allocatable)
   /// condition registers that can be used to store the results of comparisons
   /// for use by selects and conditional branches. With multiple condition
   /// registers, the code generator will not aggressively sink comparisons into
   /// the blocks of their users.
   void setHasMultipleConditionRegisters(bool hasManyRegs = true) {
     HasMultipleConditionRegisters = hasManyRegs;
   }
private:
 
   /// This indicates if the target supports unaligned atomic operations.
   bool SupportsUnalignedAtomics;
 
   /// If set to a physical register, this specifies the register that
   /// llvm.savestack/llvm.restorestack should save and restore.
   unsigned StackPointerRegisterToSaveRestore;
 
+  /// If set to a physical register, this specifies the register that
+  /// stores the POT base register.
+  unsigned POTBaseRegister;
+
   /// This indicates the default register class to use for each ValueType the
   /// target supports natively.
   const TargetRegisterClass *RegClassForVT[MVT::LAST_VALUETYPE];
   unsigned char NumRegistersForVT[MVT::LAST_VALUETYPE];
   MVT RegisterTypeForVT[MVT::LAST_VALUETYPE];
 
   /// This indicates the "representative" register class to use for each
   /// ValueType the target supports natively. This information is used by the

include/llvm/CodeGen/TargetLoweringObjectFileImpl.h

public:
   MCSection *getStaticCtorSection(unsigned Priority,
                                   const MCSymbol *KeySym) const override;
   MCSection *getStaticDtorSection(unsigned Priority,
                                   const MCSymbol *KeySym) const override;
 
   const MCExpr *lowerRelativeReference(const GlobalValue *LHS,
                                        const GlobalValue *RHS,
                                        const TargetMachine &TM) const override;
+
+  /// Return a symbol that refers to the base of the GOT. Platforms that do not
+  /// refer to the GOT as _GLOBAL_OFFSET_TABLE_ should override this function.
+  virtual MCSymbol *getGlobalOffsetTableSymbol() const override;
 };
 
 class TargetLoweringObjectFileMachO : public TargetLoweringObjectFile {
 public:
   TargetLoweringObjectFileMachO();
   ~TargetLoweringObjectFileMachO() override = default;
 
   void Initialize(MCContext &Ctx, const TargetMachine &TM) override;

include/llvm/Target/TargetLoweringObjectFile.h

public:
   }
 
   virtual void emitLinkerFlagsForGlobal(raw_ostream &OS,
                                         const GlobalValue *GV) const {}
 
   virtual void emitLinkerFlagsForUsed(raw_ostream &OS,
                                       const GlobalValue *GV) const {}
 
+  virtual MCSymbol *getGlobalOffsetTableSymbol() const {
+    return nullptr;
+  }
+
 protected:
   virtual MCSection *SelectSectionForGlobal(const GlobalObject *GO,
                                             SectionKind Kind,
                                             const TargetMachine &TM) const = 0;
 };
 
 } // end namespace llvm
 
 #endif // LLVM_CODEGEN_TARGETLOWERINGOBJECTFILE_H

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

void AsmPrinter::getNameWithPrefix(SmallVectorImpl<char> &Name,
                                    const GlobalValue *GV) const {
   TM.getNameWithPrefix(Name, GV, getObjFileLowering().getMangler());
 }
 
 MCSymbol *AsmPrinter::getSymbol(const GlobalValue *GV) const {
   return TM.getSymbol(GV);
 }
 
+MCSection *AsmPrinter::getSectionForCPE(const MachineConstantPoolEntry &CPE) const {
+  unsigned Align = CPE.getAlignment();
+
+  SectionKind Kind = CPE.getSectionKind(&getDataLayout());
+
+  const Constant *C = nullptr;
+  if (!CPE.isMachineConstantPoolEntry())
+    C = CPE.Val.ConstVal;
+
+  return getObjFileLowering().getSectionForConstant(getDataLayout(),
+                                                    Kind, C, Align);
+}
+
+MCSection *AsmPrinter::getSectionForCPI(unsigned CPID) const {
+  const MachineConstantPool *MCP = MF->getConstantPool();
+  const MachineConstantPoolEntry &CPE = MCP->getConstants()[CPID];
+
+  return getSectionForCPE(CPE);
+}
+
 /// EmitGlobalVariable - Emit the specified global variable to the .s file.
 void AsmPrinter::EmitGlobalVariable(const GlobalVariable *GV) {
   bool IsEmuTLSVar = TM.useEmulatedTLS() && GV->isThreadLocal();
   assert(!(IsEmuTLSVar && GV->hasCommonLinkage()) &&
          "No emulated TLS variables in the common section");
 
   // Never emit TLS variable xyz in emulated TLS model.
   // The initialization value is in __emutls_t.xyz instead of xyz.
bool AsmPrinter::doFinalization(Module &M) {
 
   // Emit global variables.
   for (const auto &G : M.globals())
     EmitGlobalVariable(&G);
 
   // Emit remaining GOT equivalent globals.
   emitGlobalGOTEquivs();
 
+  // Emit POT (page offset table) if Pagerando is enabled.
+  if (TM.isPagerando())
+    EmitPOT();
+
   // Emit visibility info for declarations
   for (const Function &F : M) {
     if (!F.isDeclarationForLinker())
       continue;
     GlobalValue::VisibilityTypes V = F.getVisibility();
     if (V == GlobalValue::DefaultVisibility)
       continue;
 
void AsmPrinter::EmitConstantPool() {
   if (CP.empty()) return;
 
   // Calculate sections for constant pool entries. We collect entries to go into
   // the same section together to reduce amount of section switch statements.
   SmallVector<SectionCPs, 4> CPSections;
   for (unsigned i = 0, e = CP.size(); i != e; ++i) {
     const MachineConstantPoolEntry &CPE = CP[i];
     unsigned Align = CPE.getAlignment();
-
-    SectionKind Kind = CPE.getSectionKind(&getDataLayout());
-
-    const Constant *C = nullptr;
-    if (!CPE.isMachineConstantPoolEntry())
-      C = CPE.Val.ConstVal;
-
-    MCSection *S = getObjFileLowering().getSectionForConstant(getDataLayout(),
-                                                              Kind, C, Align);
+    MCSection *S = getSectionForCPE(CPE);
 
     // The number of sections are small, just do a linear search from the
     // last section to the first.
     bool Found = false;
     unsigned SecIdx = CPSections.size();
     while (SecIdx != 0) {
       if (CPSections[--SecIdx].S == S) {
         Found = true;
if (GV->getName() == "llvm.global_dtors") {
     EmitXXStructorList(GV->getParent()->getDataLayout(), GV->getInitializer(),
                        /* isCtor */ false);
 
     return true;
   }
 
   report_fatal_error("unknown special variable");
 }
 
+/// EmitPOT - Emit the page offset table (POT), used for inter-page references
+/// in pagerando. Pagerando allows specially marked code pages (bins) in a DSO
+/// to be loaded at independent offsets from each other and the rest of the
+/// DSO. The POT stores the dynamic absolute address of each pagerando section
+/// to allow references to these pagerando sections from code that is not
+/// located at a static PC-relative offset from the target section. The first
+/// entry in the POT stores the absolute address of the global offset table
+/// (GOT) to allow code in pagerando sections to reference global data and
+/// non-pagerando code with GOT-relative relocations.
+void AsmPrinter::EmitPOT() {
+  assert(TM.getTargetTriple().isOSBinFormatELF() &&
+         "Pagerando is only supported on ELF");
+  unsigned Alignment = getDataLayout().getPointerPrefAlignment(0);
+  unsigned PtrSize = getDataLayout().getPointerSize(0);
+
+  auto *Section = OutContext.getELFSection(".pot", ELF::SHT_PROGBITS,
+                                           ELF::SHF_ALLOC | ELF::SHF_WRITE);
+  OutStreamer->SwitchSection(Section);
+
+  // Emit POT start label
+  auto *POTSym = OutContext.getOrCreateSymbol("_PAGE_OFFSET_TABLE_");
+  OutStreamer->EmitValueToAlignment(Alignment);
+  OutStreamer->EmitLabel(POTSym);
+
+  // Entry 0 is GOT reference
+  auto *GOTSym = getObjFileLowering().getGlobalOffsetTableSymbol();
+  assert(GOTSym && "Target platform must support absolute GOT symbol");
+  EmitLabelReference(GOTSym, PtrSize);
+
+  // Emit Bin references
+  for (auto *Bin : POT)
+    EmitLabelReference(Bin->getBeginSymbol(), PtrSize);
+
+  // Emit POT end label
+  auto *POTEndSym = OutContext.getOrCreateSymbol("_PAGE_OFFSET_TABLE_END_");
+  OutStreamer->EmitLabel(POTEndSym);
+
+  OutStreamer->AddBlankLine();
+}
+
 /// EmitLLVMUsedList - For targets that define a MAI::UsedDirective, mark each
 /// global in the specified llvm.used list for which emitUsedDirectiveFor
 /// is true, as being used with this directive.
 void AsmPrinter::EmitLLVMUsedList(const ConstantArray *InitList) {
   // Should be an array of 'i8*'.
   for (unsigned i = 0, e = InitList->getNumOperands(); i != e; ++i) {
     const GlobalValue *GV =
       dyn_cast<GlobalValue>(InitList->getOperand(i)->stripPointerCasts());

 
 /// Return the MCSymbol for the specified ExternalSymbol.
 MCSymbol *AsmPrinter::GetExternalSymbolSymbol(StringRef Sym) const {
   SmallString<60> NameStr;
   Mangler::getNameWithPrefix(NameStr, Sym, getDataLayout());
   return OutContext.getOrCreateSymbol(NameStr);
 }
 
+MCSymbol *AsmPrinter::GetSectionSymbol(const GlobalObject *GO) const {
+  MCSection *Sec = getObjFileLowering().SectionForGlobal(GO, TM);
+  return Sec->getBeginSymbol();
+}
+
+MCSymbol *AsmPrinter::GetSectionSymbol(unsigned CPID) const {
+  MCSection *Sec = getSectionForCPI(CPID);
+  return Sec->getBeginSymbol();
+}
+
+unsigned AsmPrinter::GetPOTIndex(const GlobalObject *GO) {
+  const MCSection *Sec = getObjFileLowering().SectionForGlobal(GO, TM);
+  return GetPOTIndex(Sec);
+}
+
+unsigned AsmPrinter::GetPOTIndex(unsigned CPID) {
+  const MCSection *Sec = getSectionForCPI(CPID);
+  return GetPOTIndex(Sec);
+}
+
+unsigned AsmPrinter::GetPOTIndex(const MCSection *Sec) {
+  auto I = std::find(POT.begin(), POT.end(), Sec);
+  auto Index = static_cast<unsigned>(I - POT.begin());
+  if (I == POT.end())
+    POT.push_back(Sec);
+  return Index + 1;  // Index 0 denotes the default bin #0
+}
+
 /// PrintParentLoopComment - Print comments about parent loops of this one.
 static void PrintParentLoopComment(raw_ostream &OS, const MachineLoop *Loop,
                                    unsigned FunctionNumber) {
   if (!Loop) return;
   PrintParentLoopComment(OS, Loop->getParentLoop(), FunctionNumber);
   OS.indent(Loop->getLoopDepth()*2)
     << "Parent Loop BB" << FunctionNumber << "_"
     << Loop->getHeader()->getNumber()

lib/CodeGen/MachineOperand.cpp

 MachinePointerInfo MachinePointerInfo::getJumpTable(MachineFunction &MF) {
   return MachinePointerInfo(MF.getPSVManager().getJumpTable());
 }
 
 MachinePointerInfo MachinePointerInfo::getGOT(MachineFunction &MF) {
   return MachinePointerInfo(MF.getPSVManager().getGOT());
 }
 
+MachinePointerInfo MachinePointerInfo::getPOT(MachineFunction &MF) {
+  return MachinePointerInfo(MF.getPSVManager().getPOT());
+}
+
 MachinePointerInfo MachinePointerInfo::getStack(MachineFunction &MF,
                                                 int64_t Offset, uint8_t ID) {
   return MachinePointerInfo(MF.getPSVManager().getStack(), Offset, ID);
 }
 
 MachinePointerInfo MachinePointerInfo::getUnknownStack(MachineFunction &MF) {
   return MachinePointerInfo(MF.getDataLayout().getAllocaAddrSpace());
 }
if (const Value *Val = getValue()) {
     assert(PVal && "Expected a pseudo source value");
     switch (PVal->kind()) {
     case PseudoSourceValue::Stack:
       OS << "stack";
       break;
     case PseudoSourceValue::GOT:
       OS << "got";
       break;
+    case PseudoSourceValue::POT:
+      OS << "pot";
+      break;
     case PseudoSourceValue::JumpTable:
       OS << "jump-table";
       break;
     case PseudoSourceValue::ConstantPool:
       OS << "constant-pool";
       break;
     case PseudoSourceValue::FixedStack: {
       int FrameIndex = cast<FixedStackPseudoSourceValue>(PVal)->getFrameIndex();

lib/CodeGen/PseudoSourceValue.cpp

 #include "llvm/CodeGen/TargetInstrInfo.h"
 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/LLVMContext.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 using namespace llvm;
 
 static const char *const PSVNames[] = {
-    "Stack", "GOT", "JumpTable", "ConstantPool", "FixedStack",
+    "Stack", "GOT", "POT", "JumpTable", "ConstantPool", "FixedStack",
     "GlobalValueCallEntry", "ExternalSymbolCallEntry"};
 
 PseudoSourceValue::PseudoSourceValue(PSVKind Kind, const TargetInstrInfo &TII)
     : Kind(Kind) {
   AddressSpace = TII.getAddressSpaceForPseudoSourceKind(Kind);
 }
 
 
 PseudoSourceValue::~PseudoSourceValue() {}
 
 void PseudoSourceValue::printCustom(raw_ostream &O) const {
   if (Kind < TargetCustom)
     O << PSVNames[Kind];
   else
     O << "TargetCustom" << Kind;
 }
 
 bool PseudoSourceValue::isConstant(const MachineFrameInfo *) const {
   if (isStack())
     return false;
-  if (isGOT() || isConstantPool() || isJumpTable())
+  if (isGOT() || isPOT() || isConstantPool() || isJumpTable())
     return true;
   llvm_unreachable("Unknown PseudoSourceValue!");
 }
 
 bool PseudoSourceValue::isAliased(const MachineFrameInfo *) const {
-  if (isStack() || isGOT() || isConstantPool() || isJumpTable())
+  if (isStack() || isGOT() || isPOT() || isConstantPool() || isJumpTable())
     return false;
   llvm_unreachable("Unknown PseudoSourceValue!");
 }
 
 bool PseudoSourceValue::mayAlias(const MachineFrameInfo *) const {
-  return !(isGOT() || isConstantPool() || isJumpTable());
+  return !(isGOT() || isPOT() || isConstantPool() || isJumpTable());
 }
 
 bool FixedStackPseudoSourceValue::isConstant(
     const MachineFrameInfo *MFI) const {
   return MFI && MFI->isImmutableObjectIndex(FI);
 }
 
 bool FixedStackPseudoSourceValue::isAliased(const MachineFrameInfo *MFI) const {
ExternalSymbolPseudoSourceValue::ExternalSymbolPseudoSourceValue(
     const char *ES, const TargetInstrInfo &TII)
     : CallEntryPseudoSourceValue(ExternalSymbolCallEntry, TII), ES(ES) {}
 
 PseudoSourceValueManager::PseudoSourceValueManager(
     const TargetInstrInfo &TIInfo)
     : TII(TIInfo),
       StackPSV(PseudoSourceValue::Stack, TII),
       GOTPSV(PseudoSourceValue::GOT, TII),
+      POTPSV(PseudoSourceValue::POT, TII),
       JumpTablePSV(PseudoSourceValue::JumpTable, TII),
       ConstantPoolPSV(PseudoSourceValue::ConstantPool, TII) {}
 
 const PseudoSourceValue *PseudoSourceValueManager::getStack() {
   return &StackPSV;
 }
 
 const PseudoSourceValue *PseudoSourceValueManager::getGOT() { return &GOTPSV; }
 
+const PseudoSourceValue *PseudoSourceValueManager::getPOT() { return &POTPSV; }
+
 const PseudoSourceValue *PseudoSourceValueManager::getConstantPool() {
   return &ConstantPoolPSV;
 }
 
 const PseudoSourceValue *PseudoSourceValueManager::getJumpTable() {
   return &JumpTablePSV;
 }
 

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

for (unsigned Idx = 0; Idx < NumElem; Idx++) {
                                     VT.getScalarType(), Ex, Sh));
     }
 
     SDValue Result = DAG.getBuildVector(Node->getValueType(0), dl, Scalars);
     ReplaceNode(SDValue(Node, 0), Result);
     break;
   }
   case ISD::GLOBAL_OFFSET_TABLE:
+  case ISD::PAGE_OFFSET_TABLE:
   case ISD::GlobalAddress:
   case ISD::GlobalTLSAddress:
   case ISD::ExternalSymbol:
   case ISD::ConstantPool:
   case ISD::JumpTable:
   case ISD::INTRINSIC_W_CHAIN:
   case ISD::INTRINSIC_WO_CHAIN:
   case ISD::INTRINSIC_VOID:

lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp

if (cast<ConstantSDNode>(this)->isOpaque())
       return "OpaqueConstant";
     return "Constant";
   case ISD::ConstantFP:                 return "ConstantFP";
   case ISD::GlobalAddress:              return "GlobalAddress";
   case ISD::GlobalTLSAddress:           return "GlobalTLSAddress";
   case ISD::FrameIndex:                 return "FrameIndex";
   case ISD::JumpTable:                  return "JumpTable";
   case ISD::GLOBAL_OFFSET_TABLE:        return "GLOBAL_OFFSET_TABLE";
+  case ISD::PAGE_OFFSET_TABLE:          return "PAGE_OFFSET_TABLE";
   case ISD::RETURNADDR:                 return "RETURNADDR";
   case ISD::ADDROFRETURNADDR:           return "ADDROFRETURNADDR";
   case ISD::FRAMEADDR:                  return "FRAMEADDR";
   case ISD::LOCAL_RECOVER:              return "LOCAL_RECOVER";
   case ISD::READ_REGISTER:              return "READ_REGISTER";
   case ISD::WRITE_REGISTER:             return "WRITE_REGISTER";
   case ISD::FRAME_TO_ARGS_OFFSET:       return "FRAME_TO_ARGS_OFFSET";
   case ISD::EH_DWARF_CFA:               return "EH_DWARF_CFA";

lib/CodeGen/TargetLoweringBase.cpp

TargetLoweringBase::TargetLoweringBase(const TargetMachine &tm) : TM(tm) {
   UseUnderscoreLongJmp = false;
   HasMultipleConditionRegisters = false;
   HasExtractBitsInsn = false;
   JumpIsExpensive = JumpIsExpensiveOverride;
   PredictableSelectIsExpensive = false;
   EnableExtLdPromotion = false;
   HasFloatingPointExceptions = true;
   StackPointerRegisterToSaveRestore = 0;
+  POTBaseRegister = 0;
   BooleanContents = UndefinedBooleanContent;
   BooleanFloatContents = UndefinedBooleanContent;
   BooleanVectorContents = UndefinedBooleanContent;
   SchedPreferenceInfo = Sched::ILP;
   JumpBufSize = 0;
   JumpBufAlignment = 0;
   MinFunctionAlignment = 0;
   PrefFunctionAlignment = 0;

lib/CodeGen/TargetLoweringObjectFileImpl.cpp

MCSection *TargetLoweringObjectFileELF::SelectSectionForGlobal(
   EmitUniqueSection |= GO->hasComdat();
 
   const MCSymbolELF *AssociatedSymbol = getAssociatedSymbol(GO, TM);
   if (AssociatedSymbol) {
     EmitUniqueSection = true;
     Flags |= ELF::SHF_LINK_ORDER;
   }
 
+  // Pagerando is incompatible with unique sections.
+  if (auto *F = dyn_cast<Function>(GO)) {
+    if (F->isPagerando())
+      EmitUniqueSection = false;
+  }
+
   MCSectionELF *Section = selectELFSectionForGlobal(
       getContext(), GO, Kind, getMangler(), TM, EmitUniqueSection, Flags,
       &NextUniqueID, AssociatedSymbol);
   assert(Section->getAssociatedSymbol() == AssociatedSymbol);
   return Section;
 }
 
 MCSection *TargetLoweringObjectFileELF::getSectionForJumpTable(
     const Function &F, const TargetMachine &TM) const {
   // If the function can be removed, produce a unique section so that
   // the table doesn't prevent the removal.
   const Comdat *C = F.getComdat();
   bool EmitUniqueSection = TM.getFunctionSections() || C;
   if (!EmitUniqueSection)
     return ReadOnlySection;
 
   return selectELFSectionForGlobal(getContext(), &F, SectionKind::getReadOnly(),
                                    getMangler(), TM, EmitUniqueSection,
                                    ELF::SHF_ALLOC, &NextUniqueID,
                                    /* AssociatedSymbol */ nullptr);
 }
 
 bool TargetLoweringObjectFileELF::shouldPutJumpTableInFunctionSection(
     bool UsesLabelDifference, const Function &F) const {
+  // If we place the function in a randomly located page, it's faster to have a
+  // local jump table, since a global one would require dynamic relocations.
+  if (F.isPagerando())
+    return true;
+
   // We can always create relative relocations, so use another section
   // that can be marked non-executable.
   return false;
 }
 
 /// Given a mergeable constant with the specified size and relocation
 /// information, return a section that it should be placed in.
 MCSection *TargetLoweringObjectFileELF::getSectionForConstant(
if (LHS->getType()->getPointerAddressSpace() != 0 ||
     return nullptr;
 
   return MCBinaryExpr::createSub(
       MCSymbolRefExpr::create(TM.getSymbol(LHS), PLTRelativeVariantKind,
                               getContext()),
       MCSymbolRefExpr::create(TM.getSymbol(RHS), getContext()), getContext());
 }
 
+MCSymbol *TargetLoweringObjectFileELF::getGlobalOffsetTableSymbol() const {
+  return getContext().getOrCreateSymbol("_GLOBAL_OFFSET_TABLE_");
+}
+
 void
 TargetLoweringObjectFileELF::InitializeELF(bool UseInitArray_) {
   UseInitArray = UseInitArray_;
   MCContext &Ctx = getContext();
   if (!UseInitArray) {
     StaticCtorSection = Ctx.getELFSection(".ctors", ELF::SHT_PROGBITS,
                                           ELF::SHF_ALLOC | ELF::SHF_WRITE);
 

lib/Target/TargetLoweringObjectFile.cpp

bool TargetLoweringObjectFile::shouldPutJumpTableInFunctionSection(
   // FIXME: Need a better predicate for this: what about custom entries?
   if (UsesLabelDifference)
     return true;
 
   // We should also do if the section name is NULL or function is declared
   // in discardable section
   // FIXME: this isn't the right predicate, should be based on the MCSection
   // for the function.
-  return F.isWeakForLinker();
+  if (F.isWeakForLinker())
+    return true;
+
+  // If we place the function in a randomly located page, it's faster to have a
+  // local jump table, since a global one would require dynamic relocations.
+  if (F.isPagerando())
+    return true;
+
+  return false;
 }
 
 /// Given a mergable constant with the specified size and relocation
 /// information, return a section that it should be placed in.
 MCSection *TargetLoweringObjectFile::getSectionForConstant(
     const DataLayout &DL, SectionKind Kind, const Constant *C,
     unsigned &Align) const {
   if (Kind.isReadOnly() && ReadOnlySection != nullptr)