[JITLink][ORC] Rename MemDeallocPolicy to MemLifetimePolicy, add NoAlloc option.

[lhames]

Summary

The original MemDeallocPolicy options were 'Standard' (the allocated memory
should live until the FinalizedAlloc is deallocated, or the InFlightAlloc is
abandoned) and 'Finalize' (the allocated memory should live until all finalize
actions have been run).

The new NoAlloc option indicates that the section should not have memory
allocated for it at all, and should be ignored by the JITLinkMemoryManager
(the memory manager should skip such sections entirely, allocating neither
working memory nor executor address space to them). The NoAlloc option is
intended to support metadata sections (e.g. debug info) that we want to keep in
the graph and have fixed up if necessary, but don't want allocated or
transmitted to the executor (or we want that transmission to be managed
manually by plugins).

Since NoAlloc blocks do not have working memory allocated for them by the
JITLinkMemoryManager, clients wishing to mutate their memory should call the
Block::getMutableMemory(G) method on them. This will copy the Block's content
to memory allocated on G's allocator (on first call, subsequent calls just
return the existing allocation(). If the block's content is not copied to
mutable memory prior to the fixup phase the generic link algorithm will do this
automatically. Block content in NoAlloc sections will remain available until
the LinkGraph is destroyed.

The addresses of blocks in NoAlloc sections are initialized by the LinkGraph
creator (a LinkGraphBuilder, if the graph is generated from an object file),
and is should not be modified by the JITLinkMemoryManager. Plugins are
responsible for updating addresses if they add/remove content from these
sections.

Blocks in Standard and Finalize sections should never contain edges pointing to
NonAlloc sections, as the addresses for these blocks aren't meaningful. The
LinkGraphBuilders should check this property in the input object files, and
plugins are required to preserve it.

[lhames]

LinkGraphBuilder changes TBD.

I'm also rethinking this requirement:

Blocks in Standard and Finalize sections should never contain edges pointing to
NonAlloc sections, as the addresses for these blocks aren't meaningful. The
LinkGraphBuilders should check this property in the input object files, and
plugins are required to preserve it.

NoAlloc sections are effectively in a different address space, but it's probably fine to have references back and forward -- something on the executor side could still use the addresses in requests to the controller.

[lhames]

Use NoAlloc lifetime in LinkGraphBuilders for COFF, ELF and MachO.

Update patch wording to clarify that edges between Standard/Finalize and
NoAlloc sections are legal in both directions, but only when the edge kinds
involved make sense.

[lhames]

New commit language on edges between Standard/Finalize and NoAlloc sections:

Addresses of blocks in NoAlloc sections are initialized by the LinkGraph
creator (a LinkGraphBuilder, if the graph is generated from an object file),
and should not be modified by the JITLinkMemoryManager. Plugins are responsible
for updating addresses if they add/remove content from these sections. The
meaning of addresses in NoAlloc-sections is backend/plugin-defined, but for
fixup purposes they will be treated the same as addresses in Standard/Finalize
sections. References from Standard/Finalize sections to NoAlloc sections are
expected to be common (these represent metadata tracking executor addresses).
References from NoAlloc sections to Standard/Finalize sections are expected to
be rare/non-existent (they would represent JIT'd code / data tracking metadata
in the controller, which would be surprising). LinkGraphBuilders and specific
backends may impose additional constraints on edges between Standard/Finalize
and NoAlloc sections where required for correctness.

llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h

public:
   StringRef getName() const { return Name; }
 
   /// Returns the protection flags for this section.
   orc::MemProt getMemProt() const { return Prot; }
 
   /// Set the protection flags for this section.
   void setMemProt(orc::MemProt Prot) { this->Prot = Prot; }
 
-  /// Get the deallocation policy for this section.
-  orc::MemDeallocPolicy getMemDeallocPolicy() const { return MDP; }
+  /// Get the memory lifetime policy for this section.
+  orc::MemLifetimePolicy getMemLifetimePolicy() const { return MLP; }
 
-  /// Set the deallocation policy for this section.
-  void setMemDeallocPolicy(orc::MemDeallocPolicy MDP) { this->MDP = MDP; }
+  /// Set the memory lifetime policy for this section.
+  void setMemLifetimePolicy(orc::MemLifetimePolicy MLP) { this->MLP = MLP; }
 
   /// Returns the ordinal for this section.
   SectionOrdinal getOrdinal() const { return SecOrdinal; }
 
   /// Returns an iterator over the blocks defined in this section.
   iterator_range<block_iterator> blocks() {
     return make_range(Blocks.begin(), Blocks.end());
   }
void transferContentTo(Section &DstSection) {
     for (auto *B : Blocks)
       DstSection.addBlock(*B);
     Symbols.clear();
     Blocks.clear();
   }
 
   StringRef Name;
   orc::MemProt Prot;
-  orc::MemDeallocPolicy MDP = orc::MemDeallocPolicy::Standard;
+  orc::MemLifetimePolicy MLP = orc::MemLifetimePolicy::Standard;
   SectionOrdinal SecOrdinal = 0;
   BlockSet Blocks;
   SymbolSet Symbols;
 };
 
 /// Represents a section address range via a pair of Block pointers
 /// to the first and last Blocks in the section.
 class SectionRange {

llvm/include/llvm/ExecutionEngine/JITLink/JITLinkMemoryManager.h

 
 /// A utility class for making simple allocations using JITLinkMemoryManager.
 ///
 /// SimpleSegementAlloc takes a mapping of AllocGroups to Segments and uses
 /// this to create a LinkGraph with one Section (containing one Block) per
 /// Segment. Clients can obtain a pointer to the working memory and executor
 /// address of that block using the Segment's AllocGroup. Once memory has been
 /// populated, clients can call finalize to finalize the memory.
+///
+/// Note: Segments with MemLifetimePolicy::NoAlloc are not permitted, since
+/// they would not be useful, and their presence is likely to indicate a bug.
 class SimpleSegmentAlloc {
 public:
   /// Describes a segment to be allocated.
   struct Segment {
     Segment() = default;
     Segment(size_t ContentSize, Align ContentAlign)
         : ContentSize(ContentSize), ContentAlign(ContentAlign) {}
 

llvm/include/llvm/ExecutionEngine/Orc/Shared/MemoryFlags.h

if (PF & sys::Memory::MF_READ)
     MP |= MemProt::Read;
   if (PF & sys::Memory::MF_WRITE)
     MP |= MemProt::Write;
   if (PF & sys::Memory::MF_EXEC)
     MP |= MemProt::None;
   return MP;
 }
 
-/// Describes a memory deallocation policy for memory to be allocated by a
+/// Describes a memory lifetime policy for memory to be allocated by a
 /// JITLinkMemoryManager.
 ///
 /// All memory allocated by a call to JITLinkMemoryManager::allocate should be
 /// deallocated if a call is made to
 /// JITLinkMemoryManager::InFlightAllocation::abandon. The policies below apply
 /// to finalized allocations.
-enum class MemDeallocPolicy {
-  /// Standard memory should be deallocated when the deallocate method is called
-  /// for the finalized allocation.
+enum class MemLifetimePolicy {
+  /// Standard memory should be allocated by the allocator and then deallocated
+  /// when the deallocate method is called for the finalized allocation.
   Standard,
 
-  /// Finalize memory should be overwritten and then deallocated after all
-  /// finalization functions have been run.
-  Finalize
+  /// Finalize memory should be allocated by the allocator, and then be
+  /// overwritten and deallocated after all finalization functions have been
+  /// run.
+  Finalize,
+
+  /// NoAlloc memory should not be allocated by the JITLinkMemoryManager at
+  /// all. It is used for sections that don't need to be transferred to the
+  /// executor process, typically metadata sections.
+  NoAlloc
 };
 
 /// Print a MemDeallocPolicy.
-inline raw_ostream &operator<<(raw_ostream &OS, MemDeallocPolicy MDP) {
-  return OS << (MDP == MemDeallocPolicy::Standard ? "standard" : "finalize");
+inline raw_ostream &operator<<(raw_ostream &OS, MemLifetimePolicy MLP) {
+  switch (MLP) {
+  case MemLifetimePolicy::Standard:
+    OS << "standard";
+    break;
+  case MemLifetimePolicy::Finalize:
+    OS << "finalize";
+    break;
+  case MemLifetimePolicy::NoAlloc:
+    OS << "noalloc";
+    break;
+  }
+  return OS;
 }
 
 /// A pair of memory protections and allocation policies.
 ///
 /// Optimized for use as a small map key.
 class AllocGroup {
   friend struct llvm::DenseMapInfo<AllocGroup>;
 
   using underlying_type = uint8_t;
   static constexpr unsigned BitsForProt = 3;
-  static constexpr unsigned BitsForDeallocPolicy = 1;
+  static constexpr unsigned BitsForLifetimePolicy = 2;
   static constexpr unsigned MaxIdentifiers =
-      1U << (BitsForProt + BitsForDeallocPolicy);
+      1U << (BitsForProt + BitsForLifetimePolicy);
 
 public:
   static constexpr unsigned NumGroups = MaxIdentifiers;
 
   /// Create a default AllocGroup. No memory protections, standard
-  /// deallocation policy.
+  /// lifetime policy.
   AllocGroup() = default;
 
   /// Create an AllocGroup from a MemProt only -- uses
-  /// MemoryDeallocationPolicy::Standard.
+  /// MemLifetimePolicy::Standard.
   AllocGroup(MemProt MP) : Id(static_cast<underlying_type>(MP)) {}
 
-  /// Create an AllocGroup from a MemProt and a MemoryDeallocationPolicy.
-  AllocGroup(MemProt MP, MemDeallocPolicy MDP)
+  /// Create an AllocGroup from a MemProt and a MemLifetimePolicy.
+  AllocGroup(MemProt MP, MemLifetimePolicy MLP)
       : Id(static_cast<underlying_type>(MP) |
-           (static_cast<underlying_type>(MDP) << BitsForProt)) {}
+           (static_cast<underlying_type>(MLP) << BitsForProt)) {}
 
   /// Returns the MemProt for this group.
   MemProt getMemProt() const {
     return static_cast<MemProt>(Id & ((1U << BitsForProt) - 1));
   }
 
-  /// Returns the MemoryDeallocationPolicy for this group.
-  MemDeallocPolicy getMemDeallocPolicy() const {
-    return static_cast<MemDeallocPolicy>(Id >> BitsForProt);
+  /// Returns the MemLifetimePolicy for this group.
+  MemLifetimePolicy getMemLifetimePolicy() const {
+    return static_cast<MemLifetimePolicy>(Id >> BitsForProt);
   }
 
   friend bool operator==(const AllocGroup &LHS, const AllocGroup &RHS) {
     return LHS.Id == RHS.Id;
   }
 
   friend bool operator!=(const AllocGroup &LHS, const AllocGroup &RHS) {
     return !(LHS == RHS);
public:
   }
 
 private:
   VectorTy Elems;
 };
 
 /// Print an AllocGroup.
 inline raw_ostream &operator<<(raw_ostream &OS, AllocGroup AG) {
-  return OS << '(' << AG.getMemProt() << ", " << AG.getMemDeallocPolicy()
+  return OS << '(' << AG.getMemProt() << ", " << AG.getMemLifetimePolicy()
             << ')';
 }
 
 } // end namespace orc
 
 template <> struct DenseMapInfo<orc::MemProt> {
   static inline orc::MemProt getEmptyKey() { return orc::MemProt(~uint8_t(0)); }
   static inline orc::MemProt getTombstoneKey() {

llvm/include/llvm/ExecutionEngine/Orc/Shared/TargetProcessControlTypes.h

 #include "llvm/Support/Memory.h"
 
 #include <vector>
 
 namespace llvm {
 namespace orc {
 namespace tpctypes {
 
+struct RemoteAllocGroup {
+  RemoteAllocGroup() = default;
+  RemoteAllocGroup(MemProt Prot) : Prot(Prot) {}
+  RemoteAllocGroup(MemProt Prot, bool FinalizeLifetime)
+      : Prot(Prot), FinalizeLifetime(FinalizeLifetime) {}
+  RemoteAllocGroup(const AllocGroup &AG) : Prot(AG.getMemProt()) {
+    assert(AG.getMemLifetimePolicy() != orc::MemLifetimePolicy::NoAlloc &&
+           "Cannot use no-alloc memory in a remote alloc request");
+    FinalizeLifetime =
+        AG.getMemLifetimePolicy() == orc::MemLifetimePolicy::Finalize;
+  }
+
+  MemProt Prot;
+  bool FinalizeLifetime = false;
+};
+
 struct SegFinalizeRequest {
-  AllocGroup AG;
+  RemoteAllocGroup RAG;
   ExecutorAddr Addr;
   uint64_t Size;
   ArrayRef<char> Content;
 };
 
 struct FinalizeRequest {
   std::vector<SegFinalizeRequest> Segments;
   shared::AllocActions Actions;
 };
 
 struct SharedMemorySegFinalizeRequest {
-  AllocGroup AG;
+  RemoteAllocGroup RAG;
   ExecutorAddr Addr;
   uint64_t Size;
 };
 
 struct SharedMemoryFinalizeRequest {
   std::vector<SharedMemorySegFinalizeRequest> Segments;
   shared::AllocActions Actions;
 };

 using DylibHandle = ExecutorAddr;
 
 using LookupResult = std::vector<ExecutorAddr>;
 
 } // end namespace tpctypes
 
 namespace shared {
 
-class SPSAllocGroup {};
+class SPSRemoteAllocGroup;
 
 using SPSSegFinalizeRequest =
-    SPSTuple<SPSAllocGroup, SPSExecutorAddr, uint64_t, SPSSequence<char>>;
+    SPSTuple<SPSRemoteAllocGroup, SPSExecutorAddr, uint64_t, SPSSequence<char>>;
 
 using SPSFinalizeRequest = SPSTuple<SPSSequence<SPSSegFinalizeRequest>,
                                     SPSSequence<SPSAllocActionCallPair>>;
 
 using SPSSharedMemorySegFinalizeRequest =
-    SPSTuple<SPSAllocGroup, SPSExecutorAddr, uint64_t>;
+    SPSTuple<SPSRemoteAllocGroup, SPSExecutorAddr, uint64_t>;
 
 using SPSSharedMemoryFinalizeRequest =
     SPSTuple<SPSSequence<SPSSharedMemorySegFinalizeRequest>,
              SPSSequence<SPSAllocActionCallPair>>;
 
 template <typename T>
 using SPSMemoryAccessUIntWrite = SPSTuple<SPSExecutorAddr, T>;
 
 using SPSMemoryAccessUInt8Write = SPSMemoryAccessUIntWrite<uint8_t>;
 using SPSMemoryAccessUInt16Write = SPSMemoryAccessUIntWrite<uint16_t>;
 using SPSMemoryAccessUInt32Write = SPSMemoryAccessUIntWrite<uint32_t>;
 using SPSMemoryAccessUInt64Write = SPSMemoryAccessUIntWrite<uint64_t>;
 
 using SPSMemoryAccessBufferWrite = SPSTuple<SPSExecutorAddr, SPSSequence<char>>;
 
-template <> class SPSSerializationTraits<SPSAllocGroup, AllocGroup> {
+template <>
+class SPSSerializationTraits<SPSRemoteAllocGroup, tpctypes::RemoteAllocGroup> {
   enum WireBits {
     ReadBit = 1 << 0,
     WriteBit = 1 << 1,
     ExecBit = 1 << 2,
     FinalizeBit = 1 << 3
   };
 
 public:
-  static size_t size(const AllocGroup &AG) {
+  static size_t size(const tpctypes::RemoteAllocGroup &RAG) {
     // All AllocGroup values encode to the same size.
     return SPSArgList<uint8_t>::size(uint8_t(0));
   }
 
-  static bool serialize(SPSOutputBuffer &OB, const AllocGroup &AG) {
+  static bool serialize(SPSOutputBuffer &OB,
+                        const tpctypes::RemoteAllocGroup &RAG) {
     uint8_t WireValue = 0;
-    if ((AG.getMemProt() & MemProt::Read) != MemProt::None)
+    if ((RAG.Prot & MemProt::Read) != MemProt::None)
       WireValue |= ReadBit;
-    if ((AG.getMemProt() & MemProt::Write) != MemProt::None)
+    if ((RAG.Prot & MemProt::Write) != MemProt::None)
       WireValue |= WriteBit;
-    if ((AG.getMemProt() & MemProt::Exec) != MemProt::None)
+    if ((RAG.Prot & MemProt::Exec) != MemProt::None)
       WireValue |= ExecBit;
-    if (AG.getMemDeallocPolicy() == MemDeallocPolicy::Finalize)
+    if (RAG.FinalizeLifetime)
       WireValue |= FinalizeBit;
     return SPSArgList<uint8_t>::serialize(OB, WireValue);
   }
 
-  static bool deserialize(SPSInputBuffer &IB, AllocGroup &AG) {
+  static bool deserialize(SPSInputBuffer &IB, tpctypes::RemoteAllocGroup &RAG) {
     uint8_t Val;
     if (!SPSArgList<uint8_t>::deserialize(IB, Val))
       return false;
     MemProt MP = MemProt::None;
     if (Val & ReadBit)
       MP |= MemProt::Read;
     if (Val & WriteBit)
       MP |= MemProt::Write;
     if (Val & ExecBit)
       MP |= MemProt::Exec;
-    MemDeallocPolicy MDP = (Val & FinalizeBit) ? MemDeallocPolicy::Finalize
-                                               : MemDeallocPolicy::Standard;
-    AG = AllocGroup(MP, MDP);
+    bool FinalizeLifetime = (Val & FinalizeBit) ? true : false;
+    RAG = {MP, FinalizeLifetime};
     return true;
   }
 };
 
 template <>
 class SPSSerializationTraits<SPSSegFinalizeRequest,
                              tpctypes::SegFinalizeRequest> {
   using SFRAL = SPSSegFinalizeRequest::AsArgList;
 
 public:
   static size_t size(const tpctypes::SegFinalizeRequest &SFR) {
-    return SFRAL::size(SFR.AG, SFR.Addr, SFR.Size, SFR.Content);
+    return SFRAL::size(SFR.RAG, SFR.Addr, SFR.Size, SFR.Content);
   }
 
   static bool serialize(SPSOutputBuffer &OB,
                         const tpctypes::SegFinalizeRequest &SFR) {
-    return SFRAL::serialize(OB, SFR.AG, SFR.Addr, SFR.Size, SFR.Content);
+    return SFRAL::serialize(OB, SFR.RAG, SFR.Addr, SFR.Size, SFR.Content);
   }
 
   static bool deserialize(SPSInputBuffer &IB,
                           tpctypes::SegFinalizeRequest &SFR) {
-    return SFRAL::deserialize(IB, SFR.AG, SFR.Addr, SFR.Size, SFR.Content);
+    return SFRAL::deserialize(IB, SFR.RAG, SFR.Addr, SFR.Size, SFR.Content);
   }
 };
 
 template <>
 class SPSSerializationTraits<SPSFinalizeRequest, tpctypes::FinalizeRequest> {
   using FRAL = SPSFinalizeRequest::AsArgList;
 
 public:

 
 template <>
 class SPSSerializationTraits<SPSSharedMemorySegFinalizeRequest,
                              tpctypes::SharedMemorySegFinalizeRequest> {
   using SFRAL = SPSSharedMemorySegFinalizeRequest::AsArgList;
 
 public:
   static size_t size(const tpctypes::SharedMemorySegFinalizeRequest &SFR) {
-    return SFRAL::size(SFR.AG, SFR.Addr, SFR.Size);
+    return SFRAL::size(SFR.RAG, SFR.Addr, SFR.Size);
   }
 
   static bool serialize(SPSOutputBuffer &OB,
                         const tpctypes::SharedMemorySegFinalizeRequest &SFR) {
-    return SFRAL::serialize(OB, SFR.AG, SFR.Addr, SFR.Size);
+    return SFRAL::serialize(OB, SFR.RAG, SFR.Addr, SFR.Size);
   }
 
   static bool deserialize(SPSInputBuffer &IB,
                           tpctypes::SharedMemorySegFinalizeRequest &SFR) {
-    return SFRAL::deserialize(IB, SFR.AG, SFR.Addr, SFR.Size);
+    return SFRAL::deserialize(IB, SFR.RAG, SFR.Addr, SFR.Size);
   }
 };
 
 template <>
 class SPSSerializationTraits<SPSSharedMemoryFinalizeRequest,
                              tpctypes::SharedMemoryFinalizeRequest> {
   using FRAL = SPSSharedMemoryFinalizeRequest::AsArgList;
 

llvm/lib/ExecutionEngine/JITLink/COFFLinkGraphBuilder.cpp

if ((*Sec)->Characteristics & COFF::IMAGE_SCN_MEM_EXECUTE)
       Prot |= orc::MemProt::Exec;
     if ((*Sec)->Characteristics & COFF::IMAGE_SCN_MEM_READ)
       Prot |= orc::MemProt::Read;
     if ((*Sec)->Characteristics & COFF::IMAGE_SCN_MEM_WRITE)
       Prot |= orc::MemProt::Write;
 
     // Look for existing sections first.
     auto *GraphSec = G->findSectionByName(SectionName);
-    if (!GraphSec)
+    if (!GraphSec) {
       GraphSec = &G->createSection(SectionName, Prot);
+      if ((*Sec)->Characteristics & COFF::IMAGE_SCN_LNK_REMOVE)
+        GraphSec->setMemLifetimePolicy(orc::MemLifetimePolicy::NoAlloc);
+    }
     if (GraphSec->getMemProt() != Prot)
       return make_error<JITLinkError>("MemProt should match");
 
     Block *B = nullptr;
     if ((*Sec)->Characteristics & COFF::IMAGE_SCN_CNT_UNINITIALIZED_DATA)
       B = &G->createZeroFillBlock(
           *GraphSec, getSectionSize(Obj, *Sec),
           orc::ExecutorAddr(getSectionAddress(Obj, *Sec)),

llvm/lib/ExecutionEngine/JITLink/ELFLinkGraphBuilder.h

if (isDwarfSection(*Name)) {
       LLVM_DEBUG({
         dbgs() << "    " << SecIndex << ": \"" << *Name
                << "\" is a debug section: "
                   "No graph section will be created.\n";
       });
       continue;
     }
 
-    // Skip non-SHF_ALLOC sections
-    if (!(Sec.sh_flags & ELF::SHF_ALLOC)) {
-      LLVM_DEBUG({
-        dbgs() << "    " << SecIndex << ": \"" << *Name
-               << "\" is not an SHF_ALLOC section: "
-                  "No graph section will be created.\n";
-      });
-      continue;
-    }
-
     LLVM_DEBUG({
       dbgs() << "    " << SecIndex << ": Creating section for \"" << *Name
              << "\"\n";
     });
 
     // Get the section's memory protection flags.
     orc::MemProt Prot;
     if (Sec.sh_flags & ELF::SHF_EXECINSTR)
       Prot = orc::MemProt::Read | orc::MemProt::Exec;
     else
       Prot = orc::MemProt::Read | orc::MemProt::Write;
 
     // Look for existing sections first.
     auto *GraphSec = G->findSectionByName(*Name);
-    if (!GraphSec)
+    if (!GraphSec) {
       GraphSec = &G->createSection(*Name, Prot);
+      // Non-SHF_ALLOC sections get NoAlloc memory lifetimes.
+      if (!(Sec.sh_flags & ELF::SHF_ALLOC)) {
+        GraphSec->setMemLifetimePolicy(orc::MemLifetimePolicy::NoAlloc);
+        LLVM_DEBUG({
+          dbgs() << "    " << SecIndex << ": \"" << *Name
+                 << "\" is not a SHF_ALLOC section. Using NoAlloc lifetime";
+        });
+      }
+      continue;
+    }
+
     assert(GraphSec->getMemProt() == Prot && "MemProt should match");
 
     Block *B = nullptr;
     if (Sec.sh_type != ELF::SHT_NOBITS) {
       auto Data = Obj.template getSectionContentsAsArray<char>(Sec);
       if (!Data)
         return Data.takeError();
 

llvm/lib/ExecutionEngine/JITLink/JITLinkGeneric.h

 private:
   const LinkerImpl &impl() const {
     return static_cast<const LinkerImpl &>(*this);
   }
 
   Error fixUpBlocks(LinkGraph &G) const override {
     LLVM_DEBUG(dbgs() << "Fixing up blocks:\n");
 
-    for (auto *B : G.blocks()) {
+    for (auto &Sec : G.sections()) {
+      bool NoAllocSection =
+          Sec.getMemLifetimePolicy() == orc::MemLifetimePolicy::NoAlloc;
+
+      for (auto *B : Sec.blocks()) {
         LLVM_DEBUG(dbgs() << "  " << *B << ":\n");
 
         // Copy Block data and apply fixups.
         LLVM_DEBUG(dbgs() << "    Applying fixups.\n");
         assert((!B->isZeroFill() || all_of(B->edges(),
                                            [](const Edge &E) {
                                              return E.getKind() ==
                                                     Edge::KeepAlive;
                                            })) &&
                "Non-KeepAlive edges in zero-fill block?");
+
+        // If this is a no-alloc section then copy the block content into
+        // memory allocated on the Graph's allocator (if it hasn't been
+        // already).
+        if (NoAllocSection)
+          (void)B->getMutableContent(G);
+
         for (auto &E : B->edges()) {
 
           // Skip non-relocation edges.
           if (!E.isRelocation())
             continue;
 
+          // If B is a block in a Standard or Finalize section then make sure
+          // that no edges point to symbols in NoAlloc sections.
+          assert(
+              (NoAllocSection || !E.getTarget().isDefined() ||
+               E.getTarget().getBlock().getSection().getMemLifetimePolicy() !=
+                   orc::MemLifetimePolicy::NoAlloc) &&
+              "Block in allocated section has edge pointing to no-alloc "
+              "section");
+
           // Dispatch to LinkerImpl for fixup.
           if (auto Err = impl().applyFixup(G, *B, E))
             return Err;
         }
       }
+    }
 
     return Error::success();
   }
 };
 
 /// Removes dead symbols/blocks/addressables.
 ///
 /// Finds the set of symbols and addressables reachable from any symbol

llvm/lib/ExecutionEngine/JITLink/JITLinkMemoryManager.cpp

 namespace jitlink {
 
 JITLinkMemoryManager::~JITLinkMemoryManager() = default;
 JITLinkMemoryManager::InFlightAlloc::~InFlightAlloc() = default;
 
 BasicLayout::BasicLayout(LinkGraph &G) : G(G) {
 
   for (auto &Sec : G.sections()) {
-    // Skip empty sections.
-    if (Sec.blocks().empty())
+    // Skip empty sections, and sections with NoAlloc lifetime policies.
+    if (Sec.blocks().empty() ||
+        Sec.getMemLifetimePolicy() == orc::MemLifetimePolicy::NoAlloc)
       continue;
 
-    auto &Seg = Segments[{Sec.getMemProt(), Sec.getMemDeallocPolicy()}];
+    auto &Seg = Segments[{Sec.getMemProt(), Sec.getMemLifetimePolicy()}];
     for (auto *B : Sec.blocks())
       if (LLVM_LIKELY(!B->isZeroFill()))
         Seg.ContentBlocks.push_back(B);
       else
         Seg.ZeroFillBlocks.push_back(B);
   }
 
   // Build Segments map.
for (auto &KV : segments()) {
     auto &AG = KV.first;
     auto &Seg = KV.second;
 
     if (Seg.Alignment > PageSize)
       return make_error<StringError>("Segment alignment greater than page size",
                                      inconvertibleErrorCode());
 
     uint64_t SegSize = alignTo(Seg.ContentSize + Seg.ZeroFillSize, PageSize);
-    if (AG.getMemDeallocPolicy() == orc::MemDeallocPolicy::Standard)
+    if (AG.getMemLifetimePolicy() == orc::MemLifetimePolicy::Standard)
       SegsSizes.StandardSegs += SegSize;
     else
       SegsSizes.FinalizeSegs += SegSize;
   }
 
   return SegsSizes;
 }
 

 orc::shared::AllocActions &BasicLayout::graphAllocActions() {
   return G.allocActions();
 }
 
 void SimpleSegmentAlloc::Create(JITLinkMemoryManager &MemMgr,
                                 const JITLinkDylib *JD, SegmentMap Segments,
                                 OnCreatedFunction OnCreated) {
 
-  static_assert(orc::AllocGroup::NumGroups == 16,
+  static_assert(orc::AllocGroup::NumGroups == 32,
                 "AllocGroup has changed. Section names below must be updated");
   StringRef AGSectionNames[] = {
       "__---.standard", "__R--.standard", "__-W-.standard", "__RW-.standard",
       "__--X.standard", "__R-X.standard", "__-WX.standard", "__RWX.standard",
       "__---.finalize", "__R--.finalize", "__-W-.finalize", "__RW-.finalize",
       "__--X.finalize", "__R-X.finalize", "__-WX.finalize", "__RWX.finalize"};
 
   auto G =
       std::make_unique<LinkGraph>("", Triple(), 0, support::native, nullptr);
   orc::AllocGroupSmallMap<Block *> ContentBlocks;
 
   orc::ExecutorAddr NextAddr(0x100000);
   for (auto &KV : Segments) {
     auto &AG = KV.first;
     auto &Seg = KV.second;
 
+    assert(AG.getMemLifetimePolicy() != orc::MemLifetimePolicy::NoAlloc &&
+           "NoAlloc segments are not supported by SimpleSegmentAlloc");
+
     auto AGSectionName =
         AGSectionNames[static_cast<unsigned>(AG.getMemProt()) |
-                       static_cast<bool>(AG.getMemDeallocPolicy()) << 3];
+                       static_cast<bool>(AG.getMemLifetimePolicy()) << 3];
 
     auto &Sec = G->createSection(AGSectionName, AG.getMemProt());
-    Sec.setMemDeallocPolicy(AG.getMemDeallocPolicy());
+    Sec.setMemLifetimePolicy(AG.getMemLifetimePolicy());
 
     if (Seg.ContentSize != 0) {
       NextAddr =
           orc::ExecutorAddr(alignTo(NextAddr.getValue(), Seg.ContentAlign));
       auto &B =
           G->createMutableContentBlock(Sec, G->allocateBuffer(Seg.ContentSize),
                                        NextAddr, Seg.ContentAlign.value(), 0);
       ContentBlocks[AG] = &B;
void InProcessMemoryManager::allocate(const JITLinkDylib *JD, LinkGraph &G,
   });
 
   // Build ProtMap, assign addresses.
   for (auto &KV : BL.segments()) {
     auto &AG = KV.first;
     auto &Seg = KV.second;
 
     auto &SegAddr =
-        (AG.getMemDeallocPolicy() == orc::MemDeallocPolicy::Standard)
+        (AG.getMemLifetimePolicy() == orc::MemLifetimePolicy::Standard)
             ? NextStandardSegAddr
             : NextFinalizeSegAddr;
 
     Seg.WorkingMem = SegAddr.toPtr<char *>();
     Seg.Addr = SegAddr;
 
     SegAddr += alignTo(Seg.ContentSize + Seg.ZeroFillSize, PageSize);
   }

llvm/lib/ExecutionEngine/JITLink/MachOLinkGraphBuilder.cpp

for (auto &SecRef : Obj.sections()) {
     else
       Prot = orc::MemProt::Read | orc::MemProt::Write;
 
     auto FullyQualifiedName =
         G->allocateContent(StringRef(NSec.SegName) + "," + NSec.SectName);
     NSec.GraphSection = &G->createSection(
         StringRef(FullyQualifiedName.data(), FullyQualifiedName.size()), Prot);
 
+    // TODO: Are there any other criteria for NoAlloc lifetime?
+    if (NSec.Flags & MachO::S_ATTR_DEBUG)
+      NSec.GraphSection->setMemLifetimePolicy(orc::MemLifetimePolicy::NoAlloc);
+
     IndexToSection.insert(std::make_pair(SecIndex, std::move(NSec)));
   }
 
   std::vector<NormalizedSection *> Sections;
   Sections.reserve(IndexToSection.size());
   for (auto &KV : IndexToSection)
     Sections.push_back(&KV.second);
 

llvm/lib/ExecutionEngine/Orc/EPCGenericRTDyldMemoryManager.cpp

std::vector<SectionAlloc> *SegSections[3] = {&SecAllocGroup.CodeAllocs,
                                                  &SecAllocGroup.RWDataAllocs};
 
     tpctypes::FinalizeRequest FR;
     std::unique_ptr<char[]> AggregateContents[3];
 
     for (unsigned I = 0; I != 3; ++I) {
       FR.Segments.push_back({});
       auto &Seg = FR.Segments.back();
-      Seg.AG = SegMemProts[I];
+      Seg.RAG = SegMemProts[I];
       Seg.Addr = RemoteAddrs[I]->Start;
       for (auto &SecAlloc : *SegSections[I]) {
         Seg.Size = alignTo(Seg.Size, SecAlloc.Align);
         Seg.Size += SecAlloc.Size;
       }
       AggregateContents[I] = std::make_unique<char[]>(Seg.Size);
       size_t SecOffset = 0;
       for (auto &SecAlloc : *SegSections[I]) {

llvm/lib/ExecutionEngine/Orc/MemoryMapper.cpp

void SharedMemoryMapper::initialize(MemoryMapper::AllocInfo &AI,
   FR.Segments.reserve(AI.Segments.size());
 
   for (auto Segment : AI.Segments) {
     char *Base = static_cast<char *>(Reservation->second.LocalAddr) +
                  AllocationOffset + Segment.Offset;
     std::memset(Base + Segment.ContentSize, 0, Segment.ZeroFillSize);
 
     tpctypes::SharedMemorySegFinalizeRequest SegReq;
-    SegReq.AG = Segment.AG;
+    SegReq.RAG = {Segment.AG.getMemProt(), Segment.AG.getMemLifetimePolicy() ==
+                                               MemLifetimePolicy::Finalize};
     SegReq.Addr = AI.MappingBase + Segment.Offset;
     SegReq.Size = Segment.ContentSize + Segment.ZeroFillSize;
 
     FR.Segments.push_back(SegReq);
   }
 
   EPC.callSPSWrapperAsync<
       rt::SPSExecutorSharedMemoryMapperServiceInitializeSignature>(

llvm/lib/ExecutionEngine/Orc/TargetProcess/ExecutorSharedMemoryMapperService.cpp

#if (defined(LLVM_ON_UNIX) && !defined(__ANDROID__)) || defined(_WIN32)
   // Contents are already in place
   for (auto &Segment : FR.Segments) {
     if (Segment.Addr < MinAddr)
       MinAddr = Segment.Addr;
 
 #if defined(LLVM_ON_UNIX)
 
     int NativeProt = 0;
-    if ((Segment.AG.getMemProt() & MemProt::Read) == MemProt::Read)
+    if ((Segment.RAG.Prot & MemProt::Read) == MemProt::Read)
       NativeProt |= PROT_READ;
-    if ((Segment.AG.getMemProt() & MemProt::Write) == MemProt::Write)
+    if ((Segment.RAG.Prot & MemProt::Write) == MemProt::Write)
       NativeProt |= PROT_WRITE;
-    if ((Segment.AG.getMemProt() & MemProt::Exec) == MemProt::Exec)
+    if ((Segment.RAG.Prot & MemProt::Exec) == MemProt::Exec)
       NativeProt |= PROT_EXEC;
 
     if (mprotect(Segment.Addr.toPtr<void *>(), Segment.Size, NativeProt))
       return errorCodeToError(std::error_code(errno, std::generic_category()));
 
 #elif defined(_WIN32)
 
-    DWORD NativeProt =
-        getWindowsProtectionFlags(Segment.AG.getMemProt());
+    DWORD NativeProt = getWindowsProtectionFlags(Segment.RAG.Prot);
 
     if (!VirtualProtect(Segment.Addr.toPtr<void *>(), Segment.Size, NativeProt,
                         &NativeProt))
       return errorCodeToError(mapWindowsError(GetLastError()));
 
 #endif
 
-    if ((Segment.AG.getMemProt() & MemProt::Exec) == MemProt::Exec)
+    if ((Segment.RAG.Prot & MemProt::Exec) == MemProt::Exec)
       sys::Memory::InvalidateInstructionCache(Segment.Addr.toPtr<void *>(),
                                               Segment.Size);
   }
 
   // Run finalization actions and get deinitlization action list.
   auto DeinitializeActions = shared::runFinalizeActions(FR.Actions);
   if (!DeinitializeActions) {
     return DeinitializeActions.takeError();

llvm/lib/ExecutionEngine/Orc/TargetProcess/SimpleExecutorMemoryManager.cpp

for (auto &Seg : FR.Segments) {
 
     char *Mem = Seg.Addr.toPtr<char *>();
     if (!Seg.Content.empty())
       memcpy(Mem, Seg.Content.data(), Seg.Content.size());
     memset(Mem + Seg.Content.size(), 0, Seg.Size - Seg.Content.size());
     assert(Seg.Size <= std::numeric_limits<size_t>::max());
     if (auto EC = sys::Memory::protectMappedMemory(
             {Mem, static_cast<size_t>(Seg.Size)},
-            toSysMemoryProtectionFlags(Seg.AG.getMemProt())))
+            toSysMemoryProtectionFlags(Seg.RAG.Prot)))
       return BailOut(errorCodeToError(EC));
-    if ((Seg.AG.getMemProt() & MemProt::Exec) == MemProt::Exec)
+    if ((Seg.RAG.Prot & MemProt::Exec) == MemProt::Exec)
       sys::Memory::InvalidateInstructionCache(Mem, Seg.Size);
   }
 
   // Run finalization actions.
   for (auto &ActPair : FR.Actions) {
     if (auto Err = ActPair.Finalize.runWithSPSRetErrorMerged())
       return BailOut(std::move(Err));
     ++SuccessfulFinalizationActions;

llvm/tools/llvm-jitlink/llvm-jitlink.cpp

public:
   }
 
   void initialize(AllocInfo &AI, OnInitializedFunction OnInitialized) override {
     // Slide mapping based on delta, make all segments read-writable, and
     // discard allocation actions.
     auto FixedAI = std::move(AI);
     FixedAI.MappingBase -= DeltaAddr;
     for (auto &Seg : FixedAI.Segments)
-      Seg.AG = AllocGroup(MemProt::Read | MemProt::Write,
-                          Seg.AG.getMemDeallocPolicy());
+      Seg.AG = {MemProt::Read | MemProt::Write, Seg.AG.getMemLifetimePolicy()};
     FixedAI.Actions.clear();
     InProcessMemoryMapper::initialize(
         FixedAI, [this, OnInitialized = std::move(OnInitialized)](
                      Expected<ExecutorAddr> Result) mutable {
           if (!Result)
             return OnInitialized(Result.takeError());
 
           OnInitialized(ExecutorAddr(Result->getValue() + DeltaAddr));

llvm/unittests/ExecutionEngine/JITLink/LinkGraphTests.cpp

TEST(LinkGraphTest, IsCStringBlockTest) {
   auto &LargerZeroFillBlock =
       G.createZeroFillBlock(Sec, 2, orc::ExecutorAddr(), 1, 0);
 
   EXPECT_TRUE(isCStringBlock(CStringBlock));
   EXPECT_FALSE(isCStringBlock(NotACStringBlock));
   EXPECT_TRUE(isCStringBlock(SizeOneZeroFillBlock));
   EXPECT_FALSE(isCStringBlock(LargerZeroFillBlock));
 }
+
+TEST(LinkGraphTest, BasicLayoutHonorsNoAlloc) {
+  // Check that BasicLayout honors NoAlloc.
+  LinkGraph G("foo", Triple("x86_64-apple-darwin"), 8, support::little,
+              getGenericEdgeKindName);
+
+  // Create a regular section and block.
+  auto &Sec1 =
+      G.createSection("__data", orc::MemProt::Read | orc::MemProt::Write);
+  G.createContentBlock(Sec1, BlockContent.slice(0, 8), orc::ExecutorAddr(), 8,
+                       0);
+
+  // Create a NoAlloc section and block.
+  auto &Sec2 =
+      G.createSection("__metadata", orc::MemProt::Read | orc::MemProt::Write);
+  Sec2.setMemLifetimePolicy(orc::MemLifetimePolicy::NoAlloc);
+  G.createContentBlock(Sec2, BlockContent.slice(0, 8), orc::ExecutorAddr(), 8,
+                       0);
+
+  BasicLayout BL(G);
+
+  EXPECT_EQ(std::distance(BL.segments().begin(), BL.segments().end()), 1U);
+  EXPECT_EQ(BL.segments().begin()->first,
+            orc::MemProt::Read | orc::MemProt::Write);
+  auto &SegInfo = BL.segments().begin()->second;
+  EXPECT_EQ(SegInfo.Alignment, 8U);
+  EXPECT_EQ(SegInfo.ContentSize, 8U);
+}

llvm/unittests/ExecutionEngine/Orc/EPCGenericJITLinkMemoryManagerTest.cpp

public:
   Error finalize(tpctypes::FinalizeRequest FR) {
     for (auto &Seg : FR.Segments) {
       char *Mem = Seg.Addr.toPtr<char *>();
       memcpy(Mem, Seg.Content.data(), Seg.Content.size());
       memset(Mem + Seg.Content.size(), 0, Seg.Size - Seg.Content.size());
       assert(Seg.Size <= std::numeric_limits<size_t>::max());
       if (auto EC = sys::Memory::protectMappedMemory(
               {Mem, static_cast<size_t>(Seg.Size)},
-              toSysMemoryProtectionFlags(Seg.AG.getMemProt())))
+              toSysMemoryProtectionFlags(Seg.RAG.Prot)))
         return errorCodeToError(EC);
-      if ((Seg.AG.getMemProt() & MemProt::Exec) != MemProt::Exec)
+      if ((Seg.RAG.Prot & MemProt::Exec) != MemProt::Exec)
         sys::Memory::InvalidateInstructionCache(Mem, Seg.Size);
     }
     return Error::success();
   }
 
   Error deallocate(std::vector<ExecutorAddr> &Bases) {
     Error Err = Error::success();
     for (auto &Base : Bases) {