[ELF] Add -z separate-loadable-segments to complement separate-code and noseparate-code

[MaskRay]

Summary

D64906 allows PT_LOAD to have overlapping p_offset ranges. In the
default R RX RW RW layout + -z noseparate-code case, we do not tail pad
segments when transiting to another segment. This can save at most
3*maxPageSize bytes.

a) Before D64906, we tail pad R, RX and the first RW.
b) With -z separate-code, we tail pad R and RX, but not the first RW (RELRO).

In some cases, b) saves one file page. In some cases, b) wastes one
virtual memory page. The waste is a concern on Fuchsia. Because it uses
compressed binaries, it doesn't benefit from the saved file page.

This patch adds -z separate-loadable-segments to restore the behavior before
D64906. It can affect section addresses and can thus be used as a
debugging mechanism (see PR43214 and ld.so partition bug in
crbug.com/998712).

[MaskRay]

ELF/Config.h

64 ↗

(On Diff #219854)

separate-segment -> separate-segments

Which one do you prefer?

[peter.smith]

ELF/Config.h

64 ↗

(On Diff #219854)

Given that there is generally only one PF_X segment separate-code being singular makes sense. There can be more than one segment involved here so personally I'd use the plural form of separate-segments. It may also be worth being more specific with "--separate-loadable-segments" as I think that is more self descriptive, if a bit long.

[MaskRay]

separate-segments -> separate-loadable-segments

ELF/Config.h

64 ↗

(On Diff #219854)

Makes sense. Thanks for the suggestion!

[jakehehrlich]

LGTM! Thanks for working on this.

[MaskRay]

@phosek @mcgrathr Is -z separate-loadable-segments a good name for you?

[MaskRay]

Rebase (bool enabled = ... has been deleted)

[MaskRay]

In D67481#1670879, @MaskRay wrote:

@phosek @mcgrathr Is -z separate-loadable-segments a good name for you?

@phosek @mcgrathr 🤔

[ruiu]

Can you add this option to the man page?

[MaskRay]

Update docs/ld.lld.1

[MaskRay]

ld.lld.1: use -compact .Bl and add .Pp (paragraph break)

[ruiu]

LGTM

[MaskRay]

ld.lld.1: fix display of execstack

lld/trunk/ELF/Config.h

 enum class SortSectionPolicy { Default, None, Alignment, Name, Priority };
 
 // For --target2
 enum class Target2Policy { Abs, Rel, GotRel };
 
 // For tracking ARM Float Argument PCS
 enum class ARMVFPArgKind { Default, Base, VFP, ToolChain };
 
+// For -z noseparate-code, -z separate-code and -z separate-loadable-segments.
+enum class SeparateSegmentKind { None, Code, Loadable };
+
 struct SymbolVersion {
   llvm::StringRef name;
   bool isExternCpp;
   bool hasWildcard;
 };
 
 // This struct contains symbols version definition that
 // can be found in version script if it is used for link.
struct Configuration {
   bool zKeepTextSectionPrefix;
   bool zNodefaultlib;
   bool zNodelete;
   bool zNodlopen;
   bool zNow;
   bool zOrigin;
   bool zRelro;
   bool zRodynamic;
-  bool zSeparateCode;
   bool zText;
   bool zRetpolineplt;
   bool zWxneeded;
   DiscardPolicy discard;
   ICFLevel icf;
   OrphanHandlingPolicy orphanHandling;
   SortSectionPolicy sortSection;
   StripPolicy strip;
   UnresolvedPolicy unresolvedSymbols;
   Target2Policy target2;
   ARMVFPArgKind armVFPArgs = ARMVFPArgKind::Default;
   BuildIdKind buildId = BuildIdKind::None;
+  SeparateSegmentKind zSeparate;
   ELFKind ekind = ELFNoneKind;
   uint16_t emachine = llvm::ELF::EM_NONE;
   llvm::Optional<uint64_t> imageBase;
   uint64_t commonPageSize;
   uint64_t maxPageSize;
   uint64_t mipsGotSize;
   uint64_t zStackSize;
   unsigned ltoPartitions;

lld/trunk/ELF/Driver.cpp

for (auto *arg : args.filtered_reverse(OPT_z)) {
     if (k1 == arg->getValue())
       return true;
     if (k2 == arg->getValue())
       return false;
   }
   return Default;
 }
 
+static SeparateSegmentKind getZSeparate(opt::InputArgList &args) {
+  for (auto *arg : args.filtered_reverse(OPT_z)) {
+    StringRef v = arg->getValue();
+    if (v == "noseparate-code")
+      return SeparateSegmentKind::None;
+    if (v == "separate-code")
+      return SeparateSegmentKind::Code;
+    if (v == "separate-loadable-segments")
+      return SeparateSegmentKind::Loadable;
+  }
+  return SeparateSegmentKind::None;
+}
+
 static bool isKnownZFlag(StringRef s) {
   return s == "combreloc" || s == "copyreloc" || s == "defs" ||
          s == "execstack" || s == "global" || s == "hazardplt" ||
          s == "ifunc-noplt" || s == "initfirst" || s == "interpose" ||
          s == "keep-text-section-prefix" || s == "lazy" || s == "muldefs" ||
-         s == "separate-code" || s == "nocombreloc" || s == "nocopyreloc" ||
-         s == "nodefaultlib" || s == "nodelete" || s == "nodlopen" ||
-         s == "noexecstack" || s == "nokeep-text-section-prefix" ||
-         s == "norelro" || s == "noseparate-code" || s == "notext" ||
-         s == "now" || s == "origin" || s == "relro" || s == "retpolineplt" ||
+         s == "separate-code" || s == "separate-loadable-segments" ||
+         s == "nocombreloc" || s == "nocopyreloc" || s == "nodefaultlib" ||
+         s == "nodelete" || s == "nodlopen" || s == "noexecstack" ||
+         s == "nokeep-text-section-prefix" || s == "norelro" ||
+         s == "noseparate-code" || s == "notext" || s == "now" ||
+         s == "origin" || s == "relro" || s == "retpolineplt" ||
          s == "rodynamic" || s == "text" || s == "undefs" || s == "wxneeded" ||
          s.startswith("common-page-size=") || s.startswith("max-page-size=") ||
          s.startswith("stack-size=");
 }
 
 // Report an error for an unknown -z option.
 static void checkZOptions(opt::InputArgList &args) {
   for (auto *arg : args.filtered(OPT_z))
static void readConfigs(opt::InputArgList &args) {
   config->zNodefaultlib = hasZOption(args, "nodefaultlib");
   config->zNodelete = hasZOption(args, "nodelete");
   config->zNodlopen = hasZOption(args, "nodlopen");
   config->zNow = getZFlag(args, "now", "lazy", false);
   config->zOrigin = hasZOption(args, "origin");
   config->zRelro = getZFlag(args, "relro", "norelro", true);
   config->zRetpolineplt = hasZOption(args, "retpolineplt");
   config->zRodynamic = hasZOption(args, "rodynamic");
-  config->zSeparateCode = getZFlag(args, "separate-code", "noseparate-code", false);
+  config->zSeparate = getZSeparate(args);
   config->zStackSize = args::getZOptionValue(args, OPT_z, "stack-size", 0);
   config->zText = getZFlag(args, "text", "notext", true);
   config->zWxneeded = hasZOption(args, "wxneeded");
 
   // Parse LTO options.
   if (auto *arg = args.getLastArg(OPT_plugin_opt_mcpu_eq))
     parseClangOption(saver.save("-mcpu=" + StringRef(arg->getValue())),
                      arg->getSpelling());

lld/trunk/ELF/Writer.cpp

template <class ELFT> void Writer<ELFT>::run() {
   if (errorCount())
     return;
   // Write the result down to a file.
   openFile();
   if (errorCount())
     return;
 
   if (!config->oFormatBinary) {
+    if (config->zSeparate != SeparateSegmentKind::None)
       writeTrapInstr();
     writeHeader();
     writeSections();
   } else {
     writeSectionsBinary();
   }
 
   // Backfill .note.gnu.build-id section content. This is done at last
   // because the content is usually a hash value of the entire output file.
if (cmd && !cmd->addrExpr) {
       // the next maximum page size boundary on transitions between executable
       // and non-executable segments.
       //
       // SHT_LLVM_PART_EHDR marks the start of a partition. The partition
       // sections will be extracted to a separate file. Align to the next
       // maximum page size boundary so that we can find the ELF header at the
       // start. We cannot benefit from overlapping p_offset ranges with the
       // previous segment anyway.
-      if ((config->zSeparateCode && prev &&
+      if (config->zSeparate == SeparateSegmentKind::Loadable ||
+          (config->zSeparate == SeparateSegmentKind::Code && prev &&
            (prev->p_flags & PF_X) != (p->p_flags & PF_X)) ||
           cmd->type == SHT_LLVM_PART_EHDR)
         cmd->addrExpr = [] {
           return alignTo(script->getDot(), config->maxPageSize);
         };
       // PT_TLS is at the start of the first RW PT_LOAD. If `p` includes PT_TLS,
       // it must be the RW. Align to p_align(PT_TLS) to make sure
       // p_vaddr(PT_LOAD)%p_align(PT_LOAD) = 0. Otherwise, if
for (PhdrEntry *p : part.phdrs)
         lastRX = p;
 
   for (OutputSection *sec : outputSections) {
     off = setFileOffset(sec, off);
 
     // If this is a last section of the last executable segment and that
     // segment is the last loadable segment, align the offset of the
     // following section to avoid loading non-segments parts of the file.
-    if (config->zSeparateCode && lastRX && lastRX->lastSec == sec)
+    if (config->zSeparate != SeparateSegmentKind::None && lastRX &&
+        lastRX->lastSec == sec)
       off = alignTo(off, config->commonPageSize);
   }
 
   sectionHeaderOff = alignTo(off, config->wordsize);
   fileSize = sectionHeaderOff + (outputSections.size() + 1) * sizeof(Elf_Shdr);
 
   // Our logic assumes that sections have rising VA within the same segment.
   // With use of linker scripts it is possible to violate this rule and get file

 
 // Fill the last page of executable segments with trap instructions
 // instead of leaving them as zero. Even though it is not required by any
 // standard, it is in general a good thing to do for security reasons.
 //
 // We'll leave other pages in segments as-is because the rest will be
 // overwritten by output sections.
 template <class ELFT> void Writer<ELFT>::writeTrapInstr() {
-  if (!config->zSeparateCode)
-    return;
-
   for (Partition &part : partitions) {
     // Fill the last page.
     for (PhdrEntry *p : part.phdrs)
       if (p->p_type == PT_LOAD && (p->p_flags & PF_X))
         fillTrap(Out::bufferStart + alignDown(p->firstSec->offset + p->p_filesz,
                                               config->commonPageSize),
                  Out::bufferStart + alignTo(p->firstSec->offset + p->p_filesz,
                                             config->commonPageSize));

lld/trunk/docs/ld.lld.1

 .It Fl -warn-unresolved-symbols
 Report unresolved symbols as warnings.
 .It Fl -whole-archive
 Force load of all members in a static library.
 .It Fl -wrap Ns = Ns Ar symbol
 Use wrapper functions for symbol.
 .It Fl z Ar option
 Linker option extensions.
-.Bl -tag -width indent
+.Bl -tag -width indent -compact
+.Pp
 .It Cm execstack
 Make the main stack executable.
 Stack permissions are recorded in the
 .Dv PT_GNU_STACK
 segment.
+.Pp
 .It Cm global
 Sets the
 .Dv DF_1_GLOBAL flag in the
 .Dv DYNAMIC
 section.
 Different loaders can decide how to handle this flag on their own.
+.Pp
 .It Cm ifunc-noplt
 Do not emit PLT entries for ifunc symbols.
 Instead, emit text relocations referencing the resolver.
 This is an experimental optimization and only suitable for standalone
 environments where text relocations do not have the usual drawbacks.
 This option must be combined with the
 .Fl z Li notext
 option.
+.Pp
 .It Cm initfirst
 Sets the
 .Dv DF_1_INITFIRST
 flag to indicate the module should be initialized first.
+.Pp
 .It Cm interpose
 Set the
 .Dv DF_1_INTERPOSE
 flag to indicate to the runtime linker that the object is an interposer.
 During symbol resolution interposers are searched after the application
 but before other dependencies.
+.Pp
 .It Cm muldefs
 Do not error if a symbol is defined multiple times.
 The first definition will be used.
 This is a synonym for
 .Fl -allow-multiple-definition.
+.Pp
 .It Cm nocombreloc
 Disable combining and sorting multiple relocation sections.
+.Pp
 .It Cm nocopyreloc
 Disable the creation of copy relocations.
+.Pp
 .It Cm nodefaultlib
 Set the
 .Dv DF_1_NODEFLIB
 flag to indicate that default library search paths should be ignored.
+.Pp
 .It Cm nodelete
 Set the
 .Dv DF_1_NODELETE
 flag to indicate that the object cannot be unloaded from a process.
+.Pp
 .It Cm nodlopen
 Set the
 .Dv DF_1_NOOPEN
 flag to indicate that the object may not be opened by
 .Xr dlopen 3 .
+.Pp
 .It Cm norelro
 Do not indicate that portions of the object shold be mapped read-only
 after initial relocation processing.
 The object will omit the
 .Dv PT_GNU_RELRO
 segment.
+.Pp
 .It Cm notext
 Allow relocations against read-only segments.
 Sets the
 .Dv DT_TEXTREL flag in the
 .Dv DYNAMIC
 section.
+.Pp
 .It Cm now
 Set the
 .Dv DF_BIND_NOW
 flag to indicate that the run-time loader should perform all relocation
 processing as part of object initialization.
 By default relocations may be performed on demand.
+.Pp
 .It Cm origin
 Set the
 .Dv DF_ORIGIN
 flag to indicate that the object requires
 $ORIGIN
 processing.
+.Pp
 .It Cm retpolineplt
 Emit retpoline format PLT entries as a mitigation for CVE-2017-5715.
+.Pp
 .It Cm rodynamic
 Make the
 .Li .dynamic
 section read-only.
 The
 .Dv DT_DEBUG
 tag will not be emitted.
+.Pp
+.It Cm separate-loadable-segments
+.It Cm separate-code
+.It Cm noseparate-code
+Specify whether two adjacent PT_LOAD segments are allowed to overlap in pages.
+.Cm noseparate-code
+(default) allows overlap.
+.Cm separate-code
+allows overlap between two executable segments, or two non-executable segments.
+.Cm separate-loadable-segments
+disallows overlap.
+.Pp
 .It Cm stack-size Ns = Ns Ar size
 Set the main thread's stack size to
 .Ar size .
 The stack size is recorded as the size of the
 .Ar size .
 .Dv PT_GNU_STACK
 program segment.
+.Pp
 .It Cm text
 Do not allow relocations against read-only segments.
 This is the default.
+.Pp
 .It Cm wxneeded
 Create a
 .Dv PT_OPENBSD_WXNEEDED
 segment.
 .El
 .El
 .Sh IMPLEMENTATION NOTES
 .Nm Ap s

lld/trunk/test/ELF/fill-trap.s

 # REQUIRES: x86
 # RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t.o
 
 ## -z noseparate-code is the default: text segment is not tail padded.
 # RUN: ld.lld %t.o -o %t
 # RUN: llvm-readobj -l %t | FileCheck %s --check-prefixes=CHECK,NOPAD
 # RUN: ld.lld %t.o -z noseparate-code -o %t
 # RUN: llvm-readobj -l %t | FileCheck %s --check-prefixes=CHECK,NOPAD
 
 ## -z separate-code pads the tail of text segment with traps.
 # RUN: ld.lld %t.o -z separate-code -o %t
 # RUN: llvm-readobj -l %t | FileCheck %s --check-prefixes=CHECK,PAD
 # RUN: od -Ax -x -N16 -j0x1ff0 %t | FileCheck %s --check-prefix=FILL
 
+## -z separate-loadable-segments pads all segments, including the text segment.
+# RUN: ld.lld %t.o -z separate-loadable-segments -o %t
+# RUN: llvm-readobj -l %t | FileCheck %s --check-prefixes=CHECK,PAD
+# RUN: od -Ax -x -N16 -j0x1ff0 %t | FileCheck %s --check-prefix=FILL
+
 # RUN: ld.lld %t.o -z separate-code -z noseparate-code -o %t
 # RUN: llvm-readobj -l %t | FileCheck %s --check-prefixes=CHECK,NOPAD
 
 # CHECK: ProgramHeader {
 # CHECK:   Type: PT_LOAD
 # PAD:     Offset: 0x1000
 # NOPAD:   Offset: 0x120
 # CHECK-NEXT:   VirtualAddress:

lld/trunk/test/ELF/separate-segments.s

+# REQUIRES: x86
+# RUN: llvm-mc -filetype=obj -triple=x86_64 %s -o %t.o
+
+## -z noseparate-code is the default. All PT_LOAD can have overlapping p_offset
+## ranges at runtime.
+# RUN: ld.lld -pie %t.o -o %t
+# RUN: llvm-readelf -l %t | FileCheck --check-prefix=NONE %s
+# NONE:      LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x000245 0x000245 R   0x1000
+# NONE-NEXT: LOAD 0x000248 0x0000000000001248 0x0000000000001248 0x000001 0x000001 R E 0x1000
+# NONE-NEXT: LOAD 0x000250 0x0000000000002250 0x0000000000002250 0x000080 0x000080 RW  0x1000
+# NONE-NEXT: LOAD 0x0002d0 0x00000000000032d0 0x00000000000032d0 0x000001 0x000001 RW  0x1000
+
+## -z separate-code makes text segment (RX) separate.
+## The two RW can have overlapping p_offset ranges at runtime.
+# RUN: ld.lld -pie %t.o -z separate-code -o %t
+# RUN: llvm-readelf -l %t | FileCheck --check-prefix=CODE %s
+# CODE:      LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x000245 0x000245 R   0x1000
+# CODE-NEXT: LOAD 0x001000 0x0000000000001000 0x0000000000001000 0x000001 0x000001 R E 0x1000
+# CODE-NEXT: LOAD 0x002000 0x0000000000002000 0x0000000000002000 0x000080 0x000080 RW  0x1000
+# CODE-NEXT: LOAD 0x002080 0x0000000000003080 0x0000000000003080 0x000001 0x000001 RW  0x1000
+
+## -z separate-loadable-segments makes all segments separate.
+# RUN: ld.lld -pie %t.o -z separate-loadable-segments -o %t
+# RUN: llvm-readelf -l %t | FileCheck --check-prefix=ALL %s
+# ALL:       LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x000245 0x000245 R   0x1000
+# ALL-NEXT:  LOAD 0x001000 0x0000000000001000 0x0000000000001000 0x000001 0x000001 R E 0x1000
+# ALL-NEXT:  LOAD 0x002000 0x0000000000002000 0x0000000000002000 0x000080 0x000080 RW  0x1000
+# ALL-NEXT:  LOAD 0x003000 0x0000000000003000 0x0000000000003000 0x000001 0x000001 RW  0x1000
+
+nop
+
+.data
+.byte 0