[ELF] Use SHF_SUNW_NODISCARD instead of SHF_GNU_RETAIN on Solaris

[ro]

Summary

Instead of the GNU extension SHF_GNU_RETAIN, Solaris provides equivalent
functionality with SHF_SUNW_NODISCARD. This patch implements the necessary
support.

Tested on sparcv9-sun-solaris2.11, amd64-pc-solaris2.11, and x86_64-pc-linux-gnu.

The patch passes existing tests, although I suspect something might still be missing.
I've inspected elf-retain.o with both Solaris' native elfdump and llvm-readelf to
check that the SHF_SUNW_NODISCARD flag is indeed present.

One caveat is that, unlike Solaris, Illumos and derivatives don't support
SHF_SUNW_NODISCARD. Unfortunately, there's currently no way to distinguish
the two related OSes and the Illumos community barely reacted to my attempts to
reach out to them.

[MaskRay]

The convention is to add lib/Object, llvm-readobj, yaml2obj, obj2yaml changes (dumper and yaml2obj) in one patch. Example: D107949. Also see needed tests there.

MC and CodeGen/TargetLoweringObjectFileImpl.cpp are less close to the above files and should be placed in a separate patch.

One caveat is that, unlike Solaris, Illumos and derivatives don't support SHF_SUNW_NODISCARD. Unfortunately, there's currently no way to distinguish the two related OSes and the Illumos community barely reacted to my attempts to reach out to them.

The ELF spirit is to ignore what the implementation don't recognize. A redundant unrecognized section flag is usually fine.

If there is a potential Illumos compatibility problem, it's clearly on their side and you are not expected to do more.

[MaskRay]

Ping for more tests:)

Request changes so that this is out of others' and my review queue.

[ro]

I've finally gotten around to this: MC and CodeGen parts split off as D120318.

[ro]

• Split off MC and CodeGen parts as D120318.
• Add testcases.

[MaskRay]

llvm/test/tools/llvm-readobj/ELF/section-flags-solaris.test

29

align:

- Name:    .os.flags.low
  Type:    SHT_PROGBITS
  ShFlags: 0x00100000

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

20

remove excessive padding while keep aligning.

llvm/tools/llvm-readobj/ELFDumper.cpp

1249

Fix clang-format lints

[ro]

llvm/tools/llvm-readobj/ELFDumper.cpp

1249

I'd rather not: clang-format-diff.py makes a total mess here: with LLVM 14 clang-format, the block for the gABI flags is changed like this:

 const EnumEntry<unsigned> ElfSectionFlags[] = {
-  ENUM_ENT(SHF_WRITE,            "W"),
-  ENUM_ENT(SHF_ALLOC,            "A"),
-  ENUM_ENT(SHF_EXECINSTR,        "X"),
-  ENUM_ENT(SHF_MERGE,            "M"),
-  ENUM_ENT(SHF_STRINGS,          "S"),
-  ENUM_ENT(SHF_INFO_LINK,        "I"),
-  ENUM_ENT(SHF_LINK_ORDER,       "L"),
-  ENUM_ENT(SHF_OS_NONCONFORMING, "O"),
-  ENUM_ENT(SHF_GROUP,            "G"),
-  ENUM_ENT(SHF_TLS,              "T"),
-  ENUM_ENT(SHF_COMPRESSED,       "C"),
-  ENUM_ENT(SHF_EXCLUDE,          "E"),
+    ENUM_ENT(SHF_WRITE, "W"),      ENUM_ENT(SHF_ALLOC, "A"),
+    ENUM_ENT(SHF_EXECINSTR, "X"),  ENUM_ENT(SHF_MERGE, "M"),
+    ENUM_ENT(SHF_STRINGS, "S"),    ENUM_ENT(SHF_INFO_LINK, "I"),
+    ENUM_ENT(SHF_LINK_ORDER, "L"), ENUM_ENT(SHF_OS_NONCONFORMING, "O"),
+    ENUM_ENT(SHF_GROUP, "G"),      ENUM_ENT(SHF_TLS, "T"),
+    ENUM_ENT(SHF_COMPRESSED, "C"), ENUM_ENT(SHF_EXCLUDE, "E"),
 };

and the next two indented further, making them inconsistent with the following ones.

[ro]

Whitespace fixes.

[MaskRay]

LGTM. @jhenderson may have further opinions.

[jhenderson]

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

2

It might be interesting to show what happens if you use SHF_GNU_RETAIN on Solaris and SHF_SUNW_NODISCARD on non-Solaris.

llvm/tools/llvm-readobj/ELFDumper.cpp

1249

You should be able to format the new enum entry lists at least though.

[ro]

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

2

Excellent point: the first silently creates what Solaris elfdump identifies as SHF_SUNW_ABSENT, but should be rejected. The second at least errors out:

YAML:68:14: error: unknown bit value
    Flags: [ SHF_SUNW_NODISCARD ]
             ^~~~~~~~~~~~~~~~~~~

although the error message could be clearer.

llvm/tools/llvm-readobj/ELFDumper.cpp

1249

Right, fixed. Indentation is still confusing, though: aligned for e.g. ElfSectionFlags, just one space for the single-entry arrays. Oh well.

[ro]

• Formatting fixes.
• Improve/test error handling.

[jhenderson]

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

47

It would be worth including the context saying which flag is the unknown value.

63

Ditto. Also add a space to match the earlier veresion.

74

Nit: too many new lines at EOF (need exactly one).

[ro]

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

47

Good point indeed. I've indented the Flags: part, matching what yaml2obj does. Is this ok or is the output supposed to be aligned in the testcase?

[ro]

• Whitespace fixes.
• Tighten error checking.

[jhenderson]

LGTM.

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

47

It doesn't really matter in this case. I'd probably keep the indentation to a minimum - see my inline edit for a suggestion.

47–48

[ro]

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

47

Thanks, I went for that: the indentation doesn't convey useful information in this case.

llvm/include/llvm/BinaryFormat/ELF.h

enum : unsigned {
 
   // This section is excluded from the final executable or shared library.
   SHF_EXCLUDE = 0x80000000U,
 
   // Start of target-specific flags.
 
   SHF_MASKOS = 0x0ff00000,
 
+  // Solaris equivalent of SHF_GNU_RETAIN.
+  SHF_SUNW_NODISCARD = 0x00100000,
+
   // Bits indicating processor-specific flags.
   SHF_MASKPROC = 0xf0000000,
 
   /// All sections with the "d" flag are grouped together by the linker to form
   /// the data section and the dp register is set to the start of the section by
   /// the boot code.
   XCORE_SHF_DP_SECTION = 0x10000000,
 

llvm/include/llvm/ObjectYAML/ELFYAML.h

struct Object {
 
   const SectionHeaderTable &getSectionHeaderTable() const {
     for (const std::unique_ptr<Chunk> &C : Chunks)
       if (auto *S = dyn_cast<ELFYAML::SectionHeaderTable>(C.get()))
         return *S;
     llvm_unreachable("the section header table chunk must always be present");
   }
 
+  ELF_ELFOSABI getOSAbi() const;
   unsigned getMachine() const;
 };
 
 bool shouldAllocateFileSpace(ArrayRef<ProgramHeader> Phdrs,
                              const NoBitsSection &S);
 
 } // end namespace ELFYAML
 } // end namespace llvm

llvm/lib/ObjectYAML/ELFYAML.cpp

 #include <cassert>
 #include <cstdint>
 
 namespace llvm {
 
 ELFYAML::Chunk::~Chunk() = default;
 
 namespace ELFYAML {
+ELF_ELFOSABI Object::getOSAbi() const { return Header.OSABI; }
+
 unsigned Object::getMachine() const {
   if (Header.Machine)
     return *Header.Machine;
   return llvm::ELF::EM_NONE;
 }
 
 constexpr StringRef SectionHeaderTable::TypeStr;
 } // namespace ELFYAML
#define BCase(X) IO.bitSetCase(Value, #X, ELF::X)
   BCase(SHF_MERGE);
   BCase(SHF_STRINGS);
   BCase(SHF_INFO_LINK);
   BCase(SHF_LINK_ORDER);
   BCase(SHF_OS_NONCONFORMING);
   BCase(SHF_GROUP);
   BCase(SHF_TLS);
   BCase(SHF_COMPRESSED);
+  switch (Object->getOSAbi()) {
+  case ELF::ELFOSABI_SOLARIS:
+    BCase(SHF_SUNW_NODISCARD);
+    break;
+  default:
     BCase(SHF_GNU_RETAIN);
+    break;
+  }
   switch (Object->getMachine()) {
   case ELF::EM_ARM:
     BCase(SHF_ARM_PURECODE);
     break;
   case ELF::EM_HEXAGON:
     BCase(SHF_HEX_GPREL);
     break;
   case ELF::EM_MIPS:

llvm/test/tools/llvm-readobj/ELF/section-flags-solaris.test

+## Here we test how Solaris specific flags are dumped.
+
+# RUN: yaml2obj %s -o %t.o
+# RUN: llvm-readobj -S %t.o | FileCheck %s
+
+# CHECK:      Name: .os.flags.low
+# CHECK-NEXT: Type: SHT_PROGBITS
+# CHECK-NEXT: Flags [ (0x100000)
+# CHECK-NEXT:   SHF_SUNW_NODISCARD (0x100000)
+# CHECK-NEXT: ]
+# CHECK:      Name: .os.flags.high
+# CHECK-NEXT: Type: SHT_PROGBITS
+# CHECK-NEXT: Flags [ (0xFE00000)
+# CHECK-NEXT: ]
+# CHECK:      Name: .os.flags.mask
+# CHECK-NEXT: Type: SHT_PROGBITS
+# CHECK-NEXT: Flags [ (0xFF00000)
+# CHECK-NEXT:   SHF_SUNW_NODISCARD (0x100000)
+# CHECK-NEXT: ]
+
+--- !ELF
+FileHeader:
+  Class: ELFCLASS64
+  Data:  ELFDATA2LSB
+  OSABI: ELFOSABI_SOLARIS
+  Type:  ET_REL
+Sections:
+  - Name:    .os.flags.low
+    Type:    SHT_PROGBITS
+    ShFlags: 0x00100000
+  - Name:    .os.flags.high
+    Type:    SHT_PROGBITS
+    ShFlags: 0x0FE00000
+  - Name:    .os.flags.mask
+    Type:    SHT_PROGBITS
+    ShFlags: 0x0FF00000

llvm/test/tools/obj2yaml/ELF/retain-section.yaml

+## Check handling of SHF_GNU_RETAIN and SHF_SUNW_NODISCARD section flags.
+
+# RUN: yaml2obj --docnum=1 %s -o %t1
+# RUN: obj2yaml %t1 | FileCheck %s --check-prefix=GNU
+
+# GNU:      Name: .gnu.retain
+# GNU-NEXT: Type: SHT_PROGBITS
+# GNU-NEXT: Flags: [ SHF_GNU_RETAIN ]
+
+--- !ELF
+FileHeader:
+  Class: ELFCLASS64
+  Data:  ELFDATA2LSB
+  Type:  ET_REL
+Sections:
+  - Name:  .gnu.retain
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_GNU_RETAIN ]
+
+# RUN: yaml2obj --docnum=2 %s -o %t2
+# RUN: obj2yaml %t2 | FileCheck %s --check-prefix=SOLARIS
+
+# SOLARIS:      Name: .sunw.nodiscard
+# SOLARIS-NEXT: Type: SHT_PROGBITS
+# SOLARIS-NEXT: Flags: [ SHF_SUNW_NODISCARD ]
+
+--- !ELF
+FileHeader:
+  Class: ELFCLASS64
+  Data:  ELFDATA2LSB
+  OSABI: ELFOSABI_SOLARIS
+  Type:  ET_REL
+Sections:
+  - Name:  .sunw.nodiscard
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_SUNW_NODISCARD ]

llvm/test/tools/yaml2obj/ELF/retain-section.yaml

+## Check how yaml2obj handles retain (SHF_GNU_RETAIN and
+## SHF_SUNW_NODISCARD) section flags.
+
+# RUN: yaml2obj --docnum=1 %s -o %t1
+# RUN: llvm-readobj --sections %t1 | FileCheck %s --check-prefix=GNU
+
+# GNU:      Name: .gnu.retain
+# GNU-NEXT: Type: SHT_PROGBITS (0x1)
+# GNU-NEXT: Flags [ (0x200000)
+# GNU-NEXT:   SHF_GNU_RETAIN (0x200000)
+# GNU-NEXT: ]
+
+--- !ELF
+FileHeader:
+  Class: ELFCLASS64
+  Data:  ELFDATA2LSB
+  Type:  ET_REL
+Sections:
+  - Name:  .gnu.retain
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_GNU_RETAIN ]
+
+# RUN: yaml2obj --docnum=2 %s -o %t2
+# RUN: llvm-readobj --sections %t2 | FileCheck %s --check-prefix=SOLARIS
+
+# SOLARIS:      Name: .sunw.nodiscard
+# SOLARIS-NEXT: Type: SHT_PROGBITS (0x1)
+# SOLARIS-NEXT: Flags [ (0x100000)
+# SOLARIS-NEXT:   SHF_SUNW_NODISCARD (0x100000)
+# SOLARIS-NEXT: ]
+
+--- !ELF
+FileHeader:
+  Class: ELFCLASS64
+  Data:  ELFDATA2LSB
+  OSABI: ELFOSABI_SOLARIS
+  Type:  ET_REL
+Sections:
+  - Name:  .sunw.nodiscard
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_SUNW_NODISCARD ]
+
+# RUN: not yaml2obj --docnum=3 %s 2>&1 | \
+# RUN:   FileCheck %s --check-prefix=SOLARIS-GNU-ERR
+
+# SOLARIS-GNU-ERR:      error: unknown bit value
+# SOLARIS-GNU-ERR-NEXT: Flags: [ SHF_GNU_RETAIN ]
+
+--- !ELF
+FileHeader:
+  Class: ELFCLASS64
+  Data:  ELFDATA2LSB
+  OSABI: ELFOSABI_SOLARIS
+  Type:  ET_REL
+Sections:
+  - Name:  .sunw.retain
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_GNU_RETAIN ]
+
+# RUN: not yaml2obj --docnum=4 %s 2>&1 | \
+# RUN:   FileCheck %s --check-prefix=GNU-SOLARIS-ERR
+
+# GNU-SOLARIS-ERR:      error: unknown bit value
+# GNU-SOLARIS-ERR-NEXT: Flags: [ SHF_SUNW_NODISCARD ]
+
+--- !ELF
+FileHeader:
+  Class: ELFCLASS64
+  Data:  ELFDATA2LSB
+  Type:  ET_REL
+Sections:
+  - Name:  .gnu.nodiscard
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_SUNW_NODISCARD ]

llvm/tools/llvm-readobj/ELFDumper.cpp

const EnumEntry<unsigned> ElfSectionFlags[] = {
   ENUM_ENT(SHF_MERGE,            "M"),
   ENUM_ENT(SHF_STRINGS,          "S"),
   ENUM_ENT(SHF_INFO_LINK,        "I"),
   ENUM_ENT(SHF_LINK_ORDER,       "L"),
   ENUM_ENT(SHF_OS_NONCONFORMING, "O"),
   ENUM_ENT(SHF_GROUP,            "G"),
   ENUM_ENT(SHF_TLS,              "T"),
   ENUM_ENT(SHF_COMPRESSED,       "C"),
-  ENUM_ENT(SHF_GNU_RETAIN,       "R"),
   ENUM_ENT(SHF_EXCLUDE,          "E"),
 };
 
+const EnumEntry<unsigned> ElfGNUSectionFlags[] = {
+  ENUM_ENT(SHF_GNU_RETAIN, "R")
+};
+
+const EnumEntry<unsigned> ElfSolarisSectionFlags[] = {
+  ENUM_ENT(SHF_SUNW_NODISCARD, "R")
+};
+
 const EnumEntry<unsigned> ElfXCoreSectionFlags[] = {
   ENUM_ENT(XCORE_SHF_CP_SECTION, ""),
   ENUM_ENT(XCORE_SHF_DP_SECTION, "")
 };
 
 const EnumEntry<unsigned> ElfARMSectionFlags[] = {
   ENUM_ENT(SHF_ARM_PURECODE, "y")
 };
const EnumEntry<unsigned> ElfMipsSectionFlags[] = {
   ENUM_ENT(SHF_MIPS_STRING,  "")
 };
 
 const EnumEntry<unsigned> ElfX86_64SectionFlags[] = {
   ENUM_ENT(SHF_X86_64_LARGE, "l")
 };
 
 static std::vector<EnumEntry<unsigned>>
-getSectionFlagsForTarget(unsigned EMachine) {
+getSectionFlagsForTarget(unsigned EOSAbi, unsigned EMachine) {
   std::vector<EnumEntry<unsigned>> Ret(std::begin(ElfSectionFlags),
                                        std::end(ElfSectionFlags));
+  switch (EOSAbi) {
+  case ELFOSABI_SOLARIS:
+    Ret.insert(Ret.end(), std::begin(ElfSolarisSectionFlags),
+               std::end(ElfSolarisSectionFlags));
+    break;
+  default:
+    Ret.insert(Ret.end(), std::begin(ElfGNUSectionFlags),
+               std::end(ElfGNUSectionFlags));
+    break;
+  }
   switch (EMachine) {
   case EM_ARM:
     Ret.insert(Ret.end(), std::begin(ElfARMSectionFlags),
                std::end(ElfARMSectionFlags));
     break;
   case EM_HEXAGON:
     Ret.insert(Ret.end(), std::begin(ElfHexagonSectionFlags),
                std::end(ElfHexagonSectionFlags));
Ret.insert(Ret.end(), std::begin(ElfXCoreSectionFlags),
                std::end(ElfXCoreSectionFlags));
     break;
   default:
     break;
   }
   return Ret;
 }
 
-static std::string getGNUFlags(unsigned EMachine, uint64_t Flags) {
+static std::string getGNUFlags(unsigned EOSAbi, unsigned EMachine,
+                               uint64_t Flags) {
   // Here we are trying to build the flags string in the same way as GNU does.
   // It is not that straightforward. Imagine we have sh_flags == 0x90000000.
   // SHF_EXCLUDE ("E") has a value of 0x80000000 and SHF_MASKPROC is 0xf0000000.
   // GNU readelf will not print "E" or "Ep" in this case, but will print just
   // "p". It only will print "E" when no other processor flag is set.
   std::string Str;
   bool HasUnknownFlag = false;
   bool HasOSFlag = false;
   bool HasProcFlag = false;
   std::vector<EnumEntry<unsigned>> FlagsList =
-      getSectionFlagsForTarget(EMachine);
+      getSectionFlagsForTarget(EOSAbi, EMachine);
   while (Flags) {
     // Take the least significant bit as a flag.
     uint64_t Flag = Flags & -Flags;
     Flags -= Flag;
 
     // Find the flag in the known flags list.
     auto I = llvm::find_if(FlagsList, [=](const EnumEntry<unsigned> &E) {
       // Flags with empty names are not printed in GNU style output.
else
           this->FileName, this->Obj.getSectionName(Sec, SecStrTable)));
     Fields[2].Str =
         getSectionTypeString(this->Obj.getHeader().e_machine, Sec.sh_type);
     Fields[3].Str =
         to_string(format_hex_no_prefix(Sec.sh_addr, ELFT::Is64Bits ? 16 : 8));
     Fields[4].Str = to_string(format_hex_no_prefix(Sec.sh_offset, 6));
     Fields[5].Str = to_string(format_hex_no_prefix(Sec.sh_size, 6));
     Fields[6].Str = to_string(format_hex_no_prefix(Sec.sh_entsize, 2));
-    Fields[7].Str = getGNUFlags(this->Obj.getHeader().e_machine, Sec.sh_flags);
+    Fields[7].Str = getGNUFlags(this->Obj.getHeader().e_ident[ELF::EI_OSABI],
+                                this->Obj.getHeader().e_machine, Sec.sh_flags);
     Fields[8].Str = to_string(Sec.sh_link);
     Fields[9].Str = to_string(Sec.sh_info);
     Fields[10].Str = to_string(Sec.sh_addralign);
 
     OS.PadToColumn(Fields[0].Column);
     OS << "[" << right_justify(Fields[0].Str, 2) << "]";
     for (int i = 1; i < 7; i++)
       printField(Fields[i]);
void LLVMELFDumper<ELFT>::printRelRelaReloc(const Relocation<ELFT> &R,
   }
 }
 
 template <class ELFT> void LLVMELFDumper<ELFT>::printSectionHeaders() {
   ListScope SectionsD(W, "Sections");
 
   int SectionIndex = -1;
   std::vector<EnumEntry<unsigned>> FlagsList =
-      getSectionFlagsForTarget(this->Obj.getHeader().e_machine);
+      getSectionFlagsForTarget(this->Obj.getHeader().e_ident[ELF::EI_OSABI],
+                               this->Obj.getHeader().e_machine);
   for (const Elf_Shdr &Sec : cantFail(this->Obj.sections())) {
     DictScope SectionD(W, "Section");
     W.printNumber("Index", ++SectionIndex);
     W.printNumber("Name", this->getPrintableSectionName(Sec), Sec.sh_name);
     W.printHex("Type",
                object::getELFSectionTypeName(this->Obj.getHeader().e_machine,
                                              Sec.sh_type),
                Sec.sh_type);