[yaml2obj/obj2yaml] - Add a basic support for extended section indexes.

[grimar]

Summary

In some cases a symbol might have section index == SHN_XINDEX.
This is an escape value indicating that the actual section header index
is too large to fit in the containing field.
Then the SHT_SYMTAB_SHNDX section is used. It contains the 32bit values
that stores section indexes.

ELF gABI says that there can be multiple SHT_SYMTAB_SHNDX sections,
i.e. for example one for .symtab and one for .dynsym
(1) https://groups.google.com/forum/#!topic/generic-abi/-XJAV5d8PRg
(2) DT_SYMTAB_SHNDX: http://www.sco.com/developers/gabi/latest/ch5.dynamic.html

In this patch I am only supporting a single SHT_SYMTAB_SHNDX associated
with a .symtab. This is a more or less common case which is used a few tests I saw in LLVM.

I decided not to create the SHT_SYMTAB_SHNDX section as "implicit",
but implement is like a kind of regular section for now.
i.e. tools do not recreate this section or its content, like they do for
symbol table sections, for example. That should allow to write all kind of
possible broken test cases for our needs and keep the output closer to requested.

[MaskRay]

test/tools/obj2yaml/elf-sht-symtab-shndx.yaml

115 ↗

(On Diff #212328)

The canonical name is .symtab_shndx

139 ↗

(On Diff #212328)

.symtab_shndx1

144 ↗

(On Diff #212328)

.symtab_shndx2

165 ↗

(On Diff #212328)

`.symtab_shndx

188 ↗

(On Diff #212328)

.symtab_shndx

[grimar]

• Addressed review comments (changed the section name).

[jhenderson]

test/tools/obj2yaml/elf-sht-symtab-shndx.yaml

21 ↗

(On Diff #213336)

I think it would be clearer for obj2yaml to emit an array of entries (and yaml2obj to consume them). What do you think? (We should allow explicit EntSize/Content in the YAML to allow creation of broken stuff too).

- Name: .symtab_shndx
  Type: SHT_SYMTAB_SHNDX
  Link: .symtab
  Entries:
    - Value: 0
    - Value: 1
    - Value: 42

51 ↗

(On Diff #213336)

dump the object that -> dump an object, which

(same throughout the test)

70 ↗

(On Diff #213336)

contain -> contains

103 ↗

(On Diff #213336)

TODO -> FIXME

125 ↗

(On Diff #213336)

Remove "and"

151 ↗

(On Diff #213336)

is not associated

174 ↗

(On Diff #213336)

with the/a

test/tools/yaml2obj/elf-sht-symtab-shndx.yaml

1 ↗

(On Diff #213336)

remove "the"

25–26 ↗

(On Diff #213336)

index. It

29 ↗

(On Diff #213336)

You could use --section-data to print the bytes of the symbol table and check those maybe?

63 ↗

(On Diff #213336)

delete "the"

64 ↗

(On Diff #213336)

missing ')'

index of a section

65 ↗

(On Diff #213336)

than the total

90 ↗

(On Diff #213336)

if a symbol index

tools/obj2yaml/elf2yaml.cpp

180 ↗

(On Diff #213336)

I'm not sure what this comment is for? It doesn't seem really relevant for the code.

[grimar]

test/tools/obj2yaml/elf-sht-symtab-shndx.yaml

21 ↗

(On Diff #213336)

Sounds good. What about me doing this in a follow-up or do you prefer to have this from the start?

tools/obj2yaml/elf2yaml.cpp

180 ↗

(On Diff #213336)

I am trying to say here that SHT_SYMTAB_SHNDX should be handled first, becase the code below (dumpSymbols)
might want to use ShndxTable. I'll reword.

[jhenderson]

test/tools/obj2yaml/elf-sht-symtab-shndx.yaml

21 ↗

(On Diff #213336)

As obj2yaml should probably print the array version, that should be in first, otherwise you just have to update this test (but it's okay to then defer adding custom content to a later change if you want).

[grimar]

• Addressed review comments.

test/tools/obj2yaml/elf-sht-symtab-shndx.yaml

21 ↗

(On Diff #213336)

I switched to Entries in this patch. This required adding a bit more code.

test/tools/yaml2obj/elf-sht-symtab-shndx.yaml

29 ↗

(On Diff #213336)

Yes, done.

[jhenderson]

LGTM, with one nit.

test/tools/yaml2obj/elf-sht-symtab-shndx.yaml

38 ↗

(On Diff #213875)

This might be a Phabricator issue, but it looks like your arrow doesn't line up with the 0xFFFF?

llvm/trunk/include/llvm/Object/ELF.h

if (!VOrErr)
     return VOrErr.takeError();
   ArrayRef<Elf_Word> V = *VOrErr;
   auto SymTableOrErr = object::getSection<ELFT>(Sections, Section.sh_link);
   if (!SymTableOrErr)
     return SymTableOrErr.takeError();
   const Elf_Shdr &SymTable = **SymTableOrErr;
   if (SymTable.sh_type != ELF::SHT_SYMTAB &&
       SymTable.sh_type != ELF::SHT_DYNSYM)
-    // TODO: this error is untested.
-    return createError("invalid sh_type");
+    return createError("SHT_SYMTAB_SHNDX section is linked with " +
+                       object::getELFSectionTypeName(getHeader()->e_machine,
+                                                     SymTable.sh_type) +
+                       " section (expected SHT_SYMTAB/SHT_DYNSYM)");
+
   if (V.size() != (SymTable.sh_size / sizeof(Elf_Sym)))
     return createError("SHT_SYMTAB_SHNDX section has sh_size (" +
                        Twine(SymTable.sh_size) +
                        ") which is not equal to the number of symbols (" +
                        Twine(V.size()) + ")");
   return V;
 }
 

llvm/trunk/include/llvm/ObjectYAML/ELFYAML.h

struct Section {
   enum class SectionKind {
     Dynamic,
     Group,
     RawContent,
     Relocation,
     NoBits,
     Verdef,
     Verneed,
+    SymtabShndxSection,
     Symver,
     MipsABIFlags
   };
   SectionKind Kind;
   StringRef Name;
   ELF_SHT Type;
   Optional<ELF_SHF> Flags;
   llvm::yaml::Hex64 Address;
struct RelocationSection : Section {
 
   RelocationSection() : Section(SectionKind::Relocation) {}
 
   static bool classof(const Section *S) {
     return S->Kind == SectionKind::Relocation;
   }
 };
 
+struct SymtabShndxSection : Section {
+  std::vector<uint32_t> Entries;
+
+  SymtabShndxSection() : Section(SectionKind::SymtabShndxSection) {}
+
+  static bool classof(const Section *S) {
+    return S->Kind == SectionKind::SymtabShndxSection;
+  }
+};
+
 // Represents .MIPS.abiflags section
 struct MipsABIFlags : Section {
   llvm::yaml::Hex16 Version;
   MIPS_ISA ISALevel;
   llvm::yaml::Hex8 ISARevision;
   MIPS_AFL_REG GPRSize;
   MIPS_AFL_REG CPR1Size;
   MIPS_AFL_REG CPR2Size;

llvm/trunk/lib/ObjectYAML/ELFEmitter.cpp

bool writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::RawContentSection &Section,
                            ContiguousBlobAccumulator &CBA);
   bool writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::RelocationSection &Section,
                            ContiguousBlobAccumulator &CBA);
   bool writeSectionContent(Elf_Shdr &SHeader, const ELFYAML::Group &Group,
                            ContiguousBlobAccumulator &CBA);
   bool writeSectionContent(Elf_Shdr &SHeader,
+                           const ELFYAML::SymtabShndxSection &Shndx,
+                           ContiguousBlobAccumulator &CBA);
+  bool writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::SymverSection &Section,
                            ContiguousBlobAccumulator &CBA);
   bool writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::VerneedSection &Section,
                            ContiguousBlobAccumulator &CBA);
   bool writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::VerdefSection &Section,
                            ContiguousBlobAccumulator &CBA);
if (I == 0) {
         if (RawSec->Info)
           SHeader.sh_info = *RawSec->Info;
       }
       if (Sec->EntSize)
         SHeader.sh_entsize = *Sec->EntSize;
     } else if (auto S = dyn_cast<ELFYAML::RawContentSection>(Sec)) {
       if (!writeSectionContent(SHeader, *S, CBA))
         return false;
+    } else if (auto S = dyn_cast<ELFYAML::SymtabShndxSection>(Sec)) {
+      if (!writeSectionContent(SHeader, *S, CBA))
+        return false;
     } else if (auto S = dyn_cast<ELFYAML::RelocationSection>(Sec)) {
       if (!writeSectionContent(SHeader, *S, CBA))
         return false;
     } else if (auto S = dyn_cast<ELFYAML::Group>(Sec)) {
       if (!writeSectionContent(SHeader, *S, CBA))
         return false;
     } else if (auto S = dyn_cast<ELFYAML::MipsABIFlags>(Sec)) {
       if (!writeSectionContent(SHeader, *S, CBA))
if (IsRela) {
       REntry.setSymbolAndType(SymIdx, Rel.Type, isMips64EL(Doc));
       OS.write((const char *)&REntry, sizeof(REntry));
     }
   }
   return true;
 }
 
 template <class ELFT>
+bool ELFState<ELFT>::writeSectionContent(
+    Elf_Shdr &SHeader, const ELFYAML::SymtabShndxSection &Shndx,
+    ContiguousBlobAccumulator &CBA) {
+  raw_ostream &OS =
+      CBA.getOSAndAlignedOffset(SHeader.sh_offset, SHeader.sh_addralign);
+
+  for (uint32_t E : Shndx.Entries)
+    support::endian::write<uint32_t>(OS, E, ELFT::TargetEndianness);
+
+  SHeader.sh_entsize = Shndx.EntSize ? (uint64_t)*Shndx.EntSize : 4;
+  SHeader.sh_size = Shndx.Entries.size() * SHeader.sh_entsize;
+  return true;
+}
+
+template <class ELFT>
 bool ELFState<ELFT>::writeSectionContent(Elf_Shdr &SHeader,
                                          const ELFYAML::Group &Section,
                                          ContiguousBlobAccumulator &CBA) {
   assert(Section.Type == llvm::ELF::SHT_GROUP &&
          "Section type is not SHT_GROUP");
 
   SHeader.sh_entsize = 4;
   SHeader.sh_size = SHeader.sh_entsize * Section.Members.size();

llvm/trunk/lib/ObjectYAML/ELFYAML.cpp

#define ECase(X) IO.enumCase(Value, #X, ELF::X)
   ECase(SHN_HIRESERVE);
   ECase(SHN_AMDGPU_LDS);
   ECase(SHN_HEXAGON_SCOMMON);
   ECase(SHN_HEXAGON_SCOMMON_1);
   ECase(SHN_HEXAGON_SCOMMON_2);
   ECase(SHN_HEXAGON_SCOMMON_4);
   ECase(SHN_HEXAGON_SCOMMON_8);
 #undef ECase
-  IO.enumFallback<Hex32>(Value);
+  IO.enumFallback<Hex16>(Value);
 }
 
 void ScalarEnumerationTraits<ELFYAML::ELF_STB>::enumeration(
     IO &IO, ELFYAML::ELF_STB &Value) {
 #define ECase(X) IO.enumCase(Value, #X, ELF::X)
   ECase(STB_LOCAL);
   ECase(STB_GLOBAL);
   ECase(STB_WEAK);
void MappingTraits<ELFYAML::Symbol>::mapping(IO &IO, ELFYAML::Symbol &Symbol) {
   IO.mapOptional("Visibility", Keys->Visibility, ELFYAML::ELF_STV(0));
   IO.mapOptional("Other", Keys->Other, ELFYAML::ELF_STO(0));
 }
 
 StringRef MappingTraits<ELFYAML::Symbol>::validate(IO &IO,
                                                    ELFYAML::Symbol &Symbol) {
   if (Symbol.Index && Symbol.Section.data())
     return "Index and Section cannot both be specified for Symbol";
-  if (Symbol.Index && *Symbol.Index == ELFYAML::ELF_SHN(ELF::SHN_XINDEX))
-    return "Large indexes are not supported";
   if (Symbol.NameIndex && !Symbol.Name.empty())
     return "Name and NameIndex cannot both be specified for Symbol";
   return StringRef();
 }
 
 static void commonSectionMapping(IO &IO, ELFYAML::Section &Section) {
   IO.mapOptional("Name", Section.Name, StringRef());
   IO.mapRequired("Type", Section.Type);

 }
 
 static void groupSectionMapping(IO &IO, ELFYAML::Group &Group) {
   commonSectionMapping(IO, Group);
   IO.mapOptional("Info", Group.Signature, StringRef());
   IO.mapRequired("Members", Group.Members);
 }
 
+static void sectionMapping(IO &IO, ELFYAML::SymtabShndxSection &Section) {
+  commonSectionMapping(IO, Section);
+  IO.mapRequired("Entries", Section.Entries);
+}
+
 void MappingTraits<ELFYAML::SectionOrType>::mapping(
     IO &IO, ELFYAML::SectionOrType &sectionOrType) {
   IO.mapRequired("SectionOrType", sectionOrType.sectionNameOrType);
 }
 
 void MappingTraits<ELFYAML::SectionName>::mapping(
     IO &IO, ELFYAML::SectionName &sectionName) {
   IO.mapRequired("Section", sectionName.Section);
if (!IO.outputting())
       Section.reset(new ELFYAML::SymverSection());
     sectionMapping(IO, *cast<ELFYAML::SymverSection>(Section.get()));
     break;
   case ELF::SHT_GNU_verneed:
     if (!IO.outputting())
       Section.reset(new ELFYAML::VerneedSection());
     sectionMapping(IO, *cast<ELFYAML::VerneedSection>(Section.get()));
     break;
+  case ELF::SHT_SYMTAB_SHNDX:
+    if (!IO.outputting())
+      Section.reset(new ELFYAML::SymtabShndxSection());
+    sectionMapping(IO, *cast<ELFYAML::SymtabShndxSection>(Section.get()));
+    break;
   default:
     if (!IO.outputting())
       Section.reset(new ELFYAML::RawContentSection());
     sectionMapping(IO, *cast<ELFYAML::RawContentSection>(Section.get()));
   }
 }
 
 StringRef MappingTraits<std::unique_ptr<ELFYAML::Section>>::validate(

llvm/trunk/test/Object/invalid.test

 
 ## Test that llvm-readobj reports an error if SHT_SYMTAB_SHNDX section has
 ## invalid sh_size which should be:
 ## sizeof(.symtab_shndx) = (sizeof(.symtab) / entsize(.symtab)) * entsize(.symtab_shndx)
 
 # RUN: yaml2obj %s --docnum=11 -o %t11
 # RUN: not llvm-readobj --symbols %t11 2>&1 | FileCheck --check-prefix=INVALID-XINDEX-SIZE %s
 
-# INVALID-XINDEX-SIZE: error: {{.*}}: SHT_SYMTAB_SHNDX section has sh_size (24) which is not equal to the number of symbols (6)
+# INVALID-XINDEX-SIZE: error: {{.*}}: SHT_SYMTAB_SHNDX section has sh_size (24) which is not equal to the number of symbols (2)
 
 --- !ELF
 FileHeader:
   Class:   ELFCLASS64
   Data:    ELFDATA2LSB
   Type:    ET_DYN
   Machine: EM_X86_64
 Sections:
   - Name: .symtab_shndx
     Type: SHT_SYMTAB_SHNDX
-    Size: 0x18
-    EntSize: 4
+    Entries: [ 0, 1 ]
     Link: .symtab
 
 ## Check that llvm-readobj reports an error if the e_phentsize field is broken.
 
 # RUN: not llvm-readobj --program-headers %p/Inputs/invalid-e_shnum.elf 2>&1 | \
 # RUN:  FileCheck --check-prefix=INVALID-PH-ENTSIZE %s
 
 # INVALID-PH-ENTSIZE: error: invalid e_phentsize: 12336

llvm/trunk/test/tools/llvm-readobj/elf-section-types.test

Sections:
   - Name: group
     Type: SHT_GROUP
     Info: foo
     Members:
         - SectionOrType: 0
   - Name: symtab_shndx
     Type: SHT_SYMTAB_SHNDX
     Link: .symtab
-    EntSize: 0x4
-    Content: "0000000000000000"
+    Entries: [ 0, 1 ]
   - Name: relr
     Type: SHT_RELR
   - Name: android_rel
     Type: SHT_ANDROID_REL
   - Name: android_rela
     Type: SHT_ANDROID_RELA
   - Name: android_relr
     Type: SHT_ANDROID_RELR

llvm/trunk/test/tools/obj2yaml/elf-sht-symtab-shndx.yaml

+## Check that obj2yaml is able to dump a normal object which
+## contains the SHT_SYMTAB_SHNDX section and symbols with
+## section index == SHN_XINDEX.
+
+# RUN: yaml2obj --docnum=1 %s -o %t1
+# RUN: obj2yaml %t1 | FileCheck %s --check-prefix=CASE1
+
+# CASE1:      --- !ELF
+# CASE1-NEXT: FileHeader:
+# CASE1-NEXT:   Class:   ELFCLASS64
+# CASE1-NEXT:   Data:    ELFDATA2LSB
+# CASE1-NEXT:   Type:    ET_REL
+# CASE1-NEXT:   Machine: EM_X86_64
+# CASE1-NEXT: Sections:
+# CASE1-NEXT:   - Name:    bar
+# CASE1-NEXT:     Type:    SHT_PROGBITS
+# CASE1-NEXT:   - Name:    .symtab_shndx
+# CASE1-NEXT:     Type:    SHT_SYMTAB_SHNDX
+# CASE1-NEXT:     Link:    .symtab
+# CASE1-NEXT:     EntSize: 0x0000000000000004
+# CASE1-NEXT:     Entries: [ 0, 1, 2 ]
+# CASE1-NEXT: Symbols:
+# CASE1-NEXT:   - Name:  bar
+# CASE1-NEXT:     Type:  STT_SECTION
+# CASE1-NEXT:     Index: SHN_XINDEX
+# CASE1-NEXT:   - Name:  .symtab_shndx
+# CASE1-NEXT:     Type:  STT_SECTION
+# CASE1-NEXT:     Index: SHN_XINDEX
+# CASE1-NEXT: ...
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name: bar
+    Type: SHT_PROGBITS
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0, 1, 2 ]
+    Link:    .symtab
+Symbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+
+## Check that yaml2obj is unable to dump an object, which has
+## symbols with section index == SHN_XINDEX, but no SHT_SYMTAB_SHNDX section.
+
+# RUN: yaml2obj --docnum=2 %s -o %t2
+# RUN: not obj2yaml %t2 2>&1 | FileCheck %s -DFILE=%t2 --check-prefix=CASE2
+
+# CASE2: Error reading file: [[FILE]]: extended symbol index (1) is past the end of the SHT_SYMTAB_SHNDX section of size 0
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Symbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+
+## Check that yaml2obj is unable to dump an object, which has symbols with
+## section index == SHN_XINDEX, but SHT_SYMTAB_SHNDX table contains invalid indices
+## that are larger than total number of the sections.
+
+# RUN: yaml2obj --docnum=3 %s -o %t3
+# RUN: not obj2yaml %t3 2>&1 | FileCheck %s -DFILE=%t3 --check-prefix=CASE3
+
+# CASE3: Error reading file: [[FILE]]: invalid section index: 254
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name: bar
+    Type: SHT_PROGBITS
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0, 254 ]
+    Link:    .symtab
+Symbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+
+## Check that yaml2obj is unable to dump an object, which has symbols with
+## section index == SHN_XINDEX, but SHT_SYMTAB_SHNDX table contains more
+## entries than the number of symbols in .symtab.
+
+# RUN: yaml2obj --docnum=4 %s -o %t4
+# RUN: not obj2yaml %t4 2>&1 | FileCheck %s -DFILE=%t4 --check-prefix=CASE4
+
+## FIXME: The error message below needs rewording. Size should not be equal to the number of symbols.
+## CASE4: Error reading file: [[FILE]]: SHT_SYMTAB_SHNDX section has sh_size (48) which is not equal to the number of symbols (3)
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name: bar
+    Type: SHT_PROGBITS
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0, 1, 2 ]
+    Link:    .symtab
+Symbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+
+## ELF gABI allows having multiple SHT_SYMTAB_SHNDX sections.
+## We only support having one associated with .symtab now.
+
+# RUN: yaml2obj --docnum=5 %s -o %t5
+# RUN: not obj2yaml %t5 2>&1 | FileCheck %s -DFILE=%t5 --check-prefix=CASE5
+
+# CASE5: Error reading file: [[FILE]]: multiple SHT_SYMTAB_SHNDX sections are not supported
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name:    .symtab_shndx1
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0 ]
+    EntSize: 4
+    Link:    .symtab
+  - Name:    .symtab_shndx2
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0 ]
+    Link:    .symtab
+
+## Check that yaml2obj can't dump the object if SHT_SYMTAB_SHNDX is
+## not associated with a SHT_SYMTAB section (this case is illegal).
+
+# RUN: yaml2obj --docnum=6 %s -o %t6
+# RUN: not obj2yaml %t6 2>&1 | FileCheck %s -DFILE=%t6 --check-prefix=CASE6
+
+# CASE6: Error reading file: [[FILE]]: SHT_SYMTAB_SHNDX section is linked with SHT_PROGBITS section (expected SHT_SYMTAB/SHT_DYNSYM)
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0 ]
+    Link:    .foo
+  - Name:    .foo
+    Type:    SHT_PROGBITS
+
+## Check that yaml2obj can't dump the object if SHT_SYMTAB_SHNDX is
+## associated with a SHT_DYNSYM section (not implemented yet).
+
+# RUN: yaml2obj --docnum=7 %s -o %t7
+# RUN: not obj2yaml %t7 2>&1 | FileCheck %s -DFILE=%t7 --check-prefix=CASE7
+
+# CASE7: Error reading file: [[FILE]]: only SHT_SYMTAB_SHNDX associated with SHT_SYMTAB are supported
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0, 1 ]
+    Link:    .dynsym
+DynamicSymbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX

llvm/trunk/test/tools/obj2yaml/special-symbol-indices.yaml

 # CHECK-NEXT:     Binding: STB_GLOBAL
 # CHECK-NEXT:   - Name:    valid_index
 # CHECK-NEXT:     Section: .text
 # CHECK-NEXT:     Binding: STB_GLOBAL
 # CHECK-NEXT:   - Name:    processor_specific_index
 # CHECK-NEXT:     Index:   SHN_HEXAGON_SCOMMON_1
 # CHECK-NEXT:     Binding: STB_GLOBAL
 # CHECK-NEXT:   - Name:    unknown_index
-# CHECK-NEXT:     Index:   0x0000FFFE
+# CHECK-NEXT:     Index:   0xFFFE
 # CHECK-NEXT:     Binding: STB_GLOBAL
 
 !ELF
 FileHeader:
   Class:           ELFCLASS64
   Data:            ELFDATA2LSB
   Type:            ET_EXEC
   Machine:         EM_HEXAGON
- Name:     valid_index
     Index:    0x1
     Binding:  STB_GLOBAL
   - Name:     processor_specific_index
     Index:    SHN_HEXAGON_SCOMMON_1
     Binding:  STB_GLOBAL
   - Name:     unknown_index
     Index:    0xfffe
     Binding:  STB_GLOBAL
-
-## shn_xindex.o contains a symbol with st_shndx == SHN_XINDEX.
-## We do not support it at this moment.
-# RUN: not obj2yaml %S/Inputs/shn_xindex.o 2>&1 | FileCheck %s --check-prefix=ERR
-# ERR: Error reading file: {{.*}}shn_xindex.o: SHN_XINDEX symbols are not supported

llvm/trunk/test/tools/yaml2obj/elf-sht-symtab-shndx.yaml

+## Check that yaml2obj is able to produce output
+## when a symbol with section index SHN_XINDEX is used,
+## but no SHT_SYMTAB_SHNDX section is defined.
+
+# RUN: yaml2obj --docnum=1 %s -o %t1
+# RUN: not llvm-readobj --symbols 2>&1 %t1 | FileCheck %s --check-prefix=CASE1
+
+# CASE1: error: extended symbol index (1) is past the end of the SHT_SYMTAB_SHNDX section of size 0
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name: bar
+    Type: SHT_PROGBITS
+Symbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+
+## Check that yaml2obj keeps the SHT_SYMTAB_SHNDX section in the output
+## even when symbol's section index value is low enough to not require the extended symtab. 
+## Also, check that symbols in .symtab still have the SHN_XINDEX index.
+
+# RUN: yaml2obj --docnum=2 %s -o %t2
+# RUN: llvm-readobj --sections --symbols --section-data 2>&1 %t2 | FileCheck %s --check-prefix=CASE2
+
+# CASE2:      Section {
+# CASE2:       Name: .symtab_shndx (1)
+# CASE2-NEXT:  Type: SHT_SYMTAB_SHNDX (0x12)
+
+# CASE2:      Name: .symtab
+# CASE2:      SectionData (
+# CASE2-NEXT:   0000: 00000000 00000000 00000000 00000000
+# CASE2-NEXT:   0010: 00000000 00000000 00000000 0300FFFF
+##                                                   ^-- 0xFFFF here is a SHN_XINDEX.
+# CASE2-NEXT:   0020: 00000000 00000000 00000000 00000000
+# CASE2-NEXT: )
+
+# CASE2:      Symbol {
+# CASE2:        Name: bar (0)
+# CASE2-NEXT:   Value: 0x0
+# CASE2-NEXT:   Size: 0
+# CASE2-NEXT:   Binding: Local (0x0)
+# CASE2-NEXT:   Type: Section (0x3)
+# CASE2-NEXT:   Other: 0
+# CASE2-NEXT:   Section: bar (0x1)
+# CASE2-NEXT: }
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name: bar
+    Type: SHT_PROGBITS
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0, 1 ]
+    Link:    .symtab
+Symbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+
+## Check that yaml2obj allows producing broken SHT_SYMTAB_SHNDX section
+## content (in the case below it contains 0xff as an index of a section,
+## which is larger than the total number of sections in the file).
+
+# RUN: yaml2obj --docnum=3 %s -o %t3
+# RUN: not llvm-readobj --symbols 2>&1 %t3 | FileCheck %s --check-prefix=CASE3
+
+# CASE3: error: invalid section index: 255
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name: bar
+    Type: SHT_PROGBITS
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0, 255 ]
+    Link:    .symtab
+Symbols:
+  - Type:  STT_SECTION
+    Index: SHN_XINDEX
+
+## Check that yaml2obj reports an error if a symbol index does not fit into 2 bytes.
+
+# RUN: not yaml2obj --docnum=4 %s -o %t4 2>&1 | FileCheck %s --check-prefix=CASE4
+
+# CASE4: error: out of range hex16 number
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Symbols:
+  - Type:  STT_SECTION
+    Index: 65536
+
+## Check we can set a custom sh_entsize for SHT_SYMTAB_SHNDX section.
+
+# RUN: yaml2obj --docnum=5 %s -o %t5
+# RUN: not llvm-readelf -S 2>&1 %t5 | FileCheck %s -DFILE=%t5 --check-prefix=CASE5
+
+# CASE5: error: '[[FILE]]': section [index 1] has an invalid sh_entsize: 2
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_REL
+  Machine: EM_X86_64
+Sections:
+  - Name:    .symtab_shndx
+    Type:    SHT_SYMTAB_SHNDX
+    Entries: [ 0 ]
+    EntSize: 2
+    Link:    .symtab

llvm/trunk/tools/obj2yaml/elf2yaml.cpp

class ELFDumper {
   template <class RelT>
   Error dumpRelocation(const RelT *Rel, const Elf_Shdr *SymTab,
                        ELFYAML::Relocation &R);
 
   Expected<ELFYAML::DynamicSection *> dumpDynamicSection(const Elf_Shdr *Shdr);
   Expected<ELFYAML::RelocationSection *> dumpRelocSection(const Elf_Shdr *Shdr);
   Expected<ELFYAML::RawContentSection *>
   dumpContentSection(const Elf_Shdr *Shdr);
+  Expected<ELFYAML::SymtabShndxSection *>
+  dumpSymtabShndxSection(const Elf_Shdr *Shdr);
   Expected<ELFYAML::NoBitsSection *> dumpNoBitsSection(const Elf_Shdr *Shdr);
   Expected<ELFYAML::VerdefSection *> dumpVerdefSection(const Elf_Shdr *Shdr);
   Expected<ELFYAML::SymverSection *> dumpSymverSection(const Elf_Shdr *Shdr);
   Expected<ELFYAML::VerneedSection *> dumpVerneedSection(const Elf_Shdr *Shdr);
   Expected<ELFYAML::Group *> dumpGroup(const Elf_Shdr *Shdr);
   Expected<ELFYAML::MipsABIFlags *> dumpMipsABIFlags(const Elf_Shdr *Shdr);
 
 public:
template <class ELFT> Expected<ELFYAML::Object *> ELFDumper<ELFT>::dump() {
   auto SectionsOrErr = Obj.sections();
   if (!SectionsOrErr)
     return SectionsOrErr.takeError();
   Sections = *SectionsOrErr;
   SectionNames.resize(Sections.size());
 
   // Dump symbols. We need to do this early because other sections might want
   // to access the deduplicated symbol names that we also create here.
+  const Elf_Shdr *SymTab = nullptr;
+  const Elf_Shdr *SymTabShndx = nullptr;
+  const Elf_Shdr *DynSymTab = nullptr;
+
   for (const Elf_Shdr &Sec : Sections) {
-    if (Sec.sh_type == ELF::SHT_SYMTAB)
-      if (Error E = dumpSymbols(&Sec, Y->Symbols))
-        return std::move(E);
-    if (Sec.sh_type == ELF::SHT_DYNSYM)
-      if (Error E = dumpSymbols(&Sec, Y->DynamicSymbols))
-        return std::move(E);
-  }
+    if (Sec.sh_type == ELF::SHT_SYMTAB) {
+      SymTab = &Sec;
+    } else if (Sec.sh_type == ELF::SHT_DYNSYM) {
+      DynSymTab = &Sec;
+    } else if (Sec.sh_type == ELF::SHT_SYMTAB_SHNDX) {
+      // ABI allows us to have one SHT_SYMTAB_SHNDX for each symbol table.
+      // We only support having the SHT_SYMTAB_SHNDX for SHT_SYMTAB now.
+      if (SymTabShndx)
+        return createStringError(obj2yaml_error::not_implemented,
+                                 "multiple SHT_SYMTAB_SHNDX sections are not supported");
+      SymTabShndx = &Sec;
+    }
+  }
+
+  // We need to locate the SHT_SYMTAB_SHNDX section early, because it might be
+  // needed for dumping symbols.
+  if (SymTabShndx) {
+    if (!SymTab || SymTabShndx->sh_link != SymTab - Sections.begin())
+      return createStringError(
+          obj2yaml_error::not_implemented,
+          "only SHT_SYMTAB_SHNDX associated with SHT_SYMTAB are supported");
+
+    auto TableOrErr = Obj.getSHNDXTable(*SymTabShndx);
+    if (!TableOrErr)
+      return TableOrErr.takeError();
+    ShndxTable = *TableOrErr;
+  }
+  if (SymTab)
+    if (Error E = dumpSymbols(SymTab, Y->Symbols))
+  if (DynSymTab)
+    if (Error E = dumpSymbols(DynSymTab, Y->DynamicSymbols))
 
   for (const Elf_Shdr &Sec : Sections) {
     switch (Sec.sh_type) {
     case ELF::SHT_DYNAMIC: {
       Expected<ELFYAML::DynamicSection *> SecOrErr = dumpDynamicSection(&Sec);
       if (!SecOrErr)
         return SecOrErr.takeError();
       Y->Sections.emplace_back(*SecOrErr);
       break;
     }
     case ELF::SHT_STRTAB:
     case ELF::SHT_SYMTAB:
     case ELF::SHT_DYNSYM:
       // Do not dump these sections.
       break;
     case ELF::SHT_SYMTAB_SHNDX: {
-      auto TableOrErr = Obj.getSHNDXTable(Sec);
-      if (!TableOrErr)
-        return TableOrErr.takeError();
-      ShndxTable = *TableOrErr;
+      Expected<ELFYAML::SymtabShndxSection *> SecOrErr =
+          dumpSymtabShndxSection(&Sec);
+      if (!SecOrErr)
+        return SecOrErr.takeError();
+      Y->Sections.emplace_back(*SecOrErr);
       break;
     }
     case ELF::SHT_REL:
     case ELF::SHT_RELA: {
       Expected<ELFYAML::RelocationSection *> SecOrErr = dumpRelocSection(&Sec);
       if (!SecOrErr)
         return SecOrErr.takeError();
       Y->Sections.emplace_back(*SecOrErr);
Error ELFDumper<ELFT>::dumpSymbol(const Elf_Sym *Sym, const Elf_Shdr *SymTab,
 
   Expected<StringRef> SymbolNameOrErr =
       getUniquedSymbolName(Sym, StrTable, SymTab);
   if (!SymbolNameOrErr)
     return SymbolNameOrErr.takeError();
   S.Name = SymbolNameOrErr.get();
 
   if (Sym->st_shndx >= ELF::SHN_LORESERVE) {
-    if (Sym->st_shndx == ELF::SHN_XINDEX)
-      return createStringError(obj2yaml_error::not_implemented,
-                               "SHN_XINDEX symbols are not supported");
     S.Index = (ELFYAML::ELF_SHN)Sym->st_shndx;
     return Error::success();
   }
 
   auto ShdrOrErr = Obj.getSection(Sym, SymTab, ShndxTable);
   if (!ShdrOrErr)
     return ShdrOrErr.takeError();
   const Elf_Shdr *Shdr = *ShdrOrErr;
ELFDumper<ELFT>::dumpContentSection(const Elf_Shdr *Shdr) {
   }
 
   if (Shdr->sh_info)
     S->Info = static_cast<llvm::yaml::Hex64>(Shdr->sh_info);
   return S.release();
 }
 
 template <class ELFT>
+Expected<ELFYAML::SymtabShndxSection *>
+ELFDumper<ELFT>::dumpSymtabShndxSection(const Elf_Shdr *Shdr) {
+  auto S = make_unique<ELFYAML::SymtabShndxSection>();
+  if (Error E = dumpCommonSection(Shdr, *S))
+    return std::move(E);
+
+  auto EntriesOrErr = Obj.template getSectionContentsAsArray<Elf_Word>(Shdr);
+  if (!EntriesOrErr)
+    return EntriesOrErr.takeError();
+  for (const Elf_Word &E : *EntriesOrErr)
+    S->Entries.push_back(E);
+  return S.release();
+}
+
+template <class ELFT>
 Expected<ELFYAML::NoBitsSection *>
 ELFDumper<ELFT>::dumpNoBitsSection(const Elf_Shdr *Shdr) {
   auto S = make_unique<ELFYAML::NoBitsSection>();
   if (Error E = dumpCommonSection(Shdr, *S))
     return std::move(E);
   S->Size = Shdr->sh_size;
 
   return S.release();