[llvm-objcopy] Add support for dynamic relocations

[jakehehrlich]

Summary

This change adds support for dynamic relocations (allocated SHT_REL/SHT_RELA sections with a dynamic symbol table as their link).

There's a bit of code duplication with RelocationSections which is unfortunate. I'm not sure how to get rid of the duplication right now. If we make RelocationSection and DynamicRelocationSection have a common base class then they wind up not really having make sure that SymbolTableSection and DynamicSymbolTableSection also have a common base class. That would be fine with me except that RelocationSection::setSymTab would have to do a cast to expose the proper methods of SymbolTableSection anyway. If we go this route then there's the cast in RelocationSection::setSymTab plus a tiny bit of code duplication with Section. If we go the route I've uploaded here we duplicate a tiny bit of code in RelocationSection but we have a bit of heavier code duplication in Object::readSectionHeaders. I wasn't really sure what the best trade off was here. I settled on this but I could still be swayed to the other way. Ideally I'd like something better than either of the ways I've proposed.

The binary I added for the test is here: https://drive.google.com/file/d/0B3gtIAmiMwZXSjJUZE9pUjd4M0k/view?usp=sharing

Unless support for dynamic symbol tables in yaml2obj is added this is needed.

[jhenderson]

I definitely agree with your TODO comment. Could we simply add a function a bit like "finalize" but for late initialization to the SectionBase interface, and implemented/overridden in the various sub-classes as required? It would, I think, just need to take the list of sections, or the Object itself as an argument. That should be another change anyway.

As for the whole code-duplication situation, I think it may be possible to derive from a templated base class like so:

class RelocationSectionBase <typename SymtabType>: public SectionBase
{
   SymtabType Symbols;
   void setSymTab(SymtabType * SymTab) { Symbols = SymTab; }
   //other shared code here, e.g. finalize, setSection etc
}

class RelocationSection : public RelocationSectionBase<SymbolTableSection>
{
  // Stuff specific to static relocation sections, e.g. addRelocation, writeSection
}

class DynamicRelocationSection : public RelocationSectionBase<DynamicSymbolTableSection>
{
  // Stuff specific to dynamic relocation sections, e.g. writeSection (which would probably just be a copy of Section's writeSection)
}

I believe this would avoid any additional casting, though without actually trying to implement it, I'm not certain.

[jakehehrlich]

Used James' idea along with https://reviews.llvm.org/D38008 to factor out almost all code duplication. Thanks James!

A couple points on the precise choices I made:

1. I don't want to just pass a vector because the only way to pass a vector/array is to copy the pointers into a new array or to pass an array of unique pointers.
2. I don't want to make the getSection* methods public but that's kind of a small point
3. Because Object takes an ELFT parameter we can't have it be an argument to a virtual method. So something without the ELFT template dependency has to be passed.

The result is that a new type needs to be made and it handles retrieving sections by index. I'm fairly happy with the overall choice here. A slight blemish is that a few of the types need special initialization beyond setting link/info fields. But there are only two of those so far (SymbolTableSection and RelocationSection) and the only other one I anticipate will ever occur is for SHT_GROUP which isn't a concern right now.

[jhenderson]

Yes, the SectionTable refactor definitely has improved things. Once the other change is ready, I'll approve this one.

[jakehehrlich]

Updated diff to be relative to origin now that the initialize methods are upstream.

[jhenderson]

LGTM.

[alekseyshl]

ASan is not happy about this patch: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/8167/steps/check-llvm%20asan/logs/stdio

[jakehehrlich]

In D37915#881460, @alekseyshl wrote:

ASan is not happy about this patch: http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/8167/steps/check-llvm%20asan/logs/stdio

The issues is that I'm not Symbols or SecToApplyRel in the relocation. I'll fix that soon.

llvm/trunk/test/tools/llvm-objcopy/dynamic-relocations.test

+# RUN: llvm-objcopy %p/Inputs/dynrel.elf %t
+# RUN: llvm-readobj -sections -section-data %t | FileCheck %s
+
+#CHECK:         Name: .rela.plt
+#CHECK-NEXT:    Type: SHT_RELA
+#CHECK-NEXT:    Flags [
+#CHECK-NEXT:      SHF_ALLOC
+#CHECK-NEXT:    ]
+#CHECK-NEXT:    Address:
+#CHECK-NEXT:    Offset:
+#CHECK-NEXT:    Size: 24
+#CHECK-NEXT:    Link:
+#CHECK-NEXT:    Info:
+#CHECK-NEXT:    AddressAlignment: 8
+#CHECK-NEXT:    EntrySize: 24
+#CHECK-NEXT:    SectionData (
+#CHECK-NEXT:       0000: 18202000 00000000 07000000 01000000  |.  .............|
+#CHECK-NEXT:       0010: 00000000 00000000                    |........|
+#CHECK-NEXT:    )

llvm/trunk/tools/llvm-objcopy/Object.h

 public:
   SectionTableRef(llvm::ArrayRef<std::unique_ptr<SectionBase>> Secs)
       : Sections(Secs) {}
   SectionTableRef(const SectionTableRef &) = default;
 
   SectionBase *getSection(uint16_t Index, llvm::Twine ErrMsg);
 
   template <class T>
-
   T *getSectionOfType(uint16_t Index, llvm::Twine IndexErrMsg,
                       llvm::Twine TypeErrMsg);
 };
 
 class SectionBase {
 public:
   llvm::StringRef Name;
   Segment *ParentSegment = nullptr;

 
 struct Relocation {
   const Symbol *RelocSymbol;
   uint64_t Offset;
   uint64_t Addend;
   uint32_t Type;
 };
 
-template <class ELFT> class RelocationSection : public SectionBase {
+template <class SymTabType> class RelocationSectionBase : public SectionBase {
+private:
+  SymTabType *Symbols;
+  SectionBase *SecToApplyRel;
+
+public:
+  void setSymTab(SymTabType *StrTab) { Symbols = StrTab; }
+  void setSection(SectionBase *Sec) { SecToApplyRel = Sec; }
+  void initialize(SectionTableRef SecTable) override;
+  void finalize() override;
+};
+
+template <class ELFT>
+class RelocationSection : public RelocationSectionBase<SymbolTableSection> {
 private:
   typedef typename ELFT::Rel Elf_Rel;
   typedef typename ELFT::Rela Elf_Rela;
 
   std::vector<Relocation> Relocations;
-  SymbolTableSection *Symbols;
-  SectionBase *SecToApplyRel;
 
   template <class T> void writeRel(T *Buf) const;
 
 public:
-  void setSymTab(SymbolTableSection *StrTab) { Symbols = StrTab; }
-  void setSection(SectionBase *Sec) { SecToApplyRel = Sec; }
   void addRelocation(Relocation Rel) { Relocations.push_back(Rel); }
-  void initialize(SectionTableRef SecTable) override;
-  void finalize() override;
   void writeSection(llvm::FileOutputBuffer &Out) const override;
+
   static bool classof(const SectionBase *S) {
+    if (S->Flags & llvm::ELF::SHF_ALLOC)
+      return false;
     return S->Type == llvm::ELF::SHT_REL || S->Type == llvm::ELF::SHT_RELA;
   }
 };
 
 class SectionWithStrTab : public Section {
 private:
   StringTableSection *StrTab;
 

 class DynamicSection : public SectionWithStrTab {
 public:
   DynamicSection(llvm::ArrayRef<uint8_t> Data) : SectionWithStrTab(Data) {}
   static bool classof(const SectionBase *S) {
     return S->Type == llvm::ELF::SHT_DYNAMIC;
   }
 };
 
+class DynamicRelocationSection
+    : public RelocationSectionBase<DynamicSymbolTableSection> {
+private:
+  llvm::ArrayRef<uint8_t> Contents;
+
+public:
+  DynamicRelocationSection(llvm::ArrayRef<uint8_t> Data) : Contents(Data) {}
+  void writeSection(llvm::FileOutputBuffer &Out) const override;
+  static bool classof(const SectionBase *S) {
+    if (!(S->Flags & llvm::ELF::SHF_ALLOC))
+      return false;
+    return S->Type == llvm::ELF::SHT_REL || S->Type == llvm::ELF::SHT_RELA;
+  }
+};
+
 template <class ELFT> class Object {
 private:
   typedef std::unique_ptr<SectionBase> SecPtr;
   typedef std::unique_ptr<Segment> SegPtr;
 
   typedef typename ELFT::Shdr Elf_Shdr;
   typedef typename ELFT::Ehdr Elf_Ehdr;
   typedef typename ELFT::Phdr Elf_Phdr;
 
   void initSymbolTable(const llvm::object::ELFFile<ELFT> &ElfFile,
                        SymbolTableSection *SymTab, SectionTableRef SecTable);
   SecPtr makeSection(const llvm::object::ELFFile<ELFT> &ElfFile,
                      const Elf_Shdr &Shdr);
   void readProgramHeaders(const llvm::object::ELFFile<ELFT> &ElfFile);
   SectionTableRef readSectionHeaders(const llvm::object::ELFFile<ELFT> &ElfFile);
 
-  SectionBase *getSection(uint16_t Index, llvm::Twine ErrMsg);
-
-  template <class T>
-  T *getSectionOfType(uint16_t Index, llvm::Twine IndexErrMsg,
-                      llvm::Twine TypeErrMsg);
-
 protected:
   StringTableSection *SectionNames;
   SymbolTableSection *SymbolTable;
   std::vector<SecPtr> Sections;
   std::vector<SegPtr> Segments;
 
   void writeHeader(llvm::FileOutputBuffer &Out) const;
   void writeProgramHeaders(llvm::FileOutputBuffer &Out) const;

llvm/trunk/tools/llvm-objcopy/Object.cpp

for (auto &Symbol : Symbols) {
     Sym->st_size = Symbol->Size;
     Sym->setBinding(Symbol->Binding);
     Sym->setType(Symbol->Type);
     Sym->st_shndx = Symbol->getShndx();
     ++Sym;
   }
 }
 
-template <class ELFT>
-void RelocationSection<ELFT>::initialize(SectionTableRef SecTable) {
-  setSymTab(SecTable.getSectionOfType<SymbolTableSection>(
+template <class SymTabType>
+void RelocationSectionBase<SymTabType>::initialize(SectionTableRef SecTable) {
+  setSymTab(SecTable.getSectionOfType<SymTabType>(
       Link,
       "Link field value " + Twine(Link) + " in section " + Name + " is invalid",
       "Link field value " + Twine(Link) + " in section " + Name +
           " is not a symbol table"));
 
+  if (Info != SHN_UNDEF)
     setSection(SecTable.getSection(Info,
                                    "Info field value " + Twine(Info) +
                                        " in section " + Name + " is invalid"));
 }
 
-template <class ELFT> void RelocationSection<ELFT>::finalize() {
+template <class SymTabType> void RelocationSectionBase<SymTabType>::finalize() {
   this->Link = Symbols->Index;
+  if (SecToApplyRel != nullptr)
     this->Info = SecToApplyRel->Index;
 }
 
 template <class ELFT>
 void setAddend(Elf_Rel_Impl<ELFT, false> &Rel, uint64_t Addend) {}
 
 template <class ELFT>
 void setAddend(Elf_Rel_Impl<ELFT, true> &Rela, uint64_t Addend) {
   Rela.r_addend = Addend;

 void RelocationSection<ELFT>::writeSection(llvm::FileOutputBuffer &Out) const {
   uint8_t *Buf = Out.getBufferStart() + Offset;
   if (Type == SHT_REL)
     writeRel(reinterpret_cast<Elf_Rel *>(Buf));
   else
     writeRel(reinterpret_cast<Elf_Rela *>(Buf));
 }
 
+void DynamicRelocationSection::writeSection(llvm::FileOutputBuffer &Out) const {
+  std::copy(std::begin(Contents), std::end(Contents),
+            Out.getBufferStart() + Offset);
+}
+
 bool SectionWithStrTab::classof(const SectionBase *S) {
   return isa<DynamicSymbolTableSection>(S) || isa<DynamicSection>(S);
 }
 
 void SectionWithStrTab::initialize(SectionTableRef SecTable) {
   setStrTab(SecTable.getSectionOfType<StringTableSection>(
       Link,
       "Link field value " + Twine(Link) + " in section " + Name + " is invalid",

 SectionBase *SectionTableRef::getSection(uint16_t Index, Twine ErrMsg) {
   if (Index == SHN_UNDEF || Index > Sections.size())
     error(ErrMsg);
   return Sections[Index - 1].get();
 }
 
 template <class T>
 T *SectionTableRef::getSectionOfType(uint16_t Index, Twine IndexErrMsg,
                                      Twine TypeErrMsg) {
   if (T *Sec = llvm::dyn_cast<T>(getSection(Index, IndexErrMsg)))
     return Sec;
   error(TypeErrMsg);
 }
 
 template <class ELFT>
-SectionBase *Object<ELFT>::getSection(uint16_t Index, Twine ErrMsg) {
-  if (Index == SHN_UNDEF || Index > Sections.size())
-    error(ErrMsg);
-  return Sections[Index - 1].get();
-}
-
-template <class ELFT>
-template <class T>
-T *Object<ELFT>::getSectionOfType(uint16_t Index, Twine IndexErrMsg,
-                                  Twine TypeErrMsg) {
-  if (T *TSec = llvm::dyn_cast<T>(getSection(Index, IndexErrMsg)))
-    return TSec;
-  error(TypeErrMsg);
-}
-
-template <class ELFT>
 std::unique_ptr<SectionBase>
 Object<ELFT>::makeSection(const llvm::object::ELFFile<ELFT> &ElfFile,
                           const Elf_Shdr &Shdr) {
   ArrayRef<uint8_t> Data;
   switch (Shdr.sh_type) {
   case SHT_REL:
   case SHT_RELA:
+    if (Shdr.sh_flags & SHF_ALLOC) {
+      Data = unwrapOrError(ElfFile.getSectionContents(&Shdr));
+      return llvm::make_unique<DynamicRelocationSection>(Data);
+    }
     return llvm::make_unique<RelocationSection<ELFT>>();
   case SHT_STRTAB:
     // If a string table is allocated we don't want to mess with it. That would
     // mean altering the memory image. There are no special link types or
     // anything so we can just use a Section.
     if (Shdr.sh_flags & SHF_ALLOC) {
       Data = unwrapOrError(ElfFile.getSectionContents(&Shdr));
       return llvm::make_unique<Section>(Data);
if (auto RelSec = dyn_cast<RelocationSection<ELFT>>(Section.get())) {
       if (RelSec->Type == SHT_REL)
         initRelocations(RelSec, SymbolTable, unwrapOrError(ElfFile.rels(Shdr)));
       else
         initRelocations(RelSec, SymbolTable,
                         unwrapOrError(ElfFile.relas(Shdr)));
     }
 
     if (auto Sec = dyn_cast<SectionWithStrTab>(Section.get())) {
-      Sec->setStrTab(getSectionOfType<StringTableSection>(
+      Sec->setStrTab(SecTable.getSectionOfType<StringTableSection>(
           Sec->Link,
           "Link field value " + Twine(Sec->Link) + " in section " + Sec->Name +
               " is invalid",
           "Link field value " + Twine(Sec->Link) + " in section " + Sec->Name +
               " is not a string table"));
     }
   }