[ELF] - Linkerscript: support combination of linkerscript and --compress-debug-sections.

[grimar]

Summary

Previously it was impossible to use linkerscript with --compress-debug-sections because of
assert failture:
Assertion failed: isFinalized(), file C:\access_softek\llvm\lib\MC\StringTableBuilder.cpp, line 64

Patch fixes the issue.

[ruiu]

test/ELF/linkerscript/compress-debug-sections.s

7

What is the point of such linker script? I wonder why do you want to control the layout of .debug_str section.

19–21

nit: Can you reduce number of repetitions of A, B, C and D? Just a few characters should suffice.

[grimar]

test/ELF/linkerscript/compress-debug-sections.s

7

Its just a sample script reduced from script of FreeBSD kernel:
https://svnweb.freebsd.org/base/head/sys/conf/ldscript.amd64?revision=315522&view=markup#l223
I do not think they really want to control layout.

Probably for this testcase I really could use just empty script and let debug section to be orphan. I'll check it when will update this patch.

19–21

Interesting that doing that revealed an issue. After reducing amount of repetitions I ended up with broken sections size or errors during decompression.
That happens because void LinkerScript::process(BaseCommand &Base) calculates size of output section wrong when doing output(). It takes each input section and updates size of output section, what works wrong for compressed case.
It's just a luck that testcase "worked" before.

I'll try to find how to fix this properly.

[grimar]

• Addressed comments.
• Improved testcase to show that woth mergeable and non-mergeable cases work fine.

[grimar]

• Addressed review comments.

[grimar]

• Fixed mistype in comment.

[ruiu]

ELF/OutputSections.cpp

154

during -> by

test/ELF/linkerscript/compress-debug-sections.s

16

I don't think you really need this much hand-crafted debug info. You can just create an object file with -g, link it with LLD and verify that the debug section doesn't have SHF_COMPRESSED flag by using llvm-objdump or readobj.

[grimar]

test/ELF/linkerscript/compress-debug-sections.s

16

But that will not cover things this patch implements.
I believe we still want to check that compressing - decompressing flow producing the same results in scripted case. When you asked in previous diff to reduce amount of repitions of A, B, C, D, that revealed an issue in calculating output section size, what produced broken zlib stream, I think it is worth to check that this case works now. Just checking the flag would not reveal it.

I think it is probably not important to check how mergeable synthetic compressed section works. I believe I can significantly simplify the testcase to use .debug_str only with merging disabled. Let me try.

[grimar]

• Simplified testcase.

[grimar]

• Fix testcase requirements.

ELF/OutputSections.cpp

154

Done.

[ruiu]

test/ELF/linkerscript/compress-debug-sections.s

18

I don't think you are actually testing the new code. Please use llvm-objdump or llvm-readelf to verify that debug sections don't SHT_COMPRESSED if linker scripts are in use. You don't need to use llvm-dwarfdump as the contents doesn't matter.

[grimar]

test/ELF/linkerscript/compress-debug-sections.s

18

Please use llvm-objdump or llvm-readelf to verify that debug sections don't SHT_COMPRESSED if linker scripts are in use.

I think I got what you want to say. Both your comments saying these sections don't compressed. But them are ! We do not support custom layout for them, but compression itself is supported by this patch. That is why I am doing that checks because content matters, as I check that them can be compressed and then decompressed and the content remains the same.

[grimar]

• Added check that shows we do compression of debug sections if linker script was used.

[ruiu]

LGTM

[grimar]

r302413

ELF/LinkerScript.cpp

if (auto *Cmd = dyn_cast<OutputSectionCommand>(Opt.Commands[I])) {
       // given value is larger or smaller than the original section alignment.
       if (Cmd->SubalignExpr) {
         uint32_t Subalign = Cmd->SubalignExpr().getValue();
         for (InputSectionBase *S : V)
           S->Alignment = Subalign;
       }
 
       // Add input sections to an output section.
-      unsigned Pos = 0;
-      for (InputSectionBase *S : V) {
-        // The actual offset will be computed during
-        // assignAddresses. For now, use the index as a very crude
-        // approximation so that it is at least easy for other code to
-        // know the section order.
-        cast<InputSection>(S)->OutSecOff = Pos++;
+      for (InputSectionBase *S : V)
         Factory.addInputSec(S, Cmd->Name, Cmd->Sec);
-      }
-    }
-  }
   CurOutSec = nullptr;
 }
 
 void LinkerScript::fabricateDefaultCommands(bool AllocateHeader) {
   std::vector<BaseCommand *> Commands;
 
   // Define start address
   uint64_t StartAddr = Config->ImageBase;
if (Cmd->LMAExpr) {
     LMAOffset = [=] { return Cmd->LMAExpr().getValue() - D; };
   }
 
   CurMemRegion = Cmd->MemRegion;
   if (CurMemRegion)
     Dot = CurMemRegion->Offset;
   switchTo(Sec);
 
+  // We do not support custom layout for compressed debug sectons.
+  // At this point we already know their size and have compressed content.
+  if (CurOutSec->Flags & SHF_COMPRESSED)
+    return;
+
   for (BaseCommand *C : Cmd->Commands)
     process(*C);
 }
 
 void LinkerScript::removeEmptyCommands() {
   // It is common practice to use very generic linker scripts. So for any
   // given run some of the output sections in the script will be empty.
   // We could create corresponding empty output sections, but that would

ELF/OutputSections.cpp

template <class ELFT> void OutputSection::finalize() {
 
   this->Link = In<ELFT>::SymTab->OutSec->SectionIndex;
   // sh_info for SHT_REL[A] sections should contain the section header index of
   // the section to which the relocation applies.
   InputSectionBase *S = First->getRelocatedSection();
   this->Info = S->OutSec->SectionIndex;
 }
 
+static uint64_t updateOffset(uint64_t Off, InputSection *S) {
+  Off = alignTo(Off, S->Alignment);
+  S->OutSecOff = Off;
+  return Off + S->getSize();
+}
+
 void OutputSection::addSection(InputSection *S) {
   assert(S->Live);
   Sections.push_back(S);
   S->OutSec = this;
   this->updateAlignment(S->Alignment);
 
+  // The actual offsets will be computed by assignAddresses. For now, use
+  // crude approximation so that it is at least easy for other code to know the
+  // section order. It is also used to calculate the output section size early
+  // for compressed debug sections.
+  this->Size = updateOffset(Size, S);
+
   // If this section contains a table of fixed-size entries, sh_entsize
   // holds the element size. Consequently, if this contains two or more
   // input sections, all of them must have the same sh_entsize. However,
   // you can put different types of input sections into one output
   // sectin by using linker scripts. I don't know what to do here.
   // Probably we sholuld handle that as an error. But for now we just
   // pick the largest sh_entsize.
   this->Entsize = std::max(this->Entsize, S->Entsize);
 }
 
 // This function is called after we sort input sections
 // and scan relocations to setup sections' offsets.
 void OutputSection::assignOffsets() {
   uint64_t Off = 0;
-  for (InputSection *S : Sections) {
-    Off = alignTo(Off, S->Alignment);
-    S->OutSecOff = Off;
-    Off += S->getSize();
-  }
+  for (InputSection *S : Sections)
+    Off = updateOffset(Off, S);
   this->Size = Off;
 }
 
 void OutputSection::sort(std::function<int(InputSectionBase *S)> Order) {
   typedef std::pair<unsigned, InputSection *> Pair;
   auto Comp = [](const Pair &A, const Pair &B) { return A.first < B.first; };
 
   std::vector<Pair> V;

test/ELF/linkerscript/Inputs/compress-debug-sections.s

+.section .debug_str
+  .asciz "CCC"
+  .asciz "DDD"

test/ELF/linkerscript/compress-debug-sections.s

+# REQUIRES: x86, zlib
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux \
+# RUN:   %S/Inputs/compress-debug-sections.s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t2.o
+
+## .debug_str section is mergeable. LLD would combine all of them into single
+## mergeable synthetic section. We use -O0 here to disable merging, that
+## allows to check that input sections has correctly assigned offsets.
+
+# RUN: echo "SECTIONS { }" > %t.script
+# RUN: ld.lld -O0 %t1.o %t2.o %t.script -o %t1 --compress-debug-sections=zlib
+# RUN: llvm-dwarfdump %t1 | FileCheck %s
+# RUN: llvm-readobj -s %t1 | FileCheck %s --check-prefix=ZLIBFLAGS
+
+# RUN: echo "SECTIONS { .debug_str 0 : { *(.debug_str) } }" > %t2.script
+# RUN: ld.lld -O0 %t1.o %t2.o %t2.script -o %t2 --compress-debug-sections=zlib
+# RUN: llvm-dwarfdump %t2 | FileCheck %s
+# RUN: llvm-readobj -s %t2 | FileCheck %s --check-prefix=ZLIBFLAGS
+
+# CHECK:       .debug_str contents:
+# CHECK-NEXT:    CCC
+# CHECK-NEXT:    DDD
+# CHECK-NEXT:    AAA
+# CHECK-NEXT:    BBB
+
+# ZLIBFLAGS:       Section {
+# ZLIBFLAGS:         Index:
+# ZLIBFLAGS:         Name: .debug_str
+# ZLIBFLAGS-NEXT:    Type: SHT_PROGBITS
+# ZLIBFLAGS-NEXT:    Flags [
+# ZLIBFLAGS-NEXT:      SHF_COMPRESSED
+
+.section .debug_str
+  .asciz "AAA"
+  .asciz "BBB"