[yaml2obj] - Verify that sections are sorted by their file offsets when creating segments.

[grimar]

Summary

This validates that sections listed for a segment in the YAML
declaration are ordered by their file offsets.

It might help to simplify the file size computation, but also
is useful by itself as helps to avoid issues in test cases and
to maintain their readability.

[grimar]

• Add the forgotten test case.

[grimar]

• Last minute fixes.

[MaskRay]

Can the tool sort program headers by p_offset so that reordered user input can be accepted? It may sometimes be useful to check how other tools behave with reordered input. This patch is dropping this feature.

[grimar]

In D78361#1988960, @MaskRay wrote:

Can the tool sort program headers by p_offset so that reordered user input can be accepted? It may sometimes be useful to check how other tools behave with reordered input. This patch is dropping this feature.

This patch fixed issues in 5 of test cases, I believe they did not expect to have reordered sections. This is what is really useful.

It may sometimes be useful to check how other tools behave with reordered input.

I am not sure what do you mean, could you elaborate? Are you going to use YAMLs as inputs for another tools? Which tools? Are them a part of LLVM? Do we have any test cases?

[jhenderson]

llvm/test/tools/obj2yaml/program-headers.yaml

419

the file offset -> their offsets

(sections only have one kind of offset field defined, so I don't think you need to say "file")

441

This isn't really a "Case", as that implies it's actually something that is testing functionality itself. I'd just get rid of the Case labels in this test.

[grimar]

• Addressed review comments.

[MaskRay]

In D78361#1990346, @grimar wrote:

In D78361#1988960, @MaskRay wrote:

Can the tool sort program headers by p_offset so that reordered user input can be accepted? It may sometimes be useful to check how other tools behave with reordered input. This patch is dropping this feature.

This patch fixed issues in 5 of test cases, I believe they did not expect to have reordered sections. This is what is really useful.

It may sometimes be useful to check how other tools behave with reordered input.

I am not sure what do you mean, could you elaborate? Are you going to use YAMLs as inputs for another tools? Which tools? Are them a part of LLVM? Do we have any test cases?

Ah, I misread the patch. The check does not prevent any feature.

[jhenderson]

LGTM.

llvm/test/tools/obj2yaml/program-headers.yaml

419

offset -> offsets

llvm/lib/ObjectYAML/ELFEmitter.cpp

 
 template <class ELFT>
 void ELFState<ELFT>::setProgramHeaderLayout(std::vector<Elf_Phdr> &PHeaders,
                                             std::vector<Elf_Shdr> &SHeaders) {
   uint32_t PhdrIdx = 0;
   for (auto &YamlPhdr : Doc.ProgramHeaders) {
     Elf_Phdr &PHeader = PHeaders[PhdrIdx++];
     std::vector<Fragment> Fragments = getPhdrFragments(YamlPhdr, SHeaders);
+    if (!llvm::is_sorted(Fragments, [](const Fragment &A, const Fragment &B) {
+          return A.Offset < B.Offset;
+        }))
+      reportError("sections in the program header with index " +
+                  Twine(PhdrIdx) + " are not sorted by their file offset");
 
     if (YamlPhdr.Offset) {
       PHeader.p_offset = *YamlPhdr.Offset;
     } else {
       if (YamlPhdr.Sections.size())
         PHeader.p_offset = UINT32_MAX;
       else
         PHeader.p_offset = 0;

llvm/test/tools/llvm-gsymutil/X86/elf-dwarf.yaml

- Type: PT_LOAD
     VAddr: 0x0000000000400000
     Align: 1024
     Sections:
       - Section: .interp
       - Section: .interp
       - Section: .note.ABI-tag
       - Section: .note.gnu.build-id
       - Section: .gnu.hash
-      - Section: .dynsym
-      - Section: .dynstr
       - Section: .gnu.version
       - Section: .gnu.version_r
       - Section: .rela.dyn
       - Section: .rela.plt
       - Section: .init
       - Section: .plt
       - Section: .plt.got
       - Section: .text
       - Section: .fini
       - Section: .rodata
       - Section: .eh_frame_hdr
       - Section: .eh_frame
+      - Section: .dynsym
+      - Section: .dynstr
 Symbols:
   - Name:            .interp
     Type:            STT_SECTION
     Section:         .interp
     Value:           0x0000000000400238
   - Name:            .note.ABI-tag
     Type:            STT_SECTION
     Section:         .note.ABI-tag

llvm/test/tools/llvm-readobj/ELF/demangle.test

 DynamicSymbols:
   - Name:    _Z3fooi
     Binding: STB_GLOBAL
 ProgramHeaders:
   - Type:  PT_LOAD
     Flags: [ PF_R, PF_X ]
     VAddr: 0x0
     Sections:
-      - Section: .dynsym
       - Section: .dynstr
+      - Section: .dynsym
       - Section: .rela.dyn
       - Section: .dynamic
       - Section: .text.foo
   - Type:  PT_DYNAMIC
     Flags: [ PF_R ]
     VAddr: 0x1000
     Sections:
       - Section: .dynamic

llvm/test/tools/llvm-readobj/ELF/hash-symbols.test

 DynamicSymbols:
   - Name:    _Z3fooi
     Binding: STB_GLOBAL
 ProgramHeaders:
   - Type:  PT_LOAD
     Flags: [ PF_R, PF_X ]
     VAddr: 0x0
     Sections:
-      - Section: .dynsym
       - Section: .dynstr
+      - Section: .dynsym
       - Section: .dynamic
       - Section: .text.foo
   - Type:  PT_DYNAMIC
     Flags: [ PF_R ]
     VAddr: 0x1000
     Sections:
       - Section: .dynamic
 

llvm/test/tools/llvm-readobj/ELF/reloc-negative-addend-no-sym.test

Relocations:
         Addend: -1
 DynamicSymbols:
   - Name:    force_dynsym
     Binding: STB_GLOBAL
 ProgramHeaders:
   - Type: PT_LOAD
     VAddr: 0x1000
     Sections:
-      - Section: .rela.dyn
       - Section: .dynamic
+      - Section: .rela.dyn
   - Type: PT_DYNAMIC
     VAddr: 0x1000
     Sections:
       - Section: .dynamic

llvm/test/tools/llvm-readobj/ELF/reloc-zero-name-or-value.test

- Name:    sym
     Binding: STB_GLOBAL
   - Value:   0x123
     Section: .text
     Binding: STB_GLOBAL
 ProgramHeaders:
   - Type: PT_LOAD
     VAddr: 0x1000
     Sections:
-      - Section: .rela.dyn
       - Section: .dynamic
+      - Section: .rela.dyn
   - Type: PT_DYNAMIC
     VAddr: 0x1000
     Sections:
       - Section: .dynamic

llvm/test/tools/obj2yaml/program-headers.yaml

- Name:    .non-alloc.2
     Flags:   [ ]
     Size:    0x10
   - Name:    .alloc.2
     Type:    SHT_PROGBITS
     Flags:   [ SHF_ALLOC ]
     Size:    0x1
 
 ## Check how we dump segments which contain SHT_NOBITS sections.
 # RUN: yaml2obj --docnum=5 %s -o %t5
 # RUN: obj2yaml %t5 | FileCheck %s --check-prefix=NOBITS
 
 # NOBITS:      ProgramHeaders:
 # NOBITS-NEXT:   - Type:  PT_LOAD
 # NOBITS-NEXT:     Flags: [ PF_W, PF_R ]
 # NOBITS-NEXT:     Sections:
 # NOBITS-NEXT:       - Section: .bss
 # NOBITS-NEXT:   - Type:  PT_LOAD
 # NOBITS-NEXT:     Flags: [ PF_W, PF_R ]
 # NOBITS-NEXT:     Sections:
 # NOBITS-NEXT:       - Section: .data.1
 # NOBITS-NEXT:       - Section: .bss
 # NOBITS-NEXT:   - Type:  PT_LOAD
 # NOBITS-NEXT:     Flags: [ PF_W, PF_R ]
 # NOBITS-NEXT:     Sections:
 # NOBITS-NEXT:       - Section: .data.1
 # NOBITS-NEXT:       - Section: .bss
 # NOBITS-NEXT:       - Section: .data.2
 # NOBITS-NEXT:   - Type:  PT_LOAD
 # NOBITS-NEXT:     Flags: [ PF_W, PF_R ]
 # NOBITS-NEXT:     Sections:
 # NOBITS-NEXT:       - Section: .bss
 # NOBITS-NEXT:       - Section: .data.2
 # NOBITS-NEXT:   - Type:  PT_LOAD
 # NOBITS-NEXT:     Flags: [ PF_W, PF_R ]
 # NOBITS-NEXT:     Sections:
 # NOBITS-NEXT:       - Section: .foo.bss
 # NOBITS-NEXT:       - Section: .bar.bss
 # NOBITS-NEXT:     VAddr: 0x0000000200000000
 # NOBITS-NEXT: Sections:
## I.e. its address is larger than addresses of previous sections.
     Size:    0x10
     Address: 0x200000000
   - Name:    .bar.bss
     Type:    SHT_NOBITS
     Flags:   [ SHF_WRITE, SHF_ALLOC ]
 ## Use an arbitrary size that is different to the size of
 ## the previous section.
     Size:    0x20
+
+## Check that we require sections in a program header
+## declaration to be sorted by their offsets.
+# RUN: not yaml2obj --docnum=6 %s -o %t6 2>&1 | \
+# RUN:   FileCheck %s --check-prefix=UNSORTED --implicit-check-not="error:"
+
+# UNSORTED:      error: sections in the program header with index 1 are not sorted by their file offset
+# UNSORTED-NEXT: error: sections in the program header with index 3 are not sorted by their file offset
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_DYN
+  Machine: EM_X86_64
+ProgramHeaders:
+## Case 1: the .bar section is placed after the .foo section in the file.
+##         Check we report an error about the violation of the order.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .bar
+      - Section: .foo
+    VAddr:  0x1000
+## There is nothing wrong with this segment. We have it to show that
+## we report correct program header indices in error messages.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .foo
+      - Section: .bar
+    VAddr:  0x1000
+## Case 2: the .bar section is placed before the .zed section in the file,
+##         but the sh_offset of .zed is less than the sh_offset of
+##         the .bar section because of the "ShOffset" property.
+##         Document we report an error for such a case.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .bar
+      - Section: .zed
+    VAddr:  0x1001
+Sections:
+  - Name:    .foo
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Size:    0x1
+    Address: 0x1000
+  - Name:    .bar
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Size:    0x1
+  - Name:     .zed
+    Type:     SHT_PROGBITS
+    Flags:    [ SHF_ALLOC ]
+    Size:     0x1
+    ShOffset: 0x0