[obj2yaml] - Teach tool to dump program headers.

[grimar]

Summary

Currently obj2yaml does not dump program headers,
this patch teaches it to do that.

[grimar]

llvm/tools/obj2yaml/elf2yaml.cpp

405

This change will go away after we land https://reviews.llvm.org/D74955

[MaskRay]

The dumpProgramHeaders() logic is similar to llvm-objcopy/ELF/Object.cpp:sectionWithSegment. It also reminds me of D74755... The attribution of an empty section on the boundary between two segments will not affect obj2yaml correctness.

llvm/tools/obj2yaml/elf2yaml.cpp

217

I haven't carefully read through the code. Just raise an opinion.

For non-SHT_NOBITS sections, sh_offset/p_offset works better than sh_addr/p_vaddr.
For SHT_NOBITS sections, sh_offset can be ignored. See D58426.

[labath]

No opinion on the code, but I think that the ability to dump program headers will be very useful. I've been wanting to have a feature like this for quite some time.

The main thing I am interested in is being able to dump core files (segments without any sections), which I guess this doesn't do yet, but it seems to be a step towards it.

[jhenderson]

llvm/include/llvm/ObjectYAML/ELFYAML.h

179 ↗

(On Diff #247225)

and is used

llvm/test/tools/obj2yaml/program-headers.yaml

5

--program-headers/--segments instead of -a

78

Some additional interesting test cases:

1. Standalone empty segments
2. Nested empty segments (at start/end/middle)
3. Empty sections in segments (at start/end/middle)
4. Misaligned segments?
5. Out-of-order PT_LOAD segments?
6. Segments with gaps in them (file size != size of all sections), where gap is at start/middle/end of segment.
7. Segments with memsize > file size.
8. Non-alloc sections within segments.

(Sorry if any of these are already covered)

llvm/tools/obj2yaml/elf2yaml.cpp

177

Maybe this shouldn't be auto?

217

+1 to using sh_offset. That is what llvm-objcopy uses, IIRC, and I believe yaml2obj places things based on the file offset too.

[jhenderson]

In D75342#1900331, @MaskRay wrote:

The dumpProgramHeaders() logic is similar to llvm-objcopy/ELF/Object.cpp:sectionWithSegment. It also reminds me of D74755... The attribution of an empty section on the boundary between two segments will not affect obj2yaml correctness.

Yes, I think in obj2yaml, it doesn't matter which segments a section is within, because the output should have the segments at exactly the same place as the input. There should be no moving of segments in obj2yaml, in my opinion, as it should provide as faithful a representation as possbile of the original input.

[grimar]

Will update after addressing review comments.

[MaskRay]

@jhenderson listed many good test cases. It requires some work. I've left some notes.

1. Standalone empty segments

GNU ld can create a p_memsz=p_filesz=0 PT_LOAD. lld doesn't.

2. Nested empty segments (at start/end/middle)

PT_TLS⊆PT_LOAD(RW)

(PT_TLS(p_memsz=0, p_align=64) is used by Android Bionic)

3. Empty sections in segments (at start/end/middle)
4. Misaligned segments?

p_offset%p_align != p_vaddr%p_align

5. Out-of-order PT_LOAD segments?

PT_LOAD not ordered by p_vaddr.

ELF spec: Loadable segment entries in the program header table appear in ascending order, sorted on the p_vaddr member.

6. Segments with gaps in them (file size != size of all sections), where gap is at start/middle/end of segment.

Type: Filler

7. Segments with memsize > file size.

Needs non-empty SHT_NOBITS.

8. Non-alloc sections within segments.

[grimar]

• Reimplemented, added test cases.

In D75342#1903392, @MaskRay wrote:

@jhenderson listed many good test cases. It requires some work. I've left some notes.

Thank you guys for these suggestions and comments, I used them in the update.

I've added tests for all cases listed except the next two.

Segments with gaps in them (file size != size of all sections), where gap is at start/middle/end of segment.

Type: Filler

I think we might want having a separate test for this. Also it needs more code I think.

Segments with memsize > file size.

Needs non-empty SHT_NOBITS.

I'd like to support SHT_NOBITS case separatelly too for the same reasons.

How does it sound?

[grimar]

llvm/test/Object/obj2yaml.test

667

.data section size == 0, I wonder if it is OK we dump it?

Can we ignore empty sections at the end of segment? I.e do what D74755 removes:

// If a section is empty it should be treated like it has a size of 1. This is
// to clarify the case when an empty section lies on a boundary between two
// segments and ensures that the section "belongs" to the second segment and
// not the first.
uint64_t SecSize = Sec.Size ? Sec.Size : 1;

[MaskRay]

llvm/test/Object/obj2yaml.test

667

I vote for removing uint64_t SecSize = Sec.Size ? Sec.Size : 1;

[MaskRay]

llvm/tools/obj2yaml/elf2yaml.cpp

202

getValueOr

224

for (; It != WorkList.end() && isInSegment<ELFT>(**It, Phdr); ++It)

The code checks VMA while isInSegment checks file offsets. It seems strange.
I think a pure O(n^2) algorithm is fine. llvm-objcopy/ELF/Object.cpp is O(n^2).

[grimar]

• Addressed review comments (simplified algorithm used).

llvm/tools/obj2yaml/elf2yaml.cpp

224

I think you are right. It is easier to use O(n*m) here, since the number of segments is not that large anyways.
Done.

[MaskRay]

In D75342#1919197, @grimar wrote:

• Reimplemented, added test cases.

In D75342#1903392, @MaskRay wrote:

@jhenderson listed many good test cases. It requires some work. I've left some notes.

Thank you guys for these suggestions and comments, I used them in the update.

I've added tests for all cases listed except the next two.

Segments with gaps in them (file size != size of all sections), where gap is at start/middle/end of segment.

Type: Filler

I think we might want having a separate test for this. Also it needs more code I think.

Segments with memsize > file size.

Needs non-empty SHT_NOBITS.

I'd like to support SHT_NOBITS case separatelly too for the same reasons.

How does it sound?

llvm/test/tools/obj2yaml/program-headers.yaml

114

used->use

llvm/tools/obj2yaml/elf2yaml.cpp

203

cast<ELFYAML::Section>(*C) is better because you assert it to be non-null.

204

The two if can be combined.

[grimar]

• Rebased, addressed comments.

[jhenderson]

In D75342#1903392, @MaskRay wrote:

@jhenderson listed many good test cases. It requires some work. I've left some notes.

1. Standalone empty segments

GNU ld can create a p_memsz=p_filesz=0 PT_LOAD. lld doesn't.

Not all segments have to be nested inside a PT_LOAD. We have several OS-specific segments in our downstream port that are not part of the memory image, but must appear in the link output, as they are used by tools other than the loader in our ecosystem, so they are not nested in PT_LOADs in our linker scripts. I imagine there are other cases too.

In D75342#1919197, @grimar wrote:

• Reimplemented, added test cases.

In D75342#1903392, @MaskRay wrote:

@jhenderson listed many good test cases. It requires some work. I've left some notes.

Thank you guys for these suggestions and comments, I used them in the update.

I've added tests for all cases listed except the next two.

Segments with gaps in them (file size != size of all sections), where gap is at start/middle/end of segment.

Type: Filler

I think we might want having a separate test for this. Also it needs more code I think.

Segments with memsize > file size.

Needs non-empty SHT_NOBITS.

I'd like to support SHT_NOBITS case separatelly too for the same reasons.

How does it sound?

Those both sound reasonable to me.

llvm/include/llvm/ObjectYAML/ELFYAML.h

179 ↗

(On Diff #247225)

holds -> hold

llvm/test/Object/obj2yaml.test

667

I think for obj2yaml, it's fine to do this, since obj2yaml should just recreate the layout, and the sections in the segment just help to configure the program headers.

llvm/test/tools/obj2yaml/program-headers.yaml

4

looks -> look

7

Thinking about it, do we really need this part? It's really testing yaml2obj and/or llvm-readelf, not obj2yaml.

9

dumped -> dump

24–33

You need {{$}} at the end of all of these lines, since there could be more segments after the last one in each of them.

133

into it.

(same throughout)

142

Might be worth saying "a nested dynamic segment" to highlight that being nested is also an interesting property.

158

Should this be moved to Part II?

173

This comment needs to explain "why" it must be the same as the previous segment. Otherwise, I think the first sentence is unnecessary.

"we and are" -> "we are"

291

It might be clearer to explicitly specify .empty.tls.end here?

Also, this appears to be missing a PT_TLS at the start of the segment, i.e. covering .empty.tls.start?

359

in a segment.

I disagree with this behaviour. A segment doesn't not have to contain only SHF_ALLOC sections (it is not mandated by the ELF gABI, and my organisation at least has linker scripts like that for segments other than PT_LOAD). If a segment's file offset + file size implies that it covers a non-alloc section, it should include it, no different to an alloc section.

llvm/tools/obj2yaml/elf2yaml.cpp

182–183

Could this return an Expected<std::vector<ELFYAML::PogramHeader>>? That would be cleaner than passing in something to be filled.

200–201

Maybe add a TODO here to add support for Fills?

204

As noted in the test, this SHF_ALLOC check is incorrect. It doesn't faithfully recreate what was in the object, and doesn't allow for non-loadable segments. Please remove it.

[grimar]

llvm/tools/obj2yaml/elf2yaml.cpp

204

The approach used by this patch won't work then.

We do not create Chunks for non-allocatable SHT_STRTAB, SHT_SYMTAB and SHT_DYNSYM,
so iterating over Chunks does not work for non-allocatable case in general.

[grimar]

llvm/tools/obj2yaml/elf2yaml.cpp

204

When I experimented with how to implement it, I had a prototype that had more complex code, but had no this limitation.
I'll try to return to what I had.

[jhenderson]

llvm/tools/obj2yaml/elf2yaml.cpp

204

FWIW, our use case doesn't have the static symbol table or string table inside a segment, although I don't know of any strict reason why this shouldn't be allowed.

[grimar]

• Changed implementation to support non-allocatable sections as requested.
• Addressed review comments.

llvm/test/tools/obj2yaml/program-headers.yaml

7

I did it for documentation purposes: (1) I find it very convenient for reading and maintaining this test: with it we can check and compare obj2yaml's output with "expected" version easily. (2) This also might allow us to refer and explain (in comments) any differences we might have in outputs.
I'd really would like to keep it because of (1) if there are no strong objections.

158

I am not sure why. This tests we can have an empty segment somewhere between other ones.
Do you want me to add an additional test to Part II that will have an object with only one empty segment?

291

Done.

359

Done.

llvm/tools/obj2yaml/elf2yaml.cpp

182–183

Done. I think I did it to be consistent with dumpSymbols, but returning Expected<std::vector<ELFYAML::ProgramHeader>> is better indeed,

200–201

I avoided any additional details as I am not sure it is clear to me how exactly we are going to support them.
(e.g. for example, I guess we might want to have a command line option, as I doubt we always want to dump a data between sections).
Just having something like "TODO: add a support for Fill chunks" is probably not very useful?

[grimar]

Note: I've changed implementation slightly. Now we dump all sections and then after dumping program headers we remove sections we do not want to see in an output.
It allows us to dump sections in one place and work with array of Chunks. It also should help to support Filler and seems makes the whole logic a bit nicer (in context of this patch).

It is possible to split this change to a separate NFC patch and I'll be happy to do it if we agree that it is a good way to go.

[jhenderson]

llvm/include/llvm/ObjectYAML/ELFYAML.h

180 ↗

(On Diff #252004)

Hold -> Holds (!)

(For reference, third-person singular verbs usually have an 's' on the end, but not third-person plural - previously multiple variables were being referenced, so plural, but now there is only one)

llvm/test/tools/obj2yaml/program-headers.yaml

7

No strong objection, but might be worth moving the comment to the llvm-readelf run line, along with a statement saying the check is here to make it clear what the layout should look like.

24–33

On second thoughts, using --match-full-lines might be just as good. I don't mind either way.

158

I don't think there's a need for that. I wouldn't expect that to be a special case.

174

the same as the previous one for this

374

to segments -> in segments

llvm/tools/obj2yaml/elf2yaml.cpp

177–178

Do we need this special case? Do you have a test case that illustrates it?

229–234

This comment refers to allocatable sections, which I don't think is quite right?

[grimar]

llvm/include/llvm/ObjectYAML/ELFYAML.h

180 ↗

(On Diff #252004)

Thanks.. :]

llvm/tools/obj2yaml/elf2yaml.cpp

229–234

We still only leave allocatable versions in the output, though I think this comment can be better now.
I'll split the mentioned NFC patch soon. It will include this piece of code and I'll rewrite this comment.

[grimar]

• Addressed review comments, rebased.

llvm/test/tools/obj2yaml/program-headers.yaml

24–33

I've kept {{$}} (have no preference here, {{$}} looks fine to me).

llvm/tools/obj2yaml/elf2yaml.cpp

177–178

I think we should ignore either all SHT_NULL sections or just the one with section index == 0.
When we don't, the first test case in program-headers.yaml includes it in the first segment and prints it:

- Type: PT_LOAD
  Flags: [ PF_R ]
  Sections:
   - Section: ''
   - Section: .hash
   - Section: .gnu.hash
   - Section: .dynsym
   - Section: .dynstr

[jhenderson]

llvm/tools/obj2yaml/elf2yaml.cpp

299–300

Is it perhaps worth a test case that shows that SHT_NULL sections, even not the first one, are not considered to be inside a program header?

313–314

I'm not sure this early out is necessary. The subsequent for loop will not trigger, and Ret will be returned anyway, without modification.

[grimar]

• Addressed review comments.

llvm/tools/obj2yaml/elf2yaml.cpp

299–300

Done.

313–314

Agree.

[jhenderson]

LGTM, I think. Please wait for @MaskRay to confirm.

[MaskRay]

Looks great!

llvm/test/Object/obj2yaml.test

 # RUN: obj2yaml %p/Inputs/trivial-object-test.elf-avr | FileCheck %s --check-prefix ELF-AVR
 
 # ELF-AVR:      FileHeader:
 # ELF-AVR-NEXT:   Class:           ELFCLASS32
 # ELF-AVR-NEXT:   Data:            ELFDATA2LSB
 # ELF-AVR-NEXT:   Type:            ET_EXEC
 # ELF-AVR-NEXT:   Machine:         EM_AVR
 # ELF-AVR-NEXT:   Flags:           [ EF_AVR_ARCH_AVR2 ]
+# ELF-AVR-NEXT: ProgramHeaders:
+# ELF-AVR-NEXT:  - Type:  PT_LOAD
+# ELF-AVR-NEXT:    Flags: [ PF_X, PF_R ]
+# ELF-AVR-NEXT:    Sections:
+# ELF-AVR-NEXT:     - Section: .text
+# ELF-AVR-NEXT:     - Section: .data
+# ELF-AVR-NEXT:    Align: 0x0000000000000002
+# ELF-AVR-NEXT:  - Type:  PT_LOAD
+# ELF-AVR-NEXT:    Flags: [ PF_W, PF_R ]
+# ELF-AVR-NEXT:    Sections:
+# ELF-AVR-NEXT:     - Section: .data
+# ELF-AVR-NEXT:    VAddr: 0x0000000000800060
+# ELF-AVR-NEXT:    PAddr: 0x0000000000000004
+# ELF-AVR-NEXT:    Align: 0x0000000000000001
 # ELF-AVR-NEXT: Sections:
 # ELF-AVR-NEXT:   - Name:            .text
 # ELF-AVR-NEXT:     Type:            SHT_PROGBITS
 # ELF-AVR-NEXT:     Flags:           [ SHF_ALLOC, SHF_EXECINSTR ]
 # ELF-AVR-NEXT:     AddressAlign:    0x0000000000000002
 # ELF-AVR-NEXT:     Content:         C20E0895
 # ELF-AVR-NEXT:   - Name:            .data
 # ELF-AVR-NEXT:     Type:            SHT_PROGBITS

llvm/test/tools/obj2yaml/program-headers.yaml

+## Show that obj2yaml is able to dump program headers.
+
+## Part I. Base check. All simple cases that look OK as a part of a single large test live here.
+
+# RUN: yaml2obj %s -o %t1
+
+## Show the layout of the object before we dump it using obj2yaml.
+## The check is here to make it clear what the layout should look like.
+# RUN: llvm-readelf --segments %t1 | FileCheck %s --check-prefix=SEGMENT-MAPPING
+
+# SEGMENT-MAPPING:      Program Headers:
+# SEGMENT-MAPPING-NEXT:   Type      Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
+# SEGMENT-MAPPING-NEXT:   LOAD      0x000000 0x0000000000000000 0x0000000000000000 0x000281 0x000281 R   0x1000
+# SEGMENT-MAPPING-NEXT:   LOAD      0x000281 0x0000000000001000 0x0000000000001000 0x000010 0x000010 R E 0x1000
+# SEGMENT-MAPPING-NEXT:   LOAD      0x000291 0x0000000000002000 0x0000000000002000 0x000009 0x000009 R   0x1000
+# SEGMENT-MAPPING-NEXT:   LOAD      0x00029a 0x0000000000003ef0 0x0000000000003ef0 0x000011 0x000011 RW  0x1000
+# SEGMENT-MAPPING-NEXT:   DYNAMIC   0x00029a 0x0000000000003ef0 0x0000000000003ef0 0x000010 0x000010 RW  0x8
+# SEGMENT-MAPPING-NEXT:   GNU_RELRO 0x00029a 0x0000000000003ef0 0x0000000000003ef0 0x000010 0x000010 R   0x1
+# SEGMENT-MAPPING-NEXT:   LOAD      0x000000 0x0000000000004000 0x0000000000004000 0x000000 0x000000 R   0x1
+# SEGMENT-MAPPING-NEXT:   LOAD      0x000248 0x00000000000001a0 0x00000000000001a0 0x000020 0x000020 R   0x1
+# SEGMENT-MAPPING-NEXT:   LOAD      0x000248 0x00000000000001a0 0x00000000000001a0 0x000020 0x000020 R   0x1
+# SEGMENT-MAPPING:      Section to Segment mapping:
+# SEGMENT-MAPPING-NEXT:  Segment Sections...
+# SEGMENT-MAPPING-NEXT:   00     .hash .gnu.hash .dynsym .dynstr {{$}}
+# SEGMENT-MAPPING-NEXT:   01     .foo .zed {{$}}
+# SEGMENT-MAPPING-NEXT:   02     .foo .baz {{$}}
+# SEGMENT-MAPPING-NEXT:   03     .dynamic .dynamic.tail {{$}}
+# SEGMENT-MAPPING-NEXT:   04     .dynamic {{$}}
+# SEGMENT-MAPPING-NEXT:   05     .dynamic {{$}}
+# SEGMENT-MAPPING-NEXT:   06{{ *$}}
+# SEGMENT-MAPPING-NEXT:   07     .gnu.hash {{$}}
+# SEGMENT-MAPPING-NEXT:   08     .gnu.hash {{$}}
+# SEGMENT-MAPPING-NEXT:   None   .symtab .strtab .shstrtab {{$}}
+
+## Check that obj2yaml produced a correct program headers description.
+
+# RUN: obj2yaml %t1 | FileCheck %s --check-prefix=YAML
+
+# YAML:      ProgramHeaders:
+# YAML-NEXT:  - Type:  PT_LOAD
+# YAML-NEXT:    Flags: [ PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: .hash
+# YAML-NEXT:      - Section: .gnu.hash
+# YAML-NEXT:      - Section: .dynsym
+# YAML-NEXT:      - Section: .dynstr
+# YAML-NEXT:    Align: 0x0000000000001000
+# YAML-NEXT:  - Type:  PT_LOAD
+# YAML-NEXT:    Flags: [ PF_X, PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: .foo
+# YAML-NEXT:      - Section: .zed
+# YAML-NEXT:    VAddr: 0x0000000000001000
+# YAML-NEXT:    Align: 0x0000000000001000
+# YAML-NEXT:  - Type:  PT_LOAD
+# YAML-NEXT:    Flags: [ PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: '.foo [1]'
+# YAML-NEXT:      - Section: .baz
+# YAML-NEXT:    VAddr: 0x0000000000002000
+# YAML-NEXT:    Align: 0x0000000000001000
+# YAML-NEXT:  - Type:  PT_LOAD
+# YAML-NEXT:    Flags: [ PF_W, PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: .dynamic
+# YAML-NEXT:      - Section: .dynamic.tail
+# YAML-NEXT:    VAddr: 0x0000000000003EF0
+# YAML-NEXT:    Align: 0x0000000000001000
+# YAML-NEXT:  - Type:  PT_DYNAMIC
+# YAML-NEXT:    Flags: [ PF_W, PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: .dynamic
+# YAML-NEXT:    VAddr: 0x0000000000003EF0
+# YAML-NEXT:    Align: 0x0000000000000008
+# YAML-NEXT:  - Type:  PT_GNU_RELRO
+# YAML-NEXT:    Flags: [ PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: .dynamic
+# YAML-NEXT:    VAddr: 0x0000000000003EF0
+# YAML-NEXT:    Align: 0x0000000000000001
+# YAML-NEXT:  - Type:  PT_LOAD
+# YAML-NEXT:    Flags: [ PF_R ]
+# YAML-NEXT:    VAddr: 0x0000000000004000
+# YAML-NEXT:    Align: 0x0000000000000001
+# YAML-NEXT:  - Type:  PT_LOAD
+# YAML-NEXT:    Flags: [ PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: .gnu.hash
+# YAML-NEXT:    VAddr: 0x00000000000001A0
+# YAML-NEXT:    Align: 0x0000000000000001
+# YAML-NEXT:  - Type:  PT_LOAD
+# YAML-NEXT:    Flags: [ PF_R ]
+# YAML-NEXT:    Sections:
+# YAML-NEXT:      - Section: .gnu.hash
+# YAML-NEXT:    VAddr: 0x00000000000001A0
+# YAML-NEXT:    Align: 0x0000000000000001
+# YAML-NEXT: Sections:
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_DYN
+  Machine: EM_X86_64
+ProgramHeaders:
+## Check we can create a PT_LOAD with arbitrary (we used .hash, .gnu.hash)
+## and implicit sections (we use .dynsym, .dynstr). It also checks that the
+## SHT_NULL section at index 0 is not included in the segment.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .hash
+      - Section: .gnu.hash
+      - Section: .dynsym
+      - Section: .dynstr
+    Align:  0x1000
+    Offset: 0x0
+## Check we can create a PT_LOAD with a different set of properties and sections.
+  - Type:  PT_LOAD
+    Flags: [ PF_X, PF_R ]
+    Sections:
+      - Section: .foo
+      - Section: .zed
+    VAddr: 0x1000
+    Align: 0x1000
+## Create a PT_LOAD to demonstate we are able to refer to output sections with the same name.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: '.foo [1]'
+      - Section: .baz
+    VAddr: 0x2000
+    Align: 0x1000
+## Show we can create a writeable PT_LOAD segment and put an arbitrary section into it.
+## Here we test both regular (SHT_PROGBITS) and a special section (SHT_DYNAMIC).
+  - Type:  PT_LOAD
+    Flags: [ PF_W, PF_R ]
+    Sections:
+      - Section: .dynamic
+      - Section: .dynamic.tail
+    VAddr: 0x3EF0
+    Align: 0x1000
+## Show we can create a nested dynamic segment and put a section into it.
+  - Type:  PT_DYNAMIC
+    Flags: [ PF_W, PF_R ]
+    Sections:
+      - Section: .dynamic
+    VAddr: 0x3EF0
+    Align: 0x8
+## Show we can create a relro segment and put a section into it.
+## We used .dynamic here and in tests above to demonstrate that
+## we can place a section in any number of segments.
+  - Type:  PT_GNU_RELRO
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .dynamic
+    VAddr: 0x3EF0
+    Align: 0x1
+## Show we can dump a standalone empty segment.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections: [ ]
+    VAddr: 0x4000
+    Align: 0x1
+## ELF specification says that loadable segment entries in the
+## program header are sorted by virtual address.
+## Show we can dump an out of order segment.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .gnu.hash
+    VAddr: 0x1A0
+    Align: 0x1
+## Test we are able to dump duplicated segments.
+## We use a segment that is the same as the previous one for this.
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .gnu.hash
+    VAddr: 0x1A0
+    Align: 0x1
+Sections:
+  - Name:    .hash
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Address: 0x190
+    Size:    0x10
+  - Name:    .gnu.hash
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Address: 0x1A0
+    Size:    0x20
+  - Name:    .dynsym
+    Type:    SHT_DYNSYM
+    Flags:   [ SHF_ALLOC ]
+    Address: 0x1C0
+    Link:    .dynstr
+    EntSize: 0x18
+  - Name:    .dynstr
+    Type:    SHT_STRTAB
+    Flags:   [ SHF_ALLOC ]
+    Address: 0x1D8
+  - Name:    .foo
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC, SHF_EXECINSTR ]
+    Address: 0x1000
+    Size:    0x8
+  - Name:    .zed
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC, SHF_EXECINSTR ]
+    Address: 0x1008
+    Size:    0x8
+  - Name:    '.foo [1]'
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Address: 0x2000
+    Size:    0x8
+  - Name:    .baz
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Address: 0x2008
+    Size:    0x1
+  - Name:    .dynamic
+    Type:    SHT_DYNAMIC
+    Flags:   [ SHF_WRITE, SHF_ALLOC ]
+    Address: 0x0000000000003EF0
+    Link:    .dynstr
+    Entries:
+      - Tag:   DT_NULL
+        Value: 0x0
+  - Name:    .dynamic.tail
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_WRITE, SHF_ALLOC ]
+    Content: "FE"
+Symbols: []
+DynamicSymbols: []
+
+## Part II. More specific tests.
+
+## Check we are able to dump segments that are empty or
+## contain empty sections.
+# RUN: yaml2obj --docnum=2 %s -o %t2
+# RUN: obj2yaml %t2 | FileCheck %s --check-prefix=EMPTY
+
+# EMPTY:      - Type:  PT_LOAD
+# EMPTY-NEXT:   Flags: [ PF_W, PF_R ]
+# EMPTY-NEXT:   Sections:
+# EMPTY-NEXT:     - Section: .empty.tls.start
+# EMPTY-NEXT:     - Section: .section.1
+# EMPTY-NEXT:     - Section: .empty.tls.middle
+# EMPTY-NEXT:     - Section: .section.2
+# EMPTY-NEXT:     - Section: .empty.tls.end
+# EMPTY-NEXT:   VAddr: 0x0000000000001000
+# EMPTY-NEXT:   Align: 0x0000000000001000
+# EMPTY-NEXT: - Type:            PT_TLS
+# EMPTY-NEXT:   Flags:           [ PF_W, PF_R ]
+# EMPTY-NEXT:   Sections:
+# EMPTY-NEXT:     - Section:         .empty.tls.start
+# EMPTY-NEXT:   VAddr:           0x0000000000001000
+# EMPTY-NEXT:   Align:           0x0000000000000001
+# EMPTY-NEXT: - Type:            PT_TLS
+# EMPTY-NEXT:   Flags:           [ PF_W, PF_R ]
+# EMPTY-NEXT:   Sections:
+# EMPTY-NEXT:     - Section:         .empty.tls.middle
+# EMPTY-NEXT:   VAddr:           0x0000000000001100
+# EMPTY-NEXT:   Align:           0x0000000000000001
+# EMPTY-NEXT: - Type:            PT_TLS
+# EMPTY-NEXT:   Flags:           [ PF_W, PF_R ]
+# EMPTY-NEXT:   Sections:
+# EMPTY-NEXT:     - Section:         .empty.tls.end
+# EMPTY-NEXT:   VAddr:           0x0000000000001200
+# EMPTY-NEXT:   Align:           0x0000000000000001
+# EMPTY-NEXT: Sections:
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_DYN
+  Machine: EM_X86_64
+ProgramHeaders:
+  - Type:  PT_LOAD
+    Flags: [ PF_W, PF_R ]
+    Sections:
+      - Section: .empty.tls.start
+      - Section: .section.1
+      - Section: .empty.tls.middle
+      - Section: .section.2
+      - Section: .empty.tls.end
+    VAddr: 0x1000
+    Align: 0x1000
+  - Type:  PT_TLS
+    Flags: [ PF_W, PF_R ]
+    Sections:
+      - Section: .empty.tls.start
+    VAddr: 0x1000
+    Align: 0x1
+  - Type:  PT_TLS
+    Flags: [ PF_W, PF_R ]
+    Sections:
+      - Section: .empty.tls.middle
+    VAddr: 0x1100
+    Align: 0x1
+  - Type:  PT_TLS
+    Flags: [ PF_W, PF_R ]
+    Sections:
+      - Section: .empty.tls.end
+    VAddr: 0x1200
+    Align: 0x1
+Sections:
+  - Name:    .empty.tls.start
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC, SHF_TLS ]
+    Size:    0x0
+    Address: 0x1000
+  - Name:  .section.1
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC ]
+    Size:  0x100
+  - Name:  .empty.tls.middle
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC, SHF_TLS ]
+    Size:  0x0
+  - Name:  .section.2
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC ]
+    Size:  0x100
+  - Name:  .empty.tls.end
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC, SHF_TLS ]
+    Size:  0x0
+
+## Document we are able to dump misaligned segments.
+## I.e. segments where (p_offset % p_align) != (p_vaddr % p_align).
+# RUN: yaml2obj --docnum=3 %s -o %t3
+# RUN: llvm-readelf --segments --sections %t3 | FileCheck %s --check-prefix=MISALIGNED-READELF
+# RUN: obj2yaml %t3 | FileCheck %s --check-prefix=MISALIGNED-YAML
+
+## As a misaligned p_offset value we use (`.foo` section offset - 1).
+# MISALIGNED-READELF:      [Nr] Name Type     Address          Off
+# MISALIGNED-READELF:      [ 1] .foo PROGBITS 0000000000001000 000078
+# MISALIGNED-READELF:      Type Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align
+# MISALIGNED-READELF-NEXT: LOAD 0x000077 0x0000000000001000 0x0000000000001000 0x000078 0x000078 R   0x1000
+
+# MISALIGNED-YAML:      ProgramHeaders:
+# MISALIGNED-YAML-NEXT:  - Type:  PT_LOAD
+# MISALIGNED-YAML-NEXT:    Flags: [ PF_R ]
+# MISALIGNED-YAML-NEXT:    Sections:
+# MISALIGNED-YAML-NEXT:     - Section: .foo
+# MISALIGNED-YAML-NEXT:    VAddr: 0x0000000000001000
+# MISALIGNED-YAML-NEXT:    Align: 0x0000000000001000
+# MISALIGNED-YAML-NEXT: Sections:
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_DYN
+  Machine: EM_X86_64
+ProgramHeaders:
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .foo
+    VAddr:  0x1000
+    Align:  0x1000
+    Offset: 0x000077
+Sections:
+  - Name:    .foo
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Size:    0x77
+    Address: 0x1000
+
+## Test we include non-allocatable sections in segments.
+## We also document that SHT_NULL sections are not considered to be inside a segment.
+# RUN: yaml2obj --docnum=4 %s -o %t4
+# RUN: obj2yaml %t4 | FileCheck %s --check-prefix=NON-ALLOC
+
+# NON-ALLOC:      ProgramHeaders:
+# NON-ALLOC-NEXT:   - Type:  PT_LOAD
+# NON-ALLOC-NEXT:     Flags: [ PF_R ]
+# NON-ALLOC-NEXT:     Sections:
+# NON-ALLOC-NEXT:      - Section: .alloc.1
+# NON-ALLOC-NEXT:      - Section: .non-alloc.1
+# NON-ALLOC-NEXT:      - Section: .alloc.2
+# NON-ALLOC-NEXT:     VAddr: 0x0000000000001000
+# NON-ALLOC-NEXT:     Align: 0x0000000000000001
+# NON-ALLOC-NEXT: Sections:
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_DYN
+  Machine: EM_X86_64
+ProgramHeaders:
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: .alloc.1
+      - Section: .alloc.2
+    VAddr:  0x1000
+Sections:
+  - Name:    .alloc.1
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Size:    0x100
+    Address: 0x1000
+  - Name:    .non-alloc.1
+    Type:    SHT_PROGBITS
+    Flags:   [ ]
+    Size:    0x10
+  - Name:    .non-alloc.2
+    Type:    SHT_NULL
+    Flags:   [ ]
+    Size:    0x10
+  - Name:    .alloc.2
+    Type:    SHT_PROGBITS
+    Flags:   [ SHF_ALLOC ]
+    Size:    0x1

llvm/tools/obj2yaml/elf2yaml.cpp

class ELFDumper {
   Expected<StringRef> getUniquedSymbolName(const Elf_Sym *Sym,
                                            StringRef StrTable,
                                            const Elf_Shdr *SymTab);
   Expected<StringRef> getSymbolName(uint32_t SymtabNdx, uint32_t SymbolNdx);
 
   const object::ELFFile<ELFT> &Obj;
   ArrayRef<Elf_Word> ShndxTable;
 
+  Expected<std::vector<ELFYAML::ProgramHeader>>
+  dumpProgramHeaders(ArrayRef<std::unique_ptr<ELFYAML::Chunk>> Sections);
+
   Error dumpSymbols(const Elf_Shdr *Symtab,
                     std::vector<ELFYAML::Symbol> &Symbols);
   Error dumpSymbol(const Elf_Sym *Sym, const Elf_Shdr *SymTab,
                    StringRef StrTable, ELFYAML::Symbol &S);
   Expected<std::vector<std::unique_ptr<ELFYAML::Chunk>>> dumpSections();
   Error dumpCommonSection(const Elf_Shdr *Shdr, ELFYAML::Section &S);
   Error dumpCommonRelocationSection(const Elf_Shdr *Shdr,
                                     ELFYAML::RelocationSection &S);
ELFDumper<ELFT>::getUniquedSymbolName(const Elf_Sym *Sym, StringRef StrTable,
 
   return Name;
 }
 
 template <class ELFT>
 bool ELFDumper<ELFT>::shouldPrintSection(const ELFYAML::Section &S,
                                          const Elf_Shdr &SHdr) {
   // We only print the SHT_NULL section at index 0 when it
   // has at least one non-null field, because yaml2obj
   // normally creates the zero section at index 0 implicitly.
   if (S.Type == ELF::SHT_NULL && (&SHdr == &Sections[0])) {
     const uint8_t *Begin = reinterpret_cast<const uint8_t *>(&SHdr);
     const uint8_t *End = Begin + sizeof(Elf_Shdr);
     return std::find_if(Begin, End, [](uint8_t V) { return V != 0; }) != End;
   }
 
   // Normally we use "Symbols:" and "DynamicSymbols:" to describe contents of
   // symbol tables. We also build and emit corresponding string tables
   // implicitly. But sometimes it is important to preserve positions and virtual
   // addresses of allocatable sections, e.g. for creating program headers.
   // Generally we are trying to reduce noise in the YAML output. Because
   // of that we do not print non-allocatable versions of such sections and
   // assume they are placed at the end.
   if (S.Type == ELF::SHT_STRTAB || S.Type == ELF::SHT_SYMTAB ||
       S.Type == ELF::SHT_DYNSYM)
     return S.Flags.getValueOr(ELFYAML::ELF_SHF(0)) & ELF::SHF_ALLOC;
 
   return true;
 }
 
 template <class ELFT> Expected<ELFYAML::Object *> ELFDumper<ELFT>::dump() {
   auto Y = std::make_unique<ELFYAML::Object>();
 
   // Dump header. We do not dump SHEntSize, SHOff, SHNum and SHStrNdx fields.
   // When not explicitly set, the values are set by yaml2obj automatically
   // and there is no need to dump them here.
   Y->Header.Class = ELFYAML::ELF_ELFCLASS(Obj.getHeader()->getFileClass());
   Y->Header.Data = ELFYAML::ELF_ELFDATA(Obj.getHeader()->getDataEncoding());
   Y->Header.OSABI = Obj.getHeader()->e_ident[ELF::EI_OSABI];
   Y->Header.ABIVersion = Obj.getHeader()->e_ident[ELF::EI_ABIVERSION];
   Y->Header.Type = Obj.getHeader()->e_type;
   Y->Header.Machine = Obj.getHeader()->e_machine;
   Y->Header.Flags = Obj.getHeader()->e_flags;
   Y->Header.Entry = Obj.getHeader()->e_entry;
 
   // Dump sections
   auto SectionsOrErr = Obj.sections();
   if (!SectionsOrErr)
     return SectionsOrErr.takeError();
   Sections = *SectionsOrErr;
   SectionNames.resize(Sections.size());
 
   // Dump symbols. We need to do this early because other sections might want
   // to access the deduplicated symbol names that we also create here.
   const Elf_Shdr *SymTab = nullptr;
   const Elf_Shdr *SymTabShndx = nullptr;
   const Elf_Shdr *DynSymTab = nullptr;
 
   for (const Elf_Shdr &Sec : Sections) {
     if (Sec.sh_type == ELF::SHT_SYMTAB) {
       SymTab = &Sec;
     } else if (Sec.sh_type == ELF::SHT_DYNSYM) {
       DynSymTab = &Sec;
     } else if (Sec.sh_type == ELF::SHT_SYMTAB_SHNDX) {
       // ABI allows us to have one SHT_SYMTAB_SHNDX for each symbol table.
       // We only support having the SHT_SYMTAB_SHNDX for SHT_SYMTAB now.
       if (SymTabShndx)
         return createStringError(obj2yaml_error::not_implemented,
                                  "multiple SHT_SYMTAB_SHNDX sections are not supported");
       SymTabShndx = &Sec;
     }
   }
 
   // We need to locate the SHT_SYMTAB_SHNDX section early, because it might be
template <class ELFT> Expected<ELFYAML::Object *> ELFDumper<ELFT>::dump() {
   // We dump all sections first. It is simple and allows us to verify that all
   // sections are valid and also to generalize the code. But we are not going to
   // keep all of them in the final output (see comments for
   // 'shouldPrintSection()'). Undesired chunks will be removed later.
   Expected<std::vector<std::unique_ptr<ELFYAML::Chunk>>> ChunksOrErr =
       dumpSections();
   if (!ChunksOrErr)
     return ChunksOrErr.takeError();
-
   std::vector<std::unique_ptr<ELFYAML::Chunk>> Chunks = std::move(*ChunksOrErr);
+
+  // Dump program headers.
+  Expected<std::vector<ELFYAML::ProgramHeader>> PhdrsOrErr =
+      dumpProgramHeaders(Chunks);
+  if (!PhdrsOrErr)
+    return PhdrsOrErr.takeError();
+  Y->ProgramHeaders = std::move(*PhdrsOrErr);
+
   llvm::erase_if(Chunks, [this](const std::unique_ptr<ELFYAML::Chunk> &C) {
     const ELFYAML::Section &S = cast<ELFYAML::Section>(*C.get());
     return !shouldPrintSection(S, Sections[S.OriginalSecNdx]);
   });
 
   Y->Chunks = std::move(Chunks);
   return Y.release();
 }
 
 template <class ELFT>
+static bool isInSegment(const ELFYAML::Section &Sec,
+                        const typename ELFT::Shdr &SHdr,
+                        const typename ELFT::Phdr &Phdr) {
+  if (Sec.Type == ELF::SHT_NULL)
+    return false;
+  return SHdr.sh_offset >= Phdr.p_offset &&
+         (SHdr.sh_offset + SHdr.sh_size <= Phdr.p_offset + Phdr.p_filesz);
+}
+
+template <class ELFT>
+Expected<std::vector<ELFYAML::ProgramHeader>>
+ELFDumper<ELFT>::dumpProgramHeaders(
+    ArrayRef<std::unique_ptr<ELFYAML::Chunk>> Chunks) {
+  std::vector<ELFYAML::ProgramHeader> Ret;
+  Expected<typename ELFT::PhdrRange> PhdrsOrErr = Obj.program_headers();
+  if (!PhdrsOrErr)
+    return PhdrsOrErr.takeError();
+
+  for (const typename ELFT::Phdr &Phdr : *PhdrsOrErr) {
+    ELFYAML::ProgramHeader PH;
+    PH.Type = Phdr.p_type;
+    PH.Flags = Phdr.p_flags;
+    PH.VAddr = Phdr.p_vaddr;
+    PH.PAddr = Phdr.p_paddr;
+    PH.Align = static_cast<llvm::yaml::Hex64>(Phdr.p_align);
+
+    // Here we match sections with segments.
+    // It is not possible to have a non-Section chunk, because
+    // obj2yaml does not create Fill chunks.
+    for (const std::unique_ptr<ELFYAML::Chunk> &C : Chunks) {
+      ELFYAML::Section &S = cast<ELFYAML::Section>(*C.get());
+      if (isInSegment<ELFT>(S, Sections[S.OriginalSecNdx], Phdr))
+        PH.Sections.push_back({S.Name});
+    }
+
+    Ret.push_back(PH);
+  }
+
+  return Ret;
+}
+
+template <class ELFT>
 Expected<ELFYAML::RawContentSection *>
 ELFDumper<ELFT>::dumpPlaceholderSection(const Elf_Shdr *Shdr) {
   auto S = std::make_unique<ELFYAML::RawContentSection>();
   if (Error E = dumpCommonSection(Shdr, *S.get()))
     return std::move(E);
   return S.release();
 }
 
case ELF::SHT_LLVM_CALL_GRAPH_PROFILE:
           [this](const Elf_Shdr *S) { return dumpCallGraphProfileSection(S); };
     case ELF::SHT_STRTAB:
     case ELF::SHT_SYMTAB:
     case ELF::SHT_DYNSYM:
       // The contents of these sections are described by other parts of the YAML
       // file. But we still want to dump them, because their properties can be
       // important. See comments for 'shouldPrintSection()' for more details.
       return [this](const Elf_Shdr *S) { return dumpPlaceholderSection(S); };
     default:
       return nullptr;
     }
   };
 
   for (const Elf_Shdr &Sec : Sections) {
     // We have dedicated dumping functions for most of the section types.
     // Try to use one of them first.
     if (std::function<Expected<ELFYAML::Chunk *>(const Elf_Shdr *)> DumpFn =