[yaml2obj] - Allocate the file space for SHT_NOBITS sections in some cases.

[grimar]

Summary

This teaches yaml2obj to allocate file space for a no-bits section
when there is a non-nobits section in the same segment that follows it.

It was discussed in D78005 thread and matches GNU linkers and LLD behavior.

[MaskRay]

This behavior seems reasonable.

It was discussed in D78005 thread and matches GNU linkers behavior.

Matches GNU linkers and LLD. Checked with:

% cat a.s
.section .data,"aw"
.byte 0

.section .dataa,"aw",@nobits
.byte 0

.section .datab,"aw",@nobits
.byte 0

.section .datac,"aw"
.byte 0

% cat a.x
SECTIONS {
  .data : { *(.data) }
  .dataa : { *(.dataa) }
  .datab : { *(.datab) }
  .datac : { *(.datac) }
  .bss : { *(.bss) }
}

% ld.lld -T a.x a.o -o a

llvm/lib/ObjectYAML/ELFEmitter.cpp

891

Inlining getChunksAfter can avoid the cost constructing a vector.

[grimar]

In D80629#2059073, @MaskRay wrote:

This behavior seems reasonable.

It was discussed in D78005 thread and matches GNU linkers behavior.

Matches GNU linkers and LLD.

Thanks! I've not tested LLD. Nice that it is consistent.

[grimar]

• Addressed review comments.

llvm/lib/ObjectYAML/ELFEmitter.cpp

891

I think the cost was very low: there is no much output sections usually, and the case when there are non-nobits sections after a nobit section is also probably rare. But it made the final code to be much shorter.

[grimar]

llvm/lib/ObjectYAML/ELFEmitter.cpp

891

Perhaps my comment is not very clear: inlining made the code shorter.

[jhenderson]

llvm/test/tools/yaml2obj/ELF/program-header-nobits.yaml

2

then -> them
have -> has

23

has the non-nobits -> has a non-nobits

25

.nobits.2 isn't in any segment, so I think this test case isn't quite right. .nobits.2 needs to be in the same segment as .nobits.1, with a non-nobits segment in between.

Also, I think you need a test-case for two trailing .nobits sections to show neither has file space allocated.

[grimar]

llvm/test/tools/yaml2obj/ELF/program-header-nobits.yaml

25

.nobits.2 isn't in any segment, so I think this test case isn't quite right. .nobits.2 needs to be in the same segment as .nobits.1, with a non-nobits segment in between.

Well. I think it is reasonable to have both then. I'll add "Case D" with the scenario you described.

Also, I think you need a test-case for two trailing .nobits sections to show neither has file space allocated.

We need to land D78005 first (which is on hold, waits for this one) to do something like this.
We have the issue with the calculation of segment size in this case.

[grimar]

• Addressed review comments.

[MaskRay]

llvm/lib/ObjectYAML/ELFEmitter.cpp

891

Inlining does seem shorter:)

Another thing about the previous approach: the worst case time complexity did not change but allocating vector worsened the expected time complexity (SHT_NOBITS is usually followed by a non-SHT_NOBITS) I know that in some places we have quadratic complexity but introducing a new one when it can be easily avoided is not very great.

900

This comment should mention that this behavior matches linkers.

llvm/test/tools/yaml2obj/ELF/program-header-nobits.yaml

163

Is -NEXT: applicable?

[jhenderson]

LGTM, with nits and @MaskRay's comments addressed.

llvm/test/tools/yaml2obj/ELF/program-header-nobits.yaml

149

a one -> one

154

Here and above, it's slightly offputting seeing a mixture of short and long options. I'd either go one or the other (i.e. -S -l or --sections --segments).

[grimar]

• Addressed review comments.

llvm/lib/ObjectYAML/ELFEmitter.cpp

891

Not sure I understood this point. A SHT_NOBITS section is often followed by a non-allocatable non-SHT_NOBITS (e.g. .comment might follow .bss), but the previous version returned getChunksAfter for a given program header. We usually include only allocatable sections to a PT_LOADs (which might contain a SHT_NOBITS section normally), so in 99% cases the returned vector was just empty I believe.

So. I believe that shorter code was the only real (but important one) benefit here.

llvm/test/tools/yaml2obj/ELF/program-header-nobits.yaml

154

I've used the long version.

163

Yes. Done.

[jhenderson]

LGTM!

[ostannard]

This is causing the 32-bit ARM bots to fail test tools/obj2yaml/ELF/program-headers.yaml. It looks like an out of memory error, but it has been happening consistently for a large number of builds now, and looks related to this change.

First failing build: http://lab.llvm.org:8011/builders/clang-cmake-armv7-quick/builds/16765

Log:

******************** TEST 'LLVM :: tools/obj2yaml/ELF/program-headers.yaml' FAILED ********************
Script:
--
: 'RUN: at line 5';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp1
: 'RUN: at line 9';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/llvm-readelf --segments /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp1 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=SEGMENT-MAPPING
: 'RUN: at line 37';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/obj2yaml /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp1 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=YAML
: 'RUN: at line 241';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=2 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp2
: 'RUN: at line 242';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/obj2yaml /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp2 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=EMPTY
: 'RUN: at line 331';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=3 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp3
: 'RUN: at line 332';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/llvm-readelf --segments --sections /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp3 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=MISALIGNED-READELF
: 'RUN: at line 333';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/obj2yaml /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp3 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=MISALIGNED-YAML
: 'RUN: at line 373';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=4 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp4
: 'RUN: at line 374';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/obj2yaml /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp4 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=NON-ALLOC
: 'RUN: at line 419';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=5 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp5
: 'RUN: at line 420';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/obj2yaml /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp5 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=NOBITS
: 'RUN: at line 523';   not /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=6 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp6 2>&1 |    /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=UNSORTED --implicit-check-not="error:"
: 'RUN: at line 579';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=7 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp7
: 'RUN: at line 587';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/llvm-readelf -sections /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp7 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=ZERO-SIZE-MAPPING
: 'RUN: at line 597';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/obj2yaml /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp7 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=ZERO-SIZE
: 'RUN: at line 648';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=8 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp8
: 'RUN: at line 649';   /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/obj2yaml /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp8 | /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/FileCheck /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml --check-prefix=BROKEN-VA
--
Exit Code: 134

Command Output (stderr):
--
LLVM ERROR: out of memory
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=5 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp5 
/home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.script: line 18: 41905 Aborted                 (core dumped) /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/bin/yaml2obj --docnum=5 /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/llvm/llvm/test/tools/obj2yaml/ELF/program-headers.yaml -o /home/tcwg-buildslave/worker/clang-cmake-armv7-quick/stage1/test/tools/obj2yaml/ELF/Output/program-headers.yaml.tmp5

--

********************

[grimar]

@ostannard, yes, sorry for the breakage. It happens because program-headers.yaml has a test that tries to produce a >4Gb object with this patch.
I am working on a fix. For now, I'll revert this.

[grimar]

Reverted in 5750f12b823448e3bb9f1fd6d896eab58d52df5b.

[grimar]

• Updated tools/obj2yaml/ELF/program-headers.yaml.

Now when we have a 10 MB limitation of the output size produced by yaml2obj,
the obj2yaml/ELF/program-headers.yaml test fails.

It has the following case:

  - Name:  .bss
    Type:  SHT_NOBITS
    Flags: [ SHF_WRITE, SHF_ALLOC ]
## Use a size that is larger than the file size.
    Size: 0x00000000FFFFFFFF

With this patch we allocate the file space for it. And
the size value is very large. I've changed the "Size" key to "ShSize" which
just overrides the value of sh_size. It keeps the logic of this test case
untouched and fixes the issue.

[MaskRay]

Looks great!

[jhenderson]

llvm/test/tools/obj2yaml/ELF/program-headers.yaml

499–500 ↗

(On Diff #269817)

Looking at this test, I'm not convinced this is the right fix for the test. My impression is that this test is supposed to be about how segments are created and their file and memory sizes calculated. By using ShSize instead of Size, we are effectively saying (in theory) "ignore the size of this section for segment calculations" when I think what is meant is to ignore it for the calculation of the file size.

I think we need to properly change this test to use a real Size value here, just much smaller than what was chosen previously. Alternatively, we probably don't need this set of tests (nobits + segments) here at all, since we have the new tests you added that highlighted the bug of not allocating file space for nobits sections.

[grimar]

llvm/test/tools/obj2yaml/ELF/program-headers.yaml

499–500 ↗

(On Diff #269817)

My impression is that this test is supposed to be about how segments are created and their file and memory sizes calculated.

This is obj2yaml test. It is about how segments are dumped. This change does not change the original behavior, which is described as
"Use a size that is larger than the file size.", i.e. it checks how we dump such situation.

I think we need to properly change this test to use a real Size value here, just much smaller than what was chosen previously.

It just will break the original intention ("Use a size that is larger than the file size.").

[jhenderson]

LGTM.

llvm/test/tools/obj2yaml/ELF/program-headers.yaml

499–500 ↗

(On Diff #269817)

Oh, I missed that this is an obj2yaml test and not a yaml2obj one, thanks.

llvm/lib/ObjectYAML/ELFEmitter.cpp

template <class ELFT> class ELFState {
 
   std::vector<Fragment>
   getPhdrFragments(const ELFYAML::ProgramHeader &Phdr,
                    ArrayRef<typename ELFT::Shdr> SHeaders);
 
   void finalizeStrings();
   void writeELFHeader(ContiguousBlobAccumulator &CBA, raw_ostream &OS);
   void writeSectionContent(Elf_Shdr &SHeader,
+                           const ELFYAML::NoBitsSection &Section,
+                           ContiguousBlobAccumulator &CBA);
+  void writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::RawContentSection &Section,
                            ContiguousBlobAccumulator &CBA);
   void writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::RelocationSection &Section,
                            ContiguousBlobAccumulator &CBA);
   void writeSectionContent(Elf_Shdr &SHeader,
                            const ELFYAML::RelrSection &Section,
                            ContiguousBlobAccumulator &CBA);
if (IsFirstUndefSection) {
       writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::RelrSection>(Sec)) {
       writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::Group>(Sec)) {
       writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::MipsABIFlags>(Sec)) {
       writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::NoBitsSection>(Sec)) {
-      // SHT_NOBITS sections do not have any content to write.
-      SHeader.sh_entsize = 0;
-      SHeader.sh_size = S->Size;
+      writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::DynamicSection>(Sec)) {
       writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::SymverSection>(Sec)) {
       writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::VerneedSection>(Sec)) {
       writeSectionContent(SHeader, *S, CBA);
     } else if (auto S = dyn_cast<ELFYAML::VerdefSection>(Sec)) {
       writeSectionContent(SHeader, *S, CBA);
if (YamlPhdr.Align) {
       // alignment.
       PHeader.p_align = 1;
       for (const Fragment &F : Fragments)
         PHeader.p_align = std::max((uint64_t)PHeader.p_align, F.AddrAlign);
     }
   }
 }
 
+static bool shouldAllocateFileSpace(ArrayRef<ELFYAML::ProgramHeader> Phdrs,
+                                    const ELFYAML::NoBitsSection &S) {
+  for (const ELFYAML::ProgramHeader &PH : Phdrs) {
+    auto It = llvm::find_if(
+        PH.Chunks, [&](ELFYAML::Chunk *C) { return C->Name == S.Name; });
+    if (std::any_of(It, PH.Chunks.end(), [](ELFYAML::Chunk *C) {
+          return (isa<ELFYAML::Fill>(C) ||
+                  cast<ELFYAML::Section>(C)->Type != ELF::SHT_NOBITS);
+        }))
+      return true;
+  }
+  return false;
+}
+
+template <class ELFT>
+void ELFState<ELFT>::writeSectionContent(Elf_Shdr &SHeader,
+                                         const ELFYAML::NoBitsSection &S,
+                                         ContiguousBlobAccumulator &CBA) {
+  // SHT_NOBITS sections do not have any content to write.
+  SHeader.sh_entsize = 0;
+  SHeader.sh_size = S.Size;
+
+  // When a nobits section is followed by a non-nobits section or fill in the
+  // same segment, we allocate the file space for it.
+  if (shouldAllocateFileSpace(Doc.ProgramHeaders, S))
+    CBA.getOS().write_zeros(S.Size);
+}
+
 template <class ELFT>
 void ELFState<ELFT>::writeSectionContent(
     Elf_Shdr &SHeader, const ELFYAML::RawContentSection &Section,
     ContiguousBlobAccumulator &CBA) {
   SHeader.sh_size = writeContent(CBA.getOS(), Section.Content, Section.Size);
 
   if (Section.EntSize)
     SHeader.sh_entsize = *Section.EntSize;

llvm/test/tools/llvm-objcopy/ELF/preserve-segment-contents.test

 # We want to preserve areas in segments that are not covered by section headers.
 # This test shows that we do this for areas at the start of a segment, between
 # sections in a segment, and after all sections in a segment.
 # To create inputs with arbitrary data in segments, not covered by sections, we
 # use yaml2obj to create segments with sections covering all areas, then remove
 # some sections in those segments, and finally write over the areas of the
 # removed sections using python.
 
 # blob* sections are the sections that will be removed to create unlabelled
 # areas and then overwritten with data to show we preserve the data.
 
 # RUN: yaml2obj %s -o %t.base
 # RUN: llvm-objcopy %t.base %t.stripped --regex -R blob.*
 # Show that the removal leaves the bytes as zeroes, as desired, for all our
 # test cases.
-# RUN: od -t x1 -j 0x2000 -N 24 %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK1 -DPATTERN="00 00 00 00"
+# RUN: od -t x1 -j 0x2000 -N 28 %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK1 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x2100 -N 12 %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK2 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x2200 -N 4  %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK3 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x2300 -N 12 %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK4 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x3000 -N 68 %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK5 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x4000 -N 60 %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK6 -DPATTERN="00 00 00 00"
 # RUN: od -t x1 -j 0x5000 -N 60 %t.stripped | FileCheck %s --ignore-case --check-prefix=CHECK7 -DPATTERN="00 00 00 00"
 
 # RUN: cp %t.stripped %t.in
 # RUN: echo "with open('%/t.in', 'rb+') as input:"                                 > %t.py
 # RUN: echo "  for offset in ["                                                   >> %t.py
-# RUN: echo "   0x2000, 0x2008, 0x200C, 0x2014, 0x2104, 0x2300,"                  >> %t.py
+# RUN: echo "   0x2000, 0x2008, 0x200C, 0x2018, 0x2104, 0x2300,"                  >> %t.py
 # RUN: echo "   0x3008, 0x3010, 0x3018, 0x3020, 0x3028, 0x302C, 0x3034, 0x303C,"  >> %t.py
 # RUN: echo "   0x4000, 0x4008, 0x4010, 0x4014, 0x401C, 0x4024, 0x4034,"          >> %t.py
 # RUN: echo "   0x5000, 0x5008, 0x5010, 0x501C, 0x5024, 0x502C, 0x5030, 0x5038]:" >> %t.py
 # RUN: echo "    input.seek(offset)"                                              >> %t.py
 # RUN: echo "    input.write(bytearray.fromhex('DEADBEEF'))"                      >> %t.py
 # RUN: %python %t.py
 # RUN: llvm-objcopy %t.in %t.out
-# RUN: od -t x1 -j 0x2000 -N 24 %t.out | FileCheck %s --ignore-case --check-prefix=CHECK1 -DPATTERN="de ad be ef"
+# RUN: od -t x1 -j 0x2000 -N 28 %t.out | FileCheck %s --ignore-case --check-prefix=CHECK1 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x2100 -N 12 %t.out | FileCheck %s --ignore-case --check-prefix=CHECK2 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x2200 -N 4  %t.out | FileCheck %s --ignore-case --check-prefix=CHECK3 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x2300 -N 12 %t.out | FileCheck %s --ignore-case --check-prefix=CHECK4 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x3000 -N 68 %t.out | FileCheck %s --ignore-case --check-prefix=CHECK5 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x4000 -N 60 %t.out | FileCheck %s --ignore-case --check-prefix=CHECK6 -DPATTERN="de ad be ef"
 # RUN: od -t x1 -j 0x5000 -N 60 %t.out | FileCheck %s --ignore-case --check-prefix=CHECK7 -DPATTERN="de ad be ef"
 
 # CHECK1:      [[PATTERN]] 11 22 33 44 [[PATTERN]] [[PATTERN]]
-# CHECK1-NEXT: 55 66 77 88 [[PATTERN]]
+# CHECK1-NEXT: 00 00 00 00 55 66 77 88 [[PATTERN]]
 # CHECK2:      99 00 aa bb [[PATTERN]] cc dd ee ff
 # CHECK3:      fe fe fe fe
 # CHECK4:      [[PATTERN]] 00 00 00 00 00 00 00 00
 # CHECK5:      ff ff ee ee dd dd cc cc [[PATTERN]] bb bb aa aa
 # CHECK5-NEXT: [[PATTERN]] 00 00 99 99 [[PATTERN]] 88 88 77 77
 # CHECK5-NEXT: [[PATTERN]] 66 66 55 55 [[PATTERN]] [[PATTERN]]
 # CHECK5-NEXT: 44 44 33 33 [[PATTERN]] 22 22 11 11 [[PATTERN]]
 # CHECK5-NEXT: 00 11 22 33

llvm/test/tools/yaml2obj/ELF/program-header-nobits.yaml

-# RUN: yaml2obj %s -o %t
-# RUN: llvm-readobj -l %t | FileCheck %s
+## In this test we check that we allocate file space for SHT_NOBITS sections when
+## there are non-nobits sections in the same segment after then. When an object have
+## multiple segments, we check each and allocate the space if at least one matches the
+## condition mentioned.
+
+## Case A. In this case there are no non-nobits sections after SHT_NOBITS sections in segments.
+##         Because of this the file space for SHT_NOBITS sections is not allocated.
+# RUN: yaml2obj %s -D SEC1=.data.before -D SEC2=.nobits.1 -o %t1
+# RUN: llvm-readelf --sections -l %t1 | FileCheck %s --check-prefix=NO-ALLOC
+
+# NO-ALLOC:      [Nr] Name         Type     Address          Off    Size
+# NO-ALLOC:      [ 1] .data.before PROGBITS 0000000000000000 0000b0 000001
+# NO-ALLOC-NEXT: [ 2] .nobits.1    NOBITS   0000000000000001 0000b1 000002
+# NO-ALLOC-NEXT: [ 3] .data.after  PROGBITS 0000000000000003 0000b1 000003
+# NO-ALLOC-NEXT: [ 4] .nobits.2    NOBITS   0000000000000006 0000b4 000004
+## .fill of size 0x5 is placed here.
+# NO-ALLOC-NEXT: [ 5] .data.last   PROGBITS 000000000000000f 0000b9 000006
+
+# NO-ALLOC:       Type Offset   VirtAddr           PhysAddr           FileSiz  MemSiz
+# NO-ALLOC-NEXT:  LOAD 0x0000b0 0x0000000000000000 0x0000000000000000 0x000001 0x000003
+# NO-ALLOC-NEXT:  LOAD 0x0000b0 0x0000000000000000 0x0000000000000000 0x000001 0x000003
+
+## Case B. We have a segment that has the non-nobits section after the SHT_NOBITS section ".nobits.1".
+##         The file space is allocated for it, but not for ".nobits.2".
+# RUN: yaml2obj %s -D SEC1=.nobits.1 -D SEC2=.data.after -o %t2
+# RUN: llvm-readelf --sections -l %t2 | FileCheck %s --check-prefix=ALLOC-FIRST
+
+# ALLOC-FIRST:      [Nr] Name         Type     Address          Off    Size
+# ALLOC-FIRST:      [ 1] .data.before PROGBITS 0000000000000000 0000b0 000001
+# ALLOC-FIRST-NEXT: [ 2] .nobits.1    NOBITS   0000000000000001 0000b1 000002
+# ALLOC-FIRST-NEXT: [ 3] .data.after  PROGBITS 0000000000000003 0000b3 000003
+# ALLOC-FIRST-NEXT: [ 4] .nobits.2    NOBITS   0000000000000006 0000b6 000004
+## .fill of size 0x5 is placed here.
+# ALLOC-FIRST-NEXT: [ 5] .data.last   PROGBITS 000000000000000f 0000bb 000006
+
+# ALLOC-FIRST:       Type Offset   VirtAddr           PhysAddr           FileSiz  MemSiz
+# ALLOC-FIRST-NEXT:  LOAD 0x0000b0 0x0000000000000000 0x0000000000000000 0x000001 0x000003
+# ALLOC-FIRST-NEXT:  LOAD 0x0000b1 0x0000000000000000 0x0000000000000000 0x000005 0x000005
+
+## Case C. We have a Fill after the ".nobits.2" section. In this case the file space is
+##         allocated for it, because fills are handled just like any other non-nobits sections.
+# RUN: yaml2obj %s -D SEC1=.nobits.2 -D SEC2=.fill -o %t3
+# RUN: llvm-readelf --sections -l %t3 | FileCheck %s --check-prefix=FILL-AT-END
+
+# FILL-AT-END: [Nr] Name       Type     Address          Off    Size
+# FILL-AT-END: [ 4] .nobits.2  NOBITS   0000000000000006 0000b4 000004
+## .fill of size 0x5 is placed here.
+# FILL-AT-END: [ 5] .data.last PROGBITS 000000000000000f 0000bd 000006
+
+# FILL-AT-END:      Type Offset   VirtAddr           PhysAddr           FileSiz  MemSiz
+# FILL-AT-END-NEXT: LOAD 0x0000b0 0x0000000000000000 0x0000000000000000 0x000001 0x000003
+# FILL-AT-END-NEXT: LOAD 0x0000b4 0x0000000000000000 0x0000000000000000 0x000009 0x000009
 
 !ELF
 FileHeader:
-  Class:           ELFCLASS64
-  Data:            ELFDATA2LSB
-  Type:            ET_EXEC
-  Machine:         EM_X86_64
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
 Sections:
-  - Name:            .data
-    Type:            SHT_PROGBITS
-    Flags:           [ SHF_ALLOC ]
-    Content:         "00000000"
-  - Name:            .after
-    Type:            SHT_NOBITS
-    Flags:           [ SHF_ALLOC ]
-    Size:            64
+  - Name:  .data.before
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC ]
+    Size:  0x1
+  - Name:  .nobits.1
+    Type:  SHT_NOBITS
+    Flags: [ SHF_ALLOC ]
+    Size:  0x2
+  - Name:  .data.after
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC ]
+    Size:  0x3
+  - Name:  .nobits.2
+    Type:  SHT_NOBITS
+    Flags: [ SHF_ALLOC ]
+    Size:  0x4
+  - Type:    Fill
+    Name:    .fill
+    Pattern: "00"
+    Size:    5
+  - Name:  .data.last
+    Type:  SHT_PROGBITS
+    Flags: [ SHF_ALLOC ]
+    Size:  0x6
 ProgramHeaders:
-  - Type: PT_LOAD
+## We have 2 segments, the first is predefined and the second can be customized.
+## We want to have more than one segment to show we check all of them when
+## trying to find a non-nobits section after a nobits one.
+  - Type:  PT_LOAD
     Flags: [ PF_R ]
     Sections:
-      - Section: .data
-      - Section: .after
-
-#CHECK:     ProgramHeaders [
-#CHECK-NEXT:  ProgramHeader {
-#CHECK-NEXT:    Type: PT_LOAD
-#CHECK-NEXT:    Offset:
-#CHECK-NEXT:    VirtualAddress:
-#CHECK-NEXT:    PhysicalAddress:
-#CHECK-NEXT:    FileSize: 4
-#CHECK-NEXT:    MemSize: 68
-#CHECK-NEXT:    Flags [
-#CHECK-NEXT:      PF_R
-#CHECK-NEXT:    ]
-#CHECK-NEXT:    Alignment:
-#CHECK-NEXT:  }
-#CHECK-NEXT:]
+      - Section: .data.before
+      - Section: .nobits.1
+  - Type:  PT_LOAD
+    Flags: [ PF_R ]
+    Sections:
+      - Section: [[SEC1]]
+      - Section: [[SEC2]]