[ObjectYAML][ELF] Add support for emitting the .debug_line section.

[Higuoxing]

Summary

This patch enables yaml2elf emit the .debug_line section.

Test cases for emitting the dwarf64 .debug_line section and opcodes will be added later.

Known issues:

• We should replace InitialLength with Format and Length
• Currently implementation of the .debug_line section only fully supports DWARFv2, some header fields in DWARFv4 and DWARFv5 is missing, e.g., header_length in DWARFv4, address_size and segment_selector_size in DWARFv5.
• Some opcodes relies on the .debug_info section, we should warn user about it.

These issues will be addressed in a follow-up patch.

[jhenderson]

llvm/test/tools/yaml2obj/ELF/DWARF/debug-line.yaml

83–88

As a suggestion, how about you make these somewhat less default values. For example:

MinInstLength:  1
DefaultIsStmt:  2
LineBase:       3
LineRange:      4
OpcodeBase:     5
StandardOpcodeLengths: [ 0, 1, 2, 3, 4 ]

You could probably even do that for the other fields too.

[Higuoxing]

Use some non-default values in the header.

[jhenderson]

Thanks, LGTM, but best wait for @grimar to check too.

[grimar]

llvm/test/tools/yaml2obj/ELF/DWARF/debug-line.yaml

174

Is it a reason not to do the same as done like in b) ?

I.e

FileCheck %s -DSIZE=10 -DADDRALIGN=0 --check-prefixes=SHDR,SIZE

337

I am not sure how much it is useful to check it, but anyways it looks probably it could be a generic test?
I.e. it could check that no .debug_* sections are emitted when there is an empty "DWARF" entry.

Perhaps it should be in a different test case file then.

[Higuoxing]

Address comments.

Thanks for reviewing!

llvm/test/tools/yaml2obj/ELF/DWARF/debug-line.yaml

337

Thanks, Can we do it in a follow-up patch? There are several testcases like this.

[grimar]

llvm/test/tools/yaml2obj/ELF/DWARF/debug-line.yaml

337

Sure.

[grimar]

LGTM.

llvm/lib/ObjectYAML/DWARFYAML.cpp

 SetVector<StringRef> DWARFYAML::Data::getUsedSectionNames() const {
   SetVector<StringRef> SecNames;
   if (!DebugStrings.empty())
     SecNames.insert("debug_str");
   if (!ARanges.empty())
     SecNames.insert("debug_aranges");
   if (!DebugRanges.empty())
     SecNames.insert("debug_ranges");
+  if (!DebugLines.empty())
+    SecNames.insert("debug_line");
   return SecNames;
 }
 
 namespace yaml {
 
 void MappingTraits<DWARFYAML::Data>::mapping(IO &IO, DWARFYAML::Data &DWARF) {
   auto oldContext = IO.getContext();
   IO.setContext(&DWARF);

llvm/lib/ObjectYAML/ELFEmitter.cpp

Expected<uint64_t> emitDWARF(typename ELFT::Shdr &SHeader, StringRef Name,
   cantFail(std::move(Err));
 
   if (Name == ".debug_str")
     Err = DWARFYAML::emitDebugStr(OS, DWARF);
   else if (Name == ".debug_aranges")
     Err = DWARFYAML::emitDebugAranges(OS, DWARF);
   else if (Name == ".debug_ranges")
     Err = DWARFYAML::emitDebugRanges(OS, DWARF);
+  else if (Name == ".debug_line")
+    Err = DWARFYAML::emitDebugLine(OS, DWARF);
   else
     llvm_unreachable("unexpected emitDWARF() call");
 
   if (Err)
     return std::move(Err);
 
   return OS.tell() - BeginOffset;
 }

llvm/test/tools/yaml2obj/ELF/DWARF/debug-line.yaml

+## Test that yaml2obj emits .debug_line section.
+
+## a) Generate the .debug_line section from the "DWARF" entry.
+
+## Generate and verify a little endian DWARF32 .debug_line section.
+
+# RUN: yaml2obj --docnum=1 -DENDIAN=ELFDATA2LSB %s -o %t1.le.o
+# RUN: llvm-readobj --sections --section-data %t1.le.o | \
+# RUN:   FileCheck %s -DSIZE=50 -DADDRALIGN=1 --check-prefixes=SHDR,DWARF-LE-CONTENT
+
+#                  SHDR: Index: 1
+#             SHDR-NEXT: Name: .debug_line (1)
+#             SHDR-NEXT: Type: SHT_PROGBITS (0x1)
+#             SHDR-NEXT: Flags [ (0x0)
+#             SHDR-NEXT: ]
+#             SHDR-NEXT: Address: 0x0
+#             SHDR-NEXT: Offset: 0x40
+#             SHDR-NEXT: Size: [[SIZE]]
+#             SHDR-NEXT: Link: 0
+#             SHDR-NEXT: Info: 0
+#             SHDR-NEXT: AddressAlignment: [[ADDRALIGN]]
+#             SHDR-NEXT: EntrySize: 0
+# DWARF-LE-CONTENT-NEXT: SectionData (
+# DWARF-LE-CONTENT-NEXT:   0000: 70000000 02003200 00000102 03040501
+##                               |        |   |        | |  | | | |
+##                               |        |   |        | |  | | | +- standard_opcode_lengths[DW_LNS_copy] (1-byte) 0x01
+##                               |        |   |        | |  | | +- opcode_base (1-byte) 0x05
+##                               |        |   |        | |  | +- line_range (1-byte) 0x04
+##                               |        |   |        | |  +- line_base (signed 1-byte) 0x03
+##                               |        |   |        | +- default_is_stmt (1-byte) 0x02
+##                               |        |   |        +- minimum_instruction_length (1-byte) 0x01
+##                               |        |   +-------- prologue_length (4-byte) 50
+##                               |        +--- version (2-byte) 0x02
+##                               +------- unit_length (4-byte) 0x70
+##
+# DWARF-LE-CONTENT-NEXT:   0010: 02030405 06070809 64697231 00646972
+##                               | | | |  | | | |  |          |
+##                               | | | |  | | | |  |          +----- include_directories[2] "dir2\0"
+##                               | | | |  | | | |  +---------- include_directories[1] "dir1\0"
+##                               | | | |  | | | +- standard_opcode_lengths[DW_LNS_fixed_advance_pc] (1-byte) 0x09
+##                               | | | |  | | +- standard_opcode_lengths[DW_LNS_const_add_pc] (1-byte) 0x08
+##                               | | | |  | +- standard_opcode_lengths[DW_LNS_set_basic_block] (1-byte) 0x07
+##                               | | | |  +- standard_opcode_lengths[DW_LNS_negate_stmt] (1-byte) 0x06
+##                               | | | +- standard_opcode_lengths[DW_LNS_set_column] (1-byte) 0x05
+##                               | | +- standard_opcode_lengths[DW_LNS_set_file] (1-byte) 0x04
+##                               | +- standard_opcode_lengths[DW_LNS_advance_line] (1-byte) 0x03
+##                               +- standard_opcode_lengths[DW_LNS_advance_pc] (1-byte) 0x02
+##
+# DWARF-LE-CONTENT-NEXT:   0020: 32000061 2E630001 0203622E 63000203
+##                               |   | |        |  | | |        | |
+##                               |   | |        |  | | |        | +- ModTime (ULEB128) 0x03
+##                               |   | |        |  | | |        +- DirIndex (ULEB128) 0x02
+##                               |   | |        |  | | +-------- file_names[2] "b.c\0"
+##                               |   | |        |  | +- Length (ULEB128) 0x03
+##                               |   | |        |  +- ModTime (ULEB128) 0x02
+##                               |   | |        +- DirIndex (ULEB128) 0x01
+##                               |   | +-------- file_names[1] "a.c\0"
+##                               |   +- terminating entry (1-byte) 0x00
+##                               +--- the last two bytes of "dir2\0"
+##
+# DWARF-LE-CONTENT-NEXT:   0030: 0400
+##                               | |
+##                               | +- terminating entry (1-byte) 0x00
+##                               +- Length (ULEB128) 0x04
+# DWARF-LE-CONTENT-NEXT: )
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    [[ENDIAN]]
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+DWARF:
+  debug_line:
+    - Length:
+        TotalLength:  0x70
+      Version:        2
+      PrologueLength: 50
+      MinInstLength:  1
+      DefaultIsStmt:  2
+      LineBase:       3
+      LineRange:      4
+      OpcodeBase:     5
+      StandardOpcodeLengths: [ 1, 2, 3, 4, 5, 6, 7, 8, 9 ]
+      IncludeDirs:
+        - dir1
+        - dir2
+      Files:
+        - Name:    "a.c"
+          DirIdx:  1
+          ModTime: 2
+          Length:  3
+        - Name:    "b.c"
+          DirIdx:  2
+          ModTime: 3
+          Length:  4
+      Opcodes: []
+
+## Generate and verify a big endian DWARF32 .debug_line section.
+
+# RUN: yaml2obj --docnum=1 -DENDIAN=ELFDATA2MSB %s -o %t1.be.o
+# RUN: llvm-readobj --sections --section-data %t1.be.o | \
+# RUN:   FileCheck %s -DSIZE=50 -DADDRALIGN=1 --check-prefixes=SHDR,DWARF-BE-CONTENT
+
+# DWARF-BE-CONTENT-NEXT: SectionData (
+# DWARF-BE-CONTENT-NEXT:   0000: 00000070 00020000 00320102 03040501
+##                               |        |   |        | |  | | | |
+##                               |        |   |        | |  | | | +- standard_opcode_lengths[DW_LNS_copy] (1-byte) 0x01
+##                               |        |   |        | |  | | +- opcode_base (1-byte) 0x05
+##                               |        |   |        | |  | +- line_range (1-byte) 0x04
+##                               |        |   |        | |  +- line_base (signed 1-byte) 0x03
+##                               |        |   |        | +- default_is_stmt (1-byte) 0x02
+##                               |        |   |        +- minimum_instruction_length (1-byte) 0x01
+##                               |        |   +-------- prologue_length (4-byte) 50
+##                               |        +--- version (2-byte) 0x02
+##                               +------- unit_length (4-byte) 0x70
+##
+# DWARF-BE-CONTENT-NEXT:   0010: 02030405 06070809 64697231 00646972
+##                               | | | |  | | | |  |          |
+##                               | | | |  | | | |  |          +----- include_directories[2] "dir2\0"
+##                               | | | |  | | | |  +---------- include_directories[1] "dir1\0"
+##                               | | | |  | | | +- standard_opcode_lengths[DW_LNS_fixed_advance_pc] (1-byte) 0x09
+##                               | | | |  | | +- standard_opcode_lengths[DW_LNS_const_add_pc] (1-byte) 0x08
+##                               | | | |  | +- standard_opcode_lengths[DW_LNS_set_basic_block] (1-byte) 0x07
+##                               | | | |  +- standard_opcode_lengths[DW_LNS_negate_stmt] (1-byte) 0x06
+##                               | | | +- standard_opcode_lengths[DW_LNS_set_column] (1-byte) 0x05
+##                               | | +- standard_opcode_lengths[DW_LNS_set_file] (1-byte) 0x04
+##                               | +- standard_opcode_lengths[DW_LNS_advance_line] (1-byte) 0x03
+##                               +- standard_opcode_lengths[DW_LNS_advance_pc] (1-byte) 0x02
+##
+# DWARF-BE-CONTENT-NEXT:   0020: 32000061 2E630001 0203622E 63000203
+##                               |   | |        |  | | |        | |
+##                               |   | |        |  | | |        | +- ModTime (ULEB128) 0x03
+##                               |   | |        |  | | |        +- DirIndex (ULEB128) 0x02
+##                               |   | |        |  | | +-------- file_names[2] "b.c\0"
+##                               |   | |        |  | +- Length (ULEB128) 0x03
+##                               |   | |        |  +- ModTime (ULEB128) 0x02
+##                               |   | |        +- DirIndex (ULEB128) 0x01
+##                               |   | +-------- file_names[1] "a.c\0"
+##                               |   +- terminating entry (1-byte) 0x00
+##                               +--- the last two bytes of "dir2\0"
+##
+# DWARF-BE-CONTENT-NEXT:   0030: 0400
+##                               | |
+##                               | +- terminating entry (1-byte) 0x00
+##                               +- Length (ULEB128) 0x04
+# DWARF-BE-CONTENT-NEXT: )
+
+## b) Generate the .debug_line section from raw section content.
+
+# RUN: yaml2obj --docnum=2 %s -o %t2.o
+# RUN: llvm-readobj --sections --section-data %t2.o | \
+# RUN:   FileCheck %s -DSIZE=3 -DADDRALIGN=0 --check-prefixes=SHDR,ARBITRARY-CONTENT
+
+#      ARBITRARY-CONTENT: SectionData (
+# ARBITRARY-CONTENT-NEXT:   0000: 112233 |."3|
+# ARBITRARY-CONTENT-NEXT: )
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:    .debug_line
+    Type:    SHT_PROGBITS
+    Content: "112233"
+
+## c) Generate the .debug_line section when the "Size" is specified.
+
+# RUN: yaml2obj --docnum=3 %s -o %t3.o
+# RUN: llvm-readobj --sections --section-data %t3.o | \
+# RUN:   FileCheck %s -DSIZE=16 -DADDRALIGN=0 --check-prefixes=SHDR,SIZE
+
+#      SIZE: SectionData (
+# SIZE-NEXT:   0000: 00000000 00000000 00000000 00000000 |................|
+# SIZE-NEXT: )
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name: .debug_line
+    Type: SHT_PROGBITS
+    Size: 0x10
+
+## d) Test that yaml2obj emits an error message when both the "Size" and the
+## "debug_line" entry are specified at the same time.
+
+# RUN: not yaml2obj --docnum=4 %s 2>&1 | FileCheck %s --check-prefix=ERROR
+
+# ERROR: yaml2obj: error: cannot specify section '.debug_line' contents in the 'DWARF' entry and the 'Content' or 'Size' in the 'Sections' entry at the same time
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name: .debug_line
+    Type: SHT_PROGBITS
+    Size: 0x10
+DWARF:
+  debug_line:
+    - Length:
+        TotalLength:         0x70
+      Version:               2
+      PrologueLength:        50
+      MinInstLength:         1
+      DefaultIsStmt:         1
+      LineBase:              1
+      LineRange:             14
+      OpcodeBase:            13
+      StandardOpcodeLengths: []
+      IncludeDirs:           []
+      Files:                 []
+      Opcodes:               []
+
+## e) Test that yaml2obj emits an error message when both the "Content" and the
+## "debug_line" entry are specified at the same time.
+
+# RUN: not yaml2obj --docnum=5 %s 2>&1 | FileCheck %s --check-prefix=ERROR
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:    .debug_line
+    Type:    SHT_PROGBITS
+    Content: "00"
+DWARF:
+  debug_line:
+    - Length:
+        TotalLength:         0x70
+      Version:               2
+      PrologueLength:        50
+      MinInstLength:         1
+      DefaultIsStmt:         1
+      LineBase:              1
+      LineRange:             14
+      OpcodeBase:            13
+      StandardOpcodeLengths: []
+      IncludeDirs:           []
+      Files:                 []
+      Opcodes:               []
+
+## f) Test that all the properties can be overridden by the section header when
+## the "debug_line" entry doesn't exist.
+
+# RUN: yaml2obj --docnum=6 %s -o %t6.o
+# RUN: llvm-readelf --sections %t6.o | FileCheck %s --check-prefix=OVERRIDDEN
+
+#      OVERRIDDEN: [Nr] Name        Type   Address          Off    Size   ES Flg Lk Inf Al
+#      OVERRIDDEN: [ 1] .debug_line STRTAB 0000000000002020 000050 000011 01   A  2   1  2
+# OVERRIDDEN-NEXT: [ 2] .sec        STRTAB 0000000000000000 000061 000000 00      0   0  0
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:         .debug_line
+    Type:         SHT_STRTAB  # SHT_PROGBITS by default.
+    Flags:        [SHF_ALLOC] # 0 by default.
+    Link:         .sec        # 0 by default.
+    EntSize:      1           # 0 by default.
+    Info:         1           # 0 by default.
+    AddressAlign: 2           # 0 by default.
+    Address:      0x2020      # 0x00 by default.
+    Offset:       0x50        # 0x40 for the first section.
+    Size:         0x11        # Set the "Size" so that we can reuse the check tag "OVERRIDDEN".
+  - Name:         .sec        # Linked by .debug_line.
+    Type:         SHT_STRTAB
+
+## g) Test that all the properties can be overridden by the section header when
+## the "debug_line" entry exists.
+
+# RUN: yaml2obj --docnum=7 %s -o %t7.o
+# RUN: llvm-readelf --sections %t7.o | FileCheck %s --check-prefix=OVERRIDDEN
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:         .debug_line
+    Type:         SHT_STRTAB  # SHT_PROGBITS by default.
+    Flags:        [SHF_ALLOC] # 0 by default.
+    Link:         .sec        # 0 by default.
+    EntSize:      1           # 0 by default.
+    Info:         1           # 0 by default.
+    AddressAlign: 2           # 1 by default.
+    Address:      0x2020      # 0x00 by default.
+    Offset:       0x50        # 0x40 for the first section.
+  - Name:         .sec        # Linked by .debug_line.
+    Type:         SHT_STRTAB
+DWARF:
+  debug_line:
+    - Length:
+        TotalLength:         0x70
+      Version:               2
+      PrologueLength:        50
+      MinInstLength:         1
+      DefaultIsStmt:         1
+      LineBase:              1
+      LineRange:             14
+      OpcodeBase:            13
+      StandardOpcodeLengths: []
+      IncludeDirs:           []
+      Files:                 []
+      Opcodes:               []
+
+## h) Test that yaml2obj will not generate the .debug_line section when the "DWARF" entry
+## exists but the "debug_line" entry doesn't exist in the "DWARF" entry or the "Sections" entry.
+
+# RUN: yaml2obj --docnum=8 %s -o %t8.o
+# RUN: llvm-readelf --sections %t8.o | FileCheck /dev/null --implicit-check-not=.debug_line
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+DWARF: