[ObjectYAML][ELF] Add support for emitting the .debug_abbrev section.

[Higuoxing]

Summary

This patch enables yaml2elf emit the .debug_abbrev section.

The generated .debug_abbrev is verified using llvm-dwarfdump.

Known issues that will be addressed later:

• Current implementation doesn't support generating multiple abbreviation tables in one .debug_abbrev section.

[grimar]

Should it have all 32/64 LE/BE set of tests?

llvm/test/tools/yaml2obj/ELF/DWARF/debug-abbrev.yaml

207

# -> ##?

[Higuoxing]

In D81820#2092347, @grimar wrote:

Should it have all 32/64 LE/BE set of tests?

I think we don't need it. Values in abbreviations tables is encoded in LEB128.

[jhenderson]

From your description: "This patch enables yaml2elf emit the .debug_line section.": s/.debug_line/.debug_abbrev

llvm/test/tools/yaml2obj/ELF/DWARF/debug-abbrev.yaml

94

It would also be good to test a Form value that requires more than one byte, e.g. 0x81.

[Higuoxing]

Address comments.

[jhenderson]

LGTM, but please wait for others to confirm.

llvm/test/tools/yaml2obj/ELF/DWARF/debug-abbrev.yaml

93–94

Fine to leave this here, but you do have that with the 0x2020 value below :)

[grimar]

LGTM too.

[MaskRay]

I asked around how we should comment individual bytes.

lichray gave an alternative:

# CONTENT-NEXT:                0000:  01 11 01 25
                                      ~~ ~~ ~~ ~~
                                       ^  ^  ^  ^
                                       |  |  |  |
      ULEB128  abbreviation code ______|  |  |  |
      ULEB128  DW_TAG_compile_unit _______|  |  |
       1-byte  DW_CHILDREN_yes ______________|  |
      ULEB128  DW_AT_producer __________________|

                                      0E 13 05 03
                                      ~~ ~~ ~~ ~~
                                       ^  ^  ^  ^
                                       |  |  |  |
      ULEB128  DW_FORM_strp ___________|  |  |  |
      ULEB128  DW_AT_language ____________|  |  |
      ULEB128  DW_FORM_data2 ________________|  |
      ULEB128  DW_AT_name ______________________|

wanders: Not radically different, but if you put the comment above instead it will read from top instead of from bottom.

(Request changes just to make sure these suggestions are considered and the comments should be improved if you think any suggestion is favorable.)

[MaskRay]

llvm/lib/ObjectYAML/DWARFYAML.cpp

35

Probably use an alphabetical order.

[Higuoxing]

In D81820#2093285, @MaskRay wrote:

I asked around how we should comment individual bytes.

lichway gave an alternative:

# CONTENT-NEXT:                0000:  01 11 01 25
                                      ~~ ~~ ~~ ~~
                                       ^  ^  ^  ^
                                       |  |  |  |
      ULEB128  abbreviation code ______|  |  |  |
      ULEB128  DW_TAG_compile_unit _______|  |  |
       1-byte  DW_CHILDREN_yes ______________|  |
      ULEB128  DW_AT_producer __________________|

                                      0E 13 05 03
                                      ~~ ~~ ~~ ~~
                                       ^  ^  ^  ^
                                       |  |  |  |
      ULEB128  DW_FORM_strp ___________|  |  |  |
      ULEB128  DW_AT_language ____________|  |  |
      ULEB128  DW_FORM_data2 ________________|  |
      ULEB128  DW_AT_name ______________________|

wanders: Not radically different, but if you put the comment above instead it will read from top instead of from bottom.

(Request changes just to make sure these suggestions are considered and the comments should be improved if you think any suggestion is favorable.)

Hi Fangrui, thanks for the suggestion.

It seems that FileCheck cannot match bytes one by one.

## Cannot match
# CONTENT-NEXT: 0000: 01 11 01 25
## Can match
# CONTENT-NEXT: 0000: 01110125

I have also checked the manual and there's no such an option that makes FileCheck happy with this match pattern. Did I miss something?

What do you think of having something like this?

# CONTENT-NEXT: 0000: 01110125
                      ^~^~^~^~
                ______| | | |
                ________| | |
                __________| |
                ____________|

OR

# CONTENT-NEXT: 0000: 01110125
                      ^~       ULEB28 abbreviation code
                        ^~     ULEB128 DW_TAG_compile_unit
                          ^~   1-byte DW_CHILDREN_yes
                            ^~ ULEB128 DW_AT_producer

llvm/lib/ObjectYAML/DWARFYAML.cpp

35

Oh, thanks for spotting it. Can I re-arrange them in a follow-up patch?

[Higuoxing]

Address @MaskRay 's comments.

• Put comments above check line.

[MaskRay]

In D81820#2094624, @Higuoxing wrote:

OR

# CONTENT-NEXT: 0000: 01110125
                      ^~       ULEB28 abbreviation code
                        ^~     ULEB128 DW_TAG_compile_unit
                          ^~   1-byte DW_CHILDREN_yes
                            ^~ ULEB128 DW_AT_producer

This style looks good to me. This is similar to lld/test/ELF/mips-gp-ext.s, which uses

# ABS:      Contents of section .text:
# ABS-NEXT:  00e0 3c080000 21080120 8f82ff08
#                 ^-- %hi(_gp_disp)
#                          ^-- %lo(_gp_disp)
#                                   ^-- 8 - (0x200 - 0x100)
#                                       G - (GP - .got)

[lichray]

In D81820#2098654, @MaskRay wrote:

In D81820#2094624, @Higuoxing wrote:

OR

# CONTENT-NEXT: 0000: 01110125
                      ^~       ULEB28 abbreviation code
                        ^~     ULEB128 DW_TAG_compile_unit
                          ^~   1-byte DW_CHILDREN_yes
                            ^~ ULEB128 DW_AT_producer

It may be more readable if the two columns are aligned separately:

# CONTENT-NEXT: 0000: 01110125
                      ^~       abbreviation code    ULEB128
                        ^~     DW_TAG_compile_unit  ULEB128
                          ^~   DW_CHILDREN_yes       1-byte
                            ^~ DW_AT_producer       ULEB128

[Higuoxing]

Address comments.

[MaskRay]

Looks great! And thanks a lot to @lichray's suggestions

[Higuoxing]

Thanks for giving suggestions and reviewing!

llvm/lib/ObjectYAML/DWARFYAML.cpp

SetVector<StringRef> DWARFYAML::Data::getUsedSectionNames() const {
   if (!DebugStrings.empty())
     SecNames.insert("debug_str");
   if (!ARanges.empty())
     SecNames.insert("debug_aranges");
   if (!DebugRanges.empty())
     SecNames.insert("debug_ranges");
   if (!DebugLines.empty())
     SecNames.insert("debug_line");
+  if (!AbbrevDecls.empty())
+    SecNames.insert("debug_abbrev");
   return SecNames;
 }
 
 namespace yaml {
 
 void MappingTraits<DWARFYAML::Data>::mapping(IO &IO, DWARFYAML::Data &DWARF) {
   auto oldContext = IO.getContext();
   IO.setContext(&DWARF);

llvm/lib/ObjectYAML/ELFEmitter.cpp

Expected<uint64_t> emitDWARF(typename ELFT::Shdr &SHeader, StringRef Name,
   if (Name == ".debug_str")
     Err = DWARFYAML::emitDebugStr(*OS, DWARF);
   else if (Name == ".debug_aranges")
     Err = DWARFYAML::emitDebugAranges(*OS, DWARF);
   else if (Name == ".debug_ranges")
     Err = DWARFYAML::emitDebugRanges(*OS, DWARF);
   else if (Name == ".debug_line")
     Err = DWARFYAML::emitDebugLine(*OS, DWARF);
+  else if (Name == ".debug_abbrev")
+    Err = DWARFYAML::emitDebugAbbrev(*OS, DWARF);
   else
     llvm_unreachable("unexpected emitDWARF() call");
 
   if (Err)
     return std::move(Err);
 
   return CBA.tell() - BeginOffset;
 }

llvm/test/tools/yaml2obj/ELF/DWARF/debug-abbrev.yaml

+## Test that yaml2obj emits .debug_abbrev section.
+
+## a) Generate the .debug_abbrev section from the "DWARF" entry.
+
+# RUN: yaml2obj --docnum=1 %s -o %t1.o
+# RUN: llvm-readobj --sections --section-data %t1.o | \
+# RUN:   FileCheck -DSIZE=38 -DADDRALIGN=1 %s --check-prefixes=SHDR,CONTENT
+
+#         SHDR: Index: 1
+#    SHDR-NEXT: Name: .debug_abbrev (1)
+#    SHDR-NEXT: Type: SHT_PROGBITS (0x1)
+#    SHDR-NEXT: Flags [ (0x0)
+#    SHDR-NEXT: ]
+#    SHDR-NEXT: Address: 0x0
+#    SHDR-NEXT: Offset: 0x40
+#    SHDR-NEXT: Size: [[SIZE]]
+#    SHDR-NEXT: Link: 0
+#    SHDR-NEXT: Info: 0
+#    SHDR-NEXT: AddressAlignment: [[ADDRALIGN]]
+#    SHDR-NEXT: EntrySize: 0
+# CONTENT-NEXT: SectionData (
+# CONTENT-NEXT:   0000: 01110125 0E130503 1A000002 2E011101
+##                      | | | |  | | | |  | |   |  | | | |
+##                      | | | |  | | | |  | |   |  | | | +- DW_FORM_addr (ULEB128) 0x01
+##                      | | | |  | | | |  | |   |  | | +- DW_AT_low_pc (ULEB128) 0x11
+##                      | | | |  | | | |  | |   |  | +- DW_CHILDREN_yes (1-byte) 0x01
+##                      | | | |  | | | |  | |   |  +- DW_TAG_subprogram (ULEB128) 0x2e
+##                      | | | |  | | | |  | |   +- abbreviation code (ULEB128) 0x02
+##                      | | | |  | | | |  | +--- terminating entry.
+##                      | | | |  | | | |  +- DW_AT_strx (UELB128) 0x1a
+##                      | | | |  | | | +- DW_AT_name (ULEB128) 0x03
+##                      | | | |  | | +- DW_FORM_data2 (ULEB128)0x05
+##                      | | | |  | +- DW_AT_language (ULEB128) 0x13
+##                      | | | |  +- DW_FORM_strp (ULEB128) 0x0e
+##                      | | | +- DW_AT_producer (ULEB128) 0x25
+##                      | | +- DW_CHILDREN_yes (1-byte) 0x01
+##                      | +- DW_TAG_compile_unit (ULEB128) 0x11
+##                      +- abbreviation code (ULEB128) 0x01
+##
+# CONTENT-NEXT:   0010: 121B0000 03060081 01810104 02A04021
+##                      | | |    | | | |    |   |  | |   |
+##                      | | |    | | | |    |   |  | |   |
+##                      | | |    | | | |    |   |  | |   +- DW_FORM_implicit_const (ULEB128) 0x21
+##                      | | |    | | | |    |   |  | +--- Attribute: reserved value (ULEB128) 0x2020
+##                      | | |    | | | |    |   |  +- Form: reserved value (0x02)
+##                      | | |    | | | |    |   +- Attribute: reserved value (0x04)
+##                      | | |    | | | |    +--- Form: invalid value (ULEB128) 0x81
+##                      | | |    | | | +---- Attribute: DW_AT_call_pc (ULEB128) 0x81
+##                      | | |    | | +- DW_CHILDREN_no (1-byte) 0x00
+##                      | | |    | +- Tag: reserved value (0x06)
+##                      | | |    +- abbreviation code (ULEB128) 0x03
+##                      | | +--- terminating entry.
+##                      | +- DW_FORM_addrx (ULEB128) 0x1b
+##                      +- DW_AT_high_pc (ULEB128) 0x12
+##
+# CONTENT-NEXT:   0020: CEC2F105 0000
+##                      |        |
+##                      |        +--- terminating entry.
+##                      +------- Value: (SLEB128) 12345678
+# CONTENT-NEXT: )
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+DWARF:
+  debug_abbrev:
+    - Code:     1
+      Tag:      DW_TAG_compile_unit
+      Children: DW_CHILDREN_yes
+      Attributes:
+        - Attribute: DW_AT_producer
+          Form:      DW_FORM_strp
+        - Attribute: DW_AT_language
+          Form:      DW_FORM_data2
+        - Attribute: DW_AT_name
+          Form:      DW_FORM_strx
+    - Code:     2
+      Tag:      DW_TAG_subprogram
+      Children: DW_CHILDREN_yes
+      Attributes:
+        - Attribute: DW_AT_low_pc
+          Form:      DW_FORM_addr
+        - Attribute: DW_AT_high_pc
+          Form:      DW_FORM_addrx
+    - Code:     3
+      ## Test a reserved tag value.
+      Tag:      0x06
+      Children: DW_CHILDREN_no
+      Attributes:
+          ## Test an attribute value that is more than one byte.
+        - Attribute: DW_AT_call_pc
+          ## Test a form value that is more than one byte.
+          Form:      0x81
+          ## Test a reserved attribute value.
+        - Attribute: 0x04
+          ## Test a reserved form value.
+          Form:      0x02
+        - Attribute: 0x2020
+          ## Test one special attribute form DW_FORM_implicit_const,
+          ## who is followed by a SLEB128 value.
+          Form:      DW_FORM_implicit_const
+          Value:     12345678
+
+## b) Generate the .debug_abbrev section from raw section content.
+
+# RUN: yaml2obj --docnum=2 %s -o %t2.o
+# RUN: llvm-readobj --sections --section-data %t2.o | \
+# RUN:   FileCheck -DADDRALIGN=0 -DSIZE=3 %s --check-prefixes=SHDR,ARBITRARY-CONTENT
+
+# ARBITRARY-CONTENT-NEXT: SectionData (
+# ARBITRARY-CONTENT-NEXT:   0000: 112233
+# ARBITRARY-CONTENT-NEXT: )
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:    .debug_abbrev
+    Type:    SHT_PROGBITS
+    Content: "112233"
+
+## c) Generate the .debug_abbrev section when the "Size" is specified.
+
+# RUN: yaml2obj --docnum=3 %s -o %t3.o
+# RUN: llvm-readobj --sections --section-data %t3.o | \
+# RUN:   FileCheck -DSIZE=16 -DADDRALIGN=0 %s --check-prefixes=SHDR,SIZE
+
+# SIZE-NEXT: SectionData (
+# SIZE-NEXT:   0000: 00000000 00000000 00000000 00000000
+# SIZE-NEXT: )
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name: .debug_abbrev
+    Type: SHT_PROGBITS
+    Size: 0x10
+
+## d) Test that yaml2obj emits an error message when both the "Size" and the
+## "debug_abbrev" entry are specified at the same time.
+
+# RUN: not yaml2obj --docnum=4 %s 2>&1 | FileCheck %s --check-prefix=ERROR
+
+# ERROR: yaml2obj: error: cannot specify section '.debug_abbrev' contents in the 'DWARF' entry and the 'Content' or 'Size' in the 'Sections' entry at the same time
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name: .debug_abbrev
+    Type: SHT_PROGBITS
+    Size: 0x10
+DWARF:
+  debug_abbrev:
+    - Code:       1
+      Tag:        DW_TAG_compile_unit
+      Children:   DW_CHILDREN_no
+      Attributes: []
+
+## e) Test that yaml2obj emits an error message when both the "Content" and the
+## "debug_abbrev" entry are specified at the same time.
+
+# RUN: not yaml2obj --docnum=5 %s 2>&1 | FileCheck %s --check-prefix=ERROR
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:    .debug_abbrev
+    Type:    SHT_PROGBITS
+    Content: "00"
+DWARF:
+  debug_abbrev:
+    - Code:       1
+      Tag:        DW_TAG_compile_unit
+      Children:   DW_CHILDREN_no
+      Attributes: []
+
+## f) Test that all the properties can be overridden by the section header when
+## the "debug_abbrev" entry doesn't exist.
+
+# RUN: yaml2obj --docnum=6 %s -o %t6.o
+# RUN: llvm-readelf --sections %t6.o | FileCheck %s --check-prefix=OVERRIDDEN
+
+#      OVERRIDDEN: [Nr] Name          Type   Address          Off    Size   ES Flg Lk Inf Al
+#      OVERRIDDEN: [ 1] .debug_abbrev STRTAB 0000000000002020 000050 000005 01   A  2   1  2
+# OVERRIDDEN-NEXT: [ 2] .sec          STRTAB 0000000000000000 000055 000000 00      0   0  0
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:         .debug_abbrev
+    Type:         SHT_STRTAB  ## SHT_PROGBITS by default.
+    Flags:        [SHF_ALLOC] ## 0 by default.
+    Link:         .sec        ## 0 by default.
+    EntSize:      1           ## 0 by default.
+    Info:         1           ## 0 by default.
+    AddressAlign: 2           ## 0 by default.
+    Address:      0x2020      ## 0x00 by default.
+    Offset:       0x50        ## 0x40 for the first section.
+    Size:         0x05        ## Set the "Size" so that we can reuse the check tag "OVERRIDDEN".
+  - Name:         .sec        ## Linked by .debug_abbrev.
+    Type:         SHT_STRTAB
+
+## g) Test that all the properties can be overridden by the section header when
+## the "debug_abbrev" entry exists.
+
+# RUN: yaml2obj --docnum=7 %s -o %t7.o
+# RUN: llvm-readelf --sections %t7.o | FileCheck %s --check-prefix=OVERRIDDEN
+
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_X86_64
+Sections:
+  - Name:         .debug_abbrev
+    Type:         SHT_STRTAB  ## SHT_PROGBITS by default.
+    Flags:        [SHF_ALLOC] ## 0 by default.
+    Link:         .sec        ## 0 by default.
+    EntSize:      1           ## 0 by default.
+    Info:         1           ## 0 by default.
+    AddressAlign: 2           ## 1 by default.
+    Address:      0x2020      ## 0x00 by default.
+    Offset:       0x50        ## 0x40 for the first section.
+  - Name:         .sec        ## Linked by .debug_abbrev.
+    Type:         SHT_STRTAB
+DWARF:
+  debug_abbrev:
+    - Code:       1
+      Tag:        DW_TAG_compile_unit
+      Children:   DW_CHILDREN_no
+      Attributes: []