[DWARFYAML][debug_aranges] Replace InitialLength with Format and Length.

[Higuoxing]

Summary

This patch addresses the comment in D80972.

Before this patch, the initial length field of .debug_aranges section should be declared as:

## 32-bit DWARF
debug_aranges:
  - Length:
      TotalLength: 0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Length:
      TotalLength:   0xffffffff
      TotalLength64: 0x20
    Version: 2
    ...

After this patch:

## 32-bit DWARF
debug_aranges:
  - [[Format:  DWARF32]] ## Optional
    Length:  0x20
    Version: 2
    ...

## 64-bit DWARF
debug_aranges:
  - Format:  DWARF64
    Length:  0x20
    Version: 2

Current implementation of generating DWARF64 .debug_aranges section is buggy. A follow-up patch will improve it and add test cases for DWARF64.

[jhenderson]

llvm/include/llvm/ObjectYAML/DWARFYAML.h

68

I wouldn't bother with a length prefix. I'm not sure it's important for testing, because either the parser will read the DWARF64 marker (i.e. 0xffffffff) and treat the next 8 bytes as the rest of the length, or it won't in which case, it will treat those 4 bytes as the actual length, regardless of their value (with possible exceptions for the rest of the reserved range, but those values can be entered as the length in the YAML document anyway).

llvm/lib/ObjectYAML/DWARFYAML.cpp

78

I'd make this optional, with DWARF32 as the default. That's going to be the common case.

[Higuoxing]

llvm/include/llvm/ObjectYAML/DWARFYAML.h

68

Sorry, I'm a little bit lost here. What do you mean by saying

"but those values can be entered as the length in the YAML document anyway"?

I'm not sure if I understand it correctly. The 12-bit initial length is valid only when the format is DWARF64 (the prefix is 0xffffffff). When the prefix is in 0xfffffff0-0xfffffffe, the DWARF parser will not interpret the last 8-byte as length?

So we don't need LengthPrefix here, right? We can test the prefix in the preserved range by crafting something like

Format: DWARF32
Length: 0xfffffff0

[jhenderson]

llvm/include/llvm/ObjectYAML/DWARFYAML.h

68

Yes, exactly. The LengthPrefix isn't needed at all, I believe. The users options are:

Format: DWARF32
Length: <some value, including special values, that will just be written in the 4 bytes range, interpreted as 32-bits>

(the parser should bail on values in the reserved range other than 0xffffffff, and not try to carry on reading)

or

Format: DWARF64 (implies 0xffffffff for first 4 bytes)
Length: <64-bit length value>

I guess there might be an edge case where people want DWARF32 bit 8-byte offsets or DWARF64 with 4-byte offsets, but let's not worry about that. If people actually need that behaviour, it can be added later.

[Higuoxing]

• Make "Format" optional. (DWARF32 by default).
• Remove LengthPrefix.

Test case for 64-bit will be added in a follow-up patch.

[Higuoxing]

This comment was removed by Higuoxing.

[jhenderson]

LGTM, thanks.

Nit: in the title: replaced ... by -> replaced ... with

(It should be either "X is replaced by Y", or "Y replaces X", along with appropriate tense modifications etc)

llvm/include/llvm/ObjectYAML/DWARFYAML.h

 };
 
 struct ARangeDescriptor {
   llvm::yaml::Hex64 Address;
   uint64_t Length;
 };
 
 struct ARange {
-  InitialLength Length;
+  dwarf::DwarfFormat Format;
+  uint64_t Length;
   uint16_t Version;
   uint32_t CuOffset;
   uint8_t AddrSize;
   uint8_t SegSize;
   std::vector<ARangeDescriptor> Descriptors;
 };
 
 /// Class that describes a range list entry, or a base address selection entry

 template <> struct MappingTraits<DWARFYAML::LineTable> {
   static void mapping(IO &IO, DWARFYAML::LineTable &LineTable);
 };
 
 template <> struct MappingTraits<DWARFYAML::InitialLength> {
   static void mapping(IO &IO, DWARFYAML::InitialLength &DWARF);
 };
 
+template <> struct ScalarEnumerationTraits<dwarf::DwarfFormat> {
+  static void enumeration(IO &IO, dwarf::DwarfFormat &Format) {
+    IO.enumCase(Format, "DWARF32", dwarf::DWARF32);
+    IO.enumCase(Format, "DWARF64", dwarf::DWARF64);
+  }
+};
+
 #define HANDLE_DW_TAG(unused, name, unused2, unused3, unused4)                 \
   io.enumCase(value, "DW_TAG_" #name, dwarf::DW_TAG_##name);
 
 template <> struct ScalarEnumerationTraits<dwarf::Tag> {
   static void enumeration(IO &io, dwarf::Tag &value) {
 #include "llvm/BinaryFormat/Dwarf.def"
     io.enumFallback<Hex16>(value);
   }

llvm/lib/ObjectYAML/DWARFEmitter.cpp

 /// The DWARF component of yaml2obj. Provided as library code for tests.
 ///
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ObjectYAML/DWARFEmitter.h"
 #include "DWARFVisitor.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/BinaryFormat/Dwarf.h"
 #include "llvm/ObjectYAML/DWARFYAML.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/Host.h"
 #include "llvm/Support/LEB128.h"
 #include "llvm/Support/MathExtras.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/SwapByteOrder.h"
 #include "llvm/Support/YAMLTraits.h"
for (auto AbbrevDecl : DI.AbbrevDecls) {
     encodeULEB128(0, OS);
     encodeULEB128(0, OS);
   }
 }
 
 void DWARFYAML::EmitDebugAranges(raw_ostream &OS, const DWARFYAML::Data &DI) {
   for (auto Range : DI.ARanges) {
     auto HeaderStart = OS.tell();
-    writeInitialLength(Range.Length, OS, DI.IsLittleEndian);
+    if (Range.Format == dwarf::DWARF64) {
+      writeInteger((uint32_t)dwarf::DW_LENGTH_DWARF64, OS, DI.IsLittleEndian);
+      writeInteger((uint64_t)Range.Length, OS, DI.IsLittleEndian);
+    } else
+      writeInteger((uint32_t)Range.Length, OS, DI.IsLittleEndian);
     writeInteger((uint16_t)Range.Version, OS, DI.IsLittleEndian);
     writeInteger((uint32_t)Range.CuOffset, OS, DI.IsLittleEndian);
     writeInteger((uint8_t)Range.AddrSize, OS, DI.IsLittleEndian);
     writeInteger((uint8_t)Range.SegSize, OS, DI.IsLittleEndian);
 
     auto HeaderSize = OS.tell() - HeaderStart;
     auto FirstDescriptor = alignTo(HeaderSize, Range.AddrSize * 2);
     ZeroFillBytes(OS, FirstDescriptor - HeaderSize);

llvm/lib/ObjectYAML/DWARFYAML.cpp

 //===- DWARFYAML.cpp - DWARF YAMLIO implementation ------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file defines classes for handling the YAML representation of DWARF Debug
 // Info.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ObjectYAML/DWARFYAML.h"
+#include "llvm/BinaryFormat/Dwarf.h"
 
 namespace llvm {
 
 bool DWARFYAML::Data::isEmpty() const {
   return DebugStrings.empty() && AbbrevDecls.empty() && ARanges.empty() &&
          DebugRanges.empty() && !PubNames && !PubTypes && !GNUPubNames &&
          !GNUPubTypes && CompileUnits.empty() && DebugLines.empty();
 }

 void MappingTraits<DWARFYAML::ARangeDescriptor>::mapping(
     IO &IO, DWARFYAML::ARangeDescriptor &Descriptor) {
   IO.mapRequired("Address", Descriptor.Address);
   IO.mapRequired("Length", Descriptor.Length);
 }
 
 void MappingTraits<DWARFYAML::ARange>::mapping(IO &IO,
                                                DWARFYAML::ARange &ARange) {
+  IO.mapOptional("Format", ARange.Format, dwarf::DWARF32);
   IO.mapRequired("Length", ARange.Length);
   IO.mapRequired("Version", ARange.Version);
   IO.mapRequired("CuOffset", ARange.CuOffset);
   IO.mapRequired("AddrSize", ARange.AddrSize);
   IO.mapRequired("SegSize", ARange.SegSize);
   IO.mapRequired("Descriptors", ARange.Descriptors);
 }
 

llvm/test/ObjectYAML/MachO/DWARF-debug_aranges.yaml

- n_strx:          22
       n_value:         4294971216
   StringTable:     
     - ''
     - ''
     - __mh_execute_header
     - _main
 DWARF:           
   debug_aranges:   
-    - Length:          
-        TotalLength:     44
+    - Length:          44
       Version:         2
       CuOffset:        0
       AddrSize:        8
       SegSize:         0
       Descriptors:     
         - Address:         0x0000000100000F50
           Length:          52
 ...
 
 #CHECK: DWARF:           
 #CHECK:   debug_aranges:   
-#CHECK:     - Length:          
-#CHECK:         TotalLength:     44
+#CHECK:     - Length:          44
 #CHECK:       Version:         2
 #CHECK:       CuOffset:        0
 #CHECK:       AddrSize:        8
 #CHECK:       SegSize:         0
 #CHECK:       Descriptors:     
 #CHECK:         - Address:         0x0000000100000F50
 #CHECK:           Length:          52

llvm/test/ObjectYAML/MachO/DWARF-debug_info.yaml

- Code:            0x00000004
           Form:            DW_FORM_data1
     - Code:            0x00000005
       Tag:             DW_TAG_pointer_type
       Children:        DW_CHILDREN_no
       Attributes:      
         - Attribute:       DW_AT_type
           Form:            DW_FORM_ref4
   debug_aranges:   
-    - Length:          
-        TotalLength:     44
+    - Length:          44
       Version:         2
       CuOffset:        0
       AddrSize:        8
       SegSize:         0
       Descriptors:     
         - Address:         0x0000000100000F50
           Length:          52
   debug_info:      

llvm/test/ObjectYAML/MachO/DWARF-debug_line.yaml

- Code:            0x00000004
           Form:            DW_FORM_data1
     - Code:            0x00000005
       Tag:             DW_TAG_pointer_type
       Children:        DW_CHILDREN_no
       Attributes:      
         - Attribute:       DW_AT_type
           Form:            DW_FORM_ref4
   debug_aranges:   
-    - Length:          
-        TotalLength:     44
+    - Length:          44
       Version:         2
       CuOffset:        0
       AddrSize:        8
       SegSize:         0
       Descriptors:     
         - Address:         0x0000000100000F50
           Length:          52
   debug_pubnames:  

llvm/test/ObjectYAML/MachO/DWARF5-debug_info.yaml

- Code:            0x00000004
           Form:            DW_FORM_data1
     - Code:            0x00000005
       Tag:             DW_TAG_pointer_type
       Children:        DW_CHILDREN_no
       Attributes:      
         - Attribute:       DW_AT_type
           Form:            DW_FORM_ref4
   debug_aranges:   
-    - Length:          
-        TotalLength:     44
+    - Length:          44
       Version:         2
       CuOffset:        0
       AddrSize:        8
       SegSize:         0
       Descriptors:     
         - Address:         0x0000000100000F50
           Length:          52
   debug_info:      

llvm/test/tools/llvm-gsymutil/ARM_AArch64/fat-macho-dwarf.yaml

DWARF:
               Form:            DW_FORM_ref_addr
         - Code:            0x00000006
           Tag:             DW_TAG_const_type
           Children:        DW_CHILDREN_no
           Attributes:
             - Attribute:       DW_AT_type
               Form:            DW_FORM_ref_addr
       debug_aranges:
-        - Length:
-            TotalLength:     28
+        - Length:          28
           Version:         2
           CuOffset:        0
           AddrSize:        4
           SegSize:         0
           Descriptors:
             - Address:         0x000000000000BFF0
               Length:          16
       debug_pubnames:
DWARF:
               Form:            DW_FORM_ref_addr
         - Code:            0x00000006
           Tag:             DW_TAG_const_type
           Children:        DW_CHILDREN_no
           Attributes:
             - Attribute:       DW_AT_type
               Form:            DW_FORM_ref_addr
       debug_aranges:
-        - Length:
-            TotalLength:     44
+        - Length:          44
           Version:         2
           CuOffset:        0
           AddrSize:        8
           SegSize:         0
           Descriptors:
             - Address:         0x0000000100007F9C
               Length:          28
       debug_pubnames:

llvm/test/tools/llvm-gsymutil/X86/mach-dwarf.yaml

- Code:            0x0000000B
           Form:            DW_FORM_ref_addr
     - Code:            0x0000000C
       Tag:             DW_TAG_const_type
       Children:        DW_CHILDREN_no
       Attributes:
         - Attribute:       DW_AT_type
           Form:            DW_FORM_ref_addr
   debug_aranges:
-    - Length:
-        TotalLength:     60
+    - Length:          60
       Version:         2
       CuOffset:        0
       AddrSize:        8
       SegSize:         0
       Descriptors:
         - Address:         0x0000000100000F70
           Length:          17
         - Address:         0x0000000100000F90

llvm/test/tools/obj2yaml/MachO/DWARF-debug_aranges-error.yaml

Sections:
         reloff:          0x00000000
         nreloc:          0
         flags:           0x00000000
         reserved1:       0x00000000
         reserved2:       0x00000000
         reserved3:       0x00000000
 DWARF:
   debug_aranges:
-    - Length:
-        TotalLength:     45
+    - Length:          45
       Version:         2
       CuOffset:        0
       AddrSize:        8
       SegSize:         0
       Descriptors:
         - Address:         0x0000000100000F50
           Length:          52
 ...
 
 #CHECK: the length of address range table at offset 0x0 exceeds section size

llvm/test/tools/yaml2obj/ELF/DWARF/debug-aranges.yaml

 ## Test that yaml2obj emits .debug_aranges section.
 
 ## a) Generate the .debug_aranges section from the "DWARF" entry.
 
-## Generate and verify a big endian object file.
+## Generate and verify a big endian DWARF32 .debug_aranges section.
 
-# RUN: yaml2obj --docnum=1 -DENDIAN=ELFDATA2MSB %s -o %t.be.o
+# RUN: yaml2obj --docnum=1 -DENDIAN=ELFDATA2MSB -DFORMAT=DWARF32 %s -o %t.be.o
 # RUN: llvm-readobj --sections --section-data %t.be.o | \
 # RUN:   FileCheck %s -DADDRALIGN=1 -DSIZE=96 --check-prefixes=DWARF-BE-HEADER,DWARF-BE-CONTENT
 
 #       DWARF-BE-HEADER: Index: 1
 #  DWARF-BE-HEADER-NEXT: Name: .debug_aranges (1)
 #  DWARF-BE-HEADER-NEXT: Type: SHT_PROGBITS (0x1)
 #  DWARF-BE-HEADER-NEXT: Flags [ (0x0)
 #  DWARF-BE-HEADER-NEXT: ]

 --- !ELF
 FileHeader:
   Class:   ELFCLASS64
   Data:    [[ENDIAN]]
   Type:    ET_EXEC
   Machine: EM_X86_64
 DWARF:
   debug_aranges:
-    - Length:
-        TotalLength: 0x2c
+    - Format:   [[FORMAT]]
+      Length:   0x2c
       Version:  2
       CuOffset: 0
       AddrSize: 0x04
       SegSize:  0x00
       Descriptors:
         - Address: 0x00001234
           Length:  0x20
-    - Length:
-        TotalLength: 0x2c
+    - Format:   [[FORMAT]]
+      Length:   0x2c
       Version:  2
       CuOffset: 0x65
       AddrSize: 0x08
       SegSize:  0x00
       Descriptors:
         - Address: 0x0000000000005678
           Length:  0x20
         - Address: 0x0000000056780000
           Length:  0x10
 
-## Generate and verify a little endian object file.
+## Generate and verify a little endian DWARF32 .debug_aranges section.
 
-# RUN: yaml2obj --docnum=1 -DENDIAN=ELFDATA2LSB %s -o %t.le.o
+# RUN: yaml2obj --docnum=1 -DENDIAN=ELFDATA2LSB -DFORMAT=DWARF32 %s -o %t.le.o
 # RUN: llvm-readobj --sections --section-data %t.le.o | \
 # RUN:   FileCheck %s --check-prefixes=DWARF-LE-DEFAULT
 
 #      DWARF-LE-DEFAULT: Section {
 #      DWARF-LE-DEFAULT:   Index: 1
 # DWARF-LE-DEFAULT-NEXT:   Name: .debug_aranges (1)
 # DWARF-LE-DEFAULT-NEXT:   Type: SHT_PROGBITS (0x1)
 # DWARF-LE-DEFAULT-NEXT:   Flags [ (0x0)
FileHeader:
   Type:    ET_EXEC
   Machine: EM_X86_64
 Sections:
   - Name: .debug_aranges
     Type: SHT_PROGBITS
     Size: 0x10
 DWARF:
   debug_aranges:
-    - Length:
-        TotalLength: 0x2c
+    - Length:   0x2c
       Version:  2
       CuOffset: 0
       AddrSize: 0x08
       SegSize:  0x00
       Descriptors:
         - Address: 0x0000000000001234
           Length:  0x20
 
FileHeader:
   Type:    ET_EXEC
   Machine: EM_X86_64
 Sections:
   - Name:    .debug_aranges
     Type:    SHT_PROGBITS
     Content: "00"
 DWARF:
   debug_aranges:
-    - Length:
-        TotalLength: 0x2c
+    - Length:   0x2c
       Version:  2
       CuOffset: 0
       AddrSize: 0x08
       SegSize:  0x00
       Descriptors:
         - Address: 0x0000000000001234
           Length:  0x20
 
- Name:         .debug_aranges
     Info:         1                  # 0 by default.
     AddressAlign: 2                  # 1 by default.
     Address:      0x0000000000002020 # 0x00 by default.
     Offset:       0x00000050         # 0x40 for the first section.
   - Name:         .sec               # Linked by .debug_aranges.
     Type:         SHT_STRTAB
 DWARF:
   debug_aranges:
-    - Length:
-        TotalLength: 0x2c
+    - Length:   0x2c
       Version:  2
       CuOffset: 0
       AddrSize: 0x08
       SegSize:  0x00
       Descriptors:
         - Address: 0x0000000000001234
           Length:  0x20
 

llvm/tools/obj2yaml/dwarf2yaml.cpp

DWARFDataExtractor ArangesData(DCtx.getDWARFObj().getArangesSection(),
                                  DCtx.isLittleEndian(), 0);
   uint64_t Offset = 0;
   DWARFDebugArangeSet Set;
 
   while (ArangesData.isValidOffset(Offset)) {
     if (Error E = Set.extract(ArangesData, &Offset))
       return E;
     DWARFYAML::ARange Range;
-    Range.Length.setLength(Set.getHeader().Length);
+    Range.Format = Set.getHeader().Format;
+    Range.Length = Set.getHeader().Length;
     Range.Version = Set.getHeader().Version;
     Range.CuOffset = Set.getHeader().CuOffset;
     Range.AddrSize = Set.getHeader().AddrSize;
     Range.SegSize = Set.getHeader().SegSize;
     for (auto Descriptor : Set.descriptors()) {
       DWARFYAML::ARangeDescriptor Desc;
       Desc.Address = Descriptor.Address;
       Desc.Length = Descriptor.Length;