[MCDwarf] Respect -fdebug-prefix-map= for generated assembly debug info (DWARF v5)

[MaskRay]

Summary

For generated assembly debug info, MCDwarfLineTableHeader::CompilationDir is an
unmapped path set in MCContext::setGenDwarfRootFile. Remap it.

A relative destination path of -fdebug-prefix-map= exposes a llvm-dwarfdump bug
which joins directories[0] into DW_AT_comp_dir.

Fix https://github.com/llvm/llvm-project/issues/56609

[raj.khem]

I am getting a compiler error on clang-15, can you confirm if this will work with clang-15 too

I am seeing

| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/ADT/Optional.h: In instantiation of ‘llvm::optional_d
etail::OptionalStorage<T, <anonymous> >& llvm::optional_detail::OptionalStorage<T, <anonymous> >::operator=(T&&) [with T = llvm::MCDwarfLineStr; bo
ol <anonymous> = false]’:
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/ADT/Optional.h:281:13:   required from ‘llvm::Optiona
l<T>& llvm::Optional<T>::operator=(T&&) [with T = llvm::MCDwarfLineStr]’
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/lib/MC/MCDwarf.cpp:269:37:   required from here
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/ADT/Optional.h:139:11: error: use of deleted function
 ‘llvm::MCDwarfLineStr& llvm::MCDwarfLineStr::operator=(llvm::MCDwarfLineStr&&)’
|   139 |       val = std::move(y);
|       |       ~~~~^~~~~~~~~~~~~~                                                                                                                 | /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/MC/MCDwarf.h:51:7: note: ‘llvm::MCDwarfLineStr& llvm:
:MCDwarfLineStr::operator=(llvm::MCDwarfLineStr&&)’ is implicitly deleted because the default definition would be ill-formed:                      |    51 | class MCDwarfLineStr {
|       |       ^~~~~~~~~~~~~~
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/MC/MCDwarf.h:51:7: error: use of deleted function ‘ll
vm::StringSaver& llvm::StringSaver::operator=(llvm::StringSaver&&)’
| In file included from /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/MC/MCDwarf.h:25:
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/Support/StringSaver.h:21:7: note: ‘llvm::StringSaver&
 llvm::StringSaver::operator=(llvm::StringSaver&&)’ is implicitly deleted because the default definition would be ill-formed:                      |    21 | class StringSaver final {
|       |       ^~~~~~~~~~~
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/Support/StringSaver.h:21:7: error: non-static referen
ce member ‘llvm::BumpPtrAllocator& llvm::StringSaver::Alloc’, cannot use default assignment operator

[MaskRay]

In D131749#3718437, @raj.khem wrote:

I am getting a compiler error on clang-15, can you confirm if this will work with clang-15 too

I am seeing

| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/ADT/Optional.h: In instantiation of ‘llvm::optional_d
etail::OptionalStorage<T, <anonymous> >& llvm::optional_detail::OptionalStorage<T, <anonymous> >::operator=(T&&) [with T = llvm::MCDwarfLineStr; bo
ol <anonymous> = false]’:
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/ADT/Optional.h:281:13:   required from ‘llvm::Optiona
l<T>& llvm::Optional<T>::operator=(T&&) [with T = llvm::MCDwarfLineStr]’
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/lib/MC/MCDwarf.cpp:269:37:   required from here
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/ADT/Optional.h:139:11: error: use of deleted function
 ‘llvm::MCDwarfLineStr& llvm::MCDwarfLineStr::operator=(llvm::MCDwarfLineStr&&)’
|   139 |       val = std::move(y);
|       |       ~~~~^~~~~~~~~~~~~~                                                                                                                 | /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/MC/MCDwarf.h:51:7: note: ‘llvm::MCDwarfLineStr& llvm:
:MCDwarfLineStr::operator=(llvm::MCDwarfLineStr&&)’ is implicitly deleted because the default definition would be ill-formed:                      |    51 | class MCDwarfLineStr {
|       |       ^~~~~~~~~~~~~~
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/MC/MCDwarf.h:51:7: error: use of deleted function ‘ll
vm::StringSaver& llvm::StringSaver::operator=(llvm::StringSaver&&)’
| In file included from /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/MC/MCDwarf.h:25:
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/Support/StringSaver.h:21:7: note: ‘llvm::StringSaver&
 llvm::StringSaver::operator=(llvm::StringSaver&&)’ is implicitly deleted because the default definition would be ill-formed:                      |    21 | class StringSaver final {
|       |       ^~~~~~~~~~~
| /mnt/b/yoe/master/build/tmp/work-shared/llvm-project-source-15.0.0-r0/git/llvm/include/llvm/Support/StringSaver.h:21:7: error: non-static referen
ce member ‘llvm::BumpPtrAllocator& llvm::StringSaver::Alloc’, cannot use default assignment operator

You need to rebase after b0c4cd35df89479ec152c1f79e18d0264dd276cc which avoids move assignment.

[dblaikie]

llvm/test/MC/ELF/debug-prefix-map.s

36–40

I'm not sure this comment is correct (perhaps there's a bug in llvm-dwarfdump if both include_directories[0] and DW_AT_comp_dir are both being joined together). From the DWARFv5 spec:

The first entry is the current directory of the compilation. Each additional path entry is either a full path name or is relative to the current directory of the compilation.
The line number program assigns a number (index) to each of the directory entries in order, beginning with 0.
Prior to DWARF Version 5, the current directory was not represented in the directories field and a directory index of 0 implicitly referred to that directory as found in the DW_AT_comp_dir attribute of the compilation unit debugging information entry. In DWARF Version 5, the current directory is explicitly present in the directories field.

So maybe llvm-dwarfdump is incorrectly joining DW_AT_comp_dir to include_directories[0]?

I'm not sure the suggestion that -fdebug-prefix-map= is meant to be absolute is the case in practice - I think Chromium folks implemented it so they could have relative debug info that could be built on a build farm and shared between multiple developers with different source tree locations. So I /think/ it's probably usually relative (the same as Google's use of -fdebug-compilation-dir=/proc/self/cwd (which, is, admittedly, technically, an absolute path - but I don't think that's the case for Chromium)... huh, maybe chromium doesn't use prefix map anymore? https://chromium.googlesource.com/chromium/src/+/71df8872b86b4d8260230742cfd3b2e4756f62b5%5E%21/#F0 )

[MaskRay]

Update comment that src_root/src_root/file is a llvm-dwarfdump bug.

[MaskRay]

llvm/test/MC/ELF/debug-prefix-map.s

36–40

Groups appear to migrate to -fdebug-compilation-dir=. which is more convenient. -fdebug-prefix-map= is used less.

I'll see about fixing llvm-dwarfdump. Thanks for noticing the bug (which appears in dwarfdump, too).

[dblaikie]

Generally seems good to me, thanks!

llvm/test/MC/ELF/debug-prefix-map.s

36–40

Groups appear to migrate to -fdebug-compilation-dir=. which is more convenient. -fdebug-prefix-map= is used less.

Hmm, yeah, seems generally better - though I guess GCC also has -fdebug-prefix-map= so maybe it's got enough use we won't be removing it any time soon. Ah well.

I'll see about fixing llvm-dwarfdump. Thanks for noticing the bug (which appears in dwarfdump, too).

Thanks!

[MaskRay]

llvm/test/MC/ELF/debug-prefix-map.s

36–40

Amend to why -fdebug-compilation-dir=. is preferred: when two distributed build harm users check out the build in different local working directories, the build can be shared because the compiler options are exactly the same (local determinism).

With -fdebug-prefix-map, the two builds will need different options.

[MaskRay]

llvm/test/MC/ELF/debug-prefix-map.s

36–40

Sent D131804 to fix llvm-dwarfdump

[srj]

This change has injected a build breakage for MSVC builds:

MCContext.cpp(859): error C2429: language feature 'structured bindings' requires compiler flag '/std:c++17'

[MaskRay]

In D131749#3729268, @srj wrote:

This change has injected a build breakage for MSVC builds:

MCContext.cpp(859): error C2429: language feature 'structured bindings' requires compiler flag '/std:c++17'

This turns out to be a backport issue. The patch was back ported to release/15.x However, release/15.x (https://github.com/llvm/llvm-project/tree/release/15.x) uses C++14 and structured bindings trigger warnings in GCC/Clang and errors in MSVC.
My bad that I did not think of the difference when crafting the patch.
Pushed a trivial commit to release/15.x to avoid structured bindings.

llvm/include/llvm/MC/MCContext.h

public:
   StringRef getCompilationDir() const { return CompilationDir; }
 
   /// Set the compilation directory for DW_AT_comp_dir
   void setCompilationDir(StringRef S) { CompilationDir = S.str(); }
 
   /// Add an entry to the debug prefix map.
   void addDebugPrefixMapEntry(const std::string &From, const std::string &To);
 
+  /// Remap one path in-place as per the debug prefix map.
+  void remapDebugPath(SmallVectorImpl<char> &Path);
+
   // Remaps all debug directory paths in-place as per the debug prefix map.
   void RemapDebugPaths();
 
   /// Get the main file name for use in error messages and debug
   /// info. This can be set to ensure we've got the correct file name
   /// after preprocessing or for -save-temps.
   const std::string &getMainFileName() const { return MainFileName; }
 

llvm/include/llvm/MC/MCDwarf.h

 #include "llvm/ADT/MapVector.h"
 #include "llvm/ADT/Optional.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringMap.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/MC/StringTableBuilder.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/MD5.h"
+#include "llvm/Support/StringSaver.h"
 #include <cassert>
 #include <cstdint>
 #include <string>
 #include <utility>
 #include <vector>
 
 namespace llvm {
 

 
 namespace mcdwarf {
 // Emit the common part of the DWARF 5 range/locations list tables header.
 MCSymbol *emitListsTableHeaderStart(MCStreamer &S);
 } // namespace mcdwarf
 
 /// Manage the .debug_line_str section contents, if we use it.
 class MCDwarfLineStr {
+  BumpPtrAllocator Alloc;
+  StringSaver Saver{Alloc};
   MCSymbol *LineStrLabel = nullptr;
   StringTableBuilder LineStrings{StringTableBuilder::DWARF};
   bool UseRelocs = false;
 
 public:
   /// Construct an instance that can emit .debug_line_str (for use in a normal
   /// v5 line table).
   explicit MCDwarfLineStr(MCContext &Ctx);
 
+  StringSaver &getSaver() { return Saver; }
+
   /// Emit a reference to the string.
   void emitRef(MCStreamer *MCOS, StringRef Path);
 
   /// Emit the .debug_line_str section if appropriate.
   void emitSection(MCStreamer *MCOS);
 
   /// Returns finalized section.
   SmallString<0> getFinalizedData();

llvm/lib/MC/MCContext.cpp

MCSubtargetInfo &MCContext::getSubtargetCopy(const MCSubtargetInfo &STI) {
   return *new (MCSubtargetAllocator.Allocate()) MCSubtargetInfo(STI);
 }
 
 void MCContext::addDebugPrefixMapEntry(const std::string &From,
                                        const std::string &To) {
   DebugPrefixMap.insert(std::make_pair(From, To));
 }
 
+void MCContext::remapDebugPath(SmallVectorImpl<char> &Path) {
+  for (const auto &[From, To] : DebugPrefixMap)
+    if (llvm::sys::path::replace_path_prefix(Path, From, To))
+      break;
+}
+
 void MCContext::RemapDebugPaths() {
   const auto &DebugPrefixMap = this->DebugPrefixMap;
   if (DebugPrefixMap.empty())
     return;
 
-  const auto RemapDebugPath = [&DebugPrefixMap](std::string &Path) {
-    SmallString<256> P(Path);
-    for (const auto &Entry : DebugPrefixMap) {
-      if (llvm::sys::path::replace_path_prefix(P, Entry.first, Entry.second)) {
-        Path = P.str().str();
-        break;
-      }
-    }
-  };
-
   // Remap compilation directory.
-  std::string CompDir = std::string(CompilationDir.str());
-  RemapDebugPath(CompDir);
-  CompilationDir = CompDir;
+  remapDebugPath(CompilationDir);
 
   // Remap MCDwarfDirs in all compilation units.
-  for (auto &CUIDTablePair : MCDwarfLineTablesCUMap)
-    for (auto &Dir : CUIDTablePair.second.getMCDwarfDirs())
-      RemapDebugPath(Dir);
+  SmallString<256> P;
+  for (auto &CUIDTablePair : MCDwarfLineTablesCUMap) {
+    for (auto &Dir : CUIDTablePair.second.getMCDwarfDirs()) {
+      P = Dir;
+      remapDebugPath(P);
+      Dir = std::string(P);
+    }
+  }
 }
 
 //===----------------------------------------------------------------------===//
 // Dwarf Management
 //===----------------------------------------------------------------------===//
 
 EmitDwarfUnwindType MCContext::emitDwarfUnwindInfo() const {
   if (!TargetOptions)

llvm/lib/MC/MCDwarf.cpp

void MCDwarfLineTableHeader::emitV5FileDirTables(
   // non-split object, these are references to .debug_line_str; in a split
   // object, they are inline strings.
   MCOS->emitInt8(1);
   MCOS->emitULEB128IntValue(dwarf::DW_LNCT_path);
   MCOS->emitULEB128IntValue(LineStr ? dwarf::DW_FORM_line_strp
                                     : dwarf::DW_FORM_string);
   MCOS->emitULEB128IntValue(MCDwarfDirs.size() + 1);
   // Try not to emit an empty compilation directory.
-  const StringRef CompDir = CompilationDir.empty()
-                                ? MCOS->getContext().getCompilationDir()
-                                : StringRef(CompilationDir);
+  SmallString<256> Dir;
+  StringRef CompDir = MCOS->getContext().getCompilationDir();
+  if (!CompilationDir.empty()) {
+    Dir = CompilationDir;
+    MCOS->getContext().remapDebugPath(Dir);
+    CompDir = Dir.str();
+    if (LineStr)
+      CompDir = LineStr->getSaver().save(CompDir);
+  }
   if (LineStr) {
     // Record path strings, emit references here.
     LineStr->emitRef(MCOS, CompDir);
     for (const auto &Dir : MCDwarfDirs)
       LineStr->emitRef(MCOS, Dir);
   } else {
     // The list of directory paths.  Compilation directory comes first.
     MCOS->emitBytes(CompDir);

llvm/test/MC/ELF/debug-prefix-map.s

 # NO_MAP_V5:      DW_AT_comp_dir [DW_FORM_string] ("{{.*}}.foo")
 # NO_MAP_V5:      include_directories[  0] =  .debug_line_str[0x00000000] = "{{.*}}.foo"
 
 # MAP_V4:      DW_AT_name [DW_FORM_string] ("src.s")
 # MAP_V4:      DW_AT_comp_dir [DW_FORM_string] ("src_root")
 # MAP_V4:      DW_AT_decl_file [DW_FORM_data4] ("src_root{{(/|\\)+}}src.s")
 # MAP_V4-NOT:  .foo
 
-## FIXME DW_AT_decl_file and include_directories are not mapped
 # MAP_V5:      DW_AT_name [DW_FORM_string] ("src.s")
 # MAP_V5:      DW_AT_comp_dir [DW_FORM_string] ("src_root")
-# MAP_V5:      DW_AT_decl_file [DW_FORM_data4] ("{{.*}}.foo{{(/|\\)+}}src.s")
-# MAP_V5:      include_directories[ 0] = .debug_line_str[0x00000000] = "{{.*}}.foo"
+## FIXME llvm-dwarfdump incorrectly joins include_directories[0] to DW_AT_comp_dir,
+## so there are two src_root path components.
+# MAP_V5:      DW_AT_decl_file [DW_FORM_data4] ("src_root{{(/|\\)+}}src_root{{(/|\\)+}}src.s")
+# MAP_V5:      include_directories[ 0] = .debug_line_str[0x00000000] = "src_root"
 
 # MAPABS_V4:      DW_AT_name [DW_FORM_string] ("src.s")
 # MAPABS_V4:      DW_AT_comp_dir [DW_FORM_string] ("{{(/|\\)+}}src_root")
 # MAPABS_V4:      DW_AT_decl_file [DW_FORM_data4] ("{{(/|\\)+}}src_root{{(/|\\)+}}src.s")
 # MAPABS_V4-NOT:  .foo
 
-## FIXME DW_AT_decl_file and include_directories are not mapped
 # MAPABS_V5:      DW_AT_name [DW_FORM_string] ("src.s")
 # MAPABS_V5:      DW_AT_comp_dir [DW_FORM_string] ("{{(/|\\)+}}src_root")
-# MAPABS_V5:      DW_AT_decl_file [DW_FORM_data4] ("{{.*}}.foo{{(/|\\)+}}src.s")
-# MAPABS_V5:      include_directories[ 0] = .debug_line_str[0x00000000] = "{{.*}}.foo"
+# MAPABS_V5:      DW_AT_decl_file [DW_FORM_data4] ("/src_root{{(/|\\)+}}src.s")
+# MAPABS_V5:      include_directories[ 0] = .debug_line_str[0x00000000] = "/src_root"