[lldb][PlatformDarwin] Parse SDK path for module compilation from debug-info

[Michael137]

Summary

When we build the Clang module compilation command (e.g., when
a user requests import of a module via expression @import Foundation),
LLDB will try to determine which SDK directory to use as the sysroot.
However, it currently does so by simply enumerating the SDKs directory
and picking the last one that's appropriate for module compilation
(see PlatformDarwin::GetSDKDirectoryForModules). That means if we have
multiple platform SDKs installed (e.g., a public and internal one), we
may pick the wrong one by chance.

On Darwin platforms we emit the SDK path that a object
file was compiled against into DWARF (using DW_AT_LLVM_sysroot
and DW_AT_APPLE_sdk). For Swift debugging, we already parse the SDK
path from debug-info if we can.

This patch mimicks the Swift behaviour for non-Swift languages. I.e.,
if we can get the SDK path from debug-info, do so. Otherwise, fall back
to the old heuristic.

rdar://110407148

[Michael137]

lldb/include/lldb/Target/Platform.h

492

I'm open to suggestions on where to put this API. Platform seems a bit too generic. But the intention is to share it with the Swift plugin. I could put it into PlatformDarwin.

[Michael137]

• Remove redundant headers

[Michael137]

lldb/include/lldb/Target/Platform.h

492

See how I intend to use it from the Swift plugin here

[aprantl]

lldb/include/lldb/Target/Platform.h

479

These flags really only make sense in the context of an XcodeSDK, so why not just return an XcodeSDK or XcodeSDK::Info object here? Otherwise we'll probably introduce subtle bugs due to a lossy translation between the flags.

[Michael137]

lldb/include/lldb/Target/Platform.h

479

Yup I think that'd be better. That'll also make it easier to use from the Swift plugin

[Michael137]

• Move into PlatformDarwin
• Return XcodeSDK from GetSDKPathFromDebugInfo so it's easier to re-use from Swift plugin
• Introduce ResolveSDKPathFromDebugInfo to be used from PlatformDarwin
• Adjust tests

[Michael137]

• Remove redundant header

[Michael137]

• Remove more headers

[Michael137]

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

1361

Only remaining question is whether we want to limit this to just Objective-C and Swift. Going through each compile unit for C++ seems like a lot of work for something that we won't use

@aprantl

[Michael137]

lldb/include/lldb/Target/Platform.h

479

Actually on second look, the XcodeSDK and XcodeSDK::Info objects represent information about a single (possibly parsed) SDK path. Whereas what the intention here was is to let the caller know whether we encountered a public/internal SDK while scanning all the CUs. Since we only return a single XcodeSDK (not all the ones we looked at) in my opinion it isn't quite right to store that information in it.

This is all really only used to print a Swift health. Maybe we could instead just log this to LLDBLog::Types? Then we don't need to worry about returning any of this information. @aprantl

[Michael137]

• Update unit-tests
• Expand function docs

[Michael137]

• Parameterize tests
• Return bool to indicate SDK mismatch

[Michael137]

• Fix test

[Michael137]

• EXPECT -> ASSERT

[aprantl]

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

1361

The work to detect the language and to get the SDK attribute is almost the same, both are stored in the top-level CU DIE.

lldb/include/lldb/Target/Platform.h

public:
   virtual bool SupportsModules() { return false; }
 
   // Appends the platform-specific options required to find the modules for the
   // current platform.
   virtual void
   AddClangModuleCompilationOptions(Target *target,
                                    std::vector<std::string> &options);
 
+  struct SDKPathSearchResult {
+    /// Full path to SDK directory that we parsed
+    /// from debug-info
+    std::string sdk_path;
+
+    /// True if one of the parsed CUs pointed
+    /// to an internal SDK
+    bool found_internal_sdk = false;
+
+    /// True if one of the parsed CUs pointed
+    /// to a public (i.e., non-interanl) SDK
+    bool found_public_sdk = false;
+  };
+
+  /// Will search each CU associated with the specified 'module'
+  /// for the SDK path the CU was compiled against.
+  ///
+  /// \returns If successful, returns a SDKPathSearchResult that
+  ///          holds the SDK path.
+  static llvm::Expected<SDKPathSearchResult>
+  GetSDKPathFromDebugInfo(Module &module);
+
   FileSpec GetWorkingDirectory();
 
   bool SetWorkingDirectory(const FileSpec &working_dir);
 
   // There may be modules that we don't want to find by default for operations
   // like "setting breakpoint by name". The platform will return "true" from
   // this call if the passed in module happens to be one of these.
 

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/LLDBLog.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/ProcessInfo.h"
 #include "lldb/Utility/Status.h"
 #include "lldb/Utility/Timer.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Threading.h"
 #include "llvm/Support/VersionTuple.h"
 
 #if defined(__APPLE__)
 #include <TargetConditionals.h>
 #endif
 
#undef OPTION
     minimum_version_option << version.getAsString();
     options.emplace_back(std::string(minimum_version_option.GetString()));
   }
 
   FileSpec sysroot_spec;
   // Scope for mutex locker below
   {
     std::lock_guard<std::mutex> guard(m_mutex);
+    if (target) {
+      if (ModuleSP exe_module_sp = target->GetExecutableModule()) {
+        if (auto path_or_err = GetSDKPathFromDebugInfo(*exe_module_sp))
+          sysroot_spec = FileSpec(path_or_err->sdk_path);
+        else
+          llvm::consumeError(path_or_err.takeError());
+      }
+    }
+
+    if (!FileSystem::Instance().IsDirectory(sysroot_spec.GetPath())) {
       sysroot_spec = GetSDKDirectoryForModules(sdk_type);
     }
+  }
 
   if (FileSystem::Instance().IsDirectory(sysroot_spec.GetPath())) {
     options.push_back("-isysroot");
     options.push_back(sysroot_spec.GetPath());
   }
 }
 
 ConstString PlatformDarwin::GetFullNameForDylib(ConstString basename) {

 #elif TARGET_OS_TV
   return llvm::Triple::TvOS;
 #elif TARGET_OS_BRIDGE
   return llvm::Triple::BridgeOS;
 #else
 #error "LLDB being compiled for an unrecognized Darwin OS"
 #endif
 #endif // __APPLE__
 }

lldb/source/Target/Platform.cpp

 #include "lldb/Host/FileCache.h"
 #include "lldb/Host/FileSystem.h"
 #include "lldb/Host/Host.h"
 #include "lldb/Host/HostInfo.h"
 #include "lldb/Host/OptionParser.h"
 #include "lldb/Interpreter/OptionValueFileSpec.h"
 #include "lldb/Interpreter/OptionValueProperties.h"
 #include "lldb/Interpreter/Property.h"
+#include "lldb/Symbol/CompileUnit.h"
 #include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Symbol/SymbolFile.h"
+#include "lldb/Target/Language.h"
 #include "lldb/Target/ModuleCache.h"
 #include "lldb/Target/Platform.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Target/UnixSignals.h"
 #include "lldb/Utility/DataBufferHeap.h"
 #include "lldb/Utility/FileSpec.h"
 #include "lldb/Utility/LLDBLog.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Status.h"
 #include "lldb/Utility/StructuredData.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 
 // Define these constants from POSIX mman.h rather than include the file so
 // that they will be correct even when compiled on Linux.
 #define MAP_PRIVATE 2
 #define MAP_ANON 0x1000
 
for (int idx = 0;
     PlatformSP platform_sp = create_callback(true, &arch);
     if (platform_sp) {
       if (platform_sp->LoadPlatformBinaryAndSetup(process, addr, notify))
         return true;
     }
   }
   return false;
 }
+
+llvm::Expected<Platform::SDKPathSearchResult>
+Platform::GetSDKPathFromDebugInfo(Module &module) {
+  SymbolFile *sym_file = module.GetSymbolFile();
+  if (!sym_file)
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        llvm::formatv("No symbol file available for module '{0}'",
+                      module.GetFileSpec().GetFilename().AsCString("")));
+
+  bool found_public_sdk = false;
+  bool found_internal_sdk = false;
+  XcodeSDK sdk;
+  for (unsigned i = 0; i < sym_file->GetNumCompileUnits(); ++i) {
+    if (auto cu_sp = sym_file->GetCompileUnitAtIndex(i)) {
+      auto cu_sdk = sym_file->ParseXcodeSDK(*cu_sp);
+      sdk.Merge(cu_sdk);
+      bool is_internal_sdk = cu_sdk.IsAppleInternalSDK();
+      found_public_sdk |= !is_internal_sdk;
+      found_internal_sdk |= is_internal_sdk;
+    }
+  }
+
+  auto path_or_err = HostInfo::GetSDKRoot(HostInfo::SDKOptions{std::move(sdk)});
+  if (!path_or_err)
+    return path_or_err.takeError();
+
+  return SDKPathSearchResult{path_or_err->str(), found_internal_sdk,
+                             found_public_sdk};
+}

lldb/unittests/SymbolFile/DWARF/XcodeSDKModuleTests.cpp

 //===-- XcodeSDKModuleTests.cpp -------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Plugins/Platform/MacOSX/PlatformMacOSX.h"
 #include "Plugins/SymbolFile/DWARF/DWARFCompileUnit.h"
 #include "Plugins/SymbolFile/DWARF/DWARFDIE.h"
 #include "Plugins/TypeSystem/Clang/TypeSystemClang.h"
 #include "TestingSupport/Symbol/YAMLModuleTester.h"
 #include "lldb/Core/PluginManager.h"
+#include "llvm/Support/Error.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 #ifdef __APPLE__
 namespace {
)";
   CompUnitSP comp_unit = sym_file.GetCompileUnitAtIndex(0);
   ASSERT_TRUE(static_cast<bool>(comp_unit.get()));
   ModuleSP module = t.GetModule();
   ASSERT_EQ(module->GetSourceMappingList().GetSize(), 0u);
   XcodeSDK sdk = sym_file.ParseXcodeSDK(*comp_unit);
   ASSERT_EQ(sdk.GetType(), XcodeSDK::Type::MacOSX);
   ASSERT_EQ(module->GetSourceMappingList().GetSize(), 1u);
 }
+
+TEST_F(XcodeSDKModuleTests, TestSDKPathFromDebugInfo_InternalAndPublicSDK) {
+  // Tests that we can parse the SDK path from debug-info.
+  // In the presence of multiple compile units, one of which
+  // points to an internal SDK, we should pick the internal SDK.
+
+  const char *yamldata = R"(
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_386
+DWARF:
+  debug_abbrev:
+    - Table:
+        - Code:            0x00000001
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_APPLE_sdk
+              Form:            DW_FORM_string
+            - Attribute:       DW_AT_LLVM_sysroot
+              Form:            DW_FORM_string
+        - Code:            0x00000002
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_APPLE_sdk
+              Form:            DW_FORM_string
+            - Attribute:       DW_AT_LLVM_sysroot
+              Form:            DW_FORM_string
+  debug_info:
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000001
+          Values:
+            - Value:       0x000000000000000C
+            - CStr:        "MacOSX10.9.sdk"
+            - CStr:        "/Library/Developer/CommandLineTools/SDKs/MacOSX10.9.sdk"
+        - AbbrCode:        0x00000000
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000002
+          Values:
+            - Value:       0x0000000000000010
+            - CStr:        "iPhoneOS14.0.Internal.sdk"
+            - CStr:        "/Library/Developer/CommandLineTools/SDKs/iPhoneOS14.0.Internal.sdk"
+        - AbbrCode:        0x00000000
+...
+)";
+
+  YAMLModuleTester t(yamldata);
+  DWARFUnit *dwarf_unit = t.GetDwarfUnit();
+  auto *dwarf_cu = llvm::cast<DWARFCompileUnit>(dwarf_unit);
+  ASSERT_TRUE(static_cast<bool>(dwarf_cu));
+  SymbolFileDWARF &sym_file = dwarf_cu->GetSymbolFileDWARF();
+  ASSERT_EQ(sym_file.GetNumCompileUnits(), 2U);
+  ModuleSP module = t.GetModule();
+  ASSERT_NE(module, nullptr);
+
+  auto path_or_err = Platform::GetSDKPathFromDebugInfo(*module);
+  ASSERT_TRUE(!!path_or_err);
+
+  auto [sdk_path, found_internal, found_public] = *path_or_err;
+
+  ASSERT_TRUE(found_internal);
+  ASSERT_TRUE(found_public);
+  ASSERT_NE(sdk_path.find("Internal.sdk"), std::string::npos);
+}
+
+TEST_F(XcodeSDKModuleTests, TestSDKPathFromDebugInfo_InvalidSDKPath) {
+  // Tests that parsing a CU with an invalid SDK directory name fails.
+
+  const char *yamldata = R"(
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_386
+DWARF:
+  debug_abbrev:
+    - Table:
+        - Code:            0x00000001
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_APPLE_sdk
+              Form:            DW_FORM_string
+  debug_info:
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000001
+          Values:
+            - Value:       0x000000000000000C
+            - CStr:        "1abc@defgh2"
+        - AbbrCode:        0x00000000
+...
+)";
+
+  YAMLModuleTester t(yamldata);
+  ModuleSP module = t.GetModule();
+  ASSERT_NE(module, nullptr);
+
+  auto path_or_err = Platform::GetSDKPathFromDebugInfo(*module);
+  auto err = path_or_err.takeError();
+  ASSERT_TRUE(static_cast<bool>(err));
+  llvm::consumeError(std::move(err));
+}
+
+TEST_F(XcodeSDKModuleTests, TestSDKPathFromDebugInfo_No_DW_AT_APPLE_sdk) {
+  // Tests that parsing a CU without a DW_AT_APPLE_sdk fails.
+
+  const char *yamldata = R"(
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_386
+DWARF:
+  debug_abbrev:
+    - Table:
+        - Code:            0x00000001
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_LLVM_sysroot
+              Form:            DW_FORM_string
+  debug_info:
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000001
+          Values:
+            - Value:       0x000000000000000C
+            - CStr:        "/Library/Developer/CommandLineTools/SDKs/iPhoneOS14.0.Internal.sdk"
+        - AbbrCode:        0x00000000
+...
+)";
+
+  YAMLModuleTester t(yamldata);
+  ModuleSP module = t.GetModule();
+  ASSERT_NE(module, nullptr);
+
+  auto path_or_err = Platform::GetSDKPathFromDebugInfo(*module);
+  auto err = path_or_err.takeError();
+  ASSERT_TRUE(static_cast<bool>(err));
+  llvm::consumeError(std::move(err));
+}
 #endif