[lldb][PlatformDarwin] Parse SDK path for module compilation from debug-info

[Michael137]

Summary

When we build the Clang module compilation command (e.g., when
a user requests import of a module via expression @import Foundation),
LLDB will try to determine which SDK directory to use as the sysroot.
However, it currently does so by simply enumerating the SDKs directory
and picking the last one that's appropriate for module compilation
(see PlatformDarwin::GetSDKDirectoryForModules). That means if we have
multiple platform SDKs installed (e.g., a public and internal one), we
may pick the wrong one by chance.

On Darwin platforms we emit the SDK path that a object
file was compiled against into DWARF (using DW_AT_LLVM_sysroot
and DW_AT_APPLE_sdk). For Swift debugging, we already parse the SDK
path from debug-info if we can.

This patch mimicks the Swift behaviour for non-Swift languages. I.e.,
if we can get the SDK path from debug-info, do so. Otherwise, fall back
to the old heuristic.

rdar://110407148

[Michael137]

lldb/include/lldb/Target/Platform.h

493

I'm open to suggestions on where to put this API. Platform seems a bit too generic. But the intention is to share it with the Swift plugin. I could put it into PlatformDarwin.

[Michael137]

• Remove redundant headers

[Michael137]

lldb/include/lldb/Target/Platform.h

493

See how I intend to use it from the Swift plugin here

[aprantl]

lldb/include/lldb/Target/Platform.h

480

These flags really only make sense in the context of an XcodeSDK, so why not just return an XcodeSDK or XcodeSDK::Info object here? Otherwise we'll probably introduce subtle bugs due to a lossy translation between the flags.

[Michael137]

lldb/include/lldb/Target/Platform.h

480

Yup I think that'd be better. That'll also make it easier to use from the Swift plugin

[Michael137]

• Move into PlatformDarwin
• Return XcodeSDK from GetSDKPathFromDebugInfo so it's easier to re-use from Swift plugin
• Introduce ResolveSDKPathFromDebugInfo to be used from PlatformDarwin
• Adjust tests

[Michael137]

• Remove redundant header

[Michael137]

• Remove more headers

[Michael137]

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

1363

Only remaining question is whether we want to limit this to just Objective-C and Swift. Going through each compile unit for C++ seems like a lot of work for something that we won't use

@aprantl

[Michael137]

lldb/include/lldb/Target/Platform.h

480

Actually on second look, the XcodeSDK and XcodeSDK::Info objects represent information about a single (possibly parsed) SDK path. Whereas what the intention here was is to let the caller know whether we encountered a public/internal SDK while scanning all the CUs. Since we only return a single XcodeSDK (not all the ones we looked at) in my opinion it isn't quite right to store that information in it.

This is all really only used to print a Swift health. Maybe we could instead just log this to LLDBLog::Types? Then we don't need to worry about returning any of this information. @aprantl

[Michael137]

• Update unit-tests
• Expand function docs

[Michael137]

• Parameterize tests
• Return bool to indicate SDK mismatch

[Michael137]

• Fix test

[Michael137]

• EXPECT -> ASSERT

[aprantl]

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

1363

The work to detect the language and to get the SDK attribute is almost the same, both are stored in the top-level CU DIE.

lldb/include/lldb/Target/Platform.h

 #include "lldb/Host/File.h"
 #include "lldb/Interpreter/Options.h"
 #include "lldb/Utility/ArchSpec.h"
 #include "lldb/Utility/ConstString.h"
 #include "lldb/Utility/FileSpec.h"
 #include "lldb/Utility/StructuredData.h"
 #include "lldb/Utility/Timeout.h"
 #include "lldb/Utility/UserIDResolver.h"
+#include "lldb/Utility/XcodeSDK.h"
 #include "lldb/lldb-private-forward.h"
 #include "lldb/lldb-public.h"
 #include "llvm/Support/VersionTuple.h"
 
 namespace lldb_private {
 
 class ProcessInstanceInfo;
 class ProcessInstanceInfoMatch;
public:
 
   FileSpec GetWorkingDirectory();
 
   bool SetWorkingDirectory(const FileSpec &working_dir);
 
   // There may be modules that we don't want to find by default for operations
   // like "setting breakpoint by name". The platform will return "true" from
   // this call if the passed in module happens to be one of these.
 
   virtual bool
   ModuleIsExcludedForUnconstrainedSearches(Target &target,
                                            const lldb::ModuleSP &module_sp) {
     return false;
   }
 
   virtual Status MakeDirectory(const FileSpec &file_spec, uint32_t permissions);
 
   virtual Status GetFilePermissions(const FileSpec &file_spec,
                                     uint32_t &file_permissions);
 
   virtual Status SetFilePermissions(const FileSpec &file_spec,
                                     uint32_t file_permissions);
 
   virtual lldb::user_id_t OpenFile(const FileSpec &file_spec,
                                    File::OpenOptions flags, uint32_t mode,
                                    Status &error);
 
   virtual bool CloseFile(lldb::user_id_t fd, Status &error);
 
   virtual lldb::user_id_t GetFileSize(const FileSpec &file_spec);

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.h

public:
 
   /// Return the toolchain directory the current LLDB instance is located in.
   static FileSpec GetCurrentToolchainDirectory();
 
   /// Return the command line tools directory the current LLDB instance is
   /// located in.
   static FileSpec GetCurrentCommandLineToolsDirectory();
 
+  /// Will search each CU associated with the specified 'module'
+  /// for the SDK paths the CUs were compiled against. In the presence
+  /// of different SDKs, we try to pick the most appropriate one using
+  /// \ref XcodeSDK::Merge.
+  ///
+  /// \returns If successful, returns a parsed XcodeSDK object
+  static llvm::Expected<XcodeSDK> GetSDKPathFromDebugInfo(Module &module);
+
+  static llvm::Expected<std::string>
+  ResolveSDKPathFromDebugInfo(Module &module);
+
 protected:
   static const char *GetCompatibleArch(ArchSpec::Core core, size_t idx);
 
   struct CrashInfoAnnotations {
     uint64_t version;          // unsigned long
     uint64_t message;          // char *
     uint64_t signature_string; // char *
     uint64_t backtrace;        // char *

lldb/source/Plugins/Platform/MacOSX/PlatformDarwin.cpp

 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/LLDBLog.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/ProcessInfo.h"
 #include "lldb/Utility/Status.h"
 #include "lldb/Utility/Timer.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Threading.h"
 #include "llvm/Support/VersionTuple.h"
 
 #if defined(__APPLE__)
 #include <TargetConditionals.h>
 #endif
 
case XcodeSDK::Type::unknown:
       }
       return;
     }
     minimum_version_option << version.getAsString();
     options.emplace_back(std::string(minimum_version_option.GetString()));
   }
 
   FileSpec sysroot_spec;
-  // Scope for mutex locker below
-  {
+
+  if (target) {
+    if (ModuleSP exe_module_sp = target->GetExecutableModule()) {
+      auto path_or_err = ResolveSDKPathFromDebugInfo(*exe_module_sp);
+      if (path_or_err) {
+        sysroot_spec = FileSpec(*path_or_err);
+      } else {
+        LLDB_LOG_ERROR(GetLog(LLDBLog::Types | LLDBLog::Host),
+                       path_or_err.takeError(),
+                       "Failed to resolve SDK path: {0}");
+      }
+    }
+  }
+
+  if (!FileSystem::Instance().IsDirectory(sysroot_spec.GetPath())) {
     std::lock_guard<std::mutex> guard(m_mutex);
     sysroot_spec = GetSDKDirectoryForModules(sdk_type);
   }
 
   if (FileSystem::Instance().IsDirectory(sysroot_spec.GetPath())) {
     options.push_back("-isysroot");
     options.push_back(sysroot_spec.GetPath());
   }
#elif TARGET_OS_TV
   return llvm::Triple::TvOS;
 #elif TARGET_OS_BRIDGE
   return llvm::Triple::BridgeOS;
 #else
 #error "LLDB being compiled for an unrecognized Darwin OS"
 #endif
 #endif // __APPLE__
 }
+
+llvm::Expected<XcodeSDK>
+PlatformDarwin::GetSDKPathFromDebugInfo(Module &module) {
+  SymbolFile *sym_file = module.GetSymbolFile();
+  if (!sym_file)
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        llvm::formatv("No symbol file available for module '{0}'",
+                      module.GetFileSpec().GetFilename().AsCString("")));
+
+  XcodeSDK sdk;
+  for (unsigned i = 0; i < sym_file->GetNumCompileUnits(); ++i)
+    if (auto cu_sp = sym_file->GetCompileUnitAtIndex(i))
+      sdk.Merge(sym_file->ParseXcodeSDK(*cu_sp));
+
+  return sdk;
+}
+
+llvm::Expected<std::string>
+PlatformDarwin::ResolveSDKPathFromDebugInfo(Module &module) {
+  auto sdk_or_err = GetSDKPathFromDebugInfo(module);
+  if (!sdk_or_err)
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        llvm::formatv("Failed to parse SDK path from debug-info: {0}",
+                      llvm::toString(sdk_or_err.takeError())));
+
+  auto path_or_err =
+      HostInfo::GetSDKRoot(HostInfo::SDKOptions{std::move(*sdk_or_err)});
+  if (!path_or_err)
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        llvm::formatv("Error while searching for SDK (XcodeSDK '{0}'): {1}",
+                      sdk_or_err->GetString(),
+                      llvm::toString(path_or_err.takeError())));
+
+  return path_or_err->str();
+}

lldb/source/Target/Platform.cpp

 #include "lldb/Host/FileCache.h"
 #include "lldb/Host/FileSystem.h"
 #include "lldb/Host/Host.h"
 #include "lldb/Host/HostInfo.h"
 #include "lldb/Host/OptionParser.h"
 #include "lldb/Interpreter/OptionValueFileSpec.h"
 #include "lldb/Interpreter/OptionValueProperties.h"
 #include "lldb/Interpreter/Property.h"
+#include "lldb/Symbol/CompileUnit.h"
 #include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Symbol/SymbolFile.h"
+#include "lldb/Target/Language.h"
 #include "lldb/Target/ModuleCache.h"
 #include "lldb/Target/Platform.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Target/UnixSignals.h"
 #include "lldb/Utility/DataBufferHeap.h"
 #include "lldb/Utility/FileSpec.h"
 #include "lldb/Utility/LLDBLog.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Status.h"
 #include "lldb/Utility/StructuredData.h"
+#include "lldb/Utility/XcodeSDK.h"
 #include "llvm/ADT/STLExtras.h"
+#include "llvm/Support/Error.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 
 // Define these constants from POSIX mman.h rather than include the file so
 // that they will be correct even when compiled on Linux.
 #define MAP_PRIVATE 2
 #define MAP_ANON 0x1000
 

lldb/unittests/SymbolFile/DWARF/XcodeSDKModuleTests.cpp

 //===-- XcodeSDKModuleTests.cpp -------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "Plugins/Platform/MacOSX/PlatformMacOSX.h"
 #include "Plugins/SymbolFile/DWARF/DWARFCompileUnit.h"
 #include "Plugins/SymbolFile/DWARF/DWARFDIE.h"
 #include "Plugins/TypeSystem/Clang/TypeSystemClang.h"
 #include "TestingSupport/Symbol/YAMLModuleTester.h"
 #include "lldb/Core/PluginManager.h"
+#include "llvm/Support/Error.h"
 #include "gmock/gmock.h"
 #include "gtest/gtest.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 #ifdef __APPLE__
 namespace {
)";
   CompUnitSP comp_unit = sym_file.GetCompileUnitAtIndex(0);
   ASSERT_TRUE(static_cast<bool>(comp_unit.get()));
   ModuleSP module = t.GetModule();
   ASSERT_EQ(module->GetSourceMappingList().GetSize(), 0u);
   XcodeSDK sdk = sym_file.ParseXcodeSDK(*comp_unit);
   ASSERT_EQ(sdk.GetType(), XcodeSDK::Type::MacOSX);
   ASSERT_EQ(module->GetSourceMappingList().GetSize(), 1u);
 }
+
+TEST_F(XcodeSDKModuleTests, TestSDKPathFromDebugInfo_InternalAndPublicSDK) {
+  // Tests that we can parse the SDK path from debug-info.
+  // In the presence of multiple compile units, one of which
+  // points to an internal SDK, we should pick the internal SDK.
+
+  const char *yamldata = R"(
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_386
+DWARF:
+  debug_abbrev:
+    - Table:
+        - Code:            0x00000001
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_APPLE_sdk
+              Form:            DW_FORM_string
+            - Attribute:       DW_AT_LLVM_sysroot
+              Form:            DW_FORM_string
+        - Code:            0x00000002
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_APPLE_sdk
+              Form:            DW_FORM_string
+            - Attribute:       DW_AT_LLVM_sysroot
+              Form:            DW_FORM_string
+  debug_info:
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000001
+          Values:
+            - Value:       0x000000000000000C
+            - CStr:        "MacOSX10.9.sdk"
+            - CStr:        "/Library/Developer/CommandLineTools/SDKs/MacOSX10.9.sdk"
+        - AbbrCode:        0x00000000
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000002
+          Values:
+            - Value:       0x0000000000000010
+            - CStr:        "iPhoneOS14.0.Internal.sdk"
+            - CStr:        "/Library/Developer/CommandLineTools/SDKs/iPhoneOS14.0.Internal.sdk"
+        - AbbrCode:        0x00000000
+...
+)";
+
+  YAMLModuleTester t(yamldata);
+  DWARFUnit *dwarf_unit = t.GetDwarfUnit();
+  auto *dwarf_cu = llvm::cast<DWARFCompileUnit>(dwarf_unit);
+  ASSERT_TRUE(static_cast<bool>(dwarf_cu));
+  SymbolFileDWARF &sym_file = dwarf_cu->GetSymbolFileDWARF();
+  ASSERT_EQ(sym_file.GetNumCompileUnits(), 2U);
+  ModuleSP module = t.GetModule();
+  ASSERT_NE(module, nullptr);
+
+  auto sdk_or_err = PlatformDarwin::GetSDKPathFromDebugInfo(*module);
+  ASSERT_TRUE(static_cast<bool>(sdk_or_err));
+
+  ASSERT_TRUE(sdk_or_err->IsAppleInternalSDK());
+  ASSERT_NE(sdk_or_err->GetString().find("Internal.sdk"), std::string::npos);
+}
+
+TEST_F(XcodeSDKModuleTests, TestSDKPathFromDebugInfo_InvalidSDKPath) {
+  // Tests that parsing a CU with an invalid SDK directory name fails.
+
+  const char *yamldata = R"(
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_386
+DWARF:
+  debug_abbrev:
+    - Table:
+        - Code:            0x00000001
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_APPLE_sdk
+              Form:            DW_FORM_string
+  debug_info:
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000001
+          Values:
+            - Value:       0x000000000000000C
+            - CStr:        "1abc@defgh2"
+        - AbbrCode:        0x00000000
+...
+)";
+
+  YAMLModuleTester t(yamldata);
+  ModuleSP module = t.GetModule();
+  ASSERT_NE(module, nullptr);
+
+  auto path_or_err = PlatformDarwin::ResolveSDKPathFromDebugInfo(*module);
+  ASSERT_FALSE(static_cast<bool>(path_or_err));
+  llvm::consumeError(path_or_err.takeError());
+}
+
+TEST_F(XcodeSDKModuleTests, TestSDKPathFromDebugInfo_No_DW_AT_APPLE_sdk) {
+  // Tests that parsing a CU without a DW_AT_APPLE_sdk fails.
+
+  const char *yamldata = R"(
+--- !ELF
+FileHeader:
+  Class:   ELFCLASS64
+  Data:    ELFDATA2LSB
+  Type:    ET_EXEC
+  Machine: EM_386
+DWARF:
+  debug_abbrev:
+    - Table:
+        - Code:            0x00000001
+          Tag:             DW_TAG_compile_unit
+          Children:        DW_CHILDREN_no
+          Attributes:
+            - Attribute:       DW_AT_language
+              Form:            DW_FORM_data2
+            - Attribute:       DW_AT_LLVM_sysroot
+              Form:            DW_FORM_string
+  debug_info:
+    - Version:         2
+      AddrSize:        8
+      AbbrevTableID:   0
+      AbbrOffset:      0x0
+      Entries:
+        - AbbrCode:        0x00000001
+          Values:
+            - Value:       0x000000000000000C
+            - CStr:        "/Library/Developer/CommandLineTools/SDKs/iPhoneOS14.0.Internal.sdk"
+        - AbbrCode:        0x00000000
+...
+)";
+
+  YAMLModuleTester t(yamldata);
+  ModuleSP module = t.GetModule();
+  ASSERT_NE(module, nullptr);
+
+  auto path_or_err = PlatformDarwin::ResolveSDKPathFromDebugInfo(*module);
+  ASSERT_FALSE(static_cast<bool>(path_or_err));
+  llvm::consumeError(path_or_err.takeError());
+}
 #endif