[lldb] integrate debuginfod

[kwk]

Summary

This first patch does the heavy lifting of bootstrapping debuginfod with
CMake and integrating it to find a source file using debuginfod when
using (lldb) source list and the file cannot be found locally.

Read more about debuginfod here:
https://sourceware.org/elfutils/Debuginfod.html

[kwk]

• Simplified return type of getBuildIDFromModule
• fixed typo

[labath]

lldb/include/lldb/Host/DebugInfoD.h

27

Expected<string> ?

lldb/source/Host/common/DebugInfoD.cpp

44–68

How is all this different from module->GetUUID() ?

98

llvm::sys::StrError(-rc)

[kwk]

Changes suggested by elfutils maintainers:

• Silently ignore error when no DEBUGINFOD_URLS was given as an environment variable (ENOSYS).
• Silently ignore error when the build ID could not be found on any server (ENOENT).
• End debuginfod client before dealing with return code.

Applied review comments from labath:

• Removed getBuildIDFromModule because we have Module->GetUUID()
• Make findSource return an llvm::Expected<std::string> instead of an error
• Various formatting issues with clang-format
• use llvm::sys::StrError instead of strerror directly

Other changes:

• Comments on functions in lldb_privat::debuginfod

[kwk]

@labath thank you for your early feedback. It was helpful even though this is still a work in progress.

lldb/source/Host/common/DebugInfoD.cpp

44–68

I didn't know about that :) . Thank you!

[kwk]

• Fix include ordering based on clang-format

[kwk]

• Added debuginfod2.py
• after running: autopep8 --in-place --aggressive --aggressive debuginfod2.py
• exponential backoff implemented
• Added http.py with doctests
• autopep8 --in-place --aggressive http.py
• change import
• Removed simulated startup time
• fixup
• Changed wording
• Using os.path.abspath on directory before using it
• Fixups
• Fixups
• Fixups
• hide port and hostname from ServeDirectoryWithHTTP
• lit test working for debuginfod and source list

[kwk]

@labath I've updated my patch and would love to hear your opinion on it. So far I've only written the python ServeDirectoryWithHTTP() function with proper doctest and documentation but since you mentioned the 0 port thingy I've tried that on the command line when using python -m http.server 0 and it works smoothly. That's why I've included the llvm-lit test I was working on. Maybe lldb/test/Shell/SymbolFile/DWARF/source-list.cpp is the wrong file for this, but we can move it around if you like it so far.

[jankratochvil]

On Fedora 31 x86_64 with LLDB using python3 I got:

llvm-lit: .../llvm-monorepo2/llvm/utils/lit/lit/TestingConfig.py:102: fatal: unable to parse config file '.../llvm-monorepo2-clangassert/tools/lldb/test/Shell/lit.site.cfg.py', traceback: Traceback (most recent call last):
  File ".../llvm-monorepo2/llvm/utils/lit/lit/TestingConfig.py", line 89, in load_from_path 
    exec(compile(data, path, 'exec'), cfg_globals, None)
  File ".../llvm-monorepo2-clangassert/tools/lldb/test/Shell/lit.site.cfg.py", line 20, in <module>
    config.lldb_enable_debuginfod = TRUE
NameError: name 'TRUE' is not defined
make[3]: *** [tools/lldb/test/CMakeFiles/check-lldb-lit.dir/build.make:58: tools/lldb/test/CMakeFiles/check-lldb-lit] Error 2

It helped to change:

-  set(Debuginfod_FOUND TRUE)
+  set(Debuginfod_FOUND 1)

[kwk]

In D75750#1929967, @jankratochvil wrote:

On Fedora 31 x86_64 with LLDB using python3 I got:

llvm-lit: .../llvm-monorepo2/llvm/utils/lit/lit/TestingConfig.py:102: fatal: unable to parse config file '.../llvm-monorepo2-clangassert/tools/lldb/test/Shell/lit.site.cfg.py', traceback: Traceback (most recent call last):
  File ".../llvm-monorepo2/llvm/utils/lit/lit/TestingConfig.py", line 89, in load_from_path 
    exec(compile(data, path, 'exec'), cfg_globals, None)
  File ".../llvm-monorepo2-clangassert/tools/lldb/test/Shell/lit.site.cfg.py", line 20, in <module>
    config.lldb_enable_debuginfod = TRUE
NameError: name 'TRUE' is not defined
make[3]: *** [tools/lldb/test/CMakeFiles/check-lldb-lit.dir/build.make:58: tools/lldb/test/CMakeFiles/check-lldb-lit] Error 2

Right, I manually fixed it locally and have forgotton to fix it. Thank you @jankratochvil for bringing it up.

It helped to change:

-  set(Debuginfod_FOUND TRUE)
+  set(Debuginfod_FOUND 1)

I'm sure this helps but we have a better way by using llvm_canonicalize_cmake_booleans in CMake. I did use this before for minidebuginfo and LZMA integration but the place in which I've put it was moved around which is why I needed some time. Expect a fix soon.

[kwk]

• Fix NameError: name 'TRUE' is not defined

[labath]

In D75750#1929124, @kwk wrote:

@labath I've updated my patch and would love to hear your opinion on it. So far I've only written the python ServeDirectoryWithHTTP() function with proper doctest and documentation but since you mentioned the 0 port thingy I've tried that on the command line when using python -m http.server 0 and it works smoothly. That's why I've included the llvm-lit test I was working on.

Being able to use 0 to auto-assign a port is definitely a big improvement, but there still the question of retrieving that port and sychronization that goes with it, which you've now done with a while loop + sniffing through the server log.

And I still haven't gotten used to how the comments in your lit tests are way longer than the test itself. If I think about that harder, I guess the thing that really bothers me about that (even though I normally like comments) is that there is no visual distinction between "comments" and "code" this way -- it all shows up as grey in the review tool. Can't say that's really your fault, but it does make it hard to see what that test is doing nonetheless. (Some areas of llvm have a convention to use ## for "real" comments, which I guess can make things slightly better, but I still haven't seen comments this big there...

So overall, I think this version is better than what you had before, but it still doesn't convince me that this is better than python.

lldb/include/lldb/Host/DebugInfoD.h

17–19

I guess this is not needed now.

lldb/packages/Python/lldbsuite/test/httpserver.py

76

What exactly is this timeout for? It seems rather small...

lldb/packages/Python/lldbsuite/test/lldbtest.py

4 ↗

(On Diff #251297)

just commit this separately. no review needed.

[kwk]

• Removed not needed forward decl
• Format comments for better readability in my test

[kwk]

@labath I've improved my test file for readability.

lldb/include/lldb/Host/DebugInfoD.h

17–19

Right.

27

Removed.

lldb/packages/Python/lldbsuite/test/httpserver.py

76

uff, I guess I had an idea when I wrote it but its lost now.

lldb/packages/Python/lldbsuite/test/lldbtest.py

4 ↗

(On Diff #251297)

Done in 44361782e2c252c8886cd77f6b7d4ebe64fb6e8d.

[kwk]

• Validate that the server received the request from debuginfod client

[jankratochvil]

lldb/source/Core/SourceManager.cpp

422

This comment should not stay there during check-in.

438

This comment should not stay there during check-in.

[jankratochvil]

lldb/source/Host/common/DebugInfoD.cpp

40

const UUID &buildID as it is even bigger (40 bytes) than std::string (32 bytes).

57

Excessive leftover comment.

60

Here it will contact the server even if the binary does not contain any build-id - LLDB then generates UUID as 4 bytes long one:

// Use 4 bytes of crc from the .gnu_debuglink section.
u32le data(gnu_debuglink_crc);
uuid = UUID::fromData(&data, sizeof(data));

That is a needless performance regression.
I sure do not like making such decision on the LLDB side. Maybe libdebuginfod could rather make such optimization - IMO as Frank Eigler.

[fche2]

lldb/source/Host/common/DebugInfoD.cpp

60

Could kkleine reject uuid of length 4 in the above test, i.e. something like:

if (!uuid.IsValid() || uuid.GetBytes().size() == sizeof(u32le)) // .gnu_debuglink crc32
   continue;

[labath]

lldb/source/Host/common/DebugInfoD.cpp

60

Ideally, lldb would not use the debug link crc as a uuid (and instead store that elsewhere), but rejecting the short uuids here does not seem _that_ bad.

[jankratochvil]

lldb/source/Host/common/DebugInfoD.cpp

60

We were discussing with @kwk that in fact sending anything stored in UUID as build-id may not be right. debuginfod wants specifically build-id, not any other identifier. Or @fche2 - does it? Would debuginfod for example accept some that Apple UUID for Apple dsym files? Maybe LLDB could store some identifier how was the UUID obtained.

[fche2]

lldb/source/Host/common/DebugInfoD.cpp

60

Would debuginfod for example accept some that Apple UUID for Apple dsym files?

The debuginfod webapi specifies that buildids simply need to be lower case hex strings. It will dutifully accept any such string, and correctly report 403's for unknown ones.

[kwk]

• check for valid UUID
• less verbose mkdir and rm output
• More explicit test and documentation
• fixup

[kwk]

@labath @fche2 @jankratochvil I've implemented the logic to ignore invalid UUIDs and the ones that are too short. Can you have another look please?

[jankratochvil]

lldb/source/Host/common/DebugInfoD.cpp

43

If it is done this way (and not in libdebuginfod.so) I think there should be <=8 because LLDB contains:

if (gnu_debuglink_crc) {
  // Use 4 bytes of crc from the .gnu_debuglink section.
  u32le data(gnu_debuglink_crc);
  uuid = UUID::fromData(&data, sizeof(data));
} else if (core_notes_crc) {
  // Use 8 bytes - first 4 bytes for *magic* prefix, mainly to make
  // it look different form .gnu_debuglink crc followed by 4 bytes
  // of note segments crc.
  u32le data[] = {u32le(g_core_uuid_magic), u32le(core_notes_crc)};
  uuid = UUID::fromData(data, sizeof(data));
}

[kwk]

• Remove commented out code
• Remove lldb/packages/Python/lldbsuite/test/httpserver.py in favor of lit test
• Removed commented out left-over code

[kwk]

• Adjust buildID verification

[labath]

The code mostly fine for me, but this should be reviewed by properly by more people, once you're ready to take down the WIP tag.

I am still not happy with the test case.

lldb/source/Host/common/DebugInfoD.cpp

43

4 would have probably been fine too, as I don't think a core file "uuid" can make its way into here. In either case, we should document what is this working around, as 4 or 8 byte uuids are technically valid.

[kwk]

• Add documentation for workaround on rejecting special build UUIDs

[kwk]

@labath @jankratochvil @fche2 I've addressed all your comments and hope the patch is good to go now.

lldb/source/Host/common/DebugInfoD.cpp

43

@labath. I've added a documentation for the workaround.

[jankratochvil]

lldb/cmake/modules/FindDebuginfod.cmake

59

"No newline at end of file", this is what saving this diff, git apply --index and git diff says to me.

lldb/include/lldb/Host/DebugInfoD.h

28

Describe what does mean a returned std::string("") - that no error happened but server does not know this UUID/path.

lldb/source/Core/SourceManager.cpp

408

I do not like this extra line as it changes behavior of LLDB unrelated to debuginfod. IIUC if the source file with fully specified directory+filename in DWARF does not exist but the same filename exists in a different directory of the sourcetree LLDB will now quietly use the different file. That's a bug.
I think it is there as you needed to initialize sc.module_sp.

462

Make the debuginfod::isAvailable() check first as it is zero-cost, FileSystem::Instance().Exists is expensive filesystem operation.
The problem with that sc.module_sp is it is initialized above with some side effects. I think you should be fine without needing any sc. The following code does not pass the testcase for me but I guess you may fix it better:

// Try finding the file using elfutils' debuginfod
if (!FileSystem::Instance().Exists(m_file_spec) &&
    debuginfod::isAvailable())
  target->GetImages().ForEach(
      [&](const ModuleSP &module_sp) -> bool {
    llvm::Expected<std::string> cache_path = debuginfod::findSource(
        module_sp->GetUUID(), file_spec.GetCString());
    if (!cache_path) {
      module_sp->ReportWarning(
          "An error occurred while finding the "
          "source file %s using debuginfod for build ID %s: %s",
          file_spec.GetCString(),
          sc.module_sp->GetUUID().GetAsString("").c_str(),
          llvm::toString(cache_path.takeError()).c_str());
    } else if (!cache_path->empty()) {
      m_file_spec = FileSpec(*cache_path);
      m_mod_time = FileSystem::Instance().GetModificationTime(*cache_path);
      return false;
    }
    return true;
  });

lldb/source/Host/common/DebugInfoD.cpp

51

It should not be an error:

echo 'int main(void) { return 0; }' >/tmp/main2.c;gcc -o /tmp/main2 /tmp/main2.c -Wall -g -Wl,--build-id=none;rm /tmp/main2.c;DEBUGINFOD_URLS=http://localhost:8002/ ./bin/lldb /tmp/main2 -o 'l main' -o q
(lldb) target create "/tmp/main2"
Current executable set to '/tmp/main2' (x86_64).
(lldb) l main
warning: (x86_64) /tmp/main2 An error occurred while finding the source file /tmp/main2.c using debuginfod for build ID A9C3D738: invalid build ID: A9C3D738
File: /tmp/main2.c
(lldb) q

lldb/test/Shell/SymbolFile/DWARF/source-list.cpp

104

s/123/{{[0-9]+}}/?

136

"No newline at end of file", this is what saving this diff, git apply --index and git diff says to me.

[jankratochvil]

lldb/source/Core/SourceManager.cpp

415

This code could be more efficient than my previously proposed GetImages.ForEach() as it should be able to find the only one Module having that source file.
But there should be passed the full pathname incl. directories to prevent wrongly chosen accidentally filename-matching source files:

FileSystem::Instance().Exists(m_file_spec) ? file_spec.GetFilename().AsCString() : file_spec.GetCString(false/*denormalize*/)

And the Exists() check should be cached in this whole function as it is expensive.

462

Please ignore this comment + code fragment, I think it should not be needed.
(Just the isAvailable() check should be moved.)

[kwk]

• Add newline to end of FindDebuginfod.cmake
• Describe empty string returned from debuginfod::findSource()
• Don't treat build IDs of len <= 8 as an error but simply as not found
• move inexpensive debuginfod::isAvailable() check to beginning of if-stmt
• Simplify line number check in test file to avoid adjusting the line number every time the test changes
• Add newline to source-list.cpp test file

[kwk]

@jankratochvil thanks for this thorough review. I have to think about one comment more precisely but the rest was fixed.

lldb/source/Host/common/DebugInfoD.cpp

51

Okay, I'll have it return just an empty string. And adjust the comment on the empty string in findSource documentation. I fully understand that an error is undesirable in your test case. My question is if the caller should sanitize it's parameters passed to findSource of if the latter should silently ignore those wrong UUIDs. For now I silently ignore them and treat a wrong build ID like a not found (e.g. empty string is returned).

lldb/test/Shell/SymbolFile/DWARF/source-list.cpp

104

Both are fine, but I'll go with your's if that helps. If you can tell me how to get a lit CHECK statement that checks for incremental numbers, that'll be awesome ;)

[labath]

Adding @jingham. Jim, what do you make of this patch and the feature overall?

I know I said this looks "mostly good", but thinking about this further (and reading Jan's comments), I do find that there are still couple of things that trouble me here. :/

The first is the module_sp searching logic. I think that was previously here mainly to support the case when one enters source list I_am_too_lazy_to_enter_the_full_path.cc, and it would not normally fire when displaying the context after the process stops. But this makes a full-fledged feature out of it, as it will run every time we look up a file (if debuginfod is enabled, etc.). It seems fine to do this for the "source list" command (though it also may be nice to give the user an option to override this logic, just like he can specify a full path if he wants to), but doing it for stop-context purposes seems wrong, as there we should already have right module somewhere up the stack.

The second is the interaction between this and the target.source-map setting. For searching the file on the local filesystem, we want to use the remapped path, but in case of debuginfod, we would want to use the original path (ideally the one which doesn't even have the per-module mappings applied). The two of these things make me wonder if this new code is plugged in at the right level.

The last one is the test case. I've already said why I don't think this is a good test. Now I'll just add one more reason. With python it would be easy to create a function which handles the details of starting a fake debug info server. With lit, each new test for this (there are going to be more that one, I hope) will have to copy the // RUN: goo needed to start the server in a separate process. Sure, maybe you could do something similar here too and move that logic into a shell script, but then this will look even less like a "normal" lit test: a RUN line, which invokes a shell script, which invokes python in a background process... -- it would be much simpler (and portable) if it was python all the way.

lldb/source/Core/SourceManager.cpp

408

Yes, that does not sound right. It may be good to break this function into smaller pieces so you can invoke the thing you need when you need it.

lldb/source/Host/common/DebugInfoD.cpp

51

It would be nice to make a test case out of that.

[jingham]

Greg originally designed the macOS equivalent of this, so I've added him.

I totally agree that you should only do wide searches for source files when there's no way to get a narrower context. Even "source list" could use the current thread & frame as a start context, and only fall back to a full search when that fails. In a big project with many shlibs, you might have multiple files of the same name, but if someone specifies foo.cpp while stopped in a method of libMyLib.dylib, they probably do mean the one in libMyLib.dylib, if it exists...

I'm not terribly happy with the way the module-level source-file remapping interacts with debug info. It overrides the source-map without a way to undo that. The dSYM's that Apple uses internally have module-level source remapping in them that point to some NFS mounted directory. We've had problems where somebody has copied over one of these dSYM's and the associated sources, but doesn't want to remote mount the directories that host the sources. The only way to point lldb at those is to either copy them over to a directory structure that matches the remote path, or edit the dSYM to remove the source remapping. That's pretty annoying.

On the other hand, I don't think we should show the build locations (or at least not as a primary thing) for source files that have come from debuginfod or from another module level remapping. That's confusing to anyone who wants to open the file in some other way (for instance if you wanted to hand the file off to an external editor.) If we a way to get at both pieces of information, the source info command could be used to show the "debug info path" and the "local path" for a given source file. We might want some API (on SBFileSpec?) to get the original path - that would actually be useful when trying to figure out what you should use for a target.source-map. But if we have a local copy of the file you need to be able to get to that path.

The test does seem like it would be much better as a Python test.

[kwk]

@labath I made a signficant simplification of starting and killing the server. I hope you like that better.

lldb/source/Core/SourceManager.cpp

408

My intention wasn't to leave this as is to be honest. I had comments in here that I removed upon request but they existed to remind myself that I haven't double checked the logic well enough. I just wanted access to the symbol context further down below and thought, that I can take it from up here.

lldb/source/Host/common/DebugInfoD.cpp

51

I agree, a test would be nice but not at this stage, where the whole patch seems to be at danger.

lldb/test/Shell/SymbolFile/DWARF/source-list.cpp

58

@labath My bad. I interpreted timeout 5 wrongly. It will kill the python server after 5 seconds no matter what. If we increase this time to timeout 5m it will kill the server after 5 minutes and we don't need the bash trap. Does that sound better? At least the only ugly part would be done this way. The whole section would look like this:

// RUN: rm -f "%t.server.log"
// RUN: timeout 5m python3 -u -m http.server 0 --directory %t.mock --bind "localhost" &> %t.server.log &

[clayborg]

Currently we have a solution for macOS to locate symbol files in the "lldb/source/Symbol/LocateSymbolFile.cpp" file in the Symbols::LocateExecutableSymbolFile(...) function:

static FileSpec Symbols::LocateExecutableSymbolFile(const ModuleSpec &module_spec, const FileSpecList &default_search_paths);

This will locate any files that are already on the system and return the symbol file. When you don't have a symbol file, we can call:

static bool Symbols::DownloadObjectAndSymbolFile(ModuleSpec &module_spec, bool force_lookup = true);

This might ping a build server and download the symbols.

As for source file remappings, as Jim stated, on mac, each dSYM has path remappings already inside of it that are applied on the Module (not a target wide setting) itself and no modifications need to be done to the SourceManager.

So my question is: can be use debuginfod to find the symbol file for a given build ID via Symbols::LocateExecutableSymbolFile(...), and when/if a symbol file is fetched from debuginfod, apply all path remappings to the module itself that we hand out? Then no changes would be needed in the SourceManager, we would just ask for a symbol file and get one back with all the remappings that are needed.

[kwk]

• Use file:// and require debuginfod 0.179
• simplify FindDebuginfod.cmake

[labath]

In D75750#1948273, @clayborg wrote:

Currently we have a solution for macOS to locate symbol files in the "lldb/source/Symbol/LocateSymbolFile.cpp" file in the Symbols::LocateExecutableSymbolFile(...) function:

static FileSpec Symbols::LocateExecutableSymbolFile(const ModuleSpec &module_spec, const FileSpecList &default_search_paths);

This will locate any files that are already on the system and return the symbol file. When you don't have a symbol file, we can call:

static bool Symbols::DownloadObjectAndSymbolFile(ModuleSpec &module_spec, bool force_lookup = true);

This might ping a build server and download the symbols.

As for source file remappings, as Jim stated, on mac, each dSYM has path remappings already inside of it that are applied on the Module (not a target wide setting) itself and no modifications need to be done to the SourceManager.

So my question is: can be use debuginfod to find the symbol file for a given build ID via Symbols::LocateExecutableSymbolFile(...), and when/if a symbol file is fetched from debuginfod, apply all path remappings to the module itself that we hand out? Then no changes would be needed in the SourceManager, we would just ask for a symbol file and get one back with all the remappings that are needed.

I've been thinking about that a lot too. The thing that's not clear to me is, does DownloadObjectAndSymbolFile download source files too? If so, how?

I am expecting that this feature will hook in very near to DownloadObjectAndSymbolFile for downloading the debug info, but it's not clear to me how would the source files fit in. Currently, debuginfod only provides an api to retrieve a single source file, so this code would have to parse all of the debug info, pry out the source files, and download them one by one -- a complicated and slow process.

Now if debuginfod provided an api to download all source files in a single request (*), that might be workable. However, in principle, I see nothing wrong with being able to download the files on demand, if we have the option to do that. (debuginfod's long term goal seems to be to provide an api to download the debug info in chunks too -- that would be very interesting, though I also expect it to be very complicated.)

(*) Though it seems very wasteful to download all files when we are going to need only a handful of them, it may not really be that way -- we're going to be downloading all of debug info anyway, and this is going to be much larger that all of source code put together.

[fche2]

In D75750#1949527, @labath wrote:

I am expecting that this feature will hook in very near to DownloadObjectAndSymbolFile for downloading the debug info, but it's not clear to me how would the source files fit in. Currently, debuginfod only provides an api to retrieve a single source file, so this code would have to parse all of the debug info, pry out the source files, and download them one by one -- a complicated and slow process.

Yeah, as debuginfod does not support a batch type of source download, maybe this particular lldb site is not an ideal fit..

(*) Though it seems very wasteful to download all files when we are going to need only a handful of them, it may not really be that way -- we're going to be downloading all of debug info anyway, and this is going to be much larger that all of source code put together.

I see your point, OTOH you only download the whole debuginfo because you currently have no choice. (Someday with debuginfod or such, you might be able to offload the DWARF searches, and then you won't have to download the whole thing.) We do have the choice to download sources on demand.

[clayborg]

In D75750#1949527, @labath wrote:

In D75750#1948273, @clayborg wrote:

Currently we have a solution for macOS to locate symbol files in the "lldb/source/Symbol/LocateSymbolFile.cpp" file in the Symbols::LocateExecutableSymbolFile(...) function:

static FileSpec Symbols::LocateExecutableSymbolFile(const ModuleSpec &module_spec, const FileSpecList &default_search_paths);

This will locate any files that are already on the system and return the symbol file. When you don't have a symbol file, we can call:

static bool Symbols::DownloadObjectAndSymbolFile(ModuleSpec &module_spec, bool force_lookup = true);

This might ping a build server and download the symbols.

As for source file remappings, as Jim stated, on mac, each dSYM has path remappings already inside of it that are applied on the Module (not a target wide setting) itself and no modifications need to be done to the SourceManager.

So my question is: can be use debuginfod to find the symbol file for a given build ID via Symbols::LocateExecutableSymbolFile(...), and when/if a symbol file is fetched from debuginfod, apply all path remappings to the module itself that we hand out? Then no changes would be needed in the SourceManager, we would just ask for a symbol file and get one back with all the remappings that are needed.

I've been thinking about that a lot too. The thing that's not clear to me is, does DownloadObjectAndSymbolFile download source files too? If so, how?

We should probably make a new SymbolServer plug-in and convert the Apple version to compile and be installed only for Apple. The API for this should be something like:

class SymbolServer {
  /// Get a cached symbol file that is already present on this machine.
  /// This gets called by all LLDB during normal debugging to fetch 
  /// and cached symbol files. 
  virtual ModuleSP GetSymbolFile(ModuleSpec module_spec);

  /// Download a symbol file when requested by the user. 
  /// This only gets run in response to the use requesting the symbols, 
  /// not as part of the normal debug work flow
  virtual FileSpec DownloadSymbolFile(ModuleSpec module_spec);

  /// New function that allows individual access to source files when 
  /// they aren't available on disk.
  virtual FileSpec GetSourceFile(FileSpec file, ....)
};

Then debuginfod would fit right in there. The one thing that this interace doesn't cover is adding source remappings to modules, but it would be great if we can do this somehow with this new interface. Maybe SymbolServer::GetSymbolFile() can take a module_sp of the existing module so it can modify the source remappings if it has any?

I am expecting that this feature will hook in very near to DownloadObjectAndSymbolFile for downloading the debug info, but it's not clear to me how would the source files fit in. Currently, debuginfod only provides an api to retrieve a single source file, so this code would have to parse all of the debug info, pry out the source files, and download them one by one -- a complicated and slow process.

This would be taken care of by the SymbolServer plug-in described above. For the Apple version, it would download the symbol file and remap the paths as it already does and the SymbolServer::GetSourceFile(FileSpec file) would just return the FileSpec that was passed in since it already is an NFS mount path. For debuginfod it can grab the source file one by one.

One possibility is to apply a module level source remapping that is unique to the symbol file that is returned from SymbolServer::GetSymbolFile(). Maybe we prepend the UUID of the module to all paths in the debug info. Something like mapping:

"/..." to "/<UUID/..."

Then when we run into this kind of path, we know we need to call into the SymbolServer to resolve it (by possibly downloading it and caching it first).

Now if debuginfod provided an api to download all source files in a single request (*), that might be workable. However, in principle, I see nothing wrong with being able to download the files on demand, if we have the option to do that. (debuginfod's long term goal seems to be to provide an api to download the debug info in chunks too -- that would be very interesting, though I also expect it to be very complicated.)

Agreed, lazy is good. I don't see the need to download sources in chunks though.

So using SymbolServer with unique path mappings ("/..." to "/<UUID/...") would allow us to accomplish lazy file access. Another option would be to mark and FileSpec objects that are handed out by any symbol files retrieved from the symbol server as needing resolution in the SymbolServer. Later code could do something like:

if file_spec.PathNeedsSymbolServerResolution():

file_spec.ResolveWithSymbolServer(symbol_server);

And the path would update itself in the SymbolFile before it makes it out to any users. So stack frames that do lookups would end up resolving these paths before handing the information out to the user.

(*) Though it seems very wasteful to download all files when we are going to need only a handful of them, it may not really be that way -- we're going to be downloading all of debug info anyway, and this is going to be much larger that all of source code put together.

I think we should be able to figure out how to make this lazy with a new plug-in interface

[clayborg]

In D75750#1949678, @fche2 wrote:

In D75750#1949527, @labath wrote:

I am expecting that this feature will hook in very near to DownloadObjectAndSymbolFile for downloading the debug info, but it's not clear to me how would the source files fit in. Currently, debuginfod only provides an api to retrieve a single source file, so this code would have to parse all of the debug info, pry out the source files, and download them one by one -- a complicated and slow process.

Yeah, as debuginfod does not support a batch type of source download, maybe this particular lldb site is not an ideal fit..

We can make it ideal. debuginfod has nice stuff in it and we should adapt LLDB for sure! See my SymbolServer plug-in interface in my previous comments and let me know what you think.

(*) Though it seems very wasteful to download all files when we are going to need only a handful of them, it may not really be that way -- we're going to be downloading all of debug info anyway, and this is going to be much larger that all of source code put together.

I see your point, OTOH you only download the whole debuginfo because you currently have no choice. (Someday with debuginfod or such, you might be able to offload the DWARF searches, and then you won't have to download the whole thing.) We do have the choice to download sources on demand.

We should be able to make this work lazily and not having to download all files.

[labath]

Making a plugin out of this sounds like a good idea to me, and I could immediately find several downstream users for it. However, it seems to me there is a great deal of overlap between this SymbolServer thingy and the existing SymbolVendor plugin (I mean, "vend" and "serve" are basically synonyms in this context). The main difference is that SymbolVendor is responsible for just finding the symbol file (in case it is not in the main executable), where as this new thing could also be used for finding the main executable too (as well as the relevant source files).

I think it would be very confusing to have both symbol "vendors" and "servers" and we should try hard to implement that with a single interface. The SymbolVendor doesn't do much nowadays (it's basically just a single function that tries to search in various locations -- the rest is boilerplate). If we add more functionality to it, it might make it seem less baroque.

That might also help the path remapping situation. Since symbol vendors sort of sit in between the SymbolFile and Module classes, it should be possible to arrange things such that they see the raw paths coming out of the symbol file, before they are mangled by various mappings.

[clayborg]

The main issue is that the symbol vendors currently are ELF, macOS and WASM. Right now we have one SymbolVendor for a triple, but I can see a SymbolVendor wanting to use multiple symbol servers to get information: one for the OS binaries (debuginfod or DebugSymbols.framework at Apple) and one for the current application with company specific symbol servers. At Apple, they can download any symbols for macOS, iOS, watchOS and tvOS OSes and applications. At Facebook we can download symbols for android, linux and iOS. Linux distros might have ways to download symbols for their OS stuff, which might work along side debuginfod? Also windows has the ability to download symbols.

So it might be good to have the SymbolVendors use one or more SymbolServer plug-ins.

[clayborg]

Another idea for the SymbolServers: be able to specify a source repository (git, svn etc) and hash or revision ID. The symbol server can grab the source from the repo and cache is locally for display.

[jankratochvil]

In D75750#1954086, @clayborg wrote:

Another idea for the SymbolServers: be able to specify a source repository (git, svn etc) and hash or revision ID. The symbol server can grab the source from the repo and cache is locally for display.

When you talk about it FYI I use "build-id" to "GIT hash" mapping text file during build to retrieve source files later according to binary's build-id. But it expects you do not strip symbols from the binaries as otherwise one cannot rebuild the binaries later ("reproducible build" problem - dependency on versions of system packages being updated in the meantime). https://www.jankratochvil.net/t/BUILDID-git-checkout
debuginfod solves this better although with higher storage requirements.

[labath]

In D75750#1953924, @clayborg wrote:

The main issue is that the symbol vendors currently are ELF, macOS and WASM. Right now we have one SymbolVendor for a triple, but I can see a SymbolVendor wanting to use multiple symbol servers to get information: one for the OS binaries (debuginfod or DebugSymbols.framework at Apple) and one for the current application with company specific symbol servers. At Apple, they can download any symbols for macOS, iOS, watchOS and tvOS OSes and applications. At Facebook we can download symbols for android, linux and iOS. Linux distros might have ways to download symbols for their OS stuff, which might work along side debuginfod? Also windows has the ability to download symbols.

So it might be good to have the SymbolVendors use one or more SymbolServer plug-ins.

I don't believe we have anything that would require all modules in a given target (or whatever) to use the same symbol vendor type. Each module gets its own instance of the object, which is obtained and manipulated through the generic interface. It is true that our current symbol vendors key off of the triple (more like object file type, really), so all modules ale likely to have the same vendor, but nothing really requires it to be that way. The symbol vendors get a ModuleSP, and they can use any information there to determine whether they are relevant. So if we had multiple symbol vendors interested in say elf files, we would just ask each of them in turn whether they can handle this module, and the first one would "win".

[fche2]

So it might be good to have the SymbolVendors use one or more SymbolServer plug-ins.

I don't believe we have anything that would require all modules in a given target (or whatever) to use the same symbol vendor type. [...]

Just for clarity, is someone proposing to undertake such a rework of that infrastructure? It sounds like this is becoming a prerequisite for Konrad's patch, but if no one's actually doing it, that means Konrad's work is on hold indefinitely. Is that the intent?

[labath]

In D75750#1967019, @fche2 wrote:

So it might be good to have the SymbolVendors use one or more SymbolServer plug-ins.

I don't believe we have anything that would require all modules in a given target (or whatever) to use the same symbol vendor type. [...]

Just for clarity, is someone proposing to undertake such a rework of that infrastructure? It sounds like this is becoming a prerequisite for Konrad's patch, but if no one's actually doing it, that means Konrad's work is on hold indefinitely. Is that the intent?

Yes, I believe that is becoming a prerequisite. I believe Konrad is willing to try to implement that, but I have advised him to hold on a bit until the exact details are hashed out.

[kwk]

In D75750#1971446, @labath wrote:

In D75750#1967019, @fche2 wrote:

So it might be good to have the SymbolVendors use one or more SymbolServer plug-ins.

I don't believe we have anything that would require all modules in a given target (or whatever) to use the same symbol vendor type. [...]

Just for clarity, is someone proposing to undertake such a rework of that infrastructure? It sounds like this is becoming a prerequisite for Konrad's patch, but if no one's actually doing it, that means Konrad's work is on hold indefinitely. Is that the intent?

Yes, I believe that is becoming a prerequisite. I believe Konrad is willing to try to implement that, but I have advised him to hold on a bit until the exact details are hashed out.

@labath, I'm not really keen on implementing the architectural changes that you mentioned because it will take ages when I do that. And the cross-platform bit makes me nervous as well. Initially I hoped we might be able to integrate my work and improve on the architecture later. Then we're not fighting on too many fronts at the same time?

Shall we maybe move the discussion about the architectural changes to lldb-dev instead of this patch? @clayborg @labath @jingham @jankratochvil ?

[labath]

lldb-dev is indeed a better place for the architectural discussion. However, moving the discussion there does not automatically unblock this patch. "get something in now and improve the architecture later" almost never works out in practice. In fact I would say that adding debuginfod is a good way to cement the status quo. The situation around finding symbols is messy enough already because one needs to understand the funky mac symbol searching mechanism, which is pretty much impossible without a mac machine. After debuginfod, one will need to understand both, and have a linux machine with some debuginfod setup. The set of such people is likely to be empty of a long time...

[kwk]

In D75750#1988330, @labath wrote:

lldb-dev is indeed a better place for the architectural discussion. However, moving the discussion there does not automatically unblock this patch. "get something in now and improve the architecture later" almost never works out in practice. In fact I would say that adding debuginfod is a good way to cement the status quo.

I get that, but hear me out...

The situation around finding symbols is messy enough already

The way in which I integrated debuginfod for now is just to find source files and not yet symbols. That being said. I don't fear the status quo so much. The status quo is probably worse for symbols than it is for source files, don't you think? So with *all* the CMake integration, the hosting inside lldb/include/lldb/Host/DebugInfoD.h and your beloved test case,

I think it is fair to say that at least some work is there that can be taken into LLDB. As long as I fix the retrieval of the module in SourceManager::File::CommonInitializer. As suggested by @jankratochvil either here or on IRC, I would like to give it a shot and try to pass down the correct module to this function. I'd say, let's see if this function can be passed a Module and if the changes are worth it. The whole part for retrieving debug information can come when the architectural changes are done. But then it's a piece of cake to extend lldb/include/lldb/Host/DebugInfoD.h with the right methods to call the debuginfod client lib.

because one needs to understand the funky mac symbol searching mechanism, which is pretty much impossible without a mac machine.

I'm setting up my old mac to compile LLDB and I guess @jankratochvil might soon also have his own Mac. This at least puts us in a position where we can verify some of our changes.

After debuginfod, one will need to understand both, and have a linux machine with some debuginfod setup. The set of such people is likely to be empty of a long time...

I'm not sure if I understand you correctly but to me the *setup* is just to point to a machine with *your* or a hosted server. At least for OS binaries @fche2 @fche (which is the correct one?) is making some effort to have those debuginfos and source files available and setup. That is a great start for most embedded systems with not much disk space to install all debug information I guess. Correct my if this is a wrong anticipation. Sure, I mean it will take a while before LLDB with debuginfod will make it into a distribution but hey, time flies by.

[jankratochvil]

The current plan discussed with @kwk is to create the new SymbolServer abstract superclass and some its inherited implementation and move there the appropriate parts of existing lldb/source/Symbol/LocateSymbolFile.cpp. Current SymbolVendor implementations would then iterate new SymbolServers by the existing LocateExecutableSymbolFile function. That may be enough for a patch of its own.

[labath]

In D75750#1988669, @kwk wrote:

In D75750#1988330, @labath wrote:

The situation around finding symbols is messy enough already

The way in which I integrated debuginfod for now is just to find source files and not yet symbols. That being said. I don't fear the status quo so much. The status quo is probably worse for symbols than it is for source files, don't you think? So with *all* the CMake integration, the hosting inside lldb/include/lldb/Host/DebugInfoD.h and your beloved test case,

<snip huge meme>

I think it is fair to say that at least some work is there that can be taken into LLDB. As long as I fix the retrieval of the module in SourceManager::File::CommonInitializer. As suggested by @jankratochvil either here or on IRC, I would like to give it a shot and try to pass down the correct module to this function. I'd say, let's see if this function can be passed a Module and if the changes are worth it. The whole part for retrieving debug information can come when the architectural changes are done.

That all sounds reasonable, and I would not really have a big problem with integrating debuginfod in this way (through @clayborg might -- you will also want to check with him). However, I have doubts about how long will it take to do the architectural changes to get symbol downloading to work (or even if it will happen). I don't want to demean the work you have done so far, but I think that stuff will be much more complicated than this.

In that light, it's not clear to me whether having the ability to download the source files without being able to get the executable and symbol files is particularly useful. It does not seem very useful to me, but maybe I am missing something.

Do you find that useful? If not, I think the changes can just as easily sit in this patch instead of in the tree. This isn't touching any areas under active development, so its not like this functionality will rot quickly.

But then it's a piece of cake to extend lldb/include/lldb/Host/DebugInfoD.h with the right methods to call the debuginfod client lib.

because one needs to understand the funky mac symbol searching mechanism, which is pretty much impossible without a mac machine.

I'm setting up my old mac to compile LLDB and I guess @jankratochvil might soon also have his own Mac. This at least puts us in a position where we can verify some of our changes.

That's great to hear. (though it's sad that is necessary)

After debuginfod, one will need to understand both, and have a linux machine with some debuginfod setup. The set of such people is likely to be empty of a long time...

I'm not sure if I understand you correctly but to me the *setup* is just to point to a machine with *your* or a hosted server. At least for OS binaries @fche2 @fche (which is the correct one?) is making some effort to have those debuginfos and source files available and setup. That is a great start for most embedded systems with not much disk space to install all debug information I guess. Correct my if this is a wrong anticipation. Sure, I mean it will take a while before LLDB with debuginfod will make it into a distribution but hey, time flies by.

I'm not worried about having symbols for all the system binaries. But just the effort to setup a environment with a foreign operating system and learn enough about it to be able to make changes to it are enough to dissuade a lot of potential developers (this is to your credit). After you start playing around with a mac, I think you'll find that a lot of things work differently there -- it took me about four years to understand how dsyms and debug maps work (I wasn't trying to learnt it all of that time, but still...).

In D75750#1988694, @jankratochvil wrote:

The current plan discussed with @kwk is to create the new SymbolServer abstract superclass and some its inherited implementation and move there the appropriate parts of existing lldb/source/Symbol/LocateSymbolFile.cpp. Current SymbolVendor implementations would then iterate new SymbolServers by the existing LocateExecutableSymbolFile function. That may be enough for a patch of its own.

I'll have to see the actual patch for a definitive opinion, but I have to say that a priori I am sceptical of this direction. And yes, that should definitely be a separate patch.

[jankratochvil]

In D75750#1991873, @labath wrote:

In D75750#1988694, @jankratochvil wrote:

The current plan discussed with @kwk is to create the new SymbolServer abstract superclass and some its inherited implementation and move there the appropriate parts of existing lldb/source/Symbol/LocateSymbolFile.cpp. Current SymbolVendor implementations would then iterate new SymbolServers by the existing LocateExecutableSymbolFile function. That may be enough for a patch of its own.

I'll have to see the actual patch for a definitive opinion, but I have to say that a priori I am sceptical of this direction. And yes, that should definitely be a separate patch.

This separate SymbolServer is following @clayborg's comment above.
You proposed to merge SymbolServer with SymbolVendor in @labath's comment above.
I found more clean the separate SymbolServer variant as there is orthogonal functionality of locating the files (on disk or from symbol server - SymbolServer) vs. extracting the unique ID from current file (extracting build-id - SymbolVendor functionality). So from the both proposed solutions I preferred the @clayborg's comment above.
I hope there is no misunderstanding which could lead to @kwk implementing a third solution nobody wants.

[labath]

In D75750#1993990, @jankratochvil wrote:

In D75750#1991873, @labath wrote:

In D75750#1988694, @jankratochvil wrote:

The current plan discussed with @kwk is to create the new SymbolServer abstract superclass and some its inherited implementation and move there the appropriate parts of existing lldb/source/Symbol/LocateSymbolFile.cpp. Current SymbolVendor implementations would then iterate new SymbolServers by the existing LocateExecutableSymbolFile function. That may be enough for a patch of its own.

I'll have to see the actual patch for a definitive opinion, but I have to say that a priori I am sceptical of this direction. And yes, that should definitely be a separate patch.

This separate SymbolServer is following @clayborg's comment above.
You proposed to merge SymbolServer with SymbolVendor in @labath's comment above.
I found more clean the separate SymbolServer variant as there is orthogonal functionality of locating the files (on disk or from symbol server - SymbolServer) vs. extracting the unique ID from current file (extracting build-id - SymbolVendor functionality). So from the both proposed solutions I preferred the @clayborg's comment above.
I hope there is no misunderstanding which could lead to @kwk implementing a third solution nobody wants.

I think that you two and Greg are mostly in sync, but I am yet to be convinced that this indeed the right solution. My reasons for that are two fold:

• the existing SymbolVendor implementations are very simple (and most importantly, stateless). In fact they are so simple, I was contemplating simply removing them and replacing by a couple of free functions. Surely, we don't need an entire class hierarchy to implement "extracting the unique ID from current file". Implementing symbol server functionality would give these classes a reason to exist, as there would now be an actual state that they need to maintain (a connection to the symbol server, or at least its url)
• I don't think the separation between "SymbolServer" and "SymbolVendor" will be as clear as you make it sound to be. The "searching" aspect is fairly trivial in SymbolVendorELF, and it does boil down to a LocateExecutableSymbolFile call. The situation is a bit fuzzier with SymbolVendorMacOSX, which also handles some path remapping aspects. But that is the job of the symbol "server" in the debuginfod world. It's not clear to me how you're going to align these two things without "vendors" and "servers" separate.

Now you can try to implement a patch and demonstrate how things will work that way and hope to convince me that way, or we can to talk about abstractly, and come up with some sort of a "design doc" for this thing. Up to you.

[iridinite]

Hello all,

I am quite interested in integrating debuginfod with LLDB; my colleagues and I are currently exploring moving away from GDB to LLDB for general development work, since it appears to outperform GDB in many respects!

It appears there has been no activity on this patch for some time, so I was wondering, is this feature still something that is being investigated? Are there perhaps any alternatives I could look into, if debuginfod is not planned to be part of the LLDB mainline?

Thank you for your time!

[kwk]

@iridinite please see these:

• https://discourse.llvm.org/t/http-library-in-llvm/56317/10
• https://discourse.llvm.org/t/rfc-building-llvm-debuginfod/59011
• https://discourse.llvm.org/t/rfc-building-llvm-debuginfod/58994
• https://discourse.llvm.org/t/debuginfod-credential-helper-rfc/64092

I suggest, you contact @phosek on the status of debuginfod implementation in LLVM. I'd love to know where it is at, so please leave a trace ;)

[kwk]

Looks like it is already there: https://github.com/llvm/llvm-project/tree/main/llvm/include/llvm/Debuginfod

[kwk]

There already is a Debuginfod implementation in LLVM by now. Abandoning revision.

[fche2]

Closing this might have been premature. I can't find an lldb debuginfod client side support. Is that somehow behind llvm-symbolicator or something like that? Is there documentation?

[iridinite]

@kwk Thanks for the links, good to know that there is a debuginfod client in the LLVM project. However, doesn't lldb itself still need to integrate with this, like @fche2 mentioned, for end-users to be able to work with it? (Please correct me if I'm wrong, I am not familiar with the LLVM project structure.)

If there is any documentation or notes on how we can set this up, that would be much appreciated!

lldb/cmake/modules/FindDebuginfod.cmake

+#.rst:
+# FindDebuginfod
+# -----------
+#
+# Find debuginfod library and headers
+#
+# The module defines the following variables:
+#
+# ::
+#
+#   Debuginfod_FOUND          - true if debuginfod was found
+#   Debuginfod_INCLUDE_DIRS   - include search path
+#   Debuginfod_LIBRARIES      - libraries to link
+#   Debuginfod_VERSION_STRING - version number
+#
+# TODO(kwk): Debuginfod_VERSION_STRING is only set if pkg-config file is
+# available. Trying to see if we can get a MAJOR, MINOR, PATCH define in the
+# debuginfod.h file.
+
+if(Debuginfod_INCLUDE_DIRS AND Debuginfod_LIBRARIES)
+  set(Debuginfod_FOUND TRUE)
+else()
+  # Utilize package config (e.g. /usr/lib64/pkgconfig/libdebuginfod.pc) to fetch
+  # version information. 
+  find_package(PkgConfig QUIET)
+  pkg_check_modules(PC_Debuginfod QUIET libdebuginfod)
+
+  find_path(Debuginfod_INCLUDE_DIRS
+            NAMES
+              elfutils/debuginfod.h
+            HINTS
+              /usr/include
+              ${PC_Debuginfod_INCLUDEDIR}
+              ${PC_Debuginfod_INCLUDE_DIRS}
+              ${CMAKE_INSTALL_FULL_INCLUDEDIR})
+  find_library(Debuginfod_LIBRARIES
+               NAMES
+                 debuginfod
+               HINTS
+                 ${PC_Debuginfod_LIBDIR}
+                 ${PC_Debuginfod_LIBRARY_DIRS}
+                 ${CMAKE_INSTALL_FULL_LIBDIR})
+
+  if(Debuginfod_INCLUDE_DIRS AND EXISTS "${Debuginfod_INCLUDE_DIRS}/debuginfod.h")
+    set(Debuginfod_VERSION_STRING "${PC_Debuginfod_VERSION}")
+  endif()
+
+  include(FindPackageHandleStandardArgs)
+  find_package_handle_standard_args(Debuginfod
+                                    FOUND_VAR
+                                      Debuginfod_FOUND
+                                    REQUIRED_VARS
+                                      Debuginfod_INCLUDE_DIRS
+                                      Debuginfod_LIBRARIES
+                                    VERSION_VAR
+                                      Debuginfod_VERSION_STRING)
+  mark_as_advanced(Debuginfod_INCLUDE_DIRS Debuginfod_LIBRARIES)
+endif()
 No newline at end of file

lldb/cmake/modules/LLDBConfig.cmake

 endmacro()
 
 add_optional_dependency(LLDB_ENABLE_LIBEDIT "Enable editline support in LLDB" LibEdit LibEdit_FOUND)
 add_optional_dependency(LLDB_ENABLE_CURSES "Enable curses support in LLDB" CursesAndPanel CURSESANDPANEL_FOUND)
 add_optional_dependency(LLDB_ENABLE_LZMA "Enable LZMA compression support in LLDB" LibLZMA LIBLZMA_FOUND)
 add_optional_dependency(LLDB_ENABLE_LUA "Enable Lua scripting support in LLDB" LuaAndSwig LUAANDSWIG_FOUND)
 add_optional_dependency(LLDB_ENABLE_PYTHON "Enable Python scripting support in LLDB" PythonInterpAndLibs PYTHONINTERPANDLIBS_FOUND)
 add_optional_dependency(LLDB_ENABLE_LIBXML2 "Enable Libxml 2 support in LLDB" LibXml2 LIBXML2_FOUND VERSION 2.8)
+add_optional_dependency(LLDB_ENABLE_DEBUGINFOD "Enable Debuginfod support in LLDB" Debuginfod Debuginfod_FOUND)
 
 option(LLDB_USE_SYSTEM_SIX "Use six.py shipped with system and do not install a copy of it" OFF)
 option(LLDB_USE_ENTITLEMENTS "When codesigning, use entitlements if available" ON)
 option(LLDB_BUILD_FRAMEWORK "Build LLDB.framework (Darwin only)" OFF)
 option(LLDB_NO_INSTALL_DEFAULT_RPATH "Disable default RPATH settings in binaries" OFF)
 option(LLDB_USE_SYSTEM_DEBUGSERVER "Use the system's debugserver for testing (Darwin only)." OFF)
 option(LLDB_SKIP_STRIP "Whether to skip stripping of binaries when installing lldb." OFF)
 

 endif()
 set(LLDB_VERSION "${LLDB_VERSION_MAJOR}.${LLDB_VERSION_MINOR}.${LLDB_VERSION_PATCH}${LLDB_VERSION_SUFFIX}")
 message(STATUS "LLDB version: ${LLDB_VERSION}")
 
 if (LLDB_ENABLE_LZMA)
   include_directories(${LIBLZMA_INCLUDE_DIRS})
 endif()
 
+if (LLDB_ENABLE_DEBUGINFOD)
+  include_directories(${Debuginfod_INCLUDE_DIRS})
+endif()
+
 if (LLDB_ENABLE_LIBXML2)
   list(APPEND system_libs ${LIBXML2_LIBRARIES})
   include_directories(${LIBXML2_INCLUDE_DIR})
 endif()
 
 include_directories(BEFORE
   ${CMAKE_CURRENT_BINARY_DIR}/include
   ${CMAKE_CURRENT_SOURCE_DIR}/include

lldb/include/lldb/Host/Config.h.cmake

 #endif
 
 #cmakedefine01 LLDB_ENABLE_POSIX
 
 #cmakedefine01 LLDB_ENABLE_TERMIOS
 
 #cmakedefine01 LLDB_ENABLE_LZMA
 
+#cmakedefine01 LLDB_ENABLE_DEBUGINFOD
+
 #cmakedefine01 LLDB_ENABLE_CURSES
 
 #cmakedefine01 LLDB_ENABLE_LIBEDIT
 
 #cmakedefine01 LLDB_ENABLE_LIBXML2
 
 #cmakedefine01 LLDB_ENABLE_LUA
 

lldb/include/lldb/Host/DebugInfoD.h

+//===-- DebugInfoD.h --------------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_HOST_DEBUGINFOD_H
+#define LLDB_HOST_DEBUGINFOD_H
+
+#include "lldb/Utility/UUID.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/Support/Error.h"
+
+namespace lldb_private {
+
+namespace debuginfod {
+
+// Returns \c true if debuginfod support was compiled-in; otherwise \c false is
+// returned.
+bool isAvailable();
+
+// Asks all servers in environment variable \c DEBUGINFOD_URLS for the \a path
+// of an artifact with a given \a buildID and returns the path to a locally
+// cached version of the file. If there  was an error, we return that instead.
+llvm::Expected<std::string> findSource(const UUID &buildID,
+                                       const std::string &path);
+
+} // End of namespace debuginfod
+
+} // End of namespace lldb_private
+
+#endif // LLDB_HOST_DEBUGINFOD_H

lldb/packages/Python/lldbsuite/test/httpserver.py

+# -*- coding: utf-8 -*-
+"""
+The httpserver module contains an easy and ready-to-use HTTP file server.
+
+o ServeDirectoryWithHTTP: Spawns an HTTP file server in a separate thread.
+"""
+
+import http.server
+from threading import Thread, current_thread
+from sys import stderr
+from functools import partial
+from os.path import abspath
+
+
+def ServeDirectoryWithHTTP(directory="."):
+    """Spawns an http.server.HTTPServer in a separate thread on the given port.
+
+    The server serves files from the given *directory*. The port listening on
+    will automatically be picked by the operating system to avoid race
+    conditions when trying to bind to an open port that turns out not to be
+    free afterall. The hostname is always "localhost".
+
+    Parameters
+    ----------
+    directory : str, optional
+        The directory to server files from. Defaults to the current directory.
+
+    Returns
+    -------
+    http.server.HTTPServer
+        The HTTP server which is serving files from a separate thread.
+
+        It is not super necessary but you might want to call shutdown() on the
+        returned HTTP server object. This will stop the inifinite request loop
+        running in the thread which in turn will then exit. The reason why this
+        is only optional is that the thread in which the server runs is a daemon
+        thread which will be terminated when the main thread ends.
+
+        By calling shutdown() you'll get a cleaner shutdown because the socket
+        is properly closed.
+
+    str
+        The address of the server as a string, e.g. "http://localhost:1234".
+
+    Examples
+    --------
+    >>> from httpserver import ServeDirectoryWithHTTP
+    >>> from urllib.request import urlopen
+    >>> httpd, address = ServeDirectoryWithHTTP()
+    >>> print("Address:", address) # doctest:+ELLIPSIS
+    ...
+    Address: http://localhost...:...
+    >>> try:
+    ...     url = address + "/httpserver.py"
+    ...     print("Getting URL:", url) # doctest:+ELLIPSIS
+    ...     with urlopen(url) as f:
+    ...         print("Code:", f.getcode())
+    ... finally:
+    ...     httpd.shutdown()
+    ...
+    Getting URL: http://localhost...:.../httpserver.py
+    Code: 200
+
+    In the example above, you can call f.read() to read the content of the file
+    you've asked for.
+
+    """
+
+    hostname = "localhost"
+    port = 0
+    directory = abspath(directory)
+    handler = partial(_SimpleRequestHandler, directory=directory)
+    httpd = http.server.HTTPServer((hostname, 0), handler, False)
+    # Block only for 0.5 seconds max
+    httpd.timeout = 0.5
+    # Allow for reusing the address
+    # HTTPServer sets this as well but I wanted to make this more obvious.
+    httpd.allow_reuse_address = True
+
+    _xprint("server about to bind to port %d on hostname '%s'" % (port, hostname))
+    httpd.server_bind()
+
+    address = "http://%s:%d" % (httpd.server_name, httpd.server_port)
+
+    _xprint("server about to listen on:", address)
+    httpd.server_activate()
+
+    def serve_forever(httpd):
+        with httpd:  # to make sure httpd.server_close is called
+            _xprint("server about to serve files from directory (infinite request loop):", directory)
+            httpd.serve_forever()
+            _xprint("server left infinite request loop")
+
+    thread = Thread(target=serve_forever, args=(httpd, ))
+    thread.setDaemon(True)
+    thread.start()
+
+    return httpd, address
+
+
+def _xprint(*args, **kwargs):
+    """Wrapper function around print() that prepends the current thread name"""
+    print("[", current_thread().name, "]",
+          " ".join(map(str, args)), **kwargs, file=stderr)
+
+
+class _SimpleRequestHandler(http.server.SimpleHTTPRequestHandler):
+    """Same as SimpleHTTPRequestHandler with adjusted logging."""
+
+    def log_message(self, format, *args):
+        """Log an arbitrary message and prepend the given thread name."""
+        stderr.write("[ " + current_thread().name + " ] ")
+        http.server.SimpleHTTPRequestHandler.log_message(self, format, *args)
+
+
+if __name__ == "__main__":
+    from doctest import testmod
+    testmod(verbose=True)

lldb/source/Core/SourceManager.cpp

 
 #include "lldb/Core/Address.h"
 #include "lldb/Core/AddressRange.h"
 #include "lldb/Core/Debugger.h"
 #include "lldb/Core/FormatEntity.h"
 #include "lldb/Core/Highlighter.h"
 #include "lldb/Core/Module.h"
 #include "lldb/Core/ModuleList.h"
+#include "lldb/Host/DebugInfoD.h"
 #include "lldb/Host/FileSystem.h"
 #include "lldb/Symbol/CompileUnit.h"
 #include "lldb/Symbol/Function.h"
 #include "lldb/Symbol/LineEntry.h"
 #include "lldb/Symbol/SymbolContext.h"
 #include "lldb/Target/PathMappingList.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/AnsiTerminal.h"

 }
 
 void SourceManager::File::CommonInitializer(const FileSpec &file_spec,
                                             Target *target) {
   if (m_mod_time == llvm::sys::TimePoint<>()) {
     if (target) {
       m_source_map_mod_id = target->GetSourcePathMap().GetModificationID();
 
-      if (!file_spec.GetDirectory() && file_spec.GetFilename()) {
+      SymbolContext sc;
+      if ((!file_spec.GetDirectory() && file_spec.GetFilename()) ||
+          !FileSystem::Instance().Exists(m_file_spec)) {
         // If this is just a file name, lets see if we can find it in the
         // target:
         bool check_inlines = false;
         SymbolContextList sc_list;
         size_t num_matches =
             target->GetImages().ResolveSymbolContextForFilePath(
                 file_spec.GetFilename().AsCString(), 0, check_inlines,
                 SymbolContextItem(eSymbolContextModule |
                                   eSymbolContextCompUnit),
                 sc_list);
         bool got_multiple = false;
         if (num_matches != 0) {
           if (num_matches > 1) {
-            SymbolContext sc;
+            // SymbolContext sc;
             CompileUnit *test_cu = nullptr;
 
             for (unsigned i = 0; i < num_matches; i++) {
               sc_list.GetContextAtIndex(i, sc);
               if (sc.comp_unit) {
                 if (test_cu) {
                   if (test_cu != sc.comp_unit)
                     got_multiple = true;
                   break;
                 } else
                   test_cu = sc.comp_unit;
               }
             }
           }
           if (!got_multiple) {
-            SymbolContext sc;
+            // SymbolContext sc;
             sc_list.GetContextAtIndex(0, sc);
             if (sc.comp_unit)
               m_file_spec = sc.comp_unit->GetPrimaryFile();
-            m_mod_time = FileSystem::Instance().GetModificationTime(m_file_spec);
+            m_mod_time =
+                FileSystem::Instance().GetModificationTime(m_file_spec);
           }
         }
       }
       // Try remapping if m_file_spec does not correspond to an existing file.
       if (!FileSystem::Instance().Exists(m_file_spec)) {
         FileSpec new_file_spec;
         // Check target specific source remappings first, then fall back to
         // modules objects can have individual path remappings that were
         // detected when the debug info for a module was found. then
         if (target->GetSourcePathMap().FindFile(m_file_spec, new_file_spec) ||
             target->GetImages().FindSourceFile(m_file_spec, new_file_spec)) {
           m_file_spec = new_file_spec;
           m_mod_time = FileSystem::Instance().GetModificationTime(m_file_spec);
         }
       }
+
+      // Try finding the file using elfutils' debuginfod
+      if (!FileSystem::Instance().Exists(m_file_spec) &&
+          debuginfod::isAvailable() && sc.module_sp) {
+        llvm::Expected<std::string> cache_path = debuginfod::findSource(
+            sc.module_sp->GetUUID(), file_spec.GetCString());
+        if (!cache_path) {
+          sc.module_sp->ReportWarning(
+              "An error occurred while finding the "
+              "source file %s using debuginfod for build ID %s: %s",
+              file_spec.GetCString(),
+              sc.module_sp->GetUUID().GetAsString("").c_str(),
+              llvm::toString(cache_path.takeError()).c_str());
+        } else if (!cache_path->empty()) {
+          m_file_spec = FileSpec(*cache_path);
+          m_mod_time = FileSystem::Instance().GetModificationTime(*cache_path);
+        }
+      }
     }
   }
 
   if (m_mod_time != llvm::sys::TimePoint<>())
     m_data_sp = FileSystem::Instance().CreateDataBuffer(m_file_spec);
 }
 
 uint32_t SourceManager::File::GetLineOffset(uint32_t line) {

lldb/source/Host/CMakeLists.txt

add_host_subdirectory(common
   common/GetOptInc.cpp
   common/Host.cpp
   common/HostInfoBase.cpp
   common/HostNativeThreadBase.cpp
   common/HostProcess.cpp
   common/HostThread.cpp
   common/LockFileBase.cpp
   common/LZMA.cpp
+  common/DebugInfoD.cpp
   common/MainLoop.cpp
   common/MonitoringProcessLauncher.cpp
   common/NativeProcessProtocol.cpp
   common/NativeRegisterContext.cpp
   common/NativeThreadProtocol.cpp
   common/NativeWatchpointList.cpp
   common/OptionParser.cpp
   common/PipeBase.cpp
if (HAVE_LIBDL)
   list(APPEND EXTRA_LIBS ${CMAKE_DL_LIBS})
 endif()
 if (LLDB_ENABLE_LIBEDIT)
   list(APPEND EXTRA_LIBS ${LibEdit_LIBRARIES})
 endif()
 if (LLDB_ENABLE_LZMA)
   list(APPEND EXTRA_LIBS ${LIBLZMA_LIBRARIES})
 endif()
+if (LLDB_ENABLE_DEBUGINFOD)
+  list(APPEND EXTRA_LIBS ${Debuginfod_LIBRARIES})
+endif()
 if (WIN32)
   list(APPEND LLDB_SYSTEM_LIBS psapi)
 endif ()
 
 if (LLDB_ENABLE_LIBEDIT)
   list(APPEND LLDB_LIBEDIT_LIBS ${LibEdit_LIBRARIES})
   if (LLVM_BUILD_STATIC)
     list(APPEND LLDB_SYSTEM_LIBS gpm)

lldb/source/Host/common/DebugInfoD.cpp

+//===-- DebugInfoD.cpp ----------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Host/DebugInfoD.h"
+#include "lldb/Core/Module.h"
+#include "lldb/Host/Config.h"
+#include "lldb/Symbol/ObjectFile.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Errno.h"
+
+#if LLDB_ENABLE_DEBUGINFOD
+#include "elfutils/debuginfod.h"
+#endif
+
+namespace lldb_private {
+
+namespace debuginfod {
+
+using namespace lldb;
+using namespace lldb_private;
+
+#if !LLDB_ENABLE_DEBUGINFOD
+bool isAvailable() { return false; }
+
+llvm::Error findSource(const UUID &buildID, const std::string &path,
+                       std::string &cache_path) {
+  llvm_unreachable("debuginfod::findSource is unavailable");
+}
+
+#else // LLDB_ENABLE_DEBUGINFOD
+
+bool isAvailable() { return true; }
+
+llvm::Expected<std::string> findSource(const UUID &buildID,
+                                       const std::string &path) {
+  if (!buildID.IsValid() ||
+      buildID.GetBytes().size() ==
+          sizeof(llvm::support::ulittle32_t)) // .gnu_debuglink crc32
+    return llvm::createStringError(llvm::inconvertibleErrorCode(),
+                                   "invalid build ID: %s",
+                                   buildID.GetAsString("").c_str());
+
+  debuginfod_client *client = debuginfod_begin();
+
+  if (!client)
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        "failed to create debuginfod connection handle: %s", strerror(errno));
+
+  // debuginfod_set_progressfn(client, [](debuginfod_client *client, long a,
+  //                                 long b) -> int {
+  //   fprintf(stderr, "KWK === a: %ld b : %ld \n", a, b);
+  //   return 0; // continue
+  // });
+
+  char *cache_path = nullptr;
+  int rc = debuginfod_find_source(client, buildID.GetBytes().data(),
+                                  buildID.GetBytes().size(), path.c_str(),
+                                  &cache_path);
+
+  debuginfod_end(client);
+
+  std::string result_path;
+  if (cache_path) {
+    result_path = std::string(cache_path);
+    free(cache_path);
+  }
+
+  if (rc < 0) {
+    if (rc == -ENOSYS) // No DEBUGINFO_URLS were specified
+      return result_path;
+    else if (rc == -ENOENT) // No such file or directory, aka build-id not
+                            // available on servers.
+      return result_path;
+    else
+      return llvm::createStringError(
+          llvm::inconvertibleErrorCode(),
+          "debuginfod_find_source query failed (CODE=%d): %s", -rc,
+          llvm::sys::StrError(-rc).c_str());
+  }
+
+  if (close(rc) < 0) {
+    return llvm::createStringError(
+        llvm::inconvertibleErrorCode(),
+        "failed to close result of call to debuginfo_find_source: %s",
+        llvm::sys::StrError(errno).c_str());
+  }
+
+  return result_path;
+}
+
+#endif // LLDB_ENABLE_DEBUGINFOD
+
+} // end of namespace debuginfod
+} // namespace lldb_private

lldb/test/CMakeLists.txt

if (CMAKE_SIZEOF_VOID_P EQUAL 8)
   set(LLDB_IS_64_BITS 1)
 endif()
 
 # These values are not canonicalized within LLVM.
 llvm_canonicalize_cmake_booleans(
   LLDB_ENABLE_PYTHON
   LLDB_ENABLE_LUA
   LLDB_ENABLE_LZMA
+  LLDB_ENABLE_DEBUGINFOD
   LLVM_ENABLE_ZLIB
   LLVM_ENABLE_SHARED_LIBS
   LLDB_IS_64_BITS)
 
 # Configure the individual test suites.
 add_subdirectory(API)
 add_subdirectory(Shell)
 add_subdirectory(Unit)

lldb/test/Shell/SymbolFile/DWARF/source-list.cpp

+// clang-format off
+// REQUIRES: debuginfod
+// UNSUPPORTED: darwin, windows
+
+//  Test that we can display the source of functions using debuginfod when the
+//  original source file is no longer present.
+//  
+//  The debuginfod client requires a buildid in the binary, so we compile one in.
+//  We can create the directory structure on disc that the client expects on a
+//  webserver that serves source files. Then we fire up a python based http
+//  server in the root of that mock directory, set the DEBUGINFOD_URLS
+//  environment variable and let LLDB do the rest. 
+//  
+//  Go here to find more about debuginfod:
+//  https://sourceware.org/elfutils/Debuginfod.html
+
+
+//    We copy this file because we want to compile and later move it away
+
+// RUN: mkdir -p %t.buildroot
+// RUN: cp %s %t.buildroot/test.cpp
+
+//    We use the prefix map in order to overwrite all DW_AT_decl_file paths
+//    in the DWARF. We cd into the directory before compiling to get
+//    DW_AT_comp_dir pickup %t.buildroot as well so it will be replaced by
+//    /my/new/path.
+
+// RUN: cd %t.buildroot
+// RUN: %clang_host \
+// RUN:   -g \
+// RUN:   -Wl,--build-id="0xaaaaaaaaaabbbbbbbbbbccccccccccdddddddddd" \
+// RUN:   -fdebug-prefix-map=%t.buildroot=/my/new/path \
+// RUN:   -o %t \
+// RUN:   %t.buildroot/test.cpp
+
+
+//    We move the original source file to a directory that looks like a debuginfod
+//    URL part.
+
+// RUN: rm -rf %t.mock
+// RUN: mkdir -p       %t.mock/buildid/aaaaaaaaaabbbbbbbbbbccccccccccdddddddddd/source/my/new/path
+// RUN: mv    %t.buildroot/test.cpp %t.mock/buildid/aaaaaaaaaabbbbbbbbbbccccccccccdddddddddd/source/my/new/path
+
+
+//    Adjust where debuginfod stores cache files:
+
+// RUN: rm -rf %t.debuginfod_cache_path
+// RUN: mkdir -p %t.debuginfod_cache_path
+// RUN: export DEBUGINFOD_CACHE_PATH=%t.debuginfod_cache_path
+
+
+//    Start HTTP file server on port picked by OS and wait until it is ready
+//    The server will be closed on exit of the test.
+
+// RUN: rm -f "%t.server.log"
+// RUN: timeout 5 python3 -u -m http.server 0 --directory %t.mock --bind "localhost" &> %t.server.log & export PID=$!
+// RUN: trap 'kill $PID;' EXIT INT
+
+
+//    Extract HTTP address from the first line of the server log
+//    (e.g. "Serving HTTP on 127.0.0.1 port 40587 (http://127.0.0.1:40587/) ..")
+
+// RUN: echo -n "Waiting for server to be ready"
+// RUN: SERVER_ADDRESS=""
+// RUN: while [ -z "$SERVER_ADDRESS" ]; do \
+// RUN:     echo -n "."; \
+// RUN:     sleep 0.01; \
+// RUN:     SERVER_ADDRESS=$(head -n1 %t.server.log | grep "http://.\+/\+" -o); \
+// RUN: done
+// RUN: echo "DONE"
+
+
+//-- TEST 1 --  No debuginfod awareness ----------------------------------------
+
+
+// RUN: DEBUGINFOD_URLS="" \
+// RUN: %lldb -f %t -o 'source list -n main' | FileCheck --dump-input=fail %s --check-prefix=TEST-1
+
+// TEST-1: (lldb) source list -n main
+// TEST-1: File: /my/new/path/test.cpp
+// TEST-1-EMPTY:
+
+
+//-- TEST 2 -- debuginfod URL pointing in wrong place --------------------------
+
+
+// RUN: DEBUGINFOD_URLS="http://example.com/debuginfod" \
+// RUN: %lldb -f %t -o 'source list -n main' | FileCheck --dump-input=fail %s --check-prefix=TEST-2
+
+// TEST-2: (lldb) source list -n main
+// TEST-2: File: /my/new/path/test.cpp
+// TEST-2-EMPTY:
+
+
+//-- TEST 3 -- debuginfod URL pointing corectly --------------------------------
+
+
+// RUN: DEBUGINFOD_URLS="$SERVER_ADDRESS" \
+// RUN: %lldb -f %t -o 'source list -n main' | FileCheck --dump-input=fail %s --check-prefix=TEST-3
+
+// TEST-3: (lldb) source list -n main
+// TEST-3: File: /my/new/path/test.cpp
+// TEST-3:         123
+// TEST-3-NEXT:    {{[0-9]+}}   // Some context lines before
+// TEST-3-NEXT:    {{[0-9]+}}   // the function.
+// TEST-3-NEXT:    {{[0-9]+}}
+// TEST-3-NEXT:    {{[0-9]+}}
+// TEST-3-NEXT:    {{[0-9]+}}   int main(int argc, char **argv) {
+// TEST-3-NEXT:    {{[0-9]+}}     // Here are some comments.
+// TEST-3-NEXT:    {{[0-9]+}}     // That we should print when listing source.
+// TEST-3-NEXT:    {{[0-9]+}}     return 0;
+// TEST-3-NEXT:    {{[0-9]+}}   }
+// TEST-3-NEXT:    {{[0-9]+}}
+// TEST-3-NEXT:    {{[0-9]+}}   // Some context lines after
+// TEST-3-NEXT:    {{[0-9]+}}   // the function.
+// TEST-3-EMPTY:
+
+//    Validate that the server received the request from debuginfod client.
+
+// RUN: cat %t.server.log | FileCheck --dump-input=fail %s --check-prefix=VALIDATION
+// VALIDATION: 127.0.0.1 - - [{{.*}}] "GET /buildid/aaaaaaaaaabbbbbbbbbbccccccccccdddddddddd/source/my/new/path/test.cpp HTTP/1.1" 200 -
+
+
+// Some context lines before
+// the function.
+
+
+int main(int argc, char **argv) {
+  // Here are some comments.
+  // That we should print when listing source.
+  return 0;
+}
+
+// Some context lines after
+// the function.
 No newline at end of file

lldb/test/Shell/lit.cfg.py

if config.lldb_enable_python:
     config.available_features.add('python')
 
 if config.lldb_enable_lua:
     config.available_features.add('lua')
 
 if config.lldb_enable_lzma:
     config.available_features.add('lzma')
 
+if config.lldb_enable_debuginfod:
+    config.available_features.add('debuginfod')
+
 if find_executable('xz') != None:
     config.available_features.add('xz')
 
 # NetBSD permits setting dbregs either if one is root
 # or if user_set_dbregs is enabled
 can_set_dbregs = True
 if platform.system() == 'NetBSD' and os.geteuid() != 0:
     try:
         output = subprocess.check_output(["/sbin/sysctl", "-n",
           "security.models.extensions.user_set_dbregs"]).decode().strip()
         if output != "1":
             can_set_dbregs = False
     except subprocess.CalledProcessError:
         can_set_dbregs = False
 if can_set_dbregs:
     config.available_features.add('dbregs-set')

lldb/test/Shell/lit.site.cfg.py.in

 config.lldb_tools_dir = "@LLDB_TOOLS_DIR@"
 # Since it comes from the command line, it may have backslashes which
 # should not need to be escaped.
 config.lldb_lit_tools_dir = r"@LLDB_LIT_TOOLS_DIR@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.python_executable = "@PYTHON_EXECUTABLE@"
 config.have_zlib = @LLVM_ENABLE_ZLIB@
 config.lldb_enable_lzma = @LLDB_ENABLE_LZMA@
+config.lldb_enable_debuginfod = @LLDB_ENABLE_DEBUGINFOD@
 config.host_triple = "@LLVM_HOST_TRIPLE@"
 config.lldb_bitness = 64 if @LLDB_IS_64_BITS@ else 32
 config.lldb_enable_python = @LLDB_ENABLE_PYTHON@
 config.lldb_enable_lua = @LLDB_ENABLE_LUA@
 config.lldb_build_directory = "@LLDB_TEST_BUILD_DIRECTORY@"
 # The shell tests use their own module caches.
 config.lldb_module_cache = os.path.join("@LLDB_TEST_MODULE_CACHE_LLDB@", "lldb-shell")
 config.clang_module_cache = os.path.join("@LLDB_TEST_MODULE_CACHE_CLANG@", "lldb-shell")