Introduce new symbol on-demand for debug info

[yinghuitan]

Summary

This diff introduces a new symbol on-demand which skips
loading a module's debug info unless explicitly asked on
demand. This provides significant performance improvement
for application with dynamic linking mode which has large
number of modules.
The feature can be turned on with:
"settings set symbols.load-on-demand true"

The feature works by creating a new SymbolFileOnDemand class for
each module which wraps the actual SymbolFIle subclass as member
variable. By default, most virtual methods on SymbolFileOnDemand are
skipped so that it looks like there is no debug info for that module.
But once the module's debug info is explicitly requested to
be enabled (in the conditions mentioned below) SymbolFileOnDemand
will allow all methods to pass through and forward to the actual SymbolFile
which would hydrate module's debug info on-demand.

In an internal benchmark, we are seeing more than 95% improvement
for a 3000 modules application.

Currently we are providing several ways to on demand hydrate
a module's debug info:

• Source line breakpoint: matching in supported files
• Stack trace: resolving symbol context for an address
• Symbolic breakpoint: symbol table match guided promotion
• Global variable: symbol table match guided promotion

In all above situations the module's debug info will be on-demand
parsed and indexed.

Some follow-ups for this feature:

• Add a command that allows users to load debug info explicitly while using a new or existing command when this feature is enabled
• Add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs
• Add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs
• Add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed)

[yinghuitan]

Fix tests line number change after formatting.

[clayborg]

I have been working with Jeffrey on this feature and wanted to explain our thoughts a bit.

We wanted to solve the issue of having too much debug information in your project and having debugging being slowed down by all of this information. Much of this information is not required at all times and a few global debug info queries can end up pulling in all of the debug information and causing performance issues.

Our solution was to create a SymbolFileOnDemand class that will own the actual SymbolFile subclass as a member variable. This SymbolFileOnDemand class can then pick and choose when the "turn on" the debug information for each module depending on what kind of queries are sent to these objects.

As Jeffrey said the main triggers for enabling debug information are:

• set a file and line breakpoint in the module. We always allow these queries to access the line tables in the debug information, and if a source file and line breakpoint are found, then debug info is turned on and all queries to SymbolFileOnDemand will be passed directly to the subclass (SymbolFileDWARF for example).
• if a stack trace has stack frames with a PC value that is in a library, then we turn on debug info. We do this by seeing when we get a "resolve this lldb_private::Address into a symbol context with debug info". We assume anyone that has resolved a section offset address (lldb_private::Address) and then asks for a symbol context to be resolved is knowing they want debug info from this module
• a call to FindFunctions will enable debug info if we find a match in the symbol table, so this on demand symbol feature will work better if users don't strip their binaries
• a call to find global variables, which will enable debug info if we find a symbol that matches. Again, this demand symbol feature will work better if users don't strip their binaries

Our goal here is to allow users to have all the debug information they want, but try to only use the debug information that we need and stem the costs of global lookups in debug information when we can. We want this feature to not require any additional settings to work well other than enabling it, but we also realize we can add settings in the future to say "always load debug info for these modules" or "never load debug info for these modules.

Since most users in IDE settings will set file and line breakpoints during debugging, we are hoping this feature "just works" for most users. Since any stack frames in any threads that are being displayed in the IDE will cause the debug info to be loaded for those modules, the most relevant code to the user will always have debug info enabled.

include/lldb/Symbol/SymbolFileOnDemand.h

1–2 ↗

(On Diff #415183)

Make this on the same line and avoid the wrapping

[jingham]

This sort of change needs to do a lot more work to help the user understand what they can and can't see if they accept the speed for visibility tradeoff.

They (and we) also need to know what operations might trigger a full "hydration" of a symbol table. It isn't useful to have all this infrastructure in place if fairly common operations trigger all the work anyway.

It also seems to me that whenever the user tells about a shared library, we should "hydrate" it. So for instance, if I do:

(lldb) break set -s someLib.dylib -f foo.c -l 12

they you should first hydrate someLib.dylib and then set the breakpoint.

In general, there should be a strategy discussion up-front before just making this change, with use cases of what won't work (not just some disabled tests) and how ordinary users might figure out something didn't work because of the on-demand loading. There needs to also be a way for lldb to inform users what to expect or the experience will just end up being frustrating.

It also seems like there should be a way to say "These libraries I'm unlikely to care about, but these I always want to have fully expanded".

Normally, it would be fine to say "we can add those details as we go along" but you're designing a new mode of interaction with the debugger, and I think we need to sketch out what we think are acceptable tradeoffs and where we need to give people the ability to manually intervene, etc. So it seems like we should first be having a strategy discussion, before getting down to the details of this patch.

include/lldb/Symbol/SymbolFileOnDemand.h

1 ↗

(On Diff #415183)

This file seems to be out of place. I don't think it's needed.

[yinghuitan]

Remove duplicate file and fix wrapping.

[labath]

I agree with everything Jim said.

One of the interesting 'strategy' questions would be the interaction between this feature and the target.preload-symbols feature. I don't think the interaction between the two is obvious or intuitive, and I'm even sure that supporting all possible combinations is useful. We could e.g. consider replacing the two settings with one ternary setting with values like "load everything upfront", "load everything the first time it's needed" and "load everything when the module is accessed in a specific way".

Also, I don't think that "run everything with the setting turned on" is a good testing strategy. Nearly all of our tests consist of a single binary and the very first action they do is set a file/line or a function breakpoint. That will immediately trigger a 'hydration' and the rest of the test will just check that the forwarders really forward all calls. Rerunning all tests to catch the few testcases that do something interesting is not a good tradeoff.

We should identify the interesting/important scenarios and write targeted tests for that. The tests in this patch are a pretty good start. My only complaint is that the shared library tests are linux-only. We currently don't have infrastructure in place to write shell tests with shared libraries portably. It may be worth rewriting those into api tests to make use of the existing shared library facilities.

[yinghuitan]

Jim, Pavel, thanks for the comment!

I am happy to start a strategy discussion for this feature. @clayborg will chrime in later to suggest the approach how we can discuss.

I would like to answer some of the questions here to share some context.

speed for visibility tradeoff

Right, that is the basic principle of this feature. We are designing this feature primarily targeting IDE use cases which, we found, majority of the users are focusing on several modules authored/depended by their own code.
Also they primarily use basic subset features of lldb.

They (and we) also need to know what operations might trigger a full "hydration" of a symbol table

We would like to make the feature work *by default* as much as possible so we decided to use breakpoint match, callstack and symbol table match to guide a module's debug info hydration. To direct answer this question: I haven't find a common case that can hydrate all process modules debug info yet unless there is callstack covering all modules or a function name matching in all module's symbol table while setting function breakpoint.

whenever the user tells about a shared library, we should "hydrate" it

I agree. Since this feature is designed for IDE scenario so we haven't focused on covering all lldb commands when a module is specified. In "break set -s someLib.dylib -f foo.c -l 12" example, if there is a matching foo.c in someLib.dylib, it will be hydrated; otherwise not.

It also seems like there should be a way to say "These libraries I'm unlikely to care about, but these I always want to have fully expanded".

I agree. It is in my plan to add a "symbols.on-demand-modules" setting which users can explicitly white list some modules for hydration.

unify with target.preload-symbols feature

I do not have strong opinion on this. One concern is that target.preload-symbols provides full experience but symbols.load-on-demand provides trade-off experience (for example some global expression evaluation may fail because we won't automatically hydrate based on type query).

I don't think that "run everything with the setting turned on" is a good testing strategy

I do not have strong opinion either. I understand the concern is extra test time. I do want to share that there are many tests that do not set breakpoints which exercised symbol ondemand code path and helped to catch real bugs in the implementation.

[clayborg]

In D121631#3380824, @jingham wrote:

This sort of change needs to do a lot more work to help the user understand what they can and can't see if they accept the speed for visibility tradeoff.

They (and we) also need to know what operations might trigger a full "hydration" of a symbol table. It isn't useful to have all this infrastructure in place if fairly common operations trigger all the work anyway.

Symbol tables are always on and are used to help us find things that will possibly be in the debug info from a function name (look for Code symbols) and global variable (Data symbols) perspective. So a few things here:

• symbol tables are always parsed for everything and help us enable debug info if we find a match for FindFunctions and FindGlobals calls on the symbol file even if these SymbolFile calls specify to not check the symbol table. If we find a symbol that matches, we turn on debug info and repeat the call by calling into the symbol file if debug info wasn't previously enabled.
• any Address -> symbol context where the user asks for debug info enables debug info: which usually means any PC from any stack frame, but can also mean if a user queries an address
• line tables are always left enabled and if any match occurs we enable debug info

It also seems to me that whenever the user tells about a shared library, we should "hydrate" it. So for instance, if I do:

(lldb) break set -s someLib.dylib -f foo.c -l 12

they you should first hydrate someLib.dylib and then set the breakpoint.

that already works. Line tables are relatively cheap and they are always enabled through SymbolFileOnDemand and if. "foo.c:12" kicks up any matches, regardless of wether the shared library was specified, will enable debug info for any libraries that match.

In general, there should be a strategy discussion up-front before just making this change, with use cases of what won't work (not just some disabled tests) and how ordinary users might figure out something didn't work because of the on-demand loading. There needs to also be a way for lldb to inform users what to expect or the experience will just end up being frustrating.

Jim: I met with you to discuss all of these strategies a month ago and I left with the impression the ideas I told you about could work pretty well and that is what Jeffrey implemented here.

It also seems like there should be a way to say "These libraries I'm unlikely to care about, but these I always want to have fully expanded".

We were hoping to see if we can automagically make this stuff work well enough that it requires no settings, but we are happy to add these settings in subsequent patches. The patch is already large as it is. Let us know if these should be added with this initial patch. We might also think about asking the Platform layer if they have any libraries that they know should be always on.

Normally, it would be fine to say "we can add those details as we go along" but you're designing a new mode of interaction with the debugger, and I think we need to sketch out what we think are acceptable tradeoffs and where we need to give people the ability to manually intervene, etc. So it seems like we should first be having a strategy discussion, before getting down to the details of this patch.

We would love to setup a Video Chat with anyone interested if that would work. Talking back and forth on email or lists takes a while and I think we can easily hash things out if any interested parties are willing to attend. We are on Pacific time zone, anyone else that wants to participate, please chime in and we will see if we can set something up?!

[clayborg]

I am personally not a fan of the preloading of symbols settings being on by default as there are large delays in debug session startup on linux due to the lack of any kind of viable acceleration tables. I understand that delays can occur later in the debug session as a result, but those are usually tied to global lookups from expressions and many people using IDEs will set file and line breakpoints and just use the variable display which doesn't need everything preloaded many times. We are also looking into trying to track who is doing these global lookups and trying to ensure that the global lookup is needed. One call to fine all "iterator" types is an example of a really bad global lookup that can cause tons of types to be pulled in.

I do think that possibly enabling this feature by changing the preload seeting to allow "preload", "on-demand" or "lazily" could easily work here.

I also agree that we should provide documentation for this feature, or possibly print out a blurb of text to the console that explains the rules if this feature gets enabled.

[labath]

In D121631#3384124, @yinghuitan wrote:

unify with target.preload-symbols feature

I do not have strong opinion on this. One concern is that target.preload-symbols provides full experience but symbols.load-on-demand provides trade-off experience (for example some global expression evaluation may fail because we won't automatically hydrate based on type query).

That is a fair point, but the having them separate also has downsides, as we have to figure out what is the meaning of on-demand=true&preload=true combo going to be. I think it would be strange to have on-demand take precedence in this setup, but then if it doesn't, then you need to change *two* settings to switch from "preload everything" (the current default) to the "on demand mode". I am not particularly invested in any solution, but I think we should make a conscious choice one way or the other.

I don't think that "run everything with the setting turned on" is a good testing strategy

I do not have strong opinion either. I understand the concern is extra test time. I do want to share that there are many tests that do not set breakpoints which exercised symbol ondemand code path and helped to catch real bugs in the implementation.

How many tests/bugs are we talking about here? Were there more than say ten distinct bugs caught there?

I see a large difference between running tests to find /new/ bugs, and running tests to find /regressions/. Our (API) test suite allows you to run it in a lot of different ways, which can be used to find new bugs. I have used that myself on several occasions. Even without making any changes to the test suite, you could run it with your feature enabled through the --setting argument to dotest. But simply because you manage to find a bug using some combination of dotest arguments it does not mean that everyone should be running the test suite with those arguments to ensure it doesn't regress. It just doesn't scale. When you find (and fix) a bug, you can make a dedicated regression test for that bug. Maybe even more than one -- to test similar edge cases that happened to work already, but could be broken in the future.

Please don't take this personally. This isn't about you or your feature -- that's my standard response to anyone tempted (in a way, it was a mistake to make it too easy to add new flavours) to add new test flavours. All of what I said applies (maybe even more so than here) to some of the existing test categories (gmodules for one), but since they're already here it becomes much harder to remove them -- which is why I'm trying to avoid adding any new ones.

[JDevlieghere]

In D121631#3386244, @labath wrote:

In D121631#3384124, @yinghuitan wrote:

unify with target.preload-symbols feature

I do not have strong opinion on this. One concern is that target.preload-symbols provides full experience but symbols.load-on-demand provides trade-off experience (for example some global expression evaluation may fail because we won't automatically hydrate based on type query).

That is a fair point, but the having them separate also has downsides, as we have to figure out what is the meaning of on-demand=true&preload=true combo going to be. I think it would be strange to have on-demand take precedence in this setup, but then if it doesn't, then you need to change *two* settings to switch from "preload everything" (the current default) to the "on demand mode". I am not particularly invested in any solution, but I think we should make a conscious choice one way or the other.

I don't think that "run everything with the setting turned on" is a good testing strategy

I do not have strong opinion either. I understand the concern is extra test time. I do want to share that there are many tests that do not set breakpoints which exercised symbol ondemand code path and helped to catch real bugs in the implementation.

How many tests/bugs are we talking about here? Were there more than say ten distinct bugs caught there?

I see a large difference between running tests to find /new/ bugs, and running tests to find /regressions/. Our (API) test suite allows you to run it in a lot of different ways, which can be used to find new bugs. I have used that myself on several occasions. Even without making any changes to the test suite, you could run it with your feature enabled through the --setting argument to dotest. But simply because you manage to find a bug using some combination of dotest arguments it does not mean that everyone should be running the test suite with those arguments to ensure it doesn't regress. It just doesn't scale. When you find (and fix) a bug, you can make a dedicated regression test for that bug. Maybe even more than one -- to test similar edge cases that happened to work already, but could be broken in the future.

Please don't take this personally. This isn't about you or your feature -- that's my standard response to anyone tempted (in a way, it was a mistake to make it too easy to add new flavours) to add new test flavours. All of what I said applies (maybe even more so than here) to some of the existing test categories (gmodules for one), but since they're already here it becomes much harder to remove them -- which is why I'm trying to avoid adding any new ones.

We have some similar wording to that extent on the "testing" page: https://lldb.llvm.org/resources/test.html#id7. I like the distinction you make between using the ability go use the flavors for new features vs using it to catch regressions. Would you consider adding that there?

[clayborg]

In D121631#3386244, @labath wrote:

In D121631#3384124, @yinghuitan wrote:

unify with target.preload-symbols feature

I do not have strong opinion on this. One concern is that target.preload-symbols provides full experience but symbols.load-on-demand provides trade-off experience (for example some global expression evaluation may fail because we won't automatically hydrate based on type query).

That is a fair point, but the having them separate also has downsides, as we have to figure out what is the meaning of on-demand=true&preload=true combo going to be. I think it would be strange to have on-demand take precedence in this setup, but then if it doesn't, then you need to change *two* settings to switch from "preload everything" (the current default) to the "on demand mode". I am not particularly invested in any solution, but I think we should make a conscious choice one way or the other.

These two settings can co-exist pretty easily. If preload is on, preloading can be done for anything that has debug info enabled, and would be done as soon as debug info was enabled for on demand. It would be like saying "please preload symbols as soon as debug info is enabled". Right now if we enable debug info due to a breakpoint, function, or global lookup that matches a symbol, we can immediately preload the symbols so they are ready regardless of wether a name lookup happens just like we do normally.

If preload is off, then we wait until someone makes a global name lookup call.

I don't think that "run everything with the setting turned on" is a good testing strategy

I do not have strong opinion either. I understand the concern is extra test time. I do want to share that there are many tests that do not set breakpoints which exercised symbol ondemand code path and helped to catch real bugs in the implementation.

How many tests/bugs are we talking about here? Were there more than say ten distinct bugs caught there?

I see a large difference between running tests to find /new/ bugs, and running tests to find /regressions/. Our (API) test suite allows you to run it in a lot of different ways, which can be used to find new bugs. I have used that myself on several occasions. Even without making any changes to the test suite, you could run it with your feature enabled through the --setting argument to dotest. But simply because you manage to find a bug using some combination of dotest arguments it does not mean that everyone should be running the test suite with those arguments to ensure it doesn't regress. It just doesn't scale. When you find (and fix) a bug, you can make a dedicated regression test for that bug. Maybe even more than one -- to test similar edge cases that happened to work already, but could be broken in the future.

Please don't take this personally. This isn't about you or your feature -- that's my standard response to anyone tempted (in a way, it was a mistake to make it too easy to add new flavours) to add new test flavours. All of what I said applies (maybe even more so than here) to some of the existing test categories (gmodules for one), but since they're already here it becomes much harder to remove them -- which is why I'm trying to avoid adding any new ones.

We knew adding a new flavor would be a contentious part of the patch and we are fine testing this however the community feels it is best tested. That being said, it would be good to catch regressions for this feature on at least one Buildbot so we don't regress the feature, I am not particular on how it happens.

Is there any interest in a VC call that anyone can choose to attend about this, or do we prefer an RFC on the website that hosts what the mailing list used to be and I can't remember right now?

[clayborg]

lldb/include/lldb/Symbol/SymbolFileOnDemand.h

202–203

lldb/source/Core/CoreProperties.td

39

We might want to put a URL to something on our website that describes this feature in detail. It isn't just breakpoints that enable debug info, and we will need to document this feature completely so users know what to expect if they enable this.

lldb/source/Core/Module.cpp

472

This should only be done if someone is asking for debug info in the "resolve_scope".

lldb/source/Symbol/SymbolFile.cpp

83

Add a comment here explaining what is happening

lldb/source/Symbol/SymbolFileOnDemand.cpp

28

We might consider letting this call always go through to the underlying symbol file and report the abilities from it.

34–36

remove braces for a single statement "if" per llvm coding guidelines

62–65

remove braces for a single statement "if" per llvm coding guidelines

79–81

remove braces for a single statement "if" per llvm coding guidelines

97

We might be able to always let this call through since the line tables will be parsed if the user sets a file and line breakpoint. We are mainly trying to stop the global name lookups in SymbolFileOnDemand

172–175

remove braces for a single statement "if" per llvm coding guidelines

206–208

remove braces for a single statement "if" per llvm coding guidelines

514–516

We might consider letting this always go through as unwind info doesn't involve any indexing of debug info.

602–604

remove braces for a single statement "if" per llvm coding guidelines

608–610

remove braces for a single statement "if" per llvm coding guidelines

lldb/source/Target/Statistics.cpp

66

Other key/value pairs for symbol table stuff start with "symbolTable",

235–237

remove braces for a single statement "if" per llvm coding guidelines

240–242

remove braces for a single statement "if" per llvm coding guidelines

[clayborg]

lldb/source/Target/Statistics.cpp

271

Other "total" key value pairs use the same string as the ones from the module level to help match them up.

272

Other "total" key value pairs use the same string as the ones from the module level to help match them up.

[yinghuitan]

Re @labath

How many tests/bugs are we talking about here? Were there more than say ten distinct bugs caught there?

It found 2~3 bugs, like source regex breakpoint (breakpoint set -p "source"), regex breakpoint filter by language (breakpoint set -r <regex> -L c++) etc.

running tests to new features vs find /regressions

Running all tests offline once to discover bugs is a good exericse. Since failing tests are understood we can put them into new tests with on-demand explicitly enabled then ondemand test category is unnecessary. This sounds good to me if Greg agrees.

Please don't take this personally

None taken :-)

[yinghuitan]

lldb/source/Symbol/SymbolFileOnDemand.cpp

28

Sure. It does not matter much in real world because ability is never calculated on SymbolFileOnDemand.

97

If the user sets a file and line breakpoint and a compile unit/supported file is matched against the breakpoint file its parent module's SymbolFileOnDemand will be hydrated before ParseLineTable() is called anyway so manual call through is unnecessary.
I prefer to keep things not pass through by default to prevent any accidental leaking.

514–516

Yes, we could. But I would like to keep all functions guarded by default unless explicitly required by a hydration scenario to prevent any accidental leakage.

lldb/source/Target/Statistics.cpp

66

Ah, thanks! Bad auto replace for variable name.

[yinghuitan]

Address review feedback.

[yinghuitan]

I will remove ondemand test flavor/category and add more new testcases next.

[yinghuitan]

Remove ondemand test category/flavor
Add new testcases:

• source text regex breakpoint
• regex breakpoint filter by language
• global variable symbol table match hydration

[yinghuitan]

Remove last ondemand test flavor testcase.

[yinghuitan]

Add documentation for the feature.

[yinghuitan]

Fix log channel typo

[labath]

In D121631#3387077, @clayborg wrote:

In D121631#3386244, @labath wrote:

In D121631#3384124, @yinghuitan wrote:

unify with target.preload-symbols feature

I do not have strong opinion on this. One concern is that target.preload-symbols provides full experience but symbols.load-on-demand provides trade-off experience (for example some global expression evaluation may fail because we won't automatically hydrate based on type query).

That is a fair point, but the having them separate also has downsides, as we have to figure out what is the meaning of on-demand=true&preload=true combo going to be. I think it would be strange to have on-demand take precedence in this setup, but then if it doesn't, then you need to change *two* settings to switch from "preload everything" (the current default) to the "on demand mode". I am not particularly invested in any solution, but I think we should make a conscious choice one way or the other.

These two settings can co-exist pretty easily. If preload is on, preloading can be done for anything that has debug info enabled, and would be done as soon as debug info was enabled for on demand. It would be like saying "please preload symbols as soon as debug info is enabled". Right now if we enable debug info due to a breakpoint, function, or global lookup that matches a symbol, we can immediately preload the symbols so they are ready regardless of wether a name lookup happens just like we do normally.

If preload is off, then we wait until someone makes a global name lookup call.

I understand your point, and I see how having a ternary setting where (exactly) one of the values creates a completely different debug experience could be confusing. When you describe it like this, I think the behavior makes sense. The part I am worried about is someone getting the relationship backwards (with preload-symbols being "on top"), and then wondering "why the %#@# is aren't my symbols loaded when I said you should preload them". And I do think one could think this is the expected behavior, since the (whether you like it or not) idea behind the preloading was to load everything up-front instead of at a random point in the debug session -- something which is no longer true when the on-demand feature is enabled.

Overall, I am still fairly fond of the ternary idea, but I am not going to push for it if there's interest in it. Having two settings makes sense too, if we can clearly communicate their relationship. One thing that might help is renaming (one or both) something so that both settings aren't using the word "load" (I'll note that you used the phrase "debug info is enabled" in your exposition), though I am not sure what would those names be.

In D121631#3397355, @yinghuitan wrote:

Remove ondemand test category/flavor
Add new testcases:

• source text regex breakpoint
• regex breakpoint filter by language
• global variable symbol table match hydration

Awesome. Thanks.

lldb/test/Shell/SymbolFile/OnDemand/shared-lib-globals-hydration.test

4 ↗

(On Diff #418708)

Is this here because there is no portable way to create shared libraries in shell tests? Would it be possible to run this test on other systems as well if this were written as an API test (where we have a shared lib portability layer)?

[yinghuitan]

lldb/test/Shell/SymbolFile/OnDemand/shared-lib-globals-hydration.test

4 ↗

(On Diff #418708)

Yeah, that's one reason. Another reason is the name of the shared library (used for checking global variables/debug info) varies across platform. For example, it can be libfoo.dylib (Mac) vs libfoo.so (linux) vs *foo*.dll (windows). I find it cumbersome to support multiple platforms.

[labath]

lldb/test/Shell/SymbolFile/OnDemand/shared-lib-globals-hydration.test

4 ↗

(On Diff #418708)

I'm not saying you have to go out of your way to support other platforms and their idiosyncrasies, but the differences in library names hardly seem like an insurmountable obstacle. We already have abstractions in the API tests (self.platformContext.{shlib_prefix,shlib_extension}) which exist precisely for this reason. There's no function to automatically construct the library name (you have to do the concatenation yourself), but I certainly wouldn't be opposed to adding one.

If it helps, think of it as courtesy towards other developers who may need to debug this feature when they break these tests -- that's much easier to achieve if you can reproduce the failure locally vs. just getting a failure email from a random bot (after you've already submitted your patch).

[yinghuitan]

lldb/test/Shell/SymbolFile/OnDemand/shared-lib-globals-hydration.test

4 ↗

(On Diff #418708)

self.platformContext.{shlib_prefix,shlib_extension}

Ah, did not know their existence, thanks! Will add API tests for them. Do you prefer to keep these linux-only shell tests or completely deprecate them?

[labath]

lldb/test/Shell/SymbolFile/OnDemand/shared-lib-globals-hydration.test

4 ↗

(On Diff #418708)

If they're going to be testing the exact same thing, then I don't see a reason for keeping these around.

[yinghuitan]

Support debug map dwarf mode.
Use zero symbol ability to skip SymbolFileOnDemand wrapping.
Add new API tests for symbol on-demand which support multi-platform.
Remove linux-only shell tests.

[clayborg]

So we added documentation on this feature explaining who will want to enable this feature and also describe exactly when and how debug info will get enabled. Does anyone else have objections or comments to make? If we need to setup a meeting to discuss, please chime in, or leave some review comments.

[labath]

I'm looking at the SymbolFileOnDemand friendship and the forwarding of protected methods (mostly dealing with compile unit lists). Does this by any chance have something to do with the fact that there are now two compile unit lists (one in the actual symbol file, and one in the wrapping ondemand class? Would it be possible to avoid that by making SymbolFile a stateless interface (just methods, no member variables), and having a putting the member variables and other utility functions into a separate class that the real symbol files would inherit from? Then only SymbolFileOnDemand would inherit from SymbolFile directly, and it would only add the state that it needs (the bool flag, and the actual symbol file pointer).

[yinghuitan]

Does this by any chance have something to do with the fact that there are now two compile unit lists (one in the actual symbol file, and one in the wrapping ondemand class?

Yes, that's the major reason. We also need make SymbolFileOnDemand friend in SymbolFile so that SymbolFileOnDemand can call/forward protected virtual methods of SymbolFile during overriding.

Would it be possible to avoid that by making SymbolFile a stateless interface?

What part do you want to avoid? We could do that by creating a new SymbolFileReal class, but we still have to make these compile unit lists methods virtual in SymbolFile class so that, like, calling SymbolFile::GetCompileUnitAtIndex() can be overridden by SymbolFileReal(touching data fields) and SymbolFileOnDemand (forwarding to real impl), right?

[labath]

I am trying to get rid of the friendship declaration and the forwarding of protected methods. I have no issue with making GetCompileUnitAtIndex virtual -- that is pretty much a necessity for this feature (*). However, SetCompileUnitAtIndex seems like an internal implementation detail of a SymbolFile instance (that's why it's protected), and it would be better if the new class didn't have to touch it. I wouldn't call the new class the "real" symbol file. I don't see it as any more real as any other subclass. The way I'd look at it is that the new class just contains some helpful implementation that can be useful to a specific symbol file class. An individual symbol file class may choose to use it, but it could also implement the pure SymbolFile interface is a completely different way, if it knows how.

(*) Well, I can also imagine an implementation where the on-demand logic lives inside the SymbolFile class (in public non-virtual methods), and the individual subclasses are implementing just the protected virtual (and maybe abstract) interface that is called from the public functions. That would be in line with the non-virtual interface design pattern, and would avoid having the GetBackingFile method, but I'm not sure if it would be cleaner overall.

[yinghuitan]

Thanks for clarifying. The friendship declaration in SymbolFile is required not only for SetCompileUnitAtIndex but also for other protected virtual methods in SymbolFile like CalculateNumCompileUnits and ParseCompileUnitAtIndex which are pure virtual and require overriding and forwarding by SymbolFileOnDemand.
One solution I can think of is creating a new SymbolFileActual class as child class of SymbolFile and move all the protected virtual member functions from SymbolFile into SymbolFileActual, then modify the callsites of these protected virtual member functions into SymbolFileActual as well. This will get rid of friend declaration and SymbolFile won't/can't have protected virtual member functions. Something like below:

// Only has public virtual functions
class SymbolFile {
};

// Override and forward the public virtual functions so no friend is needed
class SymbolFileOnDemand: public SymbolFile {}

class SymbolFileActual: public SymbolFile {
public:
  // Override any public virtual functions of SymbolFile 
  // that are callsites of protected virtual functions
protected:
  // All protected virtual function of SymbolFile are moved here
  // All member variables of SymbolFile are moved here
};

class SymbolFileDWARF: public SymbolFileActual {}
class SymbolFilePDB: public SymbolFileActual {}
class SymbolFileBreakPad: public SymbolFileActual {}
...

Is this what you are suggesting? This seems to be a big change which might introduce new issues so I took a more practical approach with minimum change instead here. Let me know if this SymbolFileActual design aligns with what you have in mind, and if so whether you want it to happen in this diff or a follow-up patch is fine. Thanks.

[labath]

Yes, I think that's exactly what I had in mind. Basically, the idea is to structure things such that the ondemand class can sit between the actual symbol file class and the outside world, but that it does not (and cannot) interfere with any of the interactions that happen inside a symbol file class. If it works, I think that kind of a setup would be much cleaner/understandable. As it stands now, one has to consider the possibility that any action inside the real symbol file can reenter the ondemand instance, which makes it harder to reason about.

I definitely wouldn't want to do it as a part of this patch, but it would be better to do it as a preparatory patch rather than a followup.

[yinghuitan]

Rebase on top of https://reviews.llvm.org/D124110
Rename test file name to avoid conflict
Added a new totalModuleCountHasDebugInfo statistics

[yinghuitan]

rebase

[yinghuitan]

rebase

[labath]

Thanks, for your patience. Overall, I'd say this looks pretty clean now -- much cleaner than I originally thought it could be. I don't have any further comments here.

Does anyone else have any thoughts?

lldb/source/Core/Module.cpp

475

Can this be moved into the if block below?

[DavidSpickett]

Bunch of random nits.

Looks like a great addition. I know debug load time is a common annoyance.

lldb/docs/use/ondemand.rst

1 ↗

(On Diff #424360)

If you can find a logical place for it, maybe define what "hydration" means in this context.

Developers can find the meaning from the source code but if "hydration" appears in logs it would be good for users to have an idea what that means.

Something like "the process of <whatever hydration means> (referred to internally as "hydration")".

92 ↗

(On Diff #424360)

"previously"

lldb/include/lldb/Symbol/SymbolFileOnDemand.h

32

Where is this call defined, and by API do you mean API like https://lldb.llvm.org/design/sbapi.html or API as in API of the SymbolFile class?

lldb/source/Symbol/SymbolFileOnDemand.cpp

48

Is there a specific reason to use "this" explicitly instead of just !m_debug_info_enabled?

Something to do with the wrapping of the underlying SymbolFile perhaps.

55

This should be langType shouldn't it?

111

What's the meaning of this comment?

134

Would return <some value> if hydrated?

lldb/test/Shell/SymbolFile/OnDemand/symbolic-breakpoint.test

2

Stray newline

[clayborg]

Thanks Pavel for taking the time to review this thoroughly. I am fine with this as I have been working with Jeffrey on this, so anyone else please chime in if you have any comments!

A few follow up patches I would love to see after this goes in:

• add a command that allows users to load debug info manually using a new or existing command when this feature is enabled
• add settings for "never load any of these executables in Symbols On Demand" that takes a list of globs
• add settings for "always load the the debug info for executables in Symbols On Demand" that takes a list of globs
• add a new column in "image list" that shows up by default when Symbols On Demand is enable to show the status for each shlib like "not enabled for this", "debug info off" and "debug info on" (with a single character to short string, not the ones I just typed)

[yinghuitan]

Thanks for all the feedback. I will address the comments and add @clayborg's follow-up suggestions into summary.

lldb/source/Symbol/SymbolFileOnDemand.cpp

48

I will remove from this diff. I guess I just got this habit from working in other languages which may explicitly require this otherwise can fail.

111

Return false from ForEachExternalModule tells the caller to not exit early. This is the default value used by SymbolFile::ForEachExternalModule(). Let me make it more obvious.

[yinghuitan]

Incorporate feedback and update summary with follow-up todos.

[clayborg]

LGTM. Anyone else have any issues?

[stella.stamenova]

Looks like this broke the windows lldb bot: https://lab.llvm.org/buildbot/#/builders/83/builds/18228

[yinghuitan]

@stella.stamenova, thanks for the heads-up. I am fixing this in https://reviews.llvm.org/D124471.

[yinghuitan]

Another patch to fix the missing import: https://reviews.llvm.org/D124479

[jasonmolenda]

Hi, I noticed that we have two tests with the same name,

lldb/test/API/lang/c/shared_lib/TestSharedLib.py
lldb/test/API/symbol_ondemand/shared_library/TestSharedLib.py

Can you rename your newly added test?

[yinghuitan]

@jasonmolenda, thanks, I discovered a bit late that testcases can't have duplicate names; I fixed one case in the patch but missed these two. Will add a follow-up patch. It would be great if we have a linter/script to detect/warn duplications, not sure how hard to add though.

[jasonmolenda]

Thanks! Yeah, I only noticed because lldb-dotest threw an error when I was trying to run a test case earlier.

[yinghuitan]

Thanks for the context. Renames conflict testcase in https://reviews.llvm.org/D124499.

lldb/include/lldb/Core/ModuleList.h

public:
   bool GetEnableLLDBIndexCache() const;
   bool SetEnableLLDBIndexCache(bool new_value);
   uint64_t GetLLDBIndexCacheMaxByteSize();
   uint64_t GetLLDBIndexCacheMaxPercent();
   uint64_t GetLLDBIndexCacheExpirationDays();
   FileSpec GetLLDBIndexCachePath() const;
   bool SetLLDBIndexCachePath(const FileSpec &path);
 
+  bool GetLoadSymbolOnDemand();
+
   PathMappingList GetSymlinkMappings() const;
 };
 
 /// \class ModuleList ModuleList.h "lldb/Core/ModuleList.h"
 /// A collection class for Module objects.
 ///
 /// Modules in the module collection class are stored as reference counted
 /// shared pointers to Module objects.

lldb/include/lldb/Symbol/SymbolFile.h

 
 namespace lldb_private {
 
 class SymbolFile : public PluginInterface {
   /// LLVM RTTI support.
   static char ID;
 
 public:
+  /// Required to call protected member functions from
+  /// SymbolFileOnDemand via forwarding.
+  friend class SymbolFileOnDemand;
+
+public:
   /// LLVM RTTI support.
   /// \{
   virtual bool isA(const void *ClassID) const { return ClassID == &ID; }
   static bool classof(const SymbolFile *obj) { return obj->isA(&ID); }
   /// \}
 
   // Symbol file ability bits.
   //
public:
   static SymbolFile *FindPlugin(lldb::ObjectFileSP objfile_sp);
 
   // Constructors and Destructors
   SymbolFile(lldb::ObjectFileSP objfile_sp)
       : m_objfile_sp(std::move(objfile_sp)) {}
 
   ~SymbolFile() override = default;
 
+  /// SymbolFileOnDemand class overrides this to return the underlying
+  /// backing SymbolFile implementation that loads on-demand.
+  virtual SymbolFile *GetBackingSymbolFile() { return this; }
+
   /// Get a mask of what this symbol file supports for the object file
   /// that it was constructed with.
   ///
   /// Each symbol file gets to respond with a mask of abilities that
   /// it supports for each object file. This happens when we are
   /// trying to figure out which symbol file plug-in will get used
   /// for a given object file. The plug-in that responds with the
   /// best mix of "SymbolFile::Abilities" bits set, will get chosen to
public:
   ///
   /// The SymbolFile object with the best set of abilities (detected
   /// in "uint32_t SymbolFile::GetAbilities()) will have this function
   /// called if it is chosen to parse an object file. More complete
   /// initialization can happen in this function which will get called
   /// prior to any other functions in the SymbolFile protocol.
   virtual void InitializeObject() {}
 
+  /// Whether debug info will be loaded or not.
+  ///
+  /// It will be true for most implementations except SymbolFileOnDemand.
+  virtual bool GetLoadDebugInfoEnabled() { return true; }
+
+  /// Specify debug info should be loaded.
+  ///
+  /// It will be no-op for most implementations except SymbolFileOnDemand.
+  virtual void SetLoadDebugInfoEnabled() { return; }
+
   // Compile Unit function calls
   // Approach 1 - iterator
   uint32_t GetNumCompileUnits();
   lldb::CompUnitSP GetCompileUnitAtIndex(uint32_t idx);
 
   Symtab *GetSymtab();
 
   virtual lldb::LanguageType ParseLanguage(CompileUnit &comp_unit) = 0;
bool GetDebugInfoIndexWasLoadedFromCache() const {
     return m_index_was_loaded_from_cache;
   }
   void SetDebugInfoIndexWasLoadedFromCache() {
     m_index_was_loaded_from_cache = true;
   }
   bool GetDebugInfoIndexWasSavedToCache() const {
     return m_index_was_saved_to_cache;
   }
-  void SetDebugInfoIndexWasSavedToCache() {
-    m_index_was_saved_to_cache = true;
-  }
+  void SetDebugInfoIndexWasSavedToCache() { m_index_was_saved_to_cache = true; }
   /// \}
 
 protected:
   void AssertModuleLock();
   virtual uint32_t CalculateNumCompileUnits() = 0;
   virtual lldb::CompUnitSP ParseCompileUnitAtIndex(uint32_t idx) = 0;
   virtual TypeList &GetTypeList() { return m_type_list; }
 

lldb/include/lldb/Symbol/SymbolFileOnDemand.h

+//===-- SymbolFileOnDemand.h ------------------------------------*- C++-*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_SYMBOL_SYMBOLFILEONDEMAND_H
+#define LLDB_SYMBOL_SYMBOLFILEONDEMAND_H
+
+#include <mutex>
+#include <vector>
+
+#include "lldb/Core/UniqueCStringMap.h"
+#include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Symbol/SymbolContext.h"
+#include "lldb/Symbol/SymbolFile.h"
+#include "lldb/Target/Statistics.h"
+#include "lldb/Utility/ConstString.h"
+#include "lldb/Utility/Flags.h"
+#include "lldb/Utility/LLDBLog.h"
+#include "lldb/Utility/Log.h"
+#include "lldb/lldb-private.h"
+
+namespace lldb_private {
+
+/// SymbolFileOnDemand wraps a real SymbolFile by providing
+/// on demand symbol parsing/indexing to improve performance.
+/// By default SymbolFileOnDemand will skip load the underlying
+/// symbols. Any client can on demand hydrate the underlying
+/// SymbolFile via SetForceLoad() API.
+class SymbolFileOnDemand : public lldb_private::SymbolFile {
+  /// LLVM RTTI support.
+  static char ID;
+
+public:
+  /// LLVM RTTI support.
+  /// \{
+  bool isA(const void *ClassID) const override {
+    return ClassID == &ID || SymbolFile::isA(ClassID);
+  }
+  static bool classof(const SymbolFile *obj) { return obj->isA(&ID); }
+  /// \}
+
+  SymbolFileOnDemand(lldb::ObjectFileSP objfile_sp,
+                     std::unique_ptr<SymbolFile> &&symbol_file);
+  virtual ~SymbolFileOnDemand() override;
+
+  // PluginInterface protocol
+  llvm::StringRef GetPluginName() override { return "ondemand"; }
+
+  bool GetLoadDebugInfoEnabled() override { return m_debug_info_enabled; }
+
+  void SetLoadDebugInfoEnabled() override;
+
+  SymbolFile *GetBackingSymbolFile() override { return m_sym_file_impl.get(); }
+
+  uint32_t CalculateAbilities() override;
+
+  std::recursive_mutex &GetModuleMutex() const override;
+
+  lldb::LanguageType
+  ParseLanguage(lldb_private::CompileUnit &comp_unit) override;
+
+  lldb_private::XcodeSDK
+  ParseXcodeSDK(lldb_private::CompileUnit &comp_unit) override;
+
+  void InitializeObject() override;
+
+  size_t ParseFunctions(lldb_private::CompileUnit &comp_unit) override;
+
+  bool ParseLineTable(lldb_private::CompileUnit &comp_unit) override;
+
+  bool ParseDebugMacros(lldb_private::CompileUnit &comp_unit) override;
+
+  bool ForEachExternalModule(
+      lldb_private::CompileUnit &, llvm::DenseSet<lldb_private::SymbolFile *> &,
+      llvm::function_ref<bool(lldb_private::Module &)>) override;
+
+  bool ParseSupportFiles(lldb_private::CompileUnit &comp_unit,
+                         lldb_private::FileSpecList &support_files) override;
+
+  bool ParseIsOptimized(lldb_private::CompileUnit &comp_unit) override;
+
+  size_t ParseTypes(lldb_private::CompileUnit &comp_unit) override;
+
+  bool ParseImportedModules(
+      const lldb_private::SymbolContext &sc,
+      std::vector<lldb_private::SourceModule> &imported_modules) override;
+
+  size_t ParseBlocksRecursive(lldb_private::Function &func) override;
+
+  size_t
+  ParseVariablesForContext(const lldb_private::SymbolContext &sc) override;
+
+  lldb_private::Type *ResolveTypeUID(lldb::user_id_t type_uid) override;
+  llvm::Optional<ArrayInfo> GetDynamicArrayInfoForUID(
+      lldb::user_id_t type_uid,
+      const lldb_private::ExecutionContext *exe_ctx) override;
+
+  bool CompleteType(lldb_private::CompilerType &compiler_type) override;
+
+  lldb_private::CompilerDecl GetDeclForUID(lldb::user_id_t uid) override;
+
+  lldb_private::CompilerDeclContext
+  GetDeclContextForUID(lldb::user_id_t uid) override;
+
+  lldb_private::CompilerDeclContext
+  GetDeclContextContainingUID(lldb::user_id_t uid) override;
+
+  void
+  ParseDeclsForContext(lldb_private::CompilerDeclContext decl_ctx) override;
+
+  uint32_t ResolveSymbolContext(const lldb_private::Address &so_addr,
+                                lldb::SymbolContextItem resolve_scope,
+                                lldb_private::SymbolContext &sc) override;
+
+  uint32_t ResolveSymbolContext(
+      const lldb_private::SourceLocationSpec &src_location_spec,
+      lldb::SymbolContextItem resolve_scope,
+      lldb_private::SymbolContextList &sc_list) override;
+
+  void Dump(lldb_private::Stream &s) override;
+  void DumpClangAST(lldb_private::Stream &s) override;
+
+  void
+  FindGlobalVariables(lldb_private::ConstString name,
+                      const lldb_private::CompilerDeclContext &parent_decl_ctx,
+                      uint32_t max_matches,
+                      lldb_private::VariableList &variables) override;
+
+  void FindGlobalVariables(const lldb_private::RegularExpression &regex,
+                           uint32_t max_matches,
+                           lldb_private::VariableList &variables) override;
+
+  void FindFunctions(lldb_private::ConstString name,
+                     const lldb_private::CompilerDeclContext &parent_decl_ctx,
+                     lldb::FunctionNameType name_type_mask,
+                     bool include_inlines,
+                     lldb_private::SymbolContextList &sc_list) override;
+
+  void FindFunctions(const lldb_private::RegularExpression &regex,
+                     bool include_inlines,
+                     lldb_private::SymbolContextList &sc_list) override;
+
+  void GetMangledNamesForFunction(
+      const std::string &scope_qualified_name,
+      std::vector<lldb_private::ConstString> &mangled_names) override;
+
+  void
+  FindTypes(lldb_private::ConstString name,
+            const lldb_private::CompilerDeclContext &parent_decl_ctx,
+            uint32_t max_matches,
+            llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
+            lldb_private::TypeMap &types) override;
+
+  void FindTypes(llvm::ArrayRef<lldb_private::CompilerContext> pattern,
+                 lldb_private::LanguageSet languages,
+                 llvm::DenseSet<SymbolFile *> &searched_symbol_files,
+                 lldb_private::TypeMap &types) override;
+
+  void GetTypes(lldb_private::SymbolContextScope *sc_scope,
+                lldb::TypeClass type_mask,
+                lldb_private::TypeList &type_list) override;
+
+  llvm::Expected<lldb_private::TypeSystem &>
+  GetTypeSystemForLanguage(lldb::LanguageType language) override;
+
+  lldb_private::CompilerDeclContext FindNamespace(
+      lldb_private::ConstString name,
+      const lldb_private::CompilerDeclContext &parent_decl_ctx) override;
+
+  std::vector<std::unique_ptr<lldb_private::CallEdge>>
+  ParseCallEdgesInFunction(UserID func_id) override;
+
+  lldb::UnwindPlanSP
+  GetUnwindPlan(const Address &address,
+                const RegisterInfoResolver &resolver) override;
+
+  llvm::Expected<lldb::addr_t> GetParameterStackSize(Symbol &symbol) override;
+
+  void PreloadSymbols() override;
+
+  uint64_t GetDebugInfoSize() override;
+  lldb_private::StatsDuration::Duration GetDebugInfoParseTime() override;
+  lldb_private::StatsDuration::Duration GetDebugInfoIndexTime() override;
+
+private:
+  Log *GetLog() const { return ::lldb_private::GetLog(LLDBLog::OnDemand); }
+
+  ConstString GetSymbolFileName() {
+    return GetObjectFile()->GetFileSpec().GetFilename();
+  }
+
+protected:
+  uint32_t CalculateNumCompileUnits() override;
+  lldb::CompUnitSP ParseCompileUnitAtIndex(uint32_t index) override;
+  lldb_private::TypeList &GetTypeList() override;
+
+private:
+  bool m_debug_info_enabled{false};
+  bool m_preload_symbols{false};
+  std::unique_ptr<SymbolFile> m_sym_file_impl;
+};
+} // namespace lldb_private
+
+#endif // LLDB_SYMBOL_SYMBOLFILEONDEMAND_H

lldb/include/lldb/Target/Statistics.h

struct ModuleStats {
   double symtab_index_time = 0.0;
   double debug_parse_time = 0.0;
   double debug_index_time = 0.0;
   uint64_t debug_info_size = 0;
   bool symtab_loaded_from_cache = false;
   bool symtab_saved_to_cache = false;
   bool debug_info_index_loaded_from_cache = false;
   bool debug_info_index_saved_to_cache = false;
+  bool debug_info_enabled = true;
+  bool symtab_stripped = false;
 };
 
 struct ConstStringStats {
   llvm::json::Value ToJSON() const;
   ConstString::MemoryStats stats = ConstString::GetMemoryStats();
 };
 
 /// A class that represents statistics for a since lldb_private::Target.

lldb/include/lldb/Utility/LLDBLog.h

enum class LLDBLog : Log::MaskType {
   Symbols = Log::ChannelFlag<23>,
   SystemRuntime = Log::ChannelFlag<24>,
   Target = Log::ChannelFlag<25>,
   Temporary = Log::ChannelFlag<26>,
   Thread = Log::ChannelFlag<27>,
   Types = Log::ChannelFlag<28>,
   Unwind = Log::ChannelFlag<29>,
   Watchpoints = Log::ChannelFlag<30>,
+  OnDemand = Log::ChannelFlag<31>,
   LLVM_MARK_AS_BITMASK_ENUM(Watchpoints),
 };
 
 LLVM_ENABLE_BITMASK_ENUMS_IN_NAMESPACE();
 
 void InitializeLldbChannel();
 
 template <> Log::Channel &LogChannelFor<LLDBLog>();
 } // namespace lldb_private
 
 #endif // LLDB_UTILITY_LLDBLOG_H

lldb/packages/Python/lldbsuite/test/builders/builder.py

def _getDebugInfoArgs(self, debug_info):
         if debug_info is None:
             return []
         if debug_info == "dwarf":
             return ["MAKE_DSYM=NO"]
         if debug_info == "dwo":
             return ["MAKE_DSYM=NO", "MAKE_DWO=YES"]
         if debug_info == "gmodules":
             return ["MAKE_DSYM=NO", "MAKE_GMODULES=YES"]
+        if debug_info == "ondemand":
+            return ["MAKE_DSYM=NO"]
         return None
 
     def getBuildCommand(self, debug_info, architecture=None, compiler=None,
             dictionary=None, testdir=None, testname=None):
         debug_info_args = self._getDebugInfoArgs(debug_info)
         if debug_info_args is None:
             return None
 

lldb/packages/Python/lldbsuite/test/lldbtest.py

class Base(unittest2.TestCase):
     def getBuildArtifact(self, name="a.out"):
         """Return absolute path to an artifact in the test's build directory."""
         return os.path.join(self.getBuildDir(), name)
 
     def getSourcePath(self, name):
         """Return absolute path to a file in the test's source directory."""
         return os.path.join(self.getSourceDir(), name)
 
+    def getTestCategories(self):
+        test_method = getattr(self, self._testMethodName)
+        if test_method is not None and hasattr(test_method, "categories"):
+            return test_method.categories
+
     @classmethod
     def setUpCommands(cls):
         commands = [
             # First of all, clear all settings to have clean state of global properties.
             "settings clear -all",
 
             # Disable Spotlight lookup. The testsuite creates
             # different binaries with the same UUID, because they only
class TestBase(Base):
     def setUp(self):
         # Works with the test driver to conditionally skip tests via
         # decorators.
         Base.setUp(self)
 
         for s in self.setUpCommands():
             self.runCmd(s)
 
+        test_categories = self.getTestCategories()
+        if test_categories is not None and 'ondemand' in test_categories:
+            self.runCmd("settings set symbols.load-on-demand true")
+
         if "LLDB_MAX_LAUNCH_COUNT" in os.environ:
             self.maxLaunchCount = int(os.environ["LLDB_MAX_LAUNCH_COUNT"])
 
         if "LLDB_TIME_WAIT_NEXT_LAUNCH" in os.environ:
             self.timeWaitNextLaunch = float(
                 os.environ["LLDB_TIME_WAIT_NEXT_LAUNCH"])
 
         # We want our debugger to be synchronous.

lldb/packages/Python/lldbsuite/test/test_categories.py

 
 # Third-party modules
 
 # LLDB modules
 from lldbsuite.support import gmodules
 
 
 debug_info_categories = [
-    'dwarf', 'dwo', 'dsym', 'gmodules'
+    'dwarf', 'dwo', 'dsym', 'gmodules', 'ondemand'
 ]
 
 all_categories = {
     'basic_process': 'Basic process execution sniff tests.',
     'cmdline': 'Tests related to the LLDB command-line interface',
     'dataformatters': 'Tests related to the type command and the data formatters subsystem',
     'debugserver': 'Debugserver tests',
     'dsym': 'Tests that can be run with DSYM debug information',
all_categories = {
     'lldb-server': 'Tests related to lldb-server',
     'lldb-vscode': 'Visual Studio Code debug adaptor tests',
     'llgs': 'Tests for the gdb-server functionality of lldb-server',
     'objc': 'Tests related to the Objective-C programming language support',
     'pyapi': 'Tests related to the Python API',
     'std-module': 'Tests related to importing the std module',
     'stresstest': 'Tests related to stressing lldb limits',
     'watchpoint': 'Watchpoint-related tests',
+    'ondemand': 'Tests that can be run with symbol on-demand loading',
 }
 
 
 def unique_string_match(yourentry, list):
     candidate = None
     for item in list:
         if not item.startswith(yourentry):
             continue

lldb/source/Core/CoreProperties.td

let Definition = "modulelist" in {
   def LLDBIndexCacheMaxPercent: Property<"lldb-index-cache-max-percent", "UInt64">,
     Global,
     DefaultUnsignedValue<0>,
     Desc<"The maximum size for the cache directory in terms of percentage of the available space on the disk. Set to 100 to indicate no limit, 50 to indicate that the cache size will not be left over half the available disk space. A value over 100 will be reduced to 100. A value of 0 disables the percentage size-based pruning.">;
   def LLDBIndexCacheExpirationDays: Property<"lldb-index-cache-expiration-days", "UInt64">,
     Global,
     DefaultUnsignedValue<7>,
     Desc<"The expiration time in days for a file. When a file hasn't been accessed for the specified amount of days, it is removed from the cache. A value of 0 disables the expiration-based pruning.">;
+  def LoadSymbolOnDemand: Property<"load-on-demand", "Boolean">,
+    Global,
+    DefaultFalse,
+    Desc<"Enable on demand symbol loading in LLDB. LLDB will load debug info on demand for each module based on breakpoint to improve performance.">;
 }
 
 let Definition = "debugger" in {
   def AutoConfirm: Property<"auto-confirm", "Boolean">,
     Global,
     DefaultFalse,
     Desc<"If true all confirmation prompts will receive their default reply.">;
   def DisassemblyFormat: Property<"disassembly-format", "FormatEntity">,

lldb/source/Core/Module.cpp

if (section_sp && section_sp->GetModule().get() == this) {
     // right module.
     sc.module_sp = shared_from_this();
     resolved_flags |= eSymbolContextModule;
 
     SymbolFile *symfile = GetSymbolFile();
     if (!symfile)
       return resolved_flags;
 
+    symfile->SetLoadDebugInfoEnabled();
+
     // Resolve the compile unit, function, block, line table or line entry if
     // requested.
     if (resolve_scope & eSymbolContextCompUnit ||
         resolve_scope & eSymbolContextFunction ||
         resolve_scope & eSymbolContextBlock ||
         resolve_scope & eSymbolContextLineEntry ||
         resolve_scope & eSymbolContextVariable) {
       resolved_flags |=
           symfile->ResolveSymbolContext(so_addr, resolve_scope, sc);
     }

lldb/source/Core/ModuleList.cpp

void ModuleListProperties::UpdateSymlinkMappings() {
   }
 }
 
 PathMappingList ModuleListProperties::GetSymlinkMappings() const {
   llvm::sys::ScopedReader lock(m_symlink_paths_mutex);
   return m_symlink_paths;
 }
 
+bool ModuleListProperties::GetLoadSymbolOnDemand() {
+  const uint32_t idx = ePropertyLoadSymbolOnDemand;
+  return m_collection_sp->GetPropertyAtIndexAsBoolean(
+      nullptr, idx, g_modulelist_properties[idx].default_uint_value != 0);
+}
+
 ModuleList::ModuleList() : m_modules(), m_modules_mutex() {}
 
 ModuleList::ModuleList(const ModuleList &rhs)
     : m_modules(), m_modules_mutex(), m_notifier(nullptr) {
   std::lock_guard<std::recursive_mutex> lhs_guard(m_modules_mutex);
   std::lock_guard<std::recursive_mutex> rhs_guard(rhs.m_modules_mutex);
   m_modules = rhs.m_modules;
 }

lldb/source/Plugins/SymbolFile/DWARF/DWARFIndex.cpp

bool DWARFIndex::ProcessFunctionDIE(
 
   return true;
 }
 
 DWARFIndex::DIERefCallbackImpl::DIERefCallbackImpl(
     const DWARFIndex &index, llvm::function_ref<bool(DWARFDIE die)> callback,
     llvm::StringRef name)
     : m_index(index),
-      m_dwarf(*llvm::cast<SymbolFileDWARF>(index.m_module.GetSymbolFile())),
+      m_dwarf(*llvm::cast<SymbolFileDWARF>(
+          index.m_module.GetSymbolFile()->GetBackingSymbolFile())),
       m_callback(callback), m_name(name) {}
 
 bool DWARFIndex::DIERefCallbackImpl::operator()(DIERef ref) const {
   if (DWARFDIE die = m_dwarf.GetDIE(ref))
     return m_callback(die);
   m_index.ReportInvalidDIERef(ref, m_name);
   return true;
 }
 
 void DWARFIndex::ReportInvalidDIERef(DIERef ref, llvm::StringRef name) const {
   m_module.ReportErrorIfModifyDetected(
       "the DWARF debug information has been modified (accelerator table had "
       "bad die 0x%8.8x for '%s')\n",
       ref.die_offset(), name.str().c_str());
 }

lldb/source/Plugins/SymbolFile/DWARF/DebugNamesDWARFIndex.cpp

 }
 
 bool DebugNamesDWARFIndex::ProcessEntry(
     const DebugNames::Entry &entry,
     llvm::function_ref<bool(DWARFDIE die)> callback, llvm::StringRef name) {
   llvm::Optional<DIERef> ref = ToDIERef(entry);
   if (!ref)
     return true;
-  SymbolFileDWARF &dwarf =
-      *llvm::cast<SymbolFileDWARF>(m_module.GetSymbolFile());
+  SymbolFileDWARF &dwarf = *llvm::cast<SymbolFileDWARF>(
+      m_module.GetSymbolFile()->GetBackingSymbolFile());
   DWARFDIE die = dwarf.GetDIE(*ref);
   if (!die)
     return true;
   return callback(die);
 }
 
 void DebugNamesDWARFIndex::MaybeLogLookupError(llvm::Error error,
                                                const DebugNames::NameIndex &ni,

lldb/source/Symbol/CMakeLists.txt

add_lldb_library(lldbSymbol
   LineEntry.cpp
   LineTable.cpp
   LocateSymbolFile.cpp
   ObjectFile.cpp
   PostfixExpression.cpp
   Symbol.cpp
   SymbolContext.cpp
   SymbolFile.cpp
+  SymbolFileOnDemand.cpp
   SymbolVendor.cpp
   Symtab.cpp
   Type.cpp
   TypeList.cpp
   TypeMap.cpp
   TypeSystem.cpp
   UnwindPlan.cpp
   UnwindTable.cpp

lldb/source/Symbol/CompileUnit.cpp

while (file_idx != UINT32_MAX) {
     file_indexes.push_back(file_idx);
     file_idx = GetSupportFiles().FindFileIndex(file_idx + 1, file_spec, true);
   }
 
   const size_t num_file_indexes = file_indexes.size();
   if (num_file_indexes == 0)
     return;
 
+  // Found a matching source file in this compile unit load its debug info.
+  GetModule()->GetSymbolFile()->SetLoadDebugInfoEnabled();
+
   LineTable *line_table = sc.comp_unit->GetLineTable();
 
   if (line_table == nullptr) {
     if (file_spec_matches_cu_file_spec && !check_inlines) {
       sc_list.Append(sc);
     }
     return;
   }
line_idx = line_table->FindLineEntryIndexByFileIndex(
         0, file_indexes.front(), src_location_spec, &line_entry);
   } else {
     // We found multiple support files that match "file_spec" so use the
     // line table function that searches for a line entries that match a
     // multiple support file indexes.
     line_idx = line_table->FindLineEntryIndexByFileIndex(
         0, file_indexes, src_location_spec, &line_entry);
   }
-  
   // If "exact == true", then "found_line" will be the same as "line". If
   // "exact == false", the "found_line" will be the closest line entry
   // with a line number greater than "line" and we will use this for our
   // subsequent line exact matches below.
   const bool inlines = false;
   const bool exact = true;
   const llvm::Optional<uint16_t> column =
       src_location_spec.GetColumn().hasValue()

lldb/source/Symbol/SymbolFile.cpp

 //===-- SymbolFile.cpp ----------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Symbol/SymbolFile.h"
 
 #include "lldb/Core/Module.h"
 #include "lldb/Core/PluginManager.h"
 #include "lldb/Symbol/CompileUnit.h"
 #include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Symbol/SymbolFileOnDemand.h"
 #include "lldb/Symbol/TypeMap.h"
 #include "lldb/Symbol/TypeSystem.h"
 #include "lldb/Symbol/VariableList.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/StreamString.h"
 #include "lldb/lldb-private.h"
 
 #include <future>
for (uint32_t idx = 0;
           // If any symbol file parser has all of the abilities, then we should
           // just stop looking.
           if ((kAllAbilities & sym_file_abilities) == kAllAbilities)
             break;
         }
       }
     }
     if (best_symfile_up) {
+      ObjectFile::Type obj_file_type = objfile_sp->CalculateType();
+      if (ModuleList::GetGlobalModuleListProperties().GetLoadSymbolOnDemand() &&
+          best_symfile_up->GetDebugInfoSize() > 0 &&
+          (obj_file_type == ObjectFile::eTypeExecutable ||
+           obj_file_type == ObjectFile::eTypeSharedLibrary ||
+           obj_file_type == ObjectFile::eTypeDebugInfo)) {
+        best_symfile_up = std::make_unique<SymbolFileOnDemand>(
+            objfile_sp, std::move(best_symfile_up));
+      }
       // Let the winning symbol file parser initialize itself more completely
       // now that it has been chosen
       best_symfile_up->InitializeObject();
     }
   }
   return best_symfile_up.release();
 }
 

lldb/source/Symbol/SymbolFileOnDemand.cpp

+//===-- SymbolFileDWARFDebugMap.cpp ---------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Symbol/SymbolFileOnDemand.h"
+
+#include "lldb/Core/Module.h"
+#include "lldb/Symbol/SymbolFile.h"
+
+#include <memory>
+
+using namespace lldb;
+using namespace lldb_private;
+
+char SymbolFileOnDemand::ID;
+
+SymbolFileOnDemand::SymbolFileOnDemand(
+    lldb::ObjectFileSP objfile_sp, std::unique_ptr<SymbolFile> &&symbol_file)
+    : SymbolFile(std::move(objfile_sp)),
+      m_sym_file_impl(std::move(symbol_file)) {}
+
+SymbolFileOnDemand::~SymbolFileOnDemand() {}
+
+uint32_t SymbolFileOnDemand::CalculateAbilities() {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      uint32_t abilities = m_sym_file_impl->CalculateAbilities();
+      if (abilities != 0) {
+        LLDB_LOG(log, "{0} abilities would return if hydrated.", abilities);
+      }
+    }
+    return 0;
+  }
+  return m_sym_file_impl->CalculateAbilities();
+}
+
+std::recursive_mutex &SymbolFileOnDemand::GetModuleMutex() const {
+  return m_sym_file_impl->GetModuleMutex();
+}
+
+void SymbolFileOnDemand::InitializeObject() {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->InitializeObject();
+}
+
+lldb::LanguageType SymbolFileOnDemand::ParseLanguage(CompileUnit &comp_unit) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      lldb::LanguageType langType = m_sym_file_impl->ParseLanguage(comp_unit);
+      if (langType != eLanguageTypeUnknown) {
+        LLDB_LOG(log, "Language {0} would return if hydrated.",
+                 eLanguageTypeUnknown);
+      }
+    }
+    return eLanguageTypeUnknown;
+  }
+  return m_sym_file_impl->ParseLanguage(comp_unit);
+}
+
+XcodeSDK SymbolFileOnDemand::ParseXcodeSDK(CompileUnit &comp_unit) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    XcodeSDK defaultValue{};
+    if (log) {
+      XcodeSDK sdk = m_sym_file_impl->ParseXcodeSDK(comp_unit);
+      if (!(sdk == defaultValue)) {
+        LLDB_LOG(log, "SDK {0} would return if hydrated.", sdk.GetString());
+      }
+    }
+    return defaultValue;
+  }
+  return m_sym_file_impl->ParseXcodeSDK(comp_unit);
+}
+
+size_t SymbolFileOnDemand::ParseFunctions(CompileUnit &comp_unit) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return 0;
+  }
+  return m_sym_file_impl->ParseFunctions(comp_unit);
+}
+
+bool SymbolFileOnDemand::ParseLineTable(CompileUnit &comp_unit) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return false;
+  }
+  return m_sym_file_impl->ParseLineTable(comp_unit);
+}
+
+bool SymbolFileOnDemand::ParseDebugMacros(CompileUnit &comp_unit) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return false;
+  }
+  return m_sym_file_impl->ParseDebugMacros(comp_unit);
+}
+
+bool SymbolFileOnDemand::ForEachExternalModule(
+    CompileUnit &comp_unit,
+    llvm::DenseSet<lldb_private::SymbolFile *> &visited_symbol_files,
+    llvm::function_ref<bool(Module &)> lambda) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    // Not early exit.
+    return false;
+  }
+  return m_sym_file_impl->ForEachExternalModule(comp_unit, visited_symbol_files,
+                                                lambda);
+}
+
+bool SymbolFileOnDemand::ParseSupportFiles(CompileUnit &comp_unit,
+                                           FileSpecList &support_files) {
+  LLDB_LOG(GetLog(),
+           "[{0}] {1} is not skipped: explicitly allowed to support breakpoint",
+           GetSymbolFileName(), __FUNCTION__);
+  // Explicitly allow this API through to support source line breakpoint.
+  return m_sym_file_impl->ParseSupportFiles(comp_unit, support_files);
+}
+
+bool SymbolFileOnDemand::ParseIsOptimized(CompileUnit &comp_unit) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      bool optimized = m_sym_file_impl->ParseIsOptimized(comp_unit);
+      if (optimized) {
+        LLDB_LOG(log, "Optimized would return if hydrated.");
+      }
+    }
+    return false;
+  }
+  return m_sym_file_impl->ParseIsOptimized(comp_unit);
+}
+
+size_t SymbolFileOnDemand::ParseTypes(CompileUnit &comp_unit) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return 0;
+  }
+  return m_sym_file_impl->ParseTypes(comp_unit);
+}
+
+bool SymbolFileOnDemand::ParseImportedModules(
+    const lldb_private::SymbolContext &sc,
+    std::vector<SourceModule> &imported_modules) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      std::vector<SourceModule> tmp_imported_modules;
+      bool succeed =
+          m_sym_file_impl->ParseImportedModules(sc, tmp_imported_modules);
+      if (succeed) {
+        LLDB_LOG(log, "{0} imported modules would be parsed if hydrated.",
+                 tmp_imported_modules.size());
+      }
+    }
+    return false;
+  }
+  return m_sym_file_impl->ParseImportedModules(sc, imported_modules);
+}
+
+size_t SymbolFileOnDemand::ParseBlocksRecursive(Function &func) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return 0;
+  }
+  return m_sym_file_impl->ParseBlocksRecursive(func);
+}
+
+size_t SymbolFileOnDemand::ParseVariablesForContext(const SymbolContext &sc) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return 0;
+  }
+  return m_sym_file_impl->ParseVariablesForContext(sc);
+}
+
+Type *SymbolFileOnDemand::ResolveTypeUID(lldb::user_id_t type_uid) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      Type *resolved_type = m_sym_file_impl->ResolveTypeUID(type_uid);
+      if (resolved_type) {
+        LLDB_LOG(log, "Type would be parsed for {0} if hydrated.", type_uid);
+      }
+    }
+    return nullptr;
+  }
+  return m_sym_file_impl->ResolveTypeUID(type_uid);
+}
+
+llvm::Optional<SymbolFile::ArrayInfo>
+SymbolFileOnDemand::GetDynamicArrayInfoForUID(
+    lldb::user_id_t type_uid, const lldb_private::ExecutionContext *exe_ctx) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return llvm::None;
+  }
+  return m_sym_file_impl->GetDynamicArrayInfoForUID(type_uid, exe_ctx);
+}
+
+bool SymbolFileOnDemand::CompleteType(CompilerType &compiler_type) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return false;
+  }
+  return m_sym_file_impl->CompleteType(compiler_type);
+}
+
+CompilerDecl SymbolFileOnDemand::GetDeclForUID(lldb::user_id_t type_uid) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      CompilerDecl parsed_decl = m_sym_file_impl->GetDeclForUID(type_uid);
+      if (parsed_decl != CompilerDecl()) {
+        LLDB_LOG(log, "CompilerDecl {0} would be parsed for {1} if hydrated.",
+                 parsed_decl.GetName(), type_uid);
+      }
+    }
+    return CompilerDecl();
+  }
+  return m_sym_file_impl->GetDeclForUID(type_uid);
+}
+
+CompilerDeclContext
+SymbolFileOnDemand::GetDeclContextForUID(lldb::user_id_t type_uid) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return CompilerDeclContext();
+  }
+  return m_sym_file_impl->GetDeclContextForUID(type_uid);
+}
+
+CompilerDeclContext
+SymbolFileOnDemand::GetDeclContextContainingUID(lldb::user_id_t type_uid) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return CompilerDeclContext();
+  }
+  return m_sym_file_impl->GetDeclContextContainingUID(type_uid);
+}
+
+void SymbolFileOnDemand::ParseDeclsForContext(CompilerDeclContext decl_ctx) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->ParseDeclsForContext(decl_ctx);
+}
+
+uint32_t
+SymbolFileOnDemand::ResolveSymbolContext(const Address &so_addr,
+                                         SymbolContextItem resolve_scope,
+                                         SymbolContext &sc) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return 0;
+  }
+  return m_sym_file_impl->ResolveSymbolContext(so_addr, resolve_scope, sc);
+}
+
+uint32_t SymbolFileOnDemand::ResolveSymbolContext(
+    const SourceLocationSpec &src_location_spec,
+    SymbolContextItem resolve_scope, SymbolContextList &sc_list) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return 0;
+  }
+  return m_sym_file_impl->ResolveSymbolContext(src_location_spec, resolve_scope,
+                                               sc_list);
+}
+
+void SymbolFileOnDemand::Dump(lldb_private::Stream &s) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->Dump(s);
+}
+
+void SymbolFileOnDemand::DumpClangAST(lldb_private::Stream &s) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->DumpClangAST(s);
+}
+
+void SymbolFileOnDemand::FindGlobalVariables(const RegularExpression &regex,
+                                             uint32_t max_matches,
+                                             VariableList &variables) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->FindGlobalVariables(regex, max_matches, variables);
+}
+
+void SymbolFileOnDemand::FindGlobalVariables(
+    ConstString name, const CompilerDeclContext &parent_decl_ctx,
+    uint32_t max_matches, VariableList &variables) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    Symtab *symtab = GetSymtab();
+    if (!symtab) {
+      LLDB_LOG(log, "[{0}] {1} is skipped - fail to get symtab",
+               GetSymbolFileName(), __FUNCTION__);
+      return;
+    }
+    Symbol *sym = symtab->FindFirstSymbolWithNameAndType(
+        name, eSymbolTypeData, Symtab::eDebugAny, Symtab::eVisibilityAny);
+    if (!sym) {
+      LLDB_LOG(log, "[{0}] {1} is skipped - fail to find match in symtab",
+               GetSymbolFileName(), __FUNCTION__);
+      return;
+    }
+    LLDB_LOG(log, "[{0}] {1} is NOT skipped - found match in symtab",
+             GetSymbolFileName(), __FUNCTION__);
+
+    // Found match in symbol table hydrate debug info and
+    // allow the FindGlobalVariables to go through.
+    SetLoadDebugInfoEnabled();
+  }
+  return m_sym_file_impl->FindGlobalVariables(name, parent_decl_ctx,
+                                              max_matches, variables);
+}
+
+void SymbolFileOnDemand::FindFunctions(const RegularExpression &regex,
+                                       bool include_inlines,
+                                       SymbolContextList &sc_list) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    Symtab *symtab = GetSymtab();
+    if (!symtab) {
+      LLDB_LOG(log, "[{0}] {1} is skipped - fail to get symtab",
+               GetSymbolFileName(), __FUNCTION__);
+      return;
+    }
+    std::vector<uint32_t> symbol_indexes;
+    symtab->AppendSymbolIndexesMatchingRegExAndType(
+        regex, eSymbolTypeAny, Symtab::eDebugAny, Symtab::eVisibilityAny,
+        symbol_indexes);
+    if (symbol_indexes.empty()) {
+      LLDB_LOG(log, "[{0}] {1} is skipped - fail to find match in symtab",
+               GetSymbolFileName(), __FUNCTION__);
+      return;
+    }
+    LLDB_LOG(log, "[{0}] {1} is NOT skipped - found match in symtab",
+             GetSymbolFileName(), __FUNCTION__);
+
+    // Found match in symbol table hydrate debug info and
+    // allow the FindFucntions to go through.
+    SetLoadDebugInfoEnabled();
+  }
+  return m_sym_file_impl->FindFunctions(regex, include_inlines, sc_list);
+}
+
+void SymbolFileOnDemand::FindFunctions(
+    ConstString name, const CompilerDeclContext &parent_decl_ctx,
+    FunctionNameType name_type_mask, bool include_inlines,
+    SymbolContextList &sc_list) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+
+    Symtab *symtab = GetSymtab();
+    if (!symtab) {
+      LLDB_LOG(log, "[{0}] {1}({2}) is skipped  - fail to get symtab",
+               GetSymbolFileName(), __FUNCTION__, name);
+      return;
+    }
+
+    SymbolContextList sc_list_helper;
+    symtab->FindFunctionSymbols(name, name_type_mask, sc_list_helper);
+    if (sc_list_helper.GetSize() == 0) {
+      LLDB_LOG(log, "[{0}] {1}({2}) is skipped - fail to find match in symtab",
+               GetSymbolFileName(), __FUNCTION__, name);
+      return;
+    }
+    LLDB_LOG(log, "[{0}] {1}({2}) is NOT skipped - found match in symtab",
+             GetSymbolFileName(), __FUNCTION__, name);
+
+    // Found match in symbol table hydrate debug info and
+    // allow the FindFucntions to go through.
+    SetLoadDebugInfoEnabled();
+  }
+  return m_sym_file_impl->FindFunctions(name, parent_decl_ctx, name_type_mask,
+                                        include_inlines, sc_list);
+}
+
+void SymbolFileOnDemand::GetMangledNamesForFunction(
+    const std::string &scope_qualified_name,
+    std::vector<ConstString> &mangled_names) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1}({2}) is skipped", GetSymbolFileName(),
+             __FUNCTION__, scope_qualified_name);
+    return;
+  }
+  return m_sym_file_impl->GetMangledNamesForFunction(scope_qualified_name,
+                                                     mangled_names);
+}
+
+void SymbolFileOnDemand::FindTypes(
+    ConstString name, const CompilerDeclContext &parent_decl_ctx,
+    uint32_t max_matches,
+    llvm::DenseSet<lldb_private::SymbolFile *> &searched_symbol_files,
+    TypeMap &types) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1}({2}) is skipped", GetSymbolFileName(),
+             __FUNCTION__, name);
+    return;
+  }
+  return m_sym_file_impl->FindTypes(name, parent_decl_ctx, max_matches,
+                                    searched_symbol_files, types);
+}
+
+void SymbolFileOnDemand::FindTypes(
+    llvm::ArrayRef<CompilerContext> pattern, LanguageSet languages,
+    llvm::DenseSet<SymbolFile *> &searched_symbol_files, TypeMap &types) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->FindTypes(pattern, languages, searched_symbol_files,
+                                    types);
+}
+
+void SymbolFileOnDemand::GetTypes(SymbolContextScope *sc_scope,
+                                  TypeClass type_mask, TypeList &type_list) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->GetTypes(sc_scope, type_mask, type_list);
+}
+
+llvm::Expected<TypeSystem &>
+SymbolFileOnDemand::GetTypeSystemForLanguage(LanguageType language) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped for language type {2}",
+             GetSymbolFileName(), __FUNCTION__, language);
+    return SymbolFile::GetTypeSystemForLanguage(language);
+  }
+  return m_sym_file_impl->GetTypeSystemForLanguage(language);
+}
+
+CompilerDeclContext
+SymbolFileOnDemand::FindNamespace(ConstString name,
+                                  const CompilerDeclContext &parent_decl_ctx) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1}({2}) is skipped", GetSymbolFileName(),
+             __FUNCTION__, name);
+    return SymbolFile::FindNamespace(name, parent_decl_ctx);
+  }
+  return m_sym_file_impl->FindNamespace(name, parent_decl_ctx);
+}
+
+std::vector<std::unique_ptr<lldb_private::CallEdge>>
+SymbolFileOnDemand::ParseCallEdgesInFunction(UserID func_id) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      std::vector<std::unique_ptr<lldb_private::CallEdge>> call_edges =
+          m_sym_file_impl->ParseCallEdgesInFunction(func_id);
+      if (call_edges.size() > 0) {
+        LLDB_LOG(log, "{0} call edges would be parsed for {1} if hydrated.",
+                 call_edges.size(), func_id.GetID());
+      }
+    }
+    return {};
+  }
+  return m_sym_file_impl->ParseCallEdgesInFunction(func_id);
+}
+
+lldb::UnwindPlanSP
+SymbolFileOnDemand::GetUnwindPlan(const Address &address,
+                                  const RegisterInfoResolver &resolver) {
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return nullptr;
+  }
+  return m_sym_file_impl->GetUnwindPlan(address, resolver);
+}
+
+llvm::Expected<lldb::addr_t>
+SymbolFileOnDemand::GetParameterStackSize(Symbol &symbol) {
+  if (!this->m_debug_info_enabled) {
+    Log *log = GetLog();
+    LLDB_LOG(log, "[{0}] {1} is skipped", GetSymbolFileName(), __FUNCTION__);
+    if (log) {
+      llvm::Expected<lldb::addr_t> stack_size =
+          m_sym_file_impl->GetParameterStackSize(symbol);
+      if (stack_size) {
+        LLDB_LOG(log, "{0} stack size would return for symbol {1} if hydrated.",
+                 *stack_size, symbol.GetName());
+      }
+    }
+    return SymbolFile::GetParameterStackSize(symbol);
+  }
+  return m_sym_file_impl->GetParameterStackSize(symbol);
+}
+
+void SymbolFileOnDemand::PreloadSymbols() {
+  m_preload_symbols = true;
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return;
+  }
+  return m_sym_file_impl->PreloadSymbols();
+}
+
+uint64_t SymbolFileOnDemand::GetDebugInfoSize() {
+  // Always return the real debug info size.
+  LLDB_LOG(GetLog(), "[{0}] {1} is not skipped", GetSymbolFileName(),
+           __FUNCTION__);
+  return m_sym_file_impl->GetDebugInfoSize();
+}
+
+StatsDuration::Duration SymbolFileOnDemand::GetDebugInfoParseTime() {
+  // Always return the real parse time.
+  LLDB_LOG(GetLog(), "[{0}] {1} is not skipped", GetSymbolFileName(),
+           __FUNCTION__);
+  return m_sym_file_impl->GetDebugInfoParseTime();
+}
+
+StatsDuration::Duration SymbolFileOnDemand::GetDebugInfoIndexTime() {
+  // Always return the real index time.
+  LLDB_LOG(GetLog(), "[{0}] {1} is not skipped", GetSymbolFileName(),
+           __FUNCTION__);
+  return m_sym_file_impl->GetDebugInfoIndexTime();
+}
+
+uint32_t SymbolFileOnDemand::CalculateNumCompileUnits() {
+  Log *log = GetLog();
+  LLDB_LOG(log, "[{0}] {1} is not skipped to support breakpoint hydration",
+           GetSymbolFileName(), __FUNCTION__);
+  // Explicitly allow this API through.
+  return m_sym_file_impl->CalculateNumCompileUnits();
+}
+
+CompUnitSP SymbolFileOnDemand::ParseCompileUnitAtIndex(uint32_t cu_idx) {
+  LLDB_LOG(
+      GetLog(),
+      "[{0}] {1} for cu_idx {2} is not skipped to support breakpoint hydration",
+      GetSymbolFileName(), __FUNCTION__, cu_idx);
+  // Explicitly allow this API through to support source line breakpoint.
+  return m_sym_file_impl->ParseCompileUnitAtIndex(cu_idx);
+}
+
+TypeList &SymbolFileOnDemand::GetTypeList() {
+  // assert(false && "Should never be called");
+  if (!this->m_debug_info_enabled) {
+    LLDB_LOG(GetLog(), "[{0}] {1} is skipped", GetSymbolFileName(),
+             __FUNCTION__);
+    return SymbolFile::GetTypeList();
+  }
+  return m_sym_file_impl->GetTypeList();
+}
+
+void SymbolFileOnDemand::SetLoadDebugInfoEnabled() {
+  if (m_debug_info_enabled) {
+    return;
+  }
+  LLDB_LOG(GetLog(), "[{0}] Hydrate debug info", GetSymbolFileName());
+  m_debug_info_enabled = true;
+  InitializeObject();
+  if (m_preload_symbols) {
+    PreloadSymbols();
+  }
+}

lldb/source/Target/Statistics.cpp

json::Value ModuleStats::ToJSON() const {
   module.try_emplace("symbolTableSavedToCache", symtab_saved_to_cache);
   module.try_emplace("debugInfoParseTime", debug_parse_time);
   module.try_emplace("debugInfoIndexTime", debug_index_time);
   module.try_emplace("debugInfoByteSize", (int64_t)debug_info_size);
   module.try_emplace("debugInfoIndexLoadedFromCache",
                      debug_info_index_loaded_from_cache);
   module.try_emplace("debugInfoIndexSavedToCache",
                      debug_info_index_saved_to_cache);
+  module.try_emplace("debugInfoEnabled", debug_info_enabled);
+  module.try_emplace("symtab_stripped", symtab_stripped);
   if (!symfile_path.empty())
     module.try_emplace("symbolFilePath", symfile_path);
 
   if (!symfile_modules.empty()) {
     json::Array symfile_ids;
     for (const auto symfile_id: symfile_modules)
       symfile_ids.emplace_back(symfile_id);
     module.try_emplace("symbolFileModuleIdentifiers", std::move(symfile_ids));
if (target) {
     json_targets.emplace_back(target->ReportStatistics());
   } else {
     for (const auto &target : debugger.GetTargetList().Targets())
       json_targets.emplace_back(target->ReportStatistics());
   }
   std::vector<ModuleStats> modules;
   std::lock_guard<std::recursive_mutex> guard(
       Module::GetAllocationModuleCollectionMutex());
-  const size_t num_modules = Module::GetNumberAllocatedModules();
+  const uint64_t num_modules = Module::GetNumberAllocatedModules();
+  uint32_t num_debug_info_enabled_modules = 0;
+  uint32_t num_stripped_modules = 0;
   for (size_t image_idx = 0; image_idx < num_modules; ++image_idx) {
     Module *module = Module::GetAllocatedModuleAtIndex(image_idx);
     ModuleStats module_stat;
     module_stat.identifier = (intptr_t)module;
     module_stat.path = module->GetFileSpec().GetPath();
     if (ConstString object_name = module->GetObjectName()) {
       module_stat.path.append(1, '(');
       module_stat.path.append(object_name.GetStringRef().str());
if (sym_file) {
         ++debug_index_loaded;
       module_stat.debug_info_index_saved_to_cache =
           sym_file->GetDebugInfoIndexWasSavedToCache();
       if (module_stat.debug_info_index_saved_to_cache)
         ++debug_index_saved;
       ModuleList symbol_modules = sym_file->GetDebugInfoModules();
       for (const auto &symbol_module: symbol_modules.Modules())
         module_stat.symfile_modules.push_back((intptr_t)symbol_module.get());
+      module_stat.symtab_stripped = module->GetObjectFile()->IsStripped();
+      if (module_stat.symtab_stripped) {
+        ++num_stripped_modules;
+      }
+      module_stat.debug_info_enabled = sym_file->GetLoadDebugInfoEnabled() &&
+                                       module_stat.debug_info_size > 0;
+      if (module_stat.debug_info_enabled) {
+        ++num_debug_info_enabled_modules;
+      }
     }
     symtab_parse_time += module_stat.symtab_parse_time;
     symtab_index_time += module_stat.symtab_index_time;
     debug_parse_time += module_stat.debug_parse_time;
     debug_index_time += module_stat.debug_index_time;
     debug_info_size += module_stat.debug_info_size;
     json_modules.emplace_back(module_stat.ToJSON());
   }
json::Object global_stats{
       {"totalSymbolTableIndexTime", symtab_index_time},
       {"totalSymbolTablesLoadedFromCache", symtabs_loaded},
       {"totalSymbolTablesSavedToCache", symtabs_saved},
       {"totalDebugInfoParseTime", debug_parse_time},
       {"totalDebugInfoIndexTime", debug_index_time},
       {"totalDebugInfoIndexLoadedFromCache", debug_index_loaded},
       {"totalDebugInfoIndexSavedToCache", debug_index_saved},
       {"totalDebugInfoByteSize", debug_info_size},
+      {"totalModuleCount", num_modules},
+      {"totalDebugInfoEnabledModules", num_debug_info_enabled_modules},
+      {"totalSymbolTableStrippedModules", num_stripped_modules},
   };
   return std::move(global_stats);
 }

lldb/source/Utility/LLDBLog.cpp

static constexpr Log::Category g_categories[] = {
     {{"symbol"}, {"log symbol related issues and warnings"}, LLDBLog::Symbols},
     {{"system-runtime"}, {"log system runtime events"}, LLDBLog::SystemRuntime},
     {{"target"}, {"log target events and activities"}, LLDBLog::Target},
     {{"temp"}, {"log internal temporary debug messages"}, LLDBLog::Temporary},
     {{"thread"}, {"log thread events and activities"}, LLDBLog::Thread},
     {{"types"}, {"log type system related activities"}, LLDBLog::Types},
     {{"unwind"}, {"log stack unwind activities"}, LLDBLog::Unwind},
     {{"watch"}, {"log watchpoint related activities"}, LLDBLog::Watchpoints},
+    {{"on-demand"},
+     {"log symbol on-demand related activities"},
+     LLDBLog::OnDemand},
 };
 
 static Log::Channel g_log_channel(g_categories,
                                   LLDBLog::Process | LLDBLog::Thread |
                                       LLDBLog::DynamicLoader |
                                       LLDBLog::Breakpoints |
                                       LLDBLog::Watchpoints | LLDBLog::Step |
                                       LLDBLog::State | LLDBLog::Symbols |

lldb/test/API/functionalities/breakpoint/breakpoint_ids/TestBreakpointIDs.py

 """
 Test lldb breakpoint ids.
 """
 
 
 
 import lldb
+from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 import lldbsuite.test.lldbutil as lldbutil
 
 
 class BreakpointIDTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
+    @skipIf(
+        debug_info="ondemand",
+        bugnumber="symbolic function for inline function won't work in on-demand mode",
+    )
     def test(self):
         self.build()
 
         exe = self.getBuildArtifact("a.out")
         self.expect("file " + exe,
                     patterns=["Current executable set to .*a.out"])
 
         bpno = lldbutil.run_break_set_by_symbol(

lldb/test/API/functionalities/find-line-entry/TestFindLineEntry.py

 """
 Test that SBCompileUnit::FindLineEntryIndex works correctly.
 """
 
 import lldb
+from lldbsuite.test.decorators import *
 import lldbsuite.test.lldbutil as lldbutil
 from lldbsuite.test.lldbtest import *
 
 class FindLineEntry(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
+    @skipIf(
+        debug_info="ondemand",
+        bugnumber="Line table parsing is disabled in on-demand mode",
+    )
     def test_compile_unit_find_line_entry_index(self):
         """ Test the CompileUnit LineEntryIndex lookup API """
         self.build()
         exe = self.getBuildArtifact("a.out")
         self.target = self.dbg.CreateTarget(exe)
         self.assertTrue(self.target.IsValid(), "Target is not valid")
 
         self.file = lldb.SBFileSpec("main.c")

lldb/test/API/functionalities/limit-debug-info/TestLimitDebugInfo.py

class LimitDebugInfoTestCase(TestBase):
     def _check_debug_info_is_limited(self, target):
         # Without other shared libraries we should only see the member declared
         # in the derived class. This serves as a sanity check that we are truly
         # building with limited debug info.
         self._check_type(target, "InheritsFromOne")
         self._check_type(target, "InheritsFromTwo")
 
     @skipIf(bugnumber="pr46284", debug_info="gmodules")
+    @skipIf(bugnumber="FindFirstType() won't work in on-demand", debug_info="ondemand")
     @skipIfWindows # Clang emits type info even with -flimit-debug-info
     # Requires DW_CC_pass_by_* attributes from Clang 7 to correctly call
     # by-value functions.
     @skipIf(compiler="clang", compiler_version=['<', '7.0'])
     def test_one_and_two_debug(self):
         self.build()
         target = self.dbg.CreateTarget(self.getBuildArtifact("a.out"))
 
def test_one_and_two_debug(self):
         self.expect_expr("get_one().member", result_value="124")
         self.expect_expr("get_two().one().member", result_value="124")
         self.expect_expr("get_two().member", result_value="224")
 
         self.expect_expr("shadowed_one.member", result_value="47")
         self.expect_expr("shadowed_one.one", result_value="142")
 
     @skipIf(bugnumber="pr46284", debug_info="gmodules")
+    @skipIf(bugnumber="FindFirstType() won't work in on-demand", debug_info="ondemand")
     @skipIfWindows # Clang emits type info even with -flimit-debug-info
     # Requires DW_CC_pass_by_* attributes from Clang 7 to correctly call
     # by-value functions.
     @skipIf(compiler="clang", compiler_version=['<', '7.0'])
     def test_two_debug(self):
         self.build(dictionary=dict(STRIP_ONE="1"))
         target = self.dbg.CreateTarget(self.getBuildArtifact("a.out"))
 
def test_two_debug(self):
 
         self.expect("expr get_one().member", error=True,
                 substrs=["calling 'get_one' with incomplete return type 'result::One'"])
         self.expect("expr get_two().one().member", error=True,
                 substrs=["calling 'one' with incomplete return type 'result::One'"])
         self.expect_expr("get_two().member", result_value="224")
 
     @skipIf(bugnumber="pr46284", debug_info="gmodules")
+    @skipIf(bugnumber="FindFirstType() won't work in on-demand", debug_info="ondemand")
     @skipIfWindows # Clang emits type info even with -flimit-debug-info
     # Requires DW_CC_pass_by_* attributes from Clang 7 to correctly call
     # by-value functions.
     @skipIf(compiler="clang", compiler_version=['<', '7.0'])
     def test_one_debug(self):
         self.build(dictionary=dict(STRIP_TWO="1"))
         target = self.dbg.CreateTarget(self.getBuildArtifact("a.out"))
 

lldb/test/API/functionalities/show_location/TestShowLocationDwarf5.py

 import lldb
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test.decorators import *
 
 # This test checks that source code location is shown correctly
 # when DWARF5 debug information is used.
 
 class TestTargetSourceMap(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
     @skipIf(archs="aarch64", oslist="linux",
             bugnumber="https://bugs.llvm.org/show_bug.cgi?id=44180")
+    @skipIf(debug_info="ondemand",
+            bugnumber="yaml generated binary does not have symbol table")
     def test_source_map(self):
         # Set the target soure map to map "./" to the current test directory.
         yaml_path = os.path.join(self.getSourceDir(), "a.yaml")
         obj_path = self.getBuildArtifact('a.out')
         self.yaml2obj(yaml_path, obj_path)
 
         # Create a target with the object file we just created from YAML
         target = self.dbg.CreateTarget(obj_path)
 
         # Check we are able to show the locations properly.
         self.expect("b main", VALID_BREAKPOINT_LOCATION,
                     substrs=['main + 13 at test.cpp:2:3, address = 0x000000000040052d'])
 
         self.expect("b foo", VALID_BREAKPOINT_LOCATION,
                     substrs=['foo() + 4 at test.cpp:6:1, address = 0x0000000000400534'])

lldb/test/API/functionalities/type_get_module/TestTypeGetModule.py

def find_type(self, type_list, name):
                 result = type
                 break
 
             index += 1
 
         self.assertTrue(result.IsValid())
         return result
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="Can't find type in on-demand mode")
     def test(self):
         self.build()
         target  = lldbutil.run_to_breakpoint_make_target(self)
         exe_module = self.find_module(target, 'a.out')
 
         num_comp_units = exe_module.GetNumCompileUnits()
         self.assertGreaterEqual(num_comp_units, 3)
 
         comp_unit = self.find_comp_unit(exe_module, 'compile_unit1.c')
         cu_type = self.find_type(comp_unit.GetTypes(), 'compile_unit1_type')
         self.assertEqual(exe_module, cu_type.GetModule())
-        
         comp_unit = self.find_comp_unit(exe_module, 'compile_unit2.c')
         cu_type = self.find_type(comp_unit.GetTypes(), 'compile_unit2_type')
         self.assertEqual(exe_module, cu_type.GetModule())

lldb/test/API/lang/c/enum_types/TestEnumTypes.py

def test_command_line(self):
                 "frame variable day",
                 'check for valid enumeration value',
                 substrs=[enum_value])
             lldbutil.continue_to_breakpoint(self.process(), bkpt)
 
     def check_enum_members(self, members):
         name_matches = ["Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday", "Sunday", "kNumDays"]
         value_matches = [-3, -2, -1, 0, 1, 2, 3, 4]
-        
         # First test that the list of members from the type works
         num_matches = len(name_matches)
         self.assertEqual(len(members), num_matches, "enum_members returns the right number of elements")
         for idx in range(0, num_matches):
             member = members[idx]
             self.assertTrue(member.IsValid(), "Got a valid member for idx: %d"%(idx))
             self.assertEqual(member.name, name_matches[idx], "Name matches for %d"%(idx))
             self.assertEqual(member.signed, value_matches[idx], "Value matches for %d"%(idx))
-        
+    @skipIf(debug_info="ondemand",
+            bugnumber="Can't find type in on-demand mode")
     def test_api(self):
         """Test that the SBTypeEnumMember API's work correctly for enum_test_days"""
         self.build()
         target = lldbutil.run_to_breakpoint_make_target(self)
 
         types = target.FindTypes("enum_test_days")
         self.assertEqual(len(types), 1, "Found more than one enum_test_days type...")
         type = types.GetTypeAtIndex(0)
 
         # First check using the Python list returned by the type:
         self.check_enum_members(type.enum_members)
 
         # Now use the SBTypeEnumMemberList.
         member_list = type.GetEnumMembers()
         self.check_enum_members(member_list)
 
         # Now check that the by name accessor works:
         for member in member_list:
             name = member.name
             check_member = member_list[name]
             self.assertTrue(check_member.IsValid(), "Got a valid member for %s."%(name))
             self.assertEqual(name, check_member.name, "Got back the right name")
             self.assertEqual(member.unsigned, check_member.unsigned)
-

lldb/test/API/lang/c/global_variables/TestGlobalVariables.py

def test_c_global_variables(self):
             substrs=[
                 'g_marked_spot.y',
                 '21'])
         self.expect(
             "target variable g_marked_spot.y",
             VARIABLES_DISPLAYED_CORRECTLY,
             matching=False,
             substrs=["can't be resolved"])
-

lldb/test/API/lang/c/shared_lib/TestSharedLib.py

def common_test_expr(self, preload_symbols):
                 "(foo)",
                 "(sub_foo)",
                 "other_element = 3"])
 
         self.expect(
             "expression GetMeASubFoo(my_foo_ptr)",
             startstr="(sub_foo *) $")
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="debug info is expected to not load for shared library")
     def test_expr(self):
         """Test that types work when defined in a shared library and forward-declared in the main executable"""
         self.common_test_expr(True)
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="debug info is expected to not load for shared library")
     def test_expr_no_preload(self):
         """Test that types work when defined in a shared library and forward-declared in the main executable, but with preloading disabled"""
         self.common_test_expr(False)
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="debug info is expected to not load for shared library")
     @expectedFailure("llvm.org/PR36712")
     def test_frame_variable(self):
         """Test that types work when defined in a shared library and forward-declared in the main executable"""
         self.build()
         self.common_setup()
 
         # This should display correctly.
         self.expect(

lldb/test/API/lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py

 """Test that types defined in shared libraries with stripped symbols work correctly."""
 
 
 
 import unittest2
 import lldb
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 
 class SharedLibStrippedTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="debug info is expected to not load for shared library")
     @expectedFailureAll(oslist=["windows"])
     def test_expr(self):
         """Test that types work when defined in a shared library and forwa/d-declared in the main executable"""
         if "clang" in self.getCompiler() and "3.4" in self.getCompilerVersion():
             self.skipTest(
                 "llvm.org/pr16214 -- clang emits partial DWARF for structures referenced via typedef")
 
         self.build()
         self.common_setup()
 
         # This should display correctly.
         self.expect(
             "expression --show-types -- *my_foo_ptr",
             VARIABLES_DISPLAYED_CORRECTLY,
             substrs=[
                 "(foo)",
                 "(sub_foo)",
                 "other_element = 3"])
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="debug info is expected to not load for shared library")
     @expectedFailureAll(oslist=["windows"])
     @expectedFailure("llvm.org/PR36712")
     def test_frame_variable(self):
         """Test that types work when defined in a shared library and forward-declared in the main executable"""
         self.build()
         self.common_setup()
 
         # This should display correctly.

lldb/test/API/lang/c/sizeof/TestCSizeof.py

 import lldb
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 class TestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="type info is not loaded by default in on-demand mode")
     def test(self):
         self.build()
         self.createTestTarget()
 
         # Empty structs are not allowed in C, but Clang/GCC allow them and
         # give them a size of 0.
         self.expect_expr("sizeof(Empty) == sizeof_empty", result_value="true")
         self.expect_expr("sizeof(EmptyMember) == sizeof_empty_member", result_value="true")
         self.expect_expr("sizeof(SingleMember) == sizeof_single", result_value="true")
         self.expect_expr("sizeof(PaddingMember) == sizeof_padding", result_value="true")

lldb/test/API/lang/c/unicode/TestUnicodeSymbols.py

 # coding=utf8
 import lldb
 from lldbsuite.test.lldbtest import *
 import lldbsuite.test.lldbutil as lldbutil
 from lldbsuite.test.decorators import *
 
 
 class TestUnicodeSymbols(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
+    @skipIf(debug_info="ondemand",
+            bugnumber="type info is not loaded by default in on-demand mode")
     @skipIf(compiler="clang", compiler_version=['<', '7.0'])
     def test_union_members(self):
         self.build()
         spec = lldb.SBModuleSpec()
         spec.SetFileSpec(lldb.SBFileSpec(self.getBuildArtifact("a.out")))
         module = lldb.SBModule(spec)
         self.assertTrue(module.IsValid())
         mytype = module.FindFirstType("foobár")
         self.assertTrue(mytype.IsValid())
         self.assertTrue(mytype.IsPointerType())

lldb/test/API/lang/cpp/breakpoint-commands/TestCPPBreakpointCommands.py

def make_breakpoint(self, name, type, expected_num_locations):
         self.assertEqual(
             num_locations, expected_num_locations,
             "Wrong number of locations for '%s', expected: %d got: %d" %
             (name,
              expected_num_locations,
              num_locations))
         return bkpt
 
+    @skipIf(
+        debug_info="ondemand",
+        bugnumber="""
+            Function name breakpoint filter by compile unit would disable
+            searching in symbol table.
+            """,
+    )
     def test_cpp_breakpoint_cmds(self):
         """Test a sequence of breakpoint command add, list, and delete."""
         self.build()
 
         exe = self.getBuildArtifact("a.out")
 
         # Create a target from the debugger.
 

lldb/test/API/lang/cpp/enum_types/TestCPP11EnumTypes.py

def check_enum(self, suffix):
 
         if unsigned:
             self.expect_expr("var_below_" + suffix, result_type=enum_name, result_value="199")
             self.expect_expr("var_above_" + suffix, result_type=enum_name, result_value="203")
         else:
             self.expect_expr("var_below_" + suffix, result_type=enum_name, result_value="-3")
             self.expect_expr("var_above_" + suffix, result_type=enum_name, result_value="1")
 
+    @skipIf(
+        debug_info="ondemand",
+        bugnumber="Type lookup is expected to fail in on-demand mode",
+    )
     @skipIf(dwarf_version=['<', '4'])
     def test(self):
         self.build()
         target = self.dbg.CreateTarget(self.getBuildArtifact("a.out"))
         self.check_enum("uc")
         self.check_enum("c")
         self.check_enum("us")
         self.check_enum("s")
         self.check_enum("ui")
         self.check_enum("i")
         self.check_enum("ul")
         self.check_enum("l")
         self.check_enum("ull")
         self.check_enum("ll")

lldb/test/API/lang/cpp/namespace_definitions/TestNamespaceDefinitions.py

 """Test that forward declarations don't cause bogus conflicts in namespaced types"""
 
 
 
 import unittest2
 import lldb
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test.decorators import *
 import lldbsuite.test.lldbutil as lldbutil
 
 
 class NamespaceDefinitionsTestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
+    @skipIf(
+        debug_info="ondemand",
+        bugnumber="Type in shared library is not parsed in on-demand mode",
+    )
     # See also llvm.org/pr28948
     @expectedFailureAll(
         bugnumber="llvm.org/pr50814",
         compiler="gcc")
     @expectedFailureAll(
         bugnumber="llvm.org/pr28948",
         oslist=['linux'], compiler="gcc", archs=['arm','aarch64'])
     @expectedFailureAll(oslist=["windows"])

lldb/test/API/lang/cpp/sizeof/TestCPPSizeof.py

 import lldb
 from lldbsuite.test.decorators import *
 from lldbsuite.test.lldbtest import *
 from lldbsuite.test import lldbutil
 
 class TestCase(TestBase):
 
     mydir = TestBase.compute_mydir(__file__)
 
+    @skipIf(
+        debug_info="ondemand",
+        bugnumber="Type is not parsed in on-demand mode",
+    )
     def test(self):
         self.build()
         self.createTestTarget()
 
         # Empty structs/classes have size 1 in C++.
         self.expect_expr("sizeof(Empty) == sizeof_empty", result_value="true")
         self.expect_expr("sizeof(EmptyClass) == sizeof_empty_class", result_value="true")
         self.expect_expr("sizeof(EmptyClassAligned) == sizeof_empty_class_aligned",
                          result_value="true")
         self.expect_expr("sizeof(ClassEmptyMember) == sizeof_class_empty_member",
                          result_value="true")
         self.expect_expr("sizeof(SingleMember) == sizeof_single", result_value="true")
         self.expect_expr("sizeof(SingleMemberClass) == sizeof_single_class", result_value="true")
         self.expect_expr("sizeof(PaddingMember) == sizeof_padding", result_value="true")
         self.expect_expr("sizeof(PaddingMemberClass) == sizeof_padding_class", result_value="true")

lldb/test/API/lang/cpp/template-function/TestTemplateFunctions.py

def do_test_template_function(self, add_cast):
           self.expect_expr("d1 << d2", result_type="bool", result_value="true")
           self.expect_expr("d1 == d2", result_type="bool", result_value="true")
 
     @skipIfWindows
     def test_template_function_with_cast(self):
         self.do_test_template_function(True)
 
     @skipIfWindows
-    @expectedFailureAll(debug_info=["dwarf", "gmodules", "dwo"])
+    @expectedFailureAll(debug_info=["dwarf", "gmodules", "dwo", "ondemand"])
     def test_template_function_without_cast(self):
         self.do_test_template_function(False)

lldb/test/Shell/SymbolFile/OnDemand/Inputs/basic.cpp

+int bar(int x, int y) { return x + y + 5; }
+
+int foo(int x, int y) { return bar(x, y) + 12; }
+
+int main(int argc, char **argv) { return foo(33, 78); }

lldb/test/Shell/SymbolFile/OnDemand/Inputs/lib.h

+#ifndef foo_h__
+#define foo_h__
+
+extern void foo(void);
+
+#endif // foo_h__

lldb/test/Shell/SymbolFile/OnDemand/Inputs/lib.cpp

+#include <stdio.h>
+
+int global_foo = 321;
+void foo(void) { puts("Hello, I am a shared library"); }

lldb/test/Shell/SymbolFile/OnDemand/Inputs/shared.cpp

+#include "lib.h"
+#include <stdio.h>
+
+int global_shared = 897;
+int main(void) {
+  puts("This is a shared library test...");
+  foo();
+  return 0;
+}

lldb/test/Shell/SymbolFile/OnDemand/shared-lib-globals.test

+# Test shows that global variables works as expected in multiple shared libraries with LLDB on demand symbol loading.
+# The global varialbes in non-hydrated shared library libFoo.so can't be found.
+
+# REQUIRES: system-linux
+
+# RUN: mkdir -p %t
+# RUN: cd %t
+# RUN: %clang_host -shared -g -fpic %p/Inputs/lib.cpp -o libFoo.so
+# RUN: %clang_host -L%t -g -o shared.out %p/Inputs/shared.cpp -lFoo -Wl,-rpath=%t
+# RUN: %lldb -b -O "settings set symbols.load-on-demand true" -s %s shared.out | FileCheck %s
+
+breakpoint list
+# CHECK: No breakpoints currently set
+
+breakpoint set -f shared.cpp -l 8
+# CHECK: at shared.cpp:8
+
+breakpoint list
+# CHECK: file = 'shared.cpp', line = 8
+
+run
+# CHECK: stop reason = breakpoint
+# CHECK: at shared.cpp:8
+
+target variable --shlib shared.out
+# CHECK: (int) ::global_shared = 897
+
+target variable --shlib libFoo.so
+# CHECK-NOT: global_foo

lldb/test/Shell/SymbolFile/OnDemand/shared-lib.test

+# Test shows that source line breakpoint works in multiple shared libraries with LLDB on demand symbol loading.
+
+# REQUIRES: system-linux
+
+# RUN: mkdir -p %t
+# RUN: cd %t
+# RUN: %clang_host -shared -g -fpic %p/Inputs/lib.cpp -o libFoo.so
+# RUN: %clang_host -L%t -g -o shared.out %p/Inputs/shared.cpp -lFoo -Wl,-rpath=%t
+# RUN: %lldb -b -O "settings set symbols.load-on-demand true" -s %s shared.out | FileCheck %s
+
+breakpoint list
+# CHECK: No breakpoints currently set
+
+breakpoint set -f lib.cpp -l 4
+# CHECK: where = {{.*}}`foo() + {{.*}} at lib.cpp:4
+
+breakpoint list
+# CHECK: file = 'lib.cpp', line = 4
+
+run
+# CHECK: stop reason = breakpoint
+# CHECK: foo() at lib.cpp:4
+
+bt
+# CHECK: {{.*}}`foo() at lib.cpp:4
+# CHECK: at shared.cpp:7
+
+breakpoint set -f shared.cpp -l 8
+# CHECK: at shared.cpp:8
+
+breakpoint list
+# CHECK: file = 'lib.cpp', line = 4
+# CHECK: file = 'shared.cpp', line = 8
+
+continue
+# CHECK: stop reason = breakpoint
+# CHECK: at shared.cpp:8

lldb/test/Shell/SymbolFile/OnDemand/source-breakpoint.test

+# Test shows that source line breakpoint works with LLDB on demand symbol loading.
+
+# RUN: mkdir -p %t
+# RUN: cd %t
+# RUN: %build %p/Inputs/basic.cpp -o basic.out
+# RUN: %lldb -b -O "settings set symbols.load-on-demand true" -s %s basic.out | FileCheck %s
+
+breakpoint list
+# CHECK: No breakpoints currently set
+
+breakpoint set -f basic.cpp -l 1
+# CHECK: where = {{.*}}`bar(int, int) + {{.*}} at basic.cpp:1
+
+breakpoint list
+# CHECK: file = 'basic.cpp'
+
+run
+# CHECK: stop reason = breakpoint
+
+bt
+# CHECK: {{.*}}`bar(x=33, y=78) at basic.cpp:1
+# CHECK: {{.*}}`foo(x=33, y=78) at basic.cpp:3
+# CHECK: {{.*}}`main(argc=1, argv={{.*}}) at basic.cpp:5

lldb/test/Shell/SymbolFile/OnDemand/symbolic-breakpoint.test

+
+# Test shows that symbolic function breakpoint works with LLDB on demand symbol loading.
+
+# RUN: mkdir -p %t
+# RUN: cd %t
+# RUN: %build %p/Inputs/basic.cpp -o basic.out
+# RUN: %lldb -b -O "settings set symbols.load-on-demand true" -s %s basic.out | FileCheck %s
+
+breakpoint list
+# CHECK: No breakpoints currently set
+
+b bar
+# CHECK: where = {{.*}}`bar(int, int) + {{.*}} at basic.cpp:1
+
+breakpoint list
+# CHECK: where = {{.*}}`bar(int, int) + {{.*}} at basic.cpp:1
+
+run
+# CHECK: stop reason = breakpoint
+
+bt
+# CHECK: {{.*}}`bar(x=33, y=78) at basic.cpp:1
+# CHECK: {{.*}}`foo(x=33, y=78) at basic.cpp:3
+# CHECK: {{.*}}`main(argc=1, argv={{.*}}) at basic.cpp:5