Looks good. Two bots failed, both unrelated to my change.
http://green.lab.llvm.org/green/job/lldb-cmake/23457/
http://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-expensive/17430/

Back to the review: I'd like to keep the test for the example and see how the build servers behave. Generally it might be useful to have tests for all the "LLJITWith..." examples right?
Do you think it makes sense to land the patch on the weekend in order to keep the number of people getting annoyed by me breaking their builds at a minimum? :)

I will get back to performance evaluation maybe in a few weeks and sure I am happy to share my progress.

Fix clang-tidy warnings

Fix clang-format issue and rebase

In D85974#2218405, @lhames wrote:

What performance issues did you run in to with threading and performance? I haven't had a chance to look in to that yet.

Test discovery should ignore subdirectories that contain test inputs.

I would like to remove the ThinLtoJIT example. It needs a more decent threading library to speed-up multithreaded compile times and that's easier to do out-of-tree.
This might be a useful (minimal) portion to keep in-tree. What do you think?

Fixed with commit 28e1015e327ec56ac4bf93967d3aa2915b215e36

It broke the BuildingAJIT-Ch5 build in http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/73310
Looking at it now.

In D85919#2216825, @lhames wrote:

Would you mind if I moved yours into a header once my patch is ready?

I didn't see a memory manager in the new TargetProcessControl API (yet). Maybe it can also be migrated there.

Remove assert(Notifiers.find(TrampolineAddr) != Notifiers.end()); from findReexport() -- NotifyResolved may have been issued from a concurrent call-through

Factor out callThroughToSymbol(). Drop example use case from ThinLtoJIT. Format change set.

In D75087#1904552, @lhames wrote:

So the aim here is to enable generation of lazy stubs without having to materialize any IR, right?

In D75090#1904569, @lhames wrote:

Is the idea here that the call-through stub is emitted as weak, and then replaced by discovery with a strong definition if one is found in time? Or have I misunderstood?

In D75083#1892549, @lhames wrote:

I have removed it in b7aa1cc3a43 and replaced it with direct checks on the symbol state, so I think you can discard this.

Thanks for your review Lang. Will land it soon.

Whether the initializers would be re-run is up to the platform.

Will initializer symbols be looked up and run again when calling initialize() a second time on a JITDylib? More specifically, is it possible for a client to break this process down and initialize specific (or simply all uninitialized) MaterializationUnits? Otherwise, wouldn't it mean modules cannot be added (and initialized) to a JITDylib after initializiation (without re-init)?

As expected this doesn't affect my code much so far. Only the symbol promotion changes in CompileOnDemandLayer (see inline comment). What do you think?

Hi Lang, I had a look at the first half of your changes. So far it looks really great!
A few mini notes inline and a larger one that's semi-related. I will continue with the second half and run this by the ThinLtoJIT example asap.

In D72176#1834446, @lhames wrote:

This seems like a nice change for ErrorList.

What are you using it for though?
[...]
In my experience, whenever I have used ErrorList it was really because I wanted to be able to report multiple errors, never because I wanted to be able to recover from them. However I think this kind of reporting is better handled by a logging idiom (like ExecutionSession::reportError).

Use numeric substitution blocks to verify the sum of submitted modules is 6

I wrote this in C, compiled to LLVM IR (on macOS with a TOT clang) and removed a lot of bulk manually. Would you recommend to commit the C sources too? For now I put them here: https://github.com/weliveindetail/ThinLtoJitTests

Polish initialization, cleanup, formatting

Refine ThinLtoModuleIndex::selectNextPaths()

Add -print-stats command line flag

Move ExplicitMemoryBarrier enum to ThinLtoJIT

Split off ThinLtoInstrumentationLayer::dump() from its dtor

Assume single entries in summary lists after checking module paths to be unique

The blocking single-module parsing function parseModuleFromFile() should use the regular primitives

Drop DiscoveryThread module requests via the definition generator. Instead trigger asynchronous module adding from there.

Let scheduleModuleParsing() process a range of paths with a single lock

Don't risk "post-mortem" access violations. Keep DiscoveryThread attached and join it on shutdown.

Prefer std::atomic fetch_add over compare-exchange in reserveDiscoveryFlags()

Accept ThinLTO global index as input file

bench script: link and disassemble global ThinLTO index file and respect LDFLAGS in static compilation

In D72486#1839833, @lhames wrote:

LGTM! I really like this. I'm all for adding it as another example if you're happy to maintain it.

The code could probably use some extra comments (including pointers to your talk), but I think that would be better dealt with in-tree.

Drop requirement to apply D72301 before this patch

Adjust for emulated-TLS changes in ce2207abaf9a

If you want to build this patch, please apply D72301 before. I will try and make it work without it soon.

Sort PathRank candidates by minimal distance

Don't dispatch materialization for trivial modules to the thread pool

Submit actual module keys and sort getAllModulePaths() by ID

Fix GUID when instrumenting functions with local linkage

Respect environment variables CFLAGS and EXEC_ARGS in the bench script

Indeed, that sounds like a better solution.

Pardon, apparently the last update was missing the actual code changes. Trying again.

Avoid concurrent addModule() calls, improve ranking algorithm in discovery, add multi-threaded bitcode parsing

Make LookaheadLevels and LookupOnAdd configurable

This was fixed with 2cdb18afda8

In D72486#1814233, @lhames wrote:

Hi Stefan. Thanks for posting this! I’m out on vacation this week, but looking forward to reviewing when I get back to the office next week.

DataLayout can be owned by MangleWrapper

Use runAsMain() utility function from ExecutionUtils

I was about to say enjoy your time off and have a look here on Monday: D72563 I guess we don't need it anymore :)
Is there a particular reason to pass lli as the program name instead of the main source file? Not saying that this made a lot of sense in the first place, just wondering.

Fix: Protect the set of parsed module IDs from concurrent access

Polishing, comments, cosmetics

Make it configurable whether the symbol generator can nugde things into discovery

Make DiscoveryFlagsPerBatch and MemFence settings configurable

Workaround: parsed module requested but waiting in JITDylib::define()