FYI The issue came up when I was trying to jitlink gzip: https://people.csail.mit.edu/smcc/projects/single-file-programs/gzip.c Next step would be an isolated test case.
Thu, Oct 1
Aug 23 2020
Looks good. Two bots failed, both unrelated to my change.
Aug 20 2020
Back to the review: I'd like to keep the test for the example and see how the build servers behave. Generally it might be useful to have tests for all the "LLJITWith..." examples right?
Do you think it makes sense to land the patch on the weekend in order to keep the number of people getting annoyed by me breaking their builds at a minimum? :)
I will get back to performance evaluation maybe in a few weeks and sure I am happy to share my progress.
Aug 15 2020
Fix clang-tidy warnings
Aug 14 2020
Fix clang-format issue and rebase
What performance issues did you run in to with threading and performance? I haven't had a chance to look in to that yet.
Test discovery should ignore subdirectories that contain test inputs.
I would like to remove the ThinLtoJIT example. It needs a more decent threading library to speed-up multithreaded compile times and that's easier to do out-of-tree.
This might be a useful (minimal) portion to keep in-tree. What do you think?
Fixed with commit 28e1015e327ec56ac4bf93967d3aa2915b215e36
It broke the BuildingAJIT-Ch5 build in http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/73310
Looking at it now.
Would you mind if I moved yours into a header once my patch is ready?
Aug 13 2020
I didn't see a memory manager in the new TargetProcessControl API (yet). Maybe it can also be migrated there.
Mar 4 2020
Remove assert(Notifiers.find(TrampolineAddr) != Notifiers.end()); from findReexport() -- NotifyResolved may have been issued from a concurrent call-through
Factor out callThroughToSymbol(). Drop example use case from ThinLtoJIT. Format change set.
So the aim here is to enable generation of lazy stubs without having to materialize any IR, right?
Is the idea here that the call-through stub is emitted as weak, and then replaced by discovery with a strong definition if one is found in time? Or have I misunderstood?
Feb 29 2020
Feb 27 2020
I have removed it in b7aa1cc3a43 and replaced it with direct checks on the symbol state, so I think you can discard this.
Thanks for your review Lang. Will land it soon.
Feb 24 2020
Feb 18 2020
Whether the initializers would be re-run is up to the platform.
Feb 16 2020
Will initializer symbols be looked up and run again when calling initialize() a second time on a JITDylib? More specifically, is it possible for a client to break this process down and initialize specific (or simply all uninitialized) MaterializationUnits? Otherwise, wouldn't it mean modules cannot be added (and initialized) to a JITDylib after initializiation (without re-init)?
Feb 12 2020
As expected this doesn't affect my code much so far. Only the symbol promotion changes in CompileOnDemandLayer (see inline comment). What do you think?
Hi Lang, I had a look at the first half of your changes. So far it looks really great!
A few mini notes inline and a larger one that's semi-related. I will continue with the second half and run this by the ThinLtoJIT example asap.
Feb 8 2020
This seems like a nice change for ErrorList.
What are you using it for though?
In my experience, whenever I have used ErrorList it was really because I wanted to be able to report multiple errors, never because I wanted to be able to recover from them. However I think this kind of reporting is better handled by a logging idiom (like ExecutionSession::reportError).
Feb 2 2020
Use numeric substitution blocks to verify the sum of submitted modules is 6
Feb 1 2020
I wrote this in C, compiled to LLVM IR (on macOS with a TOT clang) and removed a lot of bulk manually. Would you recommend to commit the C sources too? For now I put them here: https://github.com/weliveindetail/ThinLtoJitTests
Polish initialization, cleanup, formatting
Add -print-stats command line flag
Jan 31 2020
Move ExplicitMemoryBarrier enum to ThinLtoJIT
Split off ThinLtoInstrumentationLayer::dump() from its dtor
Assume single entries in summary lists after checking module paths to be unique
The blocking single-module parsing function parseModuleFromFile() should use the regular primitives
Drop DiscoveryThread module requests via the definition generator. Instead trigger asynchronous module adding from there.
Let scheduleModuleParsing() process a range of paths with a single lock
Don't risk "post-mortem" access violations. Keep DiscoveryThread attached and join it on shutdown.
Prefer std::atomic fetch_add over compare-exchange in reserveDiscoveryFlags()
Accept ThinLTO global index as input file
bench script: link and disassemble global ThinLTO index file and respect LDFLAGS in static compilation
Jan 26 2020
LGTM! I really like this. I'm all for adding it as another example if you're happy to maintain it.
The code could probably use some extra comments (including pointers to your talk), but I think that would be better dealt with in-tree.
Drop requirement to apply D72301 before this patch
Adjust for emulated-TLS changes in ce2207abaf9a
Jan 19 2020
If you want to build this patch, please apply D72301 before. I will try and make it work without it soon.
Sort PathRank candidates by minimal distance
Don't dispatch materialization for trivial modules to the thread pool
Submit actual module keys and sort getAllModulePaths() by ID
Fix GUID when instrumenting functions with local linkage
Respect environment variables CFLAGS and EXEC_ARGS in the bench script
Jan 18 2020
Indeed, that sounds like a better solution.
Jan 17 2020
Pardon, apparently the last update was missing the actual code changes. Trying again.
Jan 15 2020
Avoid concurrent addModule() calls, improve ranking algorithm in discovery, add multi-threaded bitcode parsing
Jan 14 2020
Jan 13 2020
Make LookaheadLevels and LookupOnAdd configurable
This was fixed with 2cdb18afda8
Jan 12 2020
Hi Stefan. Thanks for posting this! I’m out on vacation this week, but looking forward to reviewing when I get back to the office next week.
DataLayout can be owned by MangleWrapper
Use runAsMain() utility function from ExecutionUtils
Jan 11 2020
I was about to say enjoy your time off and have a look here on Monday: D72563 I guess we don't need it anymore :)
Is there a particular reason to pass lli as the program name instead of the main source file? Not saying that this made a lot of sense in the first place, just wondering.