Looks good, thanks!

The Interop.Dia2Lib.dll and the msdiaXXX.dll have to match. Try regsvr32.exe "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Packages\Debugger\msdia120.dll" (or look for msdia120.dll on your PC).

In D62347#1516136, @thakis wrote:

The "modernized" api is a good chunk wordier. Can you outline how composing launching would look like with and without this change?

You're right, I'll reduce it. There are a few unrelated changes that could be done after. I'll have to get back to you regarding the "without this change".

Thanks, I built it, but when I try to use it to load lld.pdb, it raises a "class not registered" exception, despite the fact that I followed the README.md instructions and registered msdia120.dll. DIA registration seems to be a continuing pain point in the Windows development world. :( I'll mess around with it.

Great stuff, thanks for doing this Reid! :) Inherently, this should make things a bit faster, it will also reduce cache misses.
We use a neat tool called Crunchersharp to have a broader overview:

If using -DLLVM_TOOL_LLDB_BUILD it now displays "Using /MD as required by LLDB."
If using -DLLVM_TOOL_LLDB_BUILD -DLLVM_USE_CRT_RELEASE=MT it displays "Disabling /MT as required by LLDB."
Otherwise /MT is the new default.

In D55056#1514451, @rnk wrote:

Previously I passed LLVM_USE_CRT_RELEASE=MT explicitly as part of my build, and I ran into the same problems with LLDB that you did. I'll try to modify this once it lands to warn when the user tries to do that. At the time, I couldn't see how to do it easily.

Reopening with fixes for LLDB.

Could you please move the test to a more approriate location? (ie. clang/trunk/test/Driver/)

Ping! Any further comments?

Ping!

In D60283#1503256, @probinson wrote:

Minor stuff. This solution is surprisingly simple, the main question being (and I don't have an answer) whether increasing the size of PresumedLoc is okay.

Thanks Paul! PresumedLoc seems to be used only as a temporary (on the stack).

Updated again with a different solution. We can't just act on Entry.getFile().hasLineDirectives() because a directive such as #line 100 simply overrides the __LINE__, not __FILE__ (we need to retain and hash the previous __FILE__ in that case). Please see an example here in the IBM documentation.
Also we can't compare by FileID because a LineEntry simply stores a string (called filename) but the buffer for that file isn't stored in the SourceManager.

In addition to the std::mutex issue @BillyONeal mentions, it seems there's a fair amount of "friction"/spinning in concrt.
As a good example, while using WIP patch D55585, about ~15% of time is spent in concrt yielding the CPU on a 6-core PC. And the more cores you have, the worse it gets, up to ~33% of time yielding the CPU on a 36-cores PC (see below).

Ping! Any further suggestions? Thanks in advance for your time.

I've commited the patch, and quoted you as the maker. You should maybe consider asking commit rights if you're planning more changes in the future?

In D61046#1486850, @rnk wrote:

The only remaining change I find questionable is the R600 backend change

Do you have commit access? Should I commit this for you?

In D61496#1489523, @probinson wrote:

Checked-in files should not have CRLF endings, in general (there are a very few exceptions, and these aren't among them).
If the checked-in files have LF endings but your tool checks them out with CRLF then that is a problem with your config, or with the tool.
Adding dos2unix to the RUN lines is the wrong fix, either way.

Changed to use FileCheck instead of grep

So it turns out this is a symlink issue. The file clang/trunk/test/Driver/Inputs/CUDA-symlinks/usr/bin/ptxas has been synchronized on my Windows 10 PC as a regular text file, not a symlink. It looks like TortoiseSVN doesn't implement symlinks. As WSL inherits of my file system, it will not find the symbolic link. I suppose REQUIRES: !system-windows isn't enough for cuda-detect-path.cu, and it would need something like REQUIRES: symlinks along with support in lit. @rnk

Ping! Is this good to go?

Thanks Paul, your solution is even better. I'll apply rL333311 locally - if everything's fine, do you mind if I re-land it again?

Yes the encoding is different. By running https://gist.github.com/zed/5898423 I get:

   Windows 10 cmd shell:                    WSL:
locale(False):     cp1252             locale(False):     UTF-8
device(stdout):    cp850              device(stdout):    UTF-8
stdout.encoding:   utf-8              stdout.encoding:   UTF-8
device(stderr):    cp850              device(stderr):    UTF-8
stderr.encoding:   utf-8              stderr.encoding:   UTF-8
device(stdin):     cp850              device(stdin):     UTF-8
stdin.encoding:    utf-8              stdin.encoding:    UTF-8

I've checked clang/llvm/lld/lldb, this seems to be the only place where raw bytes are encoded in python.

Just a quick heads-up - the cuda-detect-path.cu test fails on WSL/Ubuntu 18.04:

aganea@MTL-BJ842:/mnt/f/svn/buildWSL$ /usr/bin/python3.6 bin/llvm-lit -vv ../clang/test/Driver/cuda-detect-path.cu
llvm-lit: /mnt/f/svn/llvm/utils/lit/lit/llvm/config.py:341: note: using clang: /mnt/f/svn/buildWSL/bin/clang
-- Testing: 1 tests, single process --
FAIL: Clang :: Driver/cuda-detect-path.cu (1 of 1)
******************** TEST 'Clang :: Driver/cuda-detect-path.cu' FAILED ********************

Please let me know if other changes are needed.

I simplfied the patch by moving the PDBInputFile behavior into the existing TypeServerSource class. This makes things more elegant and keeps everything related to debug types into one place.

Adressed some of the comments.

Steal one bit from PresumedLoc::Column as suggested by @rnk.
Ping @rsmith !

In D60095#1472111, @rnk wrote:

In D60095#1471063, @ruiu wrote:

InputFile abstracts the concept of a file containing symbols. In each parse() function, we parse a symbol table and then insert symbols to the symbol table. parse() is part of a symbol resolution process step.

Since PDBInputFile doesn't contain any symbols, it doesn't fit to that concept, and it doesn't seem you need to parse them while we parse other types of input files.

So, why don't you just add a pdb file to some global list, and after resolving all symbols, parse them in a simple for loop? I guess that's conceptually much simpler.

IIUC, the point of this change is to start loading types in parallel as early as possible. The suggestion to accumulate the inputs on the side and wait until after symbol resolution won't exploit all the parallelism that's available.

Ping! I know Reid is OOO this week and Zachary might not be available for reviewing. @ruiu could you please take a quick look at this? Thank you in advance for your time.

Simplified as requested.

Thanks for the fix!

Makes sense, thank you for fixing this!

That is strange. As you can’t mix /Yc and /Yu on the command line, MSBuild should issue two different calls to clang-cl, one with /Yc and one /Yu /MP. Can you check with ProcMon that commands are issued in the right order?

I'll try profiling several solution on a large unity/jumbo file and get back with some results.

In D60283#1456497, @thakis wrote:

See http://llvm-cs.pcc.me.uk/tools/clang/lib/Frontend/VerifyDiagnosticConsumer.cpp#756 for a "is same file" example. I'm not sure adding a bool to PresumedLoc is worth it for this.