Create a new symbol file plugin for cross-platform PDB symbolization

[zturner]

Summary

The existing SymbolFilePDB only works on Windows, as it is written against a closed-source Microsoft SDK that ships with their debugging tools.

There are several reasons we want to bypass this and go straight to the bits of the PDB, but just to list a few:

1. More room for optimization. We can't see inside the implementation of the Microsoft SDK, so we don't always know if we're doing things in the most efficient way possible. For example, setting a breakpoint on main of a big program currently takes several seconds. With the implementation here, the time is unnoticeable.

2. We want to be able to symbolize Windows minidumps even if not on Windows. Someone should be able to debug Windows minidumps as if they were on Windows, given that no running process is necessary.

This patch is a very crude first attempt at filling out some of the basic pieces.

I've implemented FindFunctions, ParseCompileUnitLineTable, and ResolveSymbolContext for a limited subset of possible parameter values. Basically, enough to get it to display me something nice for the breakpoint location.

The test I've added, incidentally, is able to load a .exe into LLDB and set a breakpoint by name even on non-Windows. This is really cool because it allows us to run the test on every platform thusly increasing our test coverage, and also serving as proof that this actually does work anywhere. Of course it doesn't make sense to actually run this program on non Windows and you'll get an error if you try to do so, but all we try to do with this test is set the breakpoint and verify that symbols are resolved correctly.

I'll add many more tests of this nature later.

For now, this plugin is enabled always on non-Windows, and by setting the environment variable LLDB_USE_NATIVE_PDB_READER=1 on Windows. Eventually, once it's at parity with the Windows implementation, we'll delete the Windows DIA-based implementation.

[zturner]

Fix the test. I accidentally clang-formatted it which messed up the check lines. Turn clang-format off for this file.

[labath]

I can't say I have done a particularly thorough review, but code seems well structured, and it looks like I could make sense of it if I went through the effort of figuring out what various pdb record types mean. These are the things that came to mind while looking at this.

Also, hooray for being able to open pdb files on non-windows.

lldb/lit/SymbolFile/NativePDB/simple-breakpoints.cpp

1 ↗

(On Diff #168723)

It seems llvm has a special .clang-format file in the test folder which sets ColumnLimit: 0. Maybe we should do the same?

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

37–38 ↗

(On Diff #168723)

Maybe propagate this error to the caller and have the caller log the error when ignoring it?

lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp

53 ↗

(On Diff #168723)

Is this actually used?

lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

78–85 ↗

(On Diff #168723)

I am wondering whether it wouldn't be better to use an lldb setting instead of an env var (plugin.symbol-file.pdb.native ?). For this to work, we'd need to delay to decision of which plugin to use to until CreateInstance time, but that shouldn't be too hard.

[aleksandr.urakov]

It's great! Especially thanks for comments!

I haven't looked the whole patch yet, I'll finish it tomorrow, ok?

lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

307 ↗

(On Diff #168723)

Why shouldn't we pass a type here? Isn't it unrelated to the new plugin?

[stella.stamenova]

I looked through the code and it mostly looks good. I had a couple of nits and a couple of actual questions (especially about asserts). I agree with Aleksandr the comments are going to make a big difference in maintenance/improvements going forward (until they eventually get out of sync with the code...).

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

149 ↗

(On Diff #168723)

Nit: I've noticed you initialized the variables some places, but not all places. I'm all for initializing always.

153 ↗

(On Diff #168723)

If the assert fails, does this mean that the symbol map won't be built correctly? Should it be an actual error instead of just an assert?

lldb/source/Plugins/SymbolFile/NativePDB/PdbSymUid.h

131 ↗

(On Diff #168723)

This assert looks like it really should be an error instead of just an assert.

lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp

341 ↗

(On Diff #168723)

Did you mean to leave this comment here? Maybe with a FIXME?

592 ↗

(On Diff #168723)

It looks like a couple more functions should have FIXME comments as they are not yet implemented.

[zturner]

lldb/lit/SymbolFile/NativePDB/simple-breakpoints.cpp

1 ↗

(On Diff #168723)

We could do the same, but it won't remove the need to turn clang-format off. Especially when we're running CHECk statements against line numbers in the file, we don't want clang-format modifying the file. Even if we set column limit to 0 here, clang-format will write main on 1 line as int main(int argc, char **argv) { return 0; }. That's an interesting test case, but it's a different test case than what we have here where we want the line number to point to the first line of the function that *isn't* the declaration. It might even be interesting to have a third test case for

int main(int argc, char **argv)
{
  return 0;
}

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

37–38 ↗

(On Diff #168723)

These are a bit tricky. In all cases, there is only possible thing the caller can do, which is to give up. I don't think the caller particularly cares about the details of the error that occurred, the only way to handle it is to say "we're not going to use this plugin". So I think it's fine to consume the error and just return a generic failure here.

153 ↗

(On Diff #168723)

I think the only way this assert would fail is if two symbols in the PDB were indicated as having the same address. The only time we call BuildAddrToSymbolMap is if cci.m_symbols_by_va.empty() is true. Then, we only insert items into it at the bottom of this loop. So over the course of the loop, the same address would have had to have been added already.

If the debug info says two symbols have the same address, there isn't much we can do other than to change this to some kind of multimap, but that just pushes the decision of how to handle the incorrect debug info to the caller, which doesn't have any better strategy for resolving this kind of ambiguity. That said, down below we call insert which assumes the item is not already in the map. We could change that to a try_emplace, then assert that the insert succeeded. This would have the effect of asserting in the same case that this asserts, but the program still "working" (it would just drop anything other than the first symbol with a given address).

lldb/source/Plugins/SymbolFile/NativePDB/PdbSymUid.h

131 ↗

(On Diff #168723)

This is a record which comes from the debug info, which technically is user input. On the other hand, we need some layer at which we can assume debug info has been sanitized. Propagating errors through multiple layers of infrastructure code just complicates the control flow and doesn't really add any value. The idea is that you should have sanitized the record before you call this function.

As it turns out, I only call this function in one place, and I didn't sanitize the input yet, partly because there are only two existing implementations of things that can write PDB files, and neither one of them will emit buggy CodeView records here. But if you think it's better, I can add a sanitization check to the caller.

lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp

53 ↗

(On Diff #168723)

Nope, thanks! Neither is some of the other stuff. I started by copying all of SymbolFilePDB.cpp and then removing code. But I missed this include. I'll remove other dead code as well and re-upload.

341 ↗

(On Diff #168723)

My bad. This was from the SymbolFilePDB.cpp which I started with as the template for implementing this file. And I didn't remove this comment.

lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

78–85 ↗

(On Diff #168723)

I thought about this but decided against it. There are two reasons I didn't want to do this:

1. On non-Windows platforms, the setting doesn't make any sense. There is no other option.
2. It doesn't seem like something we want to "support" opting into or out of. It's either in experimental / development mode, like right now, in which case the only reason you have to use it is if you're working on it. Or it's complete, in which case we just delete the old one and make everything use the new one with no alternative.

So I couldn't come up with a valid use case to justify having it be a setting.

307 ↗

(On Diff #168723)

Nice catch! We actually should pass a type here. The reason I'm not doing it is because that would require implementing the PDBASTParser class. Obviously we need to do that eventually, but for now it wasn't necessary in order for LLDB to not complain. So I was trying to keep the patch small and just implement the minimum amount needed for *something* to work. But yes, we will need to do this.

[aleksandr.urakov]

lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

307 ↗

(On Diff #168723)

But here is the SymbolFilePDB part, not SymbolFileNativePDB, and PDBASTParser is already implemented for it? And we even retrieve the type in the line 297... If I understand right, it will be never called if LLDB_USE_NATIVE_PDB_READER is on?

[zturner]

lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

307 ↗

(On Diff #168723)

Oh, yea you are right. I changed this in SymbolFilePDB because I was testing whether LLDB would still display something nice. THen I forgot to change it back. So yea, I'll revert this back to how it was. Sorry about that

[aleksandr.urakov]

lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

307 ↗

(On Diff #168723)

That's ok :)

[stella.stamenova]

lldb/source/Plugins/SymbolFile/NativePDB/PdbSymUid.h

131 ↗

(On Diff #168723)

Yes, I think adding a sanitization check to the caller is a good idea. Detecting the error earlier (and when it's potentially actionable) should make maintenance down the line easier.

[zturner]

Addressed all issues pointed out by various people, and added a few more tests. There's only a limited amount of stuff we can do without a running process, so I'll probably have to resort to lldb-test soon to be able to print more detailed information.

[zturner]

lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp

571–572 ↗

(On Diff #168871)

@stella.stamenova I added the sanitization check here for the record. In theory we can provide an overload of this function for every record type that has special requirements. For now I've only implemented the function for this record type and all other types return true.

[stella.stamenova]

Thanks!

[lemo]

Yay! Nice to see this is coming along, thanks Zach!

Looks good to me, a few comments inline.

lldb/lit/SymbolFile/NativePDB/simple-breakpoints.cpp

11 ↗

(On Diff #168723)

just an idea: add a negative test as well? (break -s n non_existent_function)

12 ↗

(On Diff #168723)

Is the offset and address going to be stable enough to rely on it for testing?

lldb/source/Plugins/SymbolFile/NativePDB/CMakeLists.txt

12 ↗

(On Diff #168723)

nit: fix indentation

lldb/source/Plugins/SymbolFile/NativePDB/CompileUnitIndex.cpp

84 ↗

(On Diff #168723)

what's the meaning of this check?

103 ↗

(On Diff #168723)

is the intention to count unique kinds? can we have duplicates?

121 ↗

(On Diff #168723)

is C++ 17 available? structured bindings can make this more readable:

const auto& [it, inserted] = ...;
if (!inserted)

...

lldb/source/Plugins/SymbolFile/NativePDB/CompileUnitIndex.h

34 ↗

(On Diff #168723)

add a brief description of the struct?

71 ↗

(On Diff #168723)

ditto: class description comment?

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

34 ↗

(On Diff #168723)

lldbassert() ?

(same comment for the rest of assert()s in the CR)

37–38 ↗

(On Diff #168723)

how do you feel about adding case-specific logging?

122 ↗

(On Diff #168723)

class? (not just for the notation clarity, but there's no reason to make the visitor members public, right?

149 ↗

(On Diff #168723)

+1

151 ↗

(On Diff #168723)

C++ 17 structured bindings?

186 ↗

(On Diff #168723)

but is that what we really want? shouldn't we be looking for the "inner-most" range which contains the address?

191 ↗

(On Diff #168723)

Ok, I see what we're doing. Wouldn't make more sense to iterate backward? (efficiency wise and more importantly to locate the most "inner" range, right?)

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.h

58 ↗

(On Diff #168723)

initialize these raw pointers in-class (NSDMI) ?

llvm::pdb::DbiStream *m_dbi = nullptr;

95 ↗

(On Diff #168723)

comment?

lldb/source/Plugins/SymbolFile/NativePDB/PdbSymUid.h

162 ↗

(On Diff #168723)

should't the last argument be sizeof(result) ?

also, what is the meaning of this opaque id?

lldb/source/Plugins/SymbolFile/NativePDB/PdbUtil.cpp

124 ↗

(On Diff #168723)

I'd prefer a dedicated SegmentAndOffset struct rather than the generic pair<>, what do you think?

lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp

86 ↗

(On Diff #168723)

add logging?

116 ↗

(On Diff #168723)

logging, please?

128 ↗

(On Diff #168723)

ditto: logging? (very valuable for someone trying to troubleshoot symbol loading issues w/o debugging lldb)

131 ↗

(On Diff #168723)

sizeof(guid) ?

[zturner]

@lemo Thanks for the detailed review! I'll fix the suggestions and upload a new patch in a bit.

lldb/lit/SymbolFile/NativePDB/simple-breakpoints.cpp

12 ↗

(On Diff #168723)

No, but it sounds like you're looking at an old version of the diff. I removed this regex'ed all the address checks out.

lldb/source/Plugins/SymbolFile/NativePDB/CompileUnitIndex.cpp

121 ↗

(On Diff #168723)

Sadly no, LLVM is still on C++11.

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

34 ↗

(On Diff #168723)

lldbassert is nice to validate things that you expect should happen but want to log an error if the condition is not held. assert is better when you want to guarantee invariants of your API. In this case I use assert because the precondition of this function is "file is non-null". lldbassert wouldn't help here, because the very next line of code is to just dereference the pointer and we'd crash anyway.

37–38 ↗

(On Diff #168723)

Logging is possible. But I think there's such a thing as too verbose logging. There's a balance between code simplicity and providing useful information in the log file. This code is called at a stage when LLDB still hasn't even figured out that the PDB symbol file plugin is ultimately the right plugin for the job. It's *asking* us if we can handle it. So if we log an error and say "Invalid file, PDB doesn't have a DBI stream" it seems too verbose in my opinion. Imagine if LLDB tried the DWARF plugin first (because it just goes down the list of looking for anyone who claims they can handle this file), and DWARF logged an error saying "Invalid DWARF debug information format", but it's not even DWARF in the first place.

I don't know what the best thing to do here is other than to say "we can't handle this", for which a single nullptr return value is sufficient.

122 ↗

(On Diff #168723)

Well it's a local class defined inside of a member function, so in a sense it's already "private". But you're right, it could be a class.

186 ↗

(On Diff #168723)

The purpose of this function is to find *all* symbols that occupy a given address. Imagine a function which contains a nested block. Just because you call this function with an address inside the block doesn't necessarily mean you want the block. If we had to choose just 1 symbol to return, I'd rather choose the outermost symbol, because from there it's easier to determine all the inner symbols. But going the other direction is more complicated. This function however just returns a list of all of them and lets the caller decide what to do with that information. We could, for example, build something on top of this to allow more granular selection, but that would still need to call into this and then filter the list down.

191 ↗

(On Diff #168723)

See above, but most inner range is not necessarily what we're looking for. In this patch, for example, the goal is to find the function symbol. So if this returned the inner most range, it might return a block symbol and we'd still need to do more work to find the function symbol. So this function just returns every symbol that contains the address, and lets the caller filter that down further (if desired).

lldb/source/Plugins/SymbolFile/NativePDB/PdbSymUid.h

162 ↗

(On Diff #168723)

sizeof(result) == sizeof(m_uid). This can even be static_asserted, due to the construction of the PdbSymUid class. See the comment at the top of this header file for a description.

The idea is that LLDB represents symbols, types, compilands, and everything else using a 64-bit integer. So we partition a 64-bit integer into a 1-byte tag field that says what kind of symbol it is, and then 48 bytes of type-specific data. When LLDB asks for a 64-bit unique identifier for a particular thing, we call this function, which is basically treating the 64-bit structure as a 64-bit integer. The 1-byte tag field guarantees uniqueness across all symbol types.

lldb/source/Plugins/SymbolFile/NativePDB/PdbUtil.cpp

124 ↗

(On Diff #168723)

Reasonable, I'll fix this.

[zturner]

Addressed suggestions from Leonard.

[lemo]

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

34 ↗

(On Diff #168723)

I believe that lldbassert has been fixed (lldb_private::lldb_assert() unconditionally ends with calling abort())

[zturner]

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

34 ↗

(On Diff #168723)

Hmm, that's new. So it looks to me like the only difference between assert and lldbassert is that you can't turn lldbassert off in release mode. Do I have that right? Do we care about that?

[lemo]

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

34 ↗

(On Diff #168723)

my main concern is consistency - having multiple flavors of assert can be confusing (one should not have to think too much about the subtleties around the various flavors of "assert")

so my 2c: use lldbassert() in the LLDB codebase

[zturner]

Use lldbassert. The main reason I didn't use this originally is because it didn't used to be a hard assert. But since it seems like it is now, I've no problem standardizing on this.

(FWIW, I actually think we should call llvm::sys::report_fatal_error instead of abort, but we can have that discussion another time.

[labath]

lldb/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

34 ↗

(On Diff #168723)

It has been "fixed", though I'm not sure if people who were advocating for "lldbassert" actually saw the "fix".

37–38 ↗

(On Diff #168723)

My main reason for suggesting this was simplicity. Having this function return an error means you can turn a bunch of:

if (!Foo) {
  consumeError(Foo.takeError());
  return nullptr;
}

into

if (!Foo)
  return Foo.takeError();

Then, in the caller (which is in a better position to determine what to do with the error imho), you can discard the error with a single consumeError call (or a log message, or whatever).

153 ↗

(On Diff #168723)

It was my understanding we don't want malformed debug info to ever crash the debugger: https://reviews.llvm.org/D18646#388424 (last paragraph).

lldb/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

78–85 ↗

(On Diff #168723)

That's fine. I'll leave that up to you.

[aleksandr.urakov]

Thank you!

[stella.stamenova]

Not sure if you've noticed this yet, but a couple of the new tests are failing on Linux and one of them is failing on Windows as well. The failures on Linux are in disassembly.cpp and source-list.cpp and source-list.cpp is also failing on Windows. Let me know if you need logs from the failures.

[llvm-commits]

Ugh, I put absolute paths in the test. I can fix it in a few hours, i had
to leave tge office early. otherwise you’re free to revert, sorry :-/

• msg-1439-267.txt162 BDownload

[Hui]

lldb/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp

559 ↗

(On Diff #168723)

will be good to add a 'pdb' or 'npdb' log channel for any lookup logging etc.

[stella.stamenova]

I took a quick look at the FileCheck issue and it looks like these have been failing since mid-September!

That’s when the change to use filecheck inside the LLDB tests as a function went in: https://reviews.llvm.org/D50751. The change adds a new required input to dotests.py and updates the cmake files to pass it as well as the xcode project, but any of the bots that are calling dotests.py directly now need to pass the new input too.

Vedant, since you made the change, how do you feel about logging a warning instead of hard failing inside dotests.py and then hardfailing any tests that use filecheck when it doesn’t exist? That would limit the failures just to those tests instead of all of the tests on these bots. The bot owners will have to update the scripts as well.

Zachary, if you can’t get a local repro of the windows or linux failure with your changes, I can get you the logs from our test runs.

Thanks,
-Stella

From: Zachary Turner <[e-mail]>
Sent: Wednesday, October 10, 2018 8:23 PM
To: [e-mail]
Cc: [e-mail]; aaron.lee.smith <[e-mail]>; Stella Stamenova <[e-mail]>; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Subject: Re: [PATCH] D53002: Create a new symbol file plugin for cross-platform PDB symbolization

[vsk]

In D53002#1262143, @stella.stamenova wrote:

I took a quick look at the FileCheck issue and it looks like these have been failing since mid-September!

That’s when the change to use filecheck inside the LLDB tests as a function went in: https://reviews.llvm.org/D50751. The change adds a new required input to dotests.py and updates the cmake files to pass it as well as the xcode project, but any of the bots that are calling dotests.py directly now need to pass the new input too.

Vedant, since you made the change, how do you feel about logging a warning instead of hard failing inside dotests.py and then hardfailing any tests that use filecheck when it doesn’t exist? That would limit the failures just to those tests instead of all of the tests on these bots. The bot owners will have to update the scripts as well.

Sounds good, I'll send out a patch. Sorry, I didn't count on there being so many different ways the test harness is invoked.

Zachary, if you can’t get a local repro of the windows or linux failure with your changes, I can get you the logs from our test runs.

Thanks,
-Stella

From: Zachary Turner <[e-mail]>
Sent: Wednesday, October 10, 2018 8:23 PM
To: [e-mail]
Cc: [e-mail]; aaron.lee.smith <[e-mail]>; Stella Stamenova <[e-mail]>; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Subject: Re: [PATCH] D53002: Create a new symbol file plugin for cross-platform PDB symbolization

lldb/trunk/include/lldb/Utility/LLDBAssert.h

 //===----------------- LLDBAssert.h ------------------------------*- C++-*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef utility_LLDBAssert_h_
 #define utility_LLDBAssert_h_
 
 #ifdef LLDB_CONFIGURATION_DEBUG
 #define lldbassert(x) assert(x)
 #else
 #define lldbassert(x)                                                          \
-  lldb_private::lldb_assert(x, #x, __FUNCTION__, __FILE__, __LINE__)
+  lldb_private::lldb_assert(static_cast<bool>(x), #x, __FUNCTION__, __FILE__,  \
+                            __LINE__)
 #endif
 
 namespace lldb_private {
 void lldb_assert(bool expression, const char *expr_text, const char *func,
                  const char *file, unsigned int line);
 }
 
 #endif // utility_LLDBAssert_h_

lldb/trunk/lit/SymbolFile/NativePDB/Inputs/breakpoints.lldbinit

+break set -n main
+break set -n OvlGlobalFn
+break set -n StaticFn
+break set -n DoesntExist
+break list
+quit

lldb/trunk/lit/SymbolFile/NativePDB/Inputs/disassembly.lldbinit

+disassemble --flavor=intel -m -n main
+quit

lldb/trunk/lit/SymbolFile/NativePDB/Inputs/source-list.lldbinit

+source list -n main
+source list -n OvlGlobalFn
+quit

lldb/trunk/lit/SymbolFile/NativePDB/disassembly.cpp

+// clang-format off
+
+// Test that we can show disassembly and source.
+// RUN: clang-cl /Z7 /GS- /GR- /c %s /Fo%t.obj
+// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb %t.obj
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN:     %p/Inputs/disassembly.lldbinit | FileCheck %s
+
+// Some context lines before
+// the function.
+
+int foo() { return 42; }
+
+int main(int argc, char **argv) {
+  foo();
+  return 0;
+}
+
+
+// CHECK:      (lldb) disassemble --flavor=intel -m -n main
+// CHECK:         12   int foo() { return 42; }
+// CHECK-NEXT:    13
+// CHECK-NEXT: ** 14   int main(int argc, char **argv) {
+// CHECK:      disassembly.cpp.tmp.exe`main:
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+0>:  sub    rsp, 0x38
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+4>:  mov    dword ptr [rsp + 0x34], 0x0
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+12>: mov    qword ptr [rsp + 0x28], rdx
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+17>: mov    dword ptr [rsp + 0x24], ecx
+// CHECK:      ** 15     foo();
+// CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+21>: call   {{.*}}               ; foo at disassembly.cpp:12
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+26>: xor    ecx, ecx
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+28>: mov    dword ptr [rsp + 0x20], eax
+// CHECK:      ** 16     return 0;
+// CHECK-NEXT:    17   }
+// CHECK-NEXT:    18
+// CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+32>: mov    eax, ecx
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+34>: add    rsp, 0x38
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+38>: ret

lldb/trunk/lit/SymbolFile/NativePDB/lit.local.cfg

+config.suffixes = ['.test', '.cpp']

lldb/trunk/lit/SymbolFile/NativePDB/simple-breakpoints.cpp

+// clang-format off
+
+// Test that we can set simple breakpoints using PDB on any platform.
+// RUN: clang-cl /Z7 /GS- /GR- /c %s /Fo%t.obj
+// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb %t.obj
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN:     %p/Inputs/breakpoints.lldbinit | FileCheck %s
+
+// Use different indentation style for each overload so that the starting
+// line is in a different place.
+int OvlGlobalFn(int X) {
+  return X + 42;
+}
+
+int OvlGlobalFn(int X, int Y) { return X + Y + 42; }
+
+int OvlGlobalFn(int X, int Y, int Z)
+{
+  return X + Y + Z + 42;
+}
+
+static int StaticFn(int X) {
+  return X + 42;
+}
+
+int main(int argc, char **argv) {
+  // Make sure they don't get optimized out.
+  // Note the comments here, we want to make sure the line number reported
+  // for the breakpoint is the first actual line of code.
+  int Result = OvlGlobalFn(argc) + OvlGlobalFn(argc, argc)
+    + OvlGlobalFn(argc, argc, argc) + StaticFn(argc);
+  return Result;
+}
+
+
+// CHECK:      (lldb) target create "{{.*}}simple-breakpoints.cpp.tmp.exe"
+// CHECK:      Current executable set to '{{.*}}simple-breakpoints.cpp.tmp.exe' (x86_64).
+// CHECK:      (lldb) break set -n main
+// CHECK:      Breakpoint 1: where = simple-breakpoints.cpp.tmp.exe`main + 21
+// CHECK-SAME:               at simple-breakpoints.cpp:30
+// CHECK:      (lldb) break set -n OvlGlobalFn
+// CHECK:      Breakpoint 2: 3 locations.
+// CHECK:      (lldb) break set -n StaticFn
+// CHECK:      Breakpoint 3: where = simple-breakpoints.cpp.tmp.exe`StaticFn + 5
+// CHECK-SAME:               at simple-breakpoints.cpp:23
+// CHECK:      (lldb) break set -n DoesntExist
+// CHECK:      Breakpoint 4: no locations (pending).
+// CHECK:      (lldb) break list
+// CHECK:      Current breakpoints:
+// CHECK:      1: name = 'main', locations = 1
+// CHECK:        1.1: where = simple-breakpoints.cpp.tmp.exe`main + 21
+// CHECK-SAME:                at simple-breakpoints.cpp:30
+// CHECK:      2: name = 'OvlGlobalFn', locations = 3
+// CHECK:        2.1: where = simple-breakpoints.cpp.tmp.exe`OvlGlobalFn + 5
+// CHECK-SAME:                at simple-breakpoints.cpp:12
+// CHECK:        2.2: where = simple-breakpoints.cpp.tmp.exe`OvlGlobalFn
+// CHECK-SAME:                at simple-breakpoints.cpp:15
+// CHECK:        2.3: where = simple-breakpoints.cpp.tmp.exe`OvlGlobalFn + 17
+// CHECK-SAME:                at simple-breakpoints.cpp:19
+// CHECK:      3: name = 'StaticFn', locations = 1
+// CHECK:        3.1: where = simple-breakpoints.cpp.tmp.exe`StaticFn + 5
+// CHECK-SAME:                at simple-breakpoints.cpp:23
+// CHECK:      4: name = 'DoesntExist', locations = 0 (pending)

lldb/trunk/lit/SymbolFile/NativePDB/source-list.cpp

+// clang-format off
+
+// Test that we can set display source of functions.
+// RUN: clang-cl /Z7 /GS- /GR- /c %s /Fo%t.obj
+// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb %t.obj
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN:     %p/Inputs/source-list.lldbinit | FileCheck %s
+
+
+
+// Some context lines before
+// the function.
+
+
+int main(int argc, char **argv) {
+  // Here are some comments.
+  // That we should print when listing source.
+  return 0;
+}
+
+// Some context lines after
+// the function.
+
+// check lines go at the end so that line numbers stay stable when
+// changing this file.
+
+// CHECK: (lldb) source list -n main
+// CHECK: File: D:\src\llvm-mono\lldb\lit\SymbolFile\NativePDB\source-list.cpp
+// CHECK:    10
+// CHECK:    11    // Some context lines before
+// CHECK:    12   // the function.
+// CHECK:    13
+// CHECK:    14
+// CHECK:    15   int main(int argc, char **argv) {
+// CHECK:    16     // Here are some comments.
+// CHECK:    17     // That we should print when listing source.
+// CHECK:    18     return 0;
+// CHECK:    19   }
+// CHECK:    20
+// CHECK:    21   // Some context lines after
+// CHECK:    22   // the function.
+// CHECK:    23

lldb/trunk/lit/lit.cfg

 if platform.system() in ['Darwin']:
     debugserver = lit.util.which('debugserver', lldb_tools_dir)
 else:
     debugserver = lit.util.which('lldb-server', lldb_tools_dir)
 lldb = "%s -S %s/lit-lldb-init" % (lit.util.which('lldb', lldb_tools_dir),
                                config.test_source_root)
 
 lldbmi = lit.util.which('lldb-mi', lldb_tools_dir)
+if lldbmi:
+    config.available_features.add('lldb-mi')
 
 if not os.path.exists(config.cc):
     config.cc = lit.util.which(config.cc, config.environment['PATH'])
 
 if not os.path.exists(config.cxx):
     config.cxx = lit.util.which(config.cxx, config.environment['PATH'])
 
 if platform.system() in ['Darwin']:

 
 if platform.system() in ['OpenBSD']:
     config.cc += " -pthread"
     config.cxx += " -pthread"
 
 config.substitutions.append(('%cc', config.cc))
 config.substitutions.append(('%cxx', config.cxx))
 
+if lldbmi:
   config.substitutions.append(('%lldbmi', lldbmi + " --synchronous"))
 config.substitutions.append(('%lldb', lldb))
 
 if debugserver is not None:
     if platform.system() in ['Darwin']:
         config.substitutions.append(('%debugserver', debugserver))
     else:
         config.substitutions.append(('%debugserver', debugserver + ' gdbserver'))
 

lldb/trunk/source/Plugins/SymbolFile/CMakeLists.txt

 add_subdirectory(DWARF)
 add_subdirectory(Symtab)
+add_subdirectory(NativePDB)
 add_subdirectory(PDB)

lldb/trunk/source/Plugins/SymbolFile/NativePDB/CMakeLists.txt

+add_lldb_library(lldbPluginSymbolFileNativePDB PLUGIN
+  CompileUnitIndex.cpp
+  PdbIndex.cpp
+  PdbUtil.cpp
+  SymbolFileNativePDB.cpp
+
+  LINK_LIBS
+    clangAST
+    clangLex
+    lldbCore
+    lldbSymbol
+    lldbUtility
+  LINK_COMPONENTS
+    DebugInfoPDB
+    Support
+  )

lldb/trunk/source/Plugins/SymbolFile/NativePDB/CompileUnitIndex.h

+//===-- CompileUnitIndex.h --------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_PLUGINS_SYMBOLFILENATIVEPDB_COMPILEUNITINDEX_H
+#define LLDB_PLUGINS_SYMBOLFILENATIVEPDB_COMPILEUNITINDEX_H
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/DenseSet.h"
+#include "llvm/ADT/IntervalMap.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/DebugInfo/CodeView/StringsAndChecksums.h"
+#include "llvm/DebugInfo/CodeView/SymbolRecord.h"
+#include "llvm/DebugInfo/CodeView/TypeIndex.h"
+#include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h"
+#include "llvm/DebugInfo/PDB/Native/ModuleDebugStream.h"
+#include "llvm/DebugInfo/PDB/PDBTypes.h"
+
+#include "PdbSymUid.h"
+
+#include <map>
+#include <memory>
+
+namespace lldb_private {
+
+namespace npdb {
+class PdbIndex;
+
+/// Represents a single compile unit.  This class is useful for collecting the
+/// important accessors and information about a compile unit from disparate
+/// parts of the PDB into a single place, simplifying acess to compile unit
+/// information for the callers.
+struct CompilandIndexItem {
+  CompilandIndexItem(PdbSymUid uid,
+                     llvm::pdb::ModuleDebugStreamRef debug_stream,
+                     llvm::pdb::DbiModuleDescriptor descriptor);
+
+  // uid of this compile unit.
+  PdbSymUid m_uid;
+
+  // debug stream.
+  llvm::pdb::ModuleDebugStreamRef m_debug_stream;
+
+  // dbi module descriptor.
+  llvm::pdb::DbiModuleDescriptor m_module_descriptor;
+
+  llvm::codeview::StringsAndChecksumsRef m_strings;
+
+  // List of files which contribute to this compiland.
+  std::vector<llvm::StringRef> m_file_list;
+
+  // Maps virtual address to global symbol id, which can then be used to
+  // locate the exact compile unit and offset of the symbol.  Note that this
+  // is intentionally an ordered map so that we can find all symbols up to a
+  // given starting address.
+  std::map<lldb::addr_t, PdbSymUid> m_symbols_by_va;
+
+  // S_COMPILE3 sym describing compilation settings for the module.
+  llvm::Optional<llvm::codeview::Compile3Sym> m_compile_opts;
+
+  // S_OBJNAME sym describing object name.
+  llvm::Optional<llvm::codeview::ObjNameSym> m_obj_name;
+
+  // LF_BUILDINFO sym describing source file name, working directory,
+  // command line, etc.  This usually contains exactly 5 items which
+  // are references to other strings.
+  llvm::SmallVector<llvm::codeview::TypeIndex, 5> m_build_info;
+};
+
+/// Indexes information about all compile units.  This is really just a map of
+/// global compile unit index to |CompilandIndexItem| structures.
+class CompileUnitIndex {
+  PdbIndex &m_index;
+  llvm::DenseMap<lldb::user_id_t, std::unique_ptr<CompilandIndexItem>>
+      m_comp_units;
+
+public:
+  explicit CompileUnitIndex(PdbIndex &index) : m_index(index) {}
+
+  CompilandIndexItem &GetOrCreateCompiland(uint16_t modi);
+  CompilandIndexItem &GetOrCreateCompiland(PdbSymUid compiland_uid);
+
+  const CompilandIndexItem *GetCompiland(uint16_t modi) const;
+  const CompilandIndexItem *GetCompiland(PdbSymUid compiland_uid) const;
+
+  CompilandIndexItem *GetCompiland(uint16_t modi);
+  CompilandIndexItem *GetCompiland(PdbSymUid compiland_uid);
+
+  llvm::SmallString<64> GetMainSourceFile(const CompilandIndexItem &item) const;
+};
+} // namespace npdb
+} // namespace lldb_private
+
+#endif

lldb/trunk/source/Plugins/SymbolFile/NativePDB/CompileUnitIndex.cpp

+//===-- CompileUnitIndex.cpp ------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "CompileUnitIndex.h"
+
+#include "PdbIndex.h"
+#include "PdbUtil.h"
+
+#include "llvm/DebugInfo/CodeView/LazyRandomTypeCollection.h"
+#include "llvm/DebugInfo/CodeView/SymbolDeserializer.h"
+#include "llvm/DebugInfo/CodeView/TypeDeserializer.h"
+#include "llvm/DebugInfo/MSF/MappedBlockStream.h"
+#include "llvm/DebugInfo/PDB/Native/DbiModuleDescriptor.h"
+#include "llvm/DebugInfo/PDB/Native/DbiStream.h"
+#include "llvm/DebugInfo/PDB/Native/InfoStream.h"
+#include "llvm/DebugInfo/PDB/Native/ModuleDebugStream.h"
+#include "llvm/DebugInfo/PDB/Native/NamedStreamMap.h"
+#include "llvm/DebugInfo/PDB/Native/TpiStream.h"
+#include "llvm/Support/Path.h"
+
+#include "lldb/Utility/LLDBAssert.h"
+
+using namespace lldb;
+using namespace lldb_private;
+using namespace lldb_private::npdb;
+using namespace llvm::codeview;
+using namespace llvm::pdb;
+
+static bool IsMainFile(llvm::StringRef main, llvm::StringRef other) {
+  if (main == other)
+    return true;
+
+  // If the files refer to the local file system, we can just ask the file
+  // system if they're equivalent.  But if the source isn't present on disk
+  // then we still want to try.
+  if (llvm::sys::fs::equivalent(main, other))
+    return true;
+
+  // FIXME: If we ever want to support PDB debug info for non-Windows systems
+  // the following check will be wrong, but we need a way to store the host
+  // information in the PDB.
+  llvm::SmallString<64> normalized(other);
+  llvm::sys::path::native(normalized, llvm::sys::path::Style::windows);
+  return main.equals_lower(normalized);
+}
+
+static void ParseCompile3(const CVSymbol &sym, CompilandIndexItem &cci) {
+  cci.m_compile_opts.emplace();
+  llvm::cantFail(
+      SymbolDeserializer::deserializeAs<Compile3Sym>(sym, *cci.m_compile_opts));
+}
+
+static void ParseObjname(const CVSymbol &sym, CompilandIndexItem &cci) {
+  cci.m_obj_name.emplace();
+  llvm::cantFail(
+      SymbolDeserializer::deserializeAs<ObjNameSym>(sym, *cci.m_obj_name));
+}
+
+static void ParseBuildInfo(PdbIndex &index, const CVSymbol &sym,
+                           CompilandIndexItem &cci) {
+  BuildInfoSym bis(SymbolRecordKind::BuildInfoSym);
+  llvm::cantFail(SymbolDeserializer::deserializeAs<BuildInfoSym>(sym, bis));
+
+  // S_BUILDINFO just points to an LF_BUILDINFO in the IPI stream.  Let's do
+  // a little extra work to pull out the LF_BUILDINFO.
+  LazyRandomTypeCollection &types = index.ipi().typeCollection();
+  llvm::Optional<CVType> cvt = types.tryGetType(bis.BuildId);
+
+  if (!cvt || cvt->kind() != LF_BUILDINFO)
+    return;
+
+  BuildInfoRecord bir;
+  llvm::cantFail(TypeDeserializer::deserializeAs<BuildInfoRecord>(*cvt, bir));
+  cci.m_build_info.assign(bir.ArgIndices.begin(), bir.ArgIndices.end());
+}
+
+static void ParseExtendedInfo(PdbIndex &index, CompilandIndexItem &item) {
+  const CVSymbolArray &syms = item.m_debug_stream.getSymbolArray();
+
+  // This is a private function, it shouldn't be called if the information
+  // has already been parsed.
+  lldbassert(!item.m_obj_name);
+  lldbassert(!item.m_compile_opts);
+  lldbassert(item.m_build_info.empty());
+
+  // We're looking for 3 things.  S_COMPILE3, S_OBJNAME, and S_BUILDINFO.
+  int found = 0;
+  for (const CVSymbol &sym : syms) {
+    switch (sym.kind()) {
+    case S_COMPILE3:
+      ParseCompile3(sym, item);
+      break;
+    case S_OBJNAME:
+      ParseObjname(sym, item);
+      break;
+    case S_BUILDINFO:
+      ParseBuildInfo(index, sym, item);
+      break;
+    default:
+      continue;
+    }
+    if (++found >= 3)
+      break;
+  }
+}
+
+CompilandIndexItem::CompilandIndexItem(
+    PdbSymUid uid, llvm::pdb::ModuleDebugStreamRef debug_stream,
+    llvm::pdb::DbiModuleDescriptor descriptor)
+    : m_uid(uid), m_debug_stream(std::move(debug_stream)),
+      m_module_descriptor(std::move(descriptor)) {}
+
+CompilandIndexItem &CompileUnitIndex::GetOrCreateCompiland(uint16_t modi) {
+  PdbSymUid uid = PdbSymUid::makeCompilandId(modi);
+  return GetOrCreateCompiland(uid);
+}
+
+CompilandIndexItem &
+CompileUnitIndex::GetOrCreateCompiland(PdbSymUid compiland_uid) {
+  auto result = m_comp_units.try_emplace(compiland_uid.toOpaqueId(), nullptr);
+  if (!result.second)
+    return *result.first->second;
+
+  // Find the module list and load its debug information stream and cache it
+  // since we need to use it for almost all interesting operations.
+  const DbiModuleList &modules = m_index.dbi().modules();
+  uint16_t modi = compiland_uid.asCompiland().modi;
+  llvm::pdb::DbiModuleDescriptor descriptor = modules.getModuleDescriptor(modi);
+  uint16_t stream = descriptor.getModuleStreamIndex();
+  std::unique_ptr<llvm::msf::MappedBlockStream> stream_data =
+      m_index.pdb().createIndexedStream(stream);
+  llvm::pdb::ModuleDebugStreamRef debug_stream(descriptor,
+                                               std::move(stream_data));
+  cantFail(debug_stream.reload());
+
+  std::unique_ptr<CompilandIndexItem> &cci = result.first->second;
+
+  cci = llvm::make_unique<CompilandIndexItem>(
+      compiland_uid, std::move(debug_stream), std::move(descriptor));
+  ParseExtendedInfo(m_index, *cci);
+
+  cci->m_strings.initialize(debug_stream.getSubsectionsArray());
+  PDBStringTable &strings = cantFail(m_index.pdb().getStringTable());
+  cci->m_strings.setStrings(strings.getStringTable());
+
+  // We want the main source file to always comes first.  Note that we can't
+  // just push_back the main file onto the front because `GetMainSourceFile`
+  // computes it in such a way that it doesn't own the resulting memory.  So we
+  // have to iterate the module file list comparing each one to the main file
+  // name until we find it, and we can cache that one since the memory is backed
+  // by a contiguous chunk inside the mapped PDB.
+  llvm::SmallString<64> main_file = GetMainSourceFile(*cci);
+  llvm::sys::path::native(main_file, llvm::sys::path::Style::windows);
+
+  uint32_t file_count = modules.getSourceFileCount(modi);
+  cci->m_file_list.reserve(file_count);
+  bool found_main_file = false;
+  for (llvm::StringRef file : modules.source_files(modi)) {
+    if (!found_main_file && IsMainFile(main_file, file)) {
+      cci->m_file_list.insert(cci->m_file_list.begin(), file);
+      found_main_file = true;
+      continue;
+    }
+    cci->m_file_list.push_back(file);
+  }
+
+  return *cci;
+}
+
+const CompilandIndexItem *CompileUnitIndex::GetCompiland(uint16_t modi) const {
+  return GetCompiland(PdbSymUid::makeCompilandId(modi));
+}
+
+const CompilandIndexItem *
+CompileUnitIndex::GetCompiland(PdbSymUid compiland_uid) const {
+  auto iter = m_comp_units.find(compiland_uid.toOpaqueId());
+  if (iter == m_comp_units.end())
+    return nullptr;
+  return iter->second.get();
+}
+
+CompilandIndexItem *CompileUnitIndex::GetCompiland(uint16_t modi) {
+  return GetCompiland(PdbSymUid::makeCompilandId(modi));
+}
+
+CompilandIndexItem *CompileUnitIndex::GetCompiland(PdbSymUid compiland_uid) {
+  auto iter = m_comp_units.find(compiland_uid.toOpaqueId());
+  if (iter == m_comp_units.end())
+    return nullptr;
+  return iter->second.get();
+}
+
+llvm::SmallString<64>
+CompileUnitIndex::GetMainSourceFile(const CompilandIndexItem &item) const {
+  // LF_BUILDINFO contains a list of arg indices which point to LF_STRING_ID
+  // records in the IPI stream.  The order of the arg indices is as follows:
+  // [0] - working directory where compiler was invoked.
+  // [1] - absolute path to compiler binary
+  // [2] - source file name
+  // [3] - path to compiler generated PDB (the /Zi PDB, although this entry gets
+  //       added even when using /Z7)
+  // [4] - full command line invocation.
+  //
+  // We need to form the path [0]\[2] to generate the full path to the main
+  // file.source
+  if (item.m_build_info.size() < 3)
+    return {""};
+
+  LazyRandomTypeCollection &types = m_index.ipi().typeCollection();
+
+  StringIdRecord working_dir;
+  StringIdRecord file_name;
+  CVType dir_cvt = types.getType(item.m_build_info[0]);
+  CVType file_cvt = types.getType(item.m_build_info[2]);
+  llvm::cantFail(
+      TypeDeserializer::deserializeAs<StringIdRecord>(dir_cvt, working_dir));
+  llvm::cantFail(
+      TypeDeserializer::deserializeAs<StringIdRecord>(file_cvt, file_name));
+
+  llvm::SmallString<64> absolute_path = working_dir.String;
+  llvm::sys::path::append(absolute_path, file_name.String);
+  return absolute_path;
+}

lldb/trunk/source/Plugins/SymbolFile/NativePDB/PdbIndex.h

+//===-- PdbIndex.h ----------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_PLUGINS_SYMBOLFILENATIVEPDB_PDBINDEX_H
+#define LLDB_PLUGINS_SYMBOLFILENATIVEPDB_PDBINDEX_H
+
+#include "lldb/lldb-types.h"
+#include "llvm/ADT/IntervalMap.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/DebugInfo/PDB/Native/PDBFile.h"
+#include "llvm/DebugInfo/PDB/PDBTypes.h"
+
+#include "CompileUnitIndex.h"
+#include "PdbSymUid.h"
+
+#include <map>
+#include <memory>
+
+namespace llvm {
+namespace pdb {
+class DbiStream;
+class TpiStream;
+class TpiStream;
+class InfoStream;
+class PublicsStream;
+class GlobalsStream;
+class SymbolStream;
+} // namespace pdb
+} // namespace llvm
+
+namespace lldb_private {
+namespace npdb {
+struct SegmentOffset;
+
+/// PdbIndex - Lazy access to the important parts of a PDB file.
+///
+/// This is a layer on top of LLVM's native PDB support libraries which cache
+/// certain data when it is accessed the first time.  The entire PDB file is
+/// mapped into memory, and the underlying support libraries vend out memory
+/// that is always backed by the file, so it is safe to hold StringRefs and
+/// ArrayRefs into the backing memory as long as the PdbIndex instance is
+/// alive.
+class PdbIndex {
+
+  /// The underlying PDB file.
+  std::unique_ptr<llvm::pdb::PDBFile> m_file;
+
+  /// The DBI stream.  This contains general high level information about the
+  /// features present in the PDB file, compile units (such as the information
+  /// necessary to locate full symbol information for each compile unit),
+  /// section contributions, and other data which is not specifically symbol or
+  /// type records.
+  llvm::pdb::DbiStream *m_dbi = nullptr;
+
+  /// TPI (types) and IPI (indices) streams.  These are both in the exact same
+  /// format with different data.  Most type records are stored in the TPI
+  /// stream but certain specific types of records are stored in the IPI stream.
+  /// The IPI stream records can refer to the records in the TPI stream, but not
+  /// the other way around.
+  llvm::pdb::TpiStream *m_tpi = nullptr;
+  llvm::pdb::TpiStream *m_ipi = nullptr;
+
+  /// This is called the "PDB Stream" in the Microsoft reference implementation.
+  /// It contains information about the structure of the file, as well as fields
+  /// used to match EXE and PDB.
+  llvm::pdb::InfoStream *m_info = nullptr;
+
+  /// Publics stream.  Is actually a serialized hash table where the keys are
+  /// addresses of symbols in the executable, and values are a record containing
+  /// mangled names and an index which can be used to locate more detailed info
+  /// about the symbol in the Symbol Records stream.  The publics stream only
+  /// contains info about externally visible symbols.
+  llvm::pdb::PublicsStream *m_publics = nullptr;
+
+  /// Globals stream.  Contrary to its name, this does not contain information
+  /// about all "global variables" or "global functions".  Rather, it is the
+  /// "global symbol table", i.e. it contains information about *every* symbol
+  /// in the executable.  It is a hash table keyed on name, whose values are
+  /// indices into the symbol records stream to find the full record.
+  llvm::pdb::GlobalsStream *m_globals = nullptr;
+
+  /// Symbol records stream.  The publics and globals stream refer to records
+  /// in this stream.  For some records, like constants and typedefs, the
+  /// complete record lives in this stream.  For other symbol types, such as
+  /// functions, data, and other things that have been materialied into a
+  /// specific compile unit, the records here simply provide a reference
+  /// necessary to locate the full information.
+  llvm::pdb::SymbolStream *m_symrecords = nullptr;
+
+  /// Index of all compile units, mapping identifier to |CompilandIndexItem|
+  /// instance.
+  CompileUnitIndex m_cus;
+
+  /// An allocator for the interval maps
+  llvm::IntervalMap<lldb::addr_t, uint32_t>::Allocator m_allocator;
+
+  /// Maps virtual address to module index
+  llvm::IntervalMap<lldb::addr_t, uint16_t> m_va_to_modi;
+
+  /// The address at which the program has been loaded into memory.
+  lldb::addr_t m_load_address = 0;
+
+  PdbIndex();
+
+  void BuildAddrToSymbolMap(CompilandIndexItem &cci);
+
+public:
+  static llvm::Expected<std::unique_ptr<PdbIndex>>
+      create(std::unique_ptr<llvm::pdb::PDBFile>);
+
+  void SetLoadAddress(lldb::addr_t addr) { m_load_address = addr; }
+  void ParseSectionContribs();
+
+  llvm::pdb::PDBFile &pdb() { return *m_file; }
+  const llvm::pdb::PDBFile &pdb() const { return *m_file; }
+
+  llvm::pdb::DbiStream &dbi() { return *m_dbi; }
+  const llvm::pdb::DbiStream &dbi() const { return *m_dbi; }
+
+  llvm::pdb::TpiStream &tpi() { return *m_tpi; }
+  const llvm::pdb::TpiStream &tpi() const { return *m_tpi; }
+
+  llvm::pdb::TpiStream &ipi() { return *m_ipi; }
+  const llvm::pdb::TpiStream &ipi() const { return *m_ipi; }
+
+  llvm::pdb::InfoStream &info() { return *m_info; }
+  const llvm::pdb::InfoStream &info() const { return *m_info; }
+
+  llvm::pdb::PublicsStream &publics() { return *m_publics; }
+  const llvm::pdb::PublicsStream &publics() const { return *m_publics; }
+
+  llvm::pdb::GlobalsStream &globals() { return *m_globals; }
+  const llvm::pdb::GlobalsStream &globals() const { return *m_globals; }
+
+  llvm::pdb::SymbolStream &symrecords() { return *m_symrecords; }
+  const llvm::pdb::SymbolStream &symrecords() const { return *m_symrecords; }
+
+  CompileUnitIndex &compilands() { return m_cus; }
+  const CompileUnitIndex &compilands() const { return m_cus; }
+
+  lldb::addr_t MakeVirtualAddress(uint16_t segment, uint32_t offset) const;
+  lldb::addr_t MakeVirtualAddress(const SegmentOffset &so) const;
+
+  std::vector<SymbolAndUid> FindSymbolsByVa(lldb::addr_t va);
+
+  llvm::codeview::CVSymbol ReadSymbolRecord(PdbCuSymId cu_sym) const;
+
+  llvm::Optional<uint16_t> GetModuleIndexForAddr(uint16_t segment,
+                                                 uint32_t offset) const;
+  llvm::Optional<uint16_t> GetModuleIndexForVa(lldb::addr_t va) const;
+};
+} // namespace npdb
+} // namespace lldb_private
+
+#endif

lldb/trunk/source/Plugins/SymbolFile/NativePDB/PdbIndex.cpp

+//===-- PdbIndex.cpp --------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "PdbIndex.h"
+#include "PdbUtil.h"
+
+#include "llvm/DebugInfo/CodeView/SymbolDeserializer.h"
+#include "llvm/DebugInfo/PDB/Native/DbiStream.h"
+#include "llvm/DebugInfo/PDB/Native/GlobalsStream.h"
+#include "llvm/DebugInfo/PDB/Native/ISectionContribVisitor.h"
+#include "llvm/DebugInfo/PDB/Native/PDBFile.h"
+#include "llvm/DebugInfo/PDB/Native/PublicsStream.h"
+#include "llvm/DebugInfo/PDB/Native/SymbolStream.h"
+#include "llvm/Object/COFF.h"
+#include "llvm/Support/Error.h"
+
+#include "lldb/Utility/LLDBAssert.h"
+#include "lldb/lldb-defines.h"
+
+using namespace lldb_private;
+using namespace lldb_private::npdb;
+using namespace llvm::codeview;
+using namespace llvm::pdb;
+
+PdbIndex::PdbIndex() : m_cus(*this), m_va_to_modi(m_allocator) {}
+
+#define ASSIGN_PTR_OR_RETURN(result_ptr, expr)                                 \
+  {                                                                            \
+    auto expected_result = expr;                                               \
+    if (!expected_result)                                                      \
+      return expected_result.takeError();                                      \
+    result_ptr = &expected_result.get();                                       \
+  }
+
+llvm::Expected<std::unique_ptr<PdbIndex>>
+PdbIndex::create(std::unique_ptr<llvm::pdb::PDBFile> file) {
+  lldbassert(file);
+
+  std::unique_ptr<PdbIndex> result(new PdbIndex());
+  ASSIGN_PTR_OR_RETURN(result->m_dbi, file->getPDBDbiStream());
+  ASSIGN_PTR_OR_RETURN(result->m_tpi, file->getPDBTpiStream());
+  ASSIGN_PTR_OR_RETURN(result->m_ipi, file->getPDBIpiStream());
+  ASSIGN_PTR_OR_RETURN(result->m_info, file->getPDBInfoStream());
+  ASSIGN_PTR_OR_RETURN(result->m_publics, file->getPDBPublicsStream());
+  ASSIGN_PTR_OR_RETURN(result->m_globals, file->getPDBGlobalsStream());
+  ASSIGN_PTR_OR_RETURN(result->m_symrecords, file->getPDBSymbolStream());
+
+  result->m_file = std::move(file);
+
+  return std::move(result);
+}
+
+lldb::addr_t PdbIndex::MakeVirtualAddress(uint16_t segment,
+                                          uint32_t offset) const {
+  // Segment indices are 1-based.
+  lldbassert(segment > 0);
+
+  uint32_t max_section = dbi().getSectionHeaders().size();
+  lldbassert(segment <= max_section + 1);
+
+  // If this is an absolute symbol, it's indicated by the magic section index
+  // |max_section+1|.  In this case, the offset is meaningless, so just return.
+  if (segment == max_section + 1)
+    return LLDB_INVALID_ADDRESS;
+
+  const llvm::object::coff_section &cs = dbi().getSectionHeaders()[segment - 1];
+  return m_load_address + static_cast<lldb::addr_t>(cs.VirtualAddress) +
+         static_cast<lldb::addr_t>(offset);
+}
+
+lldb::addr_t PdbIndex::MakeVirtualAddress(const SegmentOffset &so) const {
+  return MakeVirtualAddress(so.segment, so.offset);
+}
+
+llvm::Optional<uint16_t>
+PdbIndex::GetModuleIndexForAddr(uint16_t segment, uint32_t offset) const {
+  return GetModuleIndexForVa(MakeVirtualAddress(segment, offset));
+}
+
+llvm::Optional<uint16_t> PdbIndex::GetModuleIndexForVa(lldb::addr_t va) const {
+  auto iter = m_va_to_modi.find(va);
+  if (iter == m_va_to_modi.end())
+    return llvm::None;
+
+  return iter.value();
+}
+
+void PdbIndex::ParseSectionContribs() {
+  class Visitor : public ISectionContribVisitor {
+    PdbIndex &m_ctx;
+    llvm::IntervalMap<uint64_t, uint16_t> &m_imap;
+
+  public:
+    Visitor(PdbIndex &ctx, llvm::IntervalMap<uint64_t, uint16_t> &imap)
+        : m_ctx(ctx), m_imap(imap) {}
+
+    void visit(const SectionContrib &C) override {
+      uint64_t va = m_ctx.MakeVirtualAddress(C.ISect, C.Off);
+      uint64_t end = va + C.Size;
+      // IntervalMap's start and end represent a closed range, not a half-open
+      // range, so we have to subtract 1.
+      m_imap.insert(va, end - 1, C.Imod);
+    }
+    void visit(const SectionContrib2 &C) override { visit(C.Base); }
+  };
+  Visitor v(*this, m_va_to_modi);
+  dbi().visitSectionContributions(v);
+}
+
+void PdbIndex::BuildAddrToSymbolMap(CompilandIndexItem &cci) {
+  lldbassert(cci.m_symbols_by_va.empty() &&
+             "Addr to symbol map is already built!");
+  uint16_t modi = cci.m_uid.asCompiland().modi;
+  const CVSymbolArray &syms = cci.m_debug_stream.getSymbolArray();
+  for (auto iter = syms.begin(); iter != syms.end(); ++iter) {
+    if (!SymbolHasAddress(*iter))
+      continue;
+
+    SegmentOffset so = GetSegmentAndOffset(*iter);
+    lldb::addr_t va = MakeVirtualAddress(so);
+
+    // We need to add 4 here to adjust for the codeview debug magic
+    // at the beginning of the debug info stream.
+    uint32_t sym_offset = iter.offset() + 4;
+    PdbSymUid cu_sym_uid =
+        PdbSymUid::makeCuSymId(CVSymToPDBSym(iter->kind()), modi, sym_offset);
+
+    // If the debug info is incorrect, we could have multiple symbols with the
+    // same address.  So use try_emplace instead of insert, and the first one
+    // will win.
+    auto emplace_result = cci.m_symbols_by_va.try_emplace(va, cu_sym_uid);
+    (void)emplace_result;
+
+    // The odds of an error in some function such as GetSegmentAndOffset or
+    // MakeVirtualAddress are much higher than the odds of encountering bad
+    // debug info, so assert that this item was inserted in the map as opposed
+    // to having already been there.
+    lldbassert(emplace_result.second);
+  }
+}
+
+std::vector<SymbolAndUid> PdbIndex::FindSymbolsByVa(lldb::addr_t va) {
+  std::vector<SymbolAndUid> result;
+
+  llvm::Optional<uint16_t> modi = GetModuleIndexForVa(va);
+  if (!modi)
+    return result;
+
+  CompilandIndexItem &cci = compilands().GetOrCreateCompiland(*modi);
+  if (cci.m_symbols_by_va.empty())
+    BuildAddrToSymbolMap(cci);
+
+  // The map is sorted by starting address of the symbol.  So for example
+  // we could (in theory) have this situation
+  //
+  // [------------------]
+  //    [----------]
+  //      [-----------]
+  //          [-------------]
+  //            [----]
+  //               [-----]
+  //             ^ Address we're searching for
+  // In order to find this, we use the upper_bound of the key value which would
+  // be the first symbol whose starting address is higher than the element we're
+  // searching for.
+
+  auto ub = cci.m_symbols_by_va.upper_bound(va);
+
+  for (auto iter = cci.m_symbols_by_va.begin(); iter != ub; ++iter) {
+    const PdbCuSymId &cu_sym_id = iter->second.asCuSym();
+    CVSymbol sym = ReadSymbolRecord(cu_sym_id);
+
+    SegmentOffsetLength sol;
+    if (SymbolIsCode(sym))
+      sol = GetSegmentOffsetAndLength(sym);
+    else
+      sol.so = GetSegmentAndOffset(sym);
+
+    lldb::addr_t start = MakeVirtualAddress(sol.so);
+    lldb::addr_t end = start + sol.length;
+    if (va >= start && va < end)
+      result.push_back({std::move(sym), iter->second});
+  }
+
+  return result;
+}
+
+CVSymbol PdbIndex::ReadSymbolRecord(PdbCuSymId cu_sym) const {
+  // We need to subtract 4 here to adjust for the codeview debug magic
+  // at the beginning of the debug info stream.
+  PdbSymUid cuid = PdbSymUid::makeCompilandId(cu_sym.modi);
+  const CompilandIndexItem *cci = compilands().GetCompiland(cuid);
+  auto iter = cci->m_debug_stream.getSymbolArray().at(cu_sym.offset - 4);
+  lldbassert(iter != cci->m_debug_stream.getSymbolArray().end());
+  return *iter;
+}

lldb/trunk/source/Plugins/SymbolFile/NativePDB/PdbSymUid.h

+//===-- PdbSymUid.h ---------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+// A unique identification scheme for Pdb records.
+// The scheme is to partition a 64-bit integer into an 8-bit tag field, which
+// will contain some value from the PDB_SymType enumeration.  The format of the
+// other 48-bits depend on the tag, but must be sufficient to locate the
+// corresponding entry in the underlying PDB file quickly.  For example, for
+// a compile unit, we use 2 bytes to represent the index, which allows fast
+// access to the compile unit's information.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_PLUGINS_SYMBOLFILENATIVEPDB_PDBSYMUID_H
+#define LLDB_PLUGINS_SYMBOLFILENATIVEPDB_PDBSYMUID_H
+
+#include "llvm/DebugInfo/PDB/PDBTypes.h"
+#include "llvm/Support/Compiler.h"
+
+#include "lldb/Utility/LLDBAssert.h"
+#include "lldb/lldb-types.h"
+
+namespace lldb_private {
+namespace npdb {
+
+// **important** - All concrete id types must have the 1-byte tag field at
+// the beginning so that the types are all layout-compatible with each
+// other, which is necessary in order to be able to safely access the tag
+// member through any union member.
+
+struct PdbCompilandId {
+  uint64_t tag : 8;   // PDB_SymType::Compiland
+  uint64_t modi : 16; // 0-based index of module in PDB
+  uint64_t unused : 32;
+};
+struct PdbCuSymId {
+  uint64_t tag : 8; // PDB_SymType::Data, Function, Block, etc.
+  uint64_t
+      offset : 32;    // Offset of symbol's record in module stream.  This is
+                      // offset by 4 from the CVSymbolArray's notion of offset
+                      // due to the debug magic at the beginning of the stream.
+  uint64_t modi : 16; // 0-based index of module in PDB
+};
+struct PdbTypeSymId {
+  uint64_t tag : 8;    // PDB_SymType::FunctionSig, Enum, PointerType, etc.
+  uint64_t is_ipi : 8; // 1 if this value is from the IPI stream, 0 for TPI.
+  uint64_t unused : 16;
+  uint64_t index : 32; // codeview::TypeIndex
+};
+
+static_assert(sizeof(PdbCompilandId) == 8, "invalid uid size");
+static_assert(sizeof(PdbCuSymId) == 8, "invalid uid size");
+static_assert(std::is_standard_layout<PdbCompilandId>::value,
+              "type is not standard layout!");
+static_assert(std::is_standard_layout<PdbCuSymId>::value,
+              "type is not standard layout!");
+
+class PdbSymUid {
+  union {
+    PdbCompilandId comp_id;
+    PdbCuSymId cu_sym;
+    PdbTypeSymId type_sym;
+  } m_uid;
+
+  PdbSymUid() { ::memset(&m_uid, 0, sizeof(m_uid)); }
+
+public:
+  static bool isTypeSym(llvm::pdb::PDB_SymType tag) {
+    switch (tag) {
+    case llvm::pdb::PDB_SymType::ArrayType:
+    case llvm::pdb::PDB_SymType::BaseClass:
+    case llvm::pdb::PDB_SymType::BaseInterface:
+    case llvm::pdb::PDB_SymType::BuiltinType:
+    case llvm::pdb::PDB_SymType::CustomType:
+    case llvm::pdb::PDB_SymType::Enum:
+    case llvm::pdb::PDB_SymType::FunctionArg:
+    case llvm::pdb::PDB_SymType::FunctionSig:
+    case llvm::pdb::PDB_SymType::Typedef:
+    case llvm::pdb::PDB_SymType::VectorType:
+    case llvm::pdb::PDB_SymType::VTableShape:
+    case llvm::pdb::PDB_SymType::PointerType:
+    case llvm::pdb::PDB_SymType::UDT:
+      return true;
+    default:
+      return false;
+    }
+  }
+
+  static bool isCuSym(llvm::pdb::PDB_SymType tag) {
+    switch (tag) {
+    case llvm::pdb::PDB_SymType::Block:
+    case llvm::pdb::PDB_SymType::Callee:
+    case llvm::pdb::PDB_SymType::Caller:
+    case llvm::pdb::PDB_SymType::CallSite:
+    case llvm::pdb::PDB_SymType::CoffGroup:
+    case llvm::pdb::PDB_SymType::CompilandDetails:
+    case llvm::pdb::PDB_SymType::CompilandEnv:
+    case llvm::pdb::PDB_SymType::Custom:
+    case llvm::pdb::PDB_SymType::Data:
+    case llvm::pdb::PDB_SymType::Function:
+    case llvm::pdb::PDB_SymType::Inlinee:
+    case llvm::pdb::PDB_SymType::InlineSite:
+    case llvm::pdb::PDB_SymType::Label:
+    case llvm::pdb::PDB_SymType::Thunk:
+      return true;
+    default:
+      return false;
+    }
+  }
+
+  static PdbSymUid makeCuSymId(llvm::codeview::ProcRefSym sym) {
+    return makeCuSymId(llvm::pdb::PDB_SymType::Function, sym.Module - 1,
+                       sym.SymOffset);
+  }
+
+  static PdbSymUid makeCuSymId(llvm::pdb::PDB_SymType type, uint16_t modi,
+                               uint32_t offset) {
+    lldbassert(isCuSym(type));
+
+    PdbSymUid uid;
+    uid.m_uid.cu_sym.modi = modi;
+    uid.m_uid.cu_sym.offset = offset;
+    uid.m_uid.cu_sym.tag = static_cast<uint8_t>(type);
+    return uid;
+  }
+
+  static PdbSymUid makeCompilandId(llvm::codeview::ProcRefSym sym) {
+    // S_PROCREF symbols are 1-based
+    lldbassert(sym.Module > 0);
+    return makeCompilandId(sym.Module - 1);
+  }
+
+  static PdbSymUid makeCompilandId(uint16_t modi) {
+    PdbSymUid uid;
+    uid.m_uid.comp_id.modi = modi;
+    uid.m_uid.cu_sym.tag =
+        static_cast<uint8_t>(llvm::pdb::PDB_SymType::Compiland);
+    return uid;
+  }
+
+  static PdbSymUid makeTypeSymId(llvm::pdb::PDB_SymType type,
+                                 llvm::codeview::TypeIndex index, bool is_ipi) {
+    lldbassert(isTypeSym(type));
+
+    PdbSymUid uid;
+    uid.m_uid.type_sym.tag = static_cast<uint8_t>(type);
+    uid.m_uid.type_sym.index = index.getIndex();
+    uid.m_uid.type_sym.is_ipi = static_cast<uint8_t>(is_ipi);
+    return uid;
+  }
+
+  static PdbSymUid fromOpaqueId(uint64_t value) {
+    PdbSymUid result;
+    ::memcpy(&result.m_uid, &value, sizeof(value));
+    return result;
+  }
+
+  uint64_t toOpaqueId() const {
+    uint64_t result;
+    ::memcpy(&result, &m_uid, sizeof(m_uid));
+    return result;
+  }
+
+  bool isPubSym() const {
+    return tag() == llvm::pdb::PDB_SymType::PublicSymbol;
+  }
+  bool isCompiland() const {
+    return tag() == llvm::pdb::PDB_SymType::Compiland;
+  }
+
+  llvm::pdb::PDB_SymType tag() const {
+    return static_cast<llvm::pdb::PDB_SymType>(m_uid.comp_id.tag);
+  }
+
+  const PdbCompilandId &asCompiland() const {
+    lldbassert(tag() == llvm::pdb::PDB_SymType::Compiland);
+    return m_uid.comp_id;
+  }
+
+  const PdbCuSymId &asCuSym() const {
+    lldbassert(isCuSym(tag()));
+    return m_uid.cu_sym;
+  }
+
+  const PdbTypeSymId &asTypeSym() const {
+    lldbassert(isTypeSym(tag()));
+    return m_uid.type_sym;
+  }
+};
+
+struct SymbolAndUid {
+  llvm::codeview::CVSymbol sym;
+  PdbSymUid uid;
+};
+} // namespace npdb
+} // namespace lldb_private
+
+#endif

lldb/trunk/source/Plugins/SymbolFile/NativePDB/PdbUtil.h

+//===-- PdbUtil.h -----------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_PLUGINS_SYMBOLFILENATIVEPDB_PDBUTIL_H
+#define LLDB_PLUGINS_SYMBOLFILENATIVEPDB_PDBUTIL_H
+
+#include "llvm/DebugInfo/CodeView/SymbolRecord.h"
+#include "llvm/DebugInfo/PDB/PDBTypes.h"
+
+#include <tuple>
+#include <utility>
+
+namespace lldb_private {
+namespace npdb {
+
+struct SegmentOffset {
+  uint16_t segment = 0;
+  uint32_t offset = 0;
+};
+
+struct SegmentOffsetLength {
+  SegmentOffset so;
+  uint32_t length = 0;
+};
+
+llvm::pdb::PDB_SymType CVSymToPDBSym(llvm::codeview::SymbolKind kind);
+
+bool SymbolHasAddress(const llvm::codeview::CVSymbol &sym);
+bool SymbolIsCode(const llvm::codeview::CVSymbol &sym);
+
+SegmentOffset GetSegmentAndOffset(const llvm::codeview::CVSymbol &sym);
+SegmentOffsetLength
+GetSegmentOffsetAndLength(const llvm::codeview::CVSymbol &sym);
+
+template <typename RecordT> bool IsValidRecord(const RecordT &sym) {
+  return true;
+}
+
+inline bool IsValidRecord(const llvm::codeview::ProcRefSym &sym) {
+  // S_PROCREF symbols have 1-based module indices.
+  return sym.Module > 0;
+}
+
+} // namespace npdb
+} // namespace lldb_private
+
+#endif

lldb/trunk/source/Plugins/SymbolFile/NativePDB/PdbUtil.cpp

+//===-- PdbUtil.cpp ---------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "PdbUtil.h"
+
+#include "llvm/DebugInfo/CodeView/SymbolDeserializer.h"
+
+#include "lldb/Utility/LLDBAssert.h"
+
+using namespace lldb_private;
+using namespace lldb_private::npdb;
+using namespace llvm::codeview;
+using namespace llvm::pdb;
+
+llvm::pdb::PDB_SymType
+lldb_private::npdb::CVSymToPDBSym(llvm::codeview::SymbolKind kind) {
+  switch (kind) {
+  case S_COMPILE3:
+  case S_OBJNAME:
+    return PDB_SymType::CompilandDetails;
+  case S_ENVBLOCK:
+    return PDB_SymType::CompilandEnv;
+  case S_THUNK32:
+  case S_TRAMPOLINE:
+    return PDB_SymType::Thunk;
+  case S_COFFGROUP:
+    return PDB_SymType::CoffGroup;
+  case S_EXPORT:
+    return PDB_SymType::Export;
+  case S_LPROC32:
+  case S_GPROC32:
+  case S_LPROC32_DPC:
+    return PDB_SymType::Function;
+  case S_PUB32:
+    return PDB_SymType::PublicSymbol;
+  case S_INLINESITE:
+    return PDB_SymType::InlineSite;
+  case S_LOCAL:
+  case S_BPREL32:
+  case S_REGREL32:
+  case S_MANCONSTANT:
+  case S_CONSTANT:
+  case S_LDATA32:
+  case S_GDATA32:
+  case S_LMANDATA:
+  case S_GMANDATA:
+  case S_LTHREAD32:
+  case S_GTHREAD32:
+    return PDB_SymType::Data;
+  case S_BLOCK32:
+    return PDB_SymType::Block;
+  case S_LABEL32:
+    return PDB_SymType::Label;
+  case S_CALLSITEINFO:
+    return PDB_SymType::CallSite;
+  case S_HEAPALLOCSITE:
+    return PDB_SymType::HeapAllocationSite;
+  case S_CALLEES:
+    return PDB_SymType::Callee;
+  case S_CALLERS:
+    return PDB_SymType::Caller;
+  default:
+    lldbassert(false && "Invalid symbol record kind!");
+  }
+  return PDB_SymType::None;
+}
+
+bool lldb_private::npdb::SymbolHasAddress(const llvm::codeview::CVSymbol &sym) {
+  switch (sym.kind()) {
+  case S_GPROC32:
+  case S_LPROC32:
+  case S_GPROC32_ID:
+  case S_LPROC32_ID:
+  case S_LPROC32_DPC:
+  case S_LPROC32_DPC_ID:
+  case S_THUNK32:
+  case S_TRAMPOLINE:
+  case S_COFFGROUP:
+  case S_BLOCK32:
+  case S_LABEL32:
+  case S_CALLSITEINFO:
+  case S_HEAPALLOCSITE:
+  case S_LDATA32:
+  case S_GDATA32:
+  case S_LMANDATA:
+  case S_GMANDATA:
+  case S_LTHREAD32:
+  case S_GTHREAD32:
+    return true;
+  default:
+    return false;
+  }
+}
+
+bool lldb_private::npdb::SymbolIsCode(const llvm::codeview::CVSymbol &sym) {
+  switch (sym.kind()) {
+  case S_GPROC32:
+  case S_LPROC32:
+  case S_GPROC32_ID:
+  case S_LPROC32_ID:
+  case S_LPROC32_DPC:
+  case S_LPROC32_DPC_ID:
+  case S_THUNK32:
+  case S_TRAMPOLINE:
+  case S_COFFGROUP:
+  case S_BLOCK32:
+    return true;
+  default:
+    return false;
+  }
+}
+
+template <typename RecordT> RecordT createRecord(const CVSymbol &sym) {
+  RecordT record(static_cast<SymbolRecordKind>(sym.kind()));
+  cantFail(SymbolDeserializer::deserializeAs<RecordT>(sym, record));
+  return record;
+}
+
+template <typename RecordT>
+static SegmentOffset GetSegmentAndOffset(const CVSymbol &sym) {
+  RecordT record = createRecord<RecordT>(sym);
+  return {record.Segment, record.CodeOffset};
+}
+
+template <>
+SegmentOffset GetSegmentAndOffset<TrampolineSym>(const CVSymbol &sym) {
+  TrampolineSym record = createRecord<TrampolineSym>(sym);
+  return {record.ThunkSection, record.ThunkOffset};
+}
+
+template <> SegmentOffset GetSegmentAndOffset<Thunk32Sym>(const CVSymbol &sym) {
+  Thunk32Sym record = createRecord<Thunk32Sym>(sym);
+  return {record.Segment, record.Offset};
+}
+
+template <>
+SegmentOffset GetSegmentAndOffset<CoffGroupSym>(const CVSymbol &sym) {
+  CoffGroupSym record = createRecord<CoffGroupSym>(sym);
+  return {record.Segment, record.Offset};
+}
+
+template <> SegmentOffset GetSegmentAndOffset<DataSym>(const CVSymbol &sym) {
+  DataSym record = createRecord<DataSym>(sym);
+  return {record.Segment, record.DataOffset};
+}
+
+template <>
+SegmentOffset GetSegmentAndOffset<ThreadLocalDataSym>(const CVSymbol &sym) {
+  ThreadLocalDataSym record = createRecord<ThreadLocalDataSym>(sym);
+  return {record.Segment, record.DataOffset};
+}
+
+SegmentOffset
+lldb_private::npdb::GetSegmentAndOffset(const llvm::codeview::CVSymbol &sym) {
+  switch (sym.kind()) {
+  case S_GPROC32:
+  case S_LPROC32:
+  case S_GPROC32_ID:
+  case S_LPROC32_ID:
+  case S_LPROC32_DPC:
+  case S_LPROC32_DPC_ID:
+    return ::GetSegmentAndOffset<ProcSym>(sym);
+  case S_THUNK32:
+    return ::GetSegmentAndOffset<Thunk32Sym>(sym);
+    break;
+  case S_TRAMPOLINE:
+    return ::GetSegmentAndOffset<TrampolineSym>(sym);
+    break;
+  case S_COFFGROUP:
+    return ::GetSegmentAndOffset<CoffGroupSym>(sym);
+    break;
+  case S_BLOCK32:
+    return ::GetSegmentAndOffset<BlockSym>(sym);
+    break;
+  case S_LABEL32:
+    return ::GetSegmentAndOffset<LabelSym>(sym);
+    break;
+  case S_CALLSITEINFO:
+    return ::GetSegmentAndOffset<CallSiteInfoSym>(sym);
+    break;
+  case S_HEAPALLOCSITE:
+    return ::GetSegmentAndOffset<HeapAllocationSiteSym>(sym);
+    break;
+  case S_LDATA32:
+  case S_GDATA32:
+  case S_LMANDATA:
+  case S_GMANDATA:
+    return ::GetSegmentAndOffset<DataSym>(sym);
+    break;
+  case S_LTHREAD32:
+  case S_GTHREAD32:
+    return ::GetSegmentAndOffset<ThreadLocalDataSym>(sym);
+    break;
+  default:
+    lldbassert(false && "Record does not have a segment/offset!");
+  }
+  return {0, 0};
+}
+
+template <typename RecordT>
+SegmentOffsetLength GetSegmentOffsetAndLength(const CVSymbol &sym) {
+  RecordT record = createRecord<RecordT>(sym);
+  return {record.Segment, record.CodeOffset, record.CodeSize};
+}
+
+template <>
+SegmentOffsetLength
+GetSegmentOffsetAndLength<TrampolineSym>(const CVSymbol &sym) {
+  TrampolineSym record = createRecord<TrampolineSym>(sym);
+  return {record.ThunkSection, record.ThunkOffset, record.Size};
+}
+
+template <>
+SegmentOffsetLength GetSegmentOffsetAndLength<Thunk32Sym>(const CVSymbol &sym) {
+  Thunk32Sym record = createRecord<Thunk32Sym>(sym);
+  return {record.Segment, record.Offset, record.Length};
+}
+
+template <>
+SegmentOffsetLength
+GetSegmentOffsetAndLength<CoffGroupSym>(const CVSymbol &sym) {
+  CoffGroupSym record = createRecord<CoffGroupSym>(sym);
+  return {record.Segment, record.Offset, record.Size};
+}
+
+SegmentOffsetLength lldb_private::npdb::GetSegmentOffsetAndLength(
+    const llvm::codeview::CVSymbol &sym) {
+  switch (sym.kind()) {
+  case S_GPROC32:
+  case S_LPROC32:
+  case S_GPROC32_ID:
+  case S_LPROC32_ID:
+  case S_LPROC32_DPC:
+  case S_LPROC32_DPC_ID:
+    return ::GetSegmentOffsetAndLength<ProcSym>(sym);
+  case S_THUNK32:
+    return ::GetSegmentOffsetAndLength<Thunk32Sym>(sym);
+    break;
+  case S_TRAMPOLINE:
+    return ::GetSegmentOffsetAndLength<TrampolineSym>(sym);
+    break;
+  case S_COFFGROUP:
+    return ::GetSegmentOffsetAndLength<CoffGroupSym>(sym);
+    break;
+  case S_BLOCK32:
+    return ::GetSegmentOffsetAndLength<BlockSym>(sym);
+    break;
+  default:
+    lldbassert(false && "Record does not have a segment/offset/length triple!");
+  }
+  return {0, 0, 0};
+}

lldb/trunk/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.h

+//===-- SymbolFileNativePDB.h -----------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef lldb_Plugins_SymbolFile_PDB_SymbolFileNativePDB_h_
+#define lldb_Plugins_SymbolFile_PDB_SymbolFileNativePDB_h_
+
+#include "lldb/Core/UniqueCStringMap.h"
+#include "lldb/Symbol/SymbolFile.h"
+#include "lldb/Symbol/VariableList.h"
+#include "lldb/Utility/UserID.h"
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/DebugInfo/CodeView/CVRecord.h"
+#include "llvm/DebugInfo/CodeView/StringsAndChecksums.h"
+#include "llvm/DebugInfo/CodeView/SymbolRecord.h"
+#include "llvm/DebugInfo/PDB/Native/ModuleDebugStream.h"
+#include "llvm/DebugInfo/PDB/PDBTypes.h"
+
+#include "CompileUnitIndex.h"
+#include "PdbIndex.h"
+
+#include <unordered_map>
+
+namespace llvm {
+namespace pdb {
+class PDBFile;
+class PDBSymbol;
+class PDBSymbolCompiland;
+class PDBSymbolData;
+class PDBSymbolFunc;
+
+class DbiStream;
+class TpiStream;
+class TpiStream;
+class InfoStream;
+class PublicsStream;
+class GlobalsStream;
+class SymbolStream;
+class ModuleDebugStreamRef;
+} // namespace pdb
+} // namespace llvm
+
+namespace lldb_private {
+namespace npdb {
+
+class SymbolFileNativePDB : public lldb_private::SymbolFile {
+public:
+  //------------------------------------------------------------------
+  // Static Functions
+  //------------------------------------------------------------------
+  static void Initialize();
+
+  static void Terminate();
+
+  static void DebuggerInitialize(lldb_private::Debugger &debugger);
+
+  static lldb_private::ConstString GetPluginNameStatic();
+
+  static const char *GetPluginDescriptionStatic();
+
+  static lldb_private::SymbolFile *
+  CreateInstance(lldb_private::ObjectFile *obj_file);
+
+  //------------------------------------------------------------------
+  // Constructors and Destructors
+  //------------------------------------------------------------------
+  SymbolFileNativePDB(lldb_private::ObjectFile *ofile);
+
+  ~SymbolFileNativePDB() override;
+
+  uint32_t CalculateAbilities() override;
+
+  void InitializeObject() override;
+
+  //------------------------------------------------------------------
+  // Compile Unit function calls
+  //------------------------------------------------------------------
+
+  uint32_t GetNumCompileUnits() override;
+
+  lldb::CompUnitSP ParseCompileUnitAtIndex(uint32_t index) override;
+
+  lldb::LanguageType
+  ParseCompileUnitLanguage(const lldb_private::SymbolContext &sc) override;
+
+  size_t
+  ParseCompileUnitFunctions(const lldb_private::SymbolContext &sc) override;
+
+  bool
+  ParseCompileUnitLineTable(const lldb_private::SymbolContext &sc) override;
+
+  bool
+  ParseCompileUnitDebugMacros(const lldb_private::SymbolContext &sc) override;
+
+  bool ParseCompileUnitSupportFiles(
+      const lldb_private::SymbolContext &sc,
+      lldb_private::FileSpecList &support_files) override;
+
+  bool ParseImportedModules(
+      const lldb_private::SymbolContext &sc,
+      std::vector<lldb_private::ConstString> &imported_modules) override;
+
+  size_t ParseFunctionBlocks(const lldb_private::SymbolContext &sc) override;
+
+  size_t ParseTypes(const lldb_private::SymbolContext &sc) override {
+    return 0;
+  }
+  size_t
+  ParseVariablesForContext(const lldb_private::SymbolContext &sc) override {
+    return 0;
+  }
+  lldb_private::Type *ResolveTypeUID(lldb::user_id_t type_uid) override {
+    return nullptr;
+  }
+  bool CompleteType(lldb_private::CompilerType &compiler_type) override {
+    return false;
+  }
+  uint32_t ResolveSymbolContext(const lldb_private::Address &so_addr,
+                                uint32_t resolve_scope,
+                                lldb_private::SymbolContext &sc) override;
+
+  virtual size_t GetTypes(lldb_private::SymbolContextScope *sc_scope,
+                          uint32_t type_mask,
+                          lldb_private::TypeList &type_list) override {
+    return 0;
+  }
+
+  uint32_t
+  FindFunctions(const lldb_private::ConstString &name,
+                const lldb_private::CompilerDeclContext *parent_decl_ctx,
+                uint32_t name_type_mask, bool include_inlines, bool append,
+                lldb_private::SymbolContextList &sc_list) override;
+
+  uint32_t FindFunctions(const lldb_private::RegularExpression &regex,
+                         bool include_inlines, bool append,
+                         lldb_private::SymbolContextList &sc_list) override;
+
+  lldb_private::TypeSystem *
+  GetTypeSystemForLanguage(lldb::LanguageType language) override;
+
+  lldb_private::CompilerDeclContext FindNamespace(
+      const lldb_private::SymbolContext &sc,
+      const lldb_private::ConstString &name,
+      const lldb_private::CompilerDeclContext *parent_decl_ctx) override;
+
+  lldb_private::ConstString GetPluginName() override;
+
+  uint32_t GetPluginVersion() override;
+
+  llvm::pdb::PDBFile &GetPDBFile() { return m_index->pdb(); }
+  const llvm::pdb::PDBFile &GetPDBFile() const { return m_index->pdb(); }
+
+private:
+  lldb::FunctionSP GetOrCreateFunction(PdbSymUid func_uid,
+                                       const SymbolContext &sc);
+  lldb::CompUnitSP GetOrCreateCompileUnit(const CompilandIndexItem &cci);
+
+  lldb::FunctionSP CreateFunction(PdbSymUid func_uid, const SymbolContext &sc);
+  lldb::CompUnitSP CreateCompileUnit(const CompilandIndexItem &cci);
+
+  llvm::BumpPtrAllocator m_allocator;
+
+  lldb::addr_t m_obj_load_address = 0;
+
+  std::unique_ptr<PdbIndex> m_index;
+
+  llvm::DenseMap<lldb::user_id_t, lldb::FunctionSP> m_functions;
+  llvm::DenseMap<lldb::user_id_t, lldb::CompUnitSP> m_compilands;
+};
+
+} // namespace npdb
+} // namespace lldb_private
+
+#endif // lldb_Plugins_SymbolFile_PDB_SymbolFilePDB_h_

lldb/trunk/source/Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.cpp

+//===-- SymbolFileNativePDB.cpp ---------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "SymbolFileNativePDB.h"
+
+#include "clang/Lex/Lexer.h"
+
+#include "lldb/Core/Module.h"
+#include "lldb/Core/PluginManager.h"
+#include "lldb/Symbol/CompileUnit.h"
+#include "lldb/Symbol/LineTable.h"
+#include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Symbol/SymbolContext.h"
+#include "lldb/Symbol/SymbolVendor.h"
+
+#include "llvm/DebugInfo/CodeView/CVRecord.h"
+#include "llvm/DebugInfo/CodeView/DebugLinesSubsection.h"
+#include "llvm/DebugInfo/CodeView/RecordName.h"
+#include "llvm/DebugInfo/CodeView/SymbolDeserializer.h"
+#include "llvm/DebugInfo/PDB/Native/DbiStream.h"
+#include "llvm/DebugInfo/PDB/Native/GlobalsStream.h"
+#include "llvm/DebugInfo/PDB/Native/InfoStream.h"
+#include "llvm/DebugInfo/PDB/Native/ModuleDebugStream.h"
+#include "llvm/DebugInfo/PDB/Native/PDBFile.h"
+#include "llvm/DebugInfo/PDB/Native/SymbolStream.h"
+#include "llvm/DebugInfo/PDB/PDBTypes.h"
+#include "llvm/Object/COFF.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/BinaryStreamReader.h"
+#include "llvm/Support/ErrorOr.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+#include "PdbSymUid.h"
+#include "PdbUtil.h"
+
+using namespace lldb;
+using namespace lldb_private;
+using namespace lldb_private::npdb;
+using namespace llvm::codeview;
+using namespace llvm::pdb;
+
+static lldb::LanguageType TranslateLanguage(PDB_Lang lang) {
+  switch (lang) {
+  case PDB_Lang::Cpp:
+    return lldb::LanguageType::eLanguageTypeC_plus_plus;
+  case PDB_Lang::C:
+    return lldb::LanguageType::eLanguageTypeC;
+  default:
+    return lldb::LanguageType::eLanguageTypeUnknown;
+  }
+}
+
+static std::unique_ptr<PDBFile> loadPDBFile(std::string PdbPath,
+                                            llvm::BumpPtrAllocator &Allocator) {
+  llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> ErrorOrBuffer =
+      llvm::MemoryBuffer::getFile(PdbPath, /*FileSize=*/-1,
+                                  /*RequiresNullTerminator=*/false);
+  if (!ErrorOrBuffer)
+    return nullptr;
+  std::unique_ptr<llvm::MemoryBuffer> Buffer = std::move(*ErrorOrBuffer);
+
+  llvm::StringRef Path = Buffer->getBufferIdentifier();
+  auto Stream = llvm::make_unique<llvm::MemoryBufferByteStream>(
+      std::move(Buffer), llvm::support::little);
+
+  auto File = llvm::make_unique<PDBFile>(Path, std::move(Stream), Allocator);
+  if (auto EC = File->parseFileHeaders())
+    return nullptr;
+  if (auto EC = File->parseStreamData())
+    return nullptr;
+
+  return std::move(File);
+}
+
+static std::unique_ptr<PDBFile>
+loadMatchingPDBFile(std::string exe_path, llvm::BumpPtrAllocator &allocator) {
+  // Try to find a matching PDB for an EXE.
+  using namespace llvm::object;
+  auto expected_binary = createBinary(exe_path);
+
+  // If the file isn't a PE/COFF executable, fail.
+  if (!expected_binary) {
+    llvm::consumeError(expected_binary.takeError());
+    return nullptr;
+  }
+  OwningBinary<Binary> binary = std::move(*expected_binary);
+
+  auto *obj = llvm::dyn_cast<llvm::object::COFFObjectFile>(binary.getBinary());
+  if (!obj)
+    return nullptr;
+  const llvm::codeview::DebugInfo *pdb_info = nullptr;
+
+  // If it doesn't have a debug directory, fail.
+  llvm::StringRef pdb_file;
+  auto ec = obj->getDebugPDBInfo(pdb_info, pdb_file);
+  if (ec)
+    return nullptr;
+
+  // if the file doesn't exist, is not a pdb, or doesn't have a matching guid,
+  // fail.
+  llvm::file_magic magic;
+  ec = llvm::identify_magic(pdb_file, magic);
+  if (ec || magic != llvm::file_magic::pdb)
+    return nullptr;
+  std::unique_ptr<PDBFile> pdb = loadPDBFile(pdb_file, allocator);
+  auto expected_info = pdb->getPDBInfoStream();
+  if (!expected_info) {
+    llvm::consumeError(expected_info.takeError());
+    return nullptr;
+  }
+  llvm::codeview::GUID guid;
+  memcpy(&guid, pdb_info->PDB70.Signature, 16);
+
+  if (expected_info->getGuid() != guid)
+    return nullptr;
+  return std::move(pdb);
+}
+
+static bool IsFunctionPrologue(const CompilandIndexItem &cci,
+                               lldb::addr_t addr) {
+  // FIXME: Implement this.
+  return false;
+}
+
+static bool IsFunctionEpilogue(const CompilandIndexItem &cci,
+                               lldb::addr_t addr) {
+  // FIXME: Implement this.
+  return false;
+}
+
+void SymbolFileNativePDB::Initialize() {
+  PluginManager::RegisterPlugin(GetPluginNameStatic(),
+                                GetPluginDescriptionStatic(), CreateInstance,
+                                DebuggerInitialize);
+}
+
+void SymbolFileNativePDB::Terminate() {
+  PluginManager::UnregisterPlugin(CreateInstance);
+}
+
+void SymbolFileNativePDB::DebuggerInitialize(lldb_private::Debugger &debugger) {
+}
+
+lldb_private::ConstString SymbolFileNativePDB::GetPluginNameStatic() {
+  static ConstString g_name("native-pdb");
+  return g_name;
+}
+
+const char *SymbolFileNativePDB::GetPluginDescriptionStatic() {
+  return "Microsoft PDB debug symbol cross-platform file reader.";
+}
+
+lldb_private::SymbolFile *
+SymbolFileNativePDB::CreateInstance(lldb_private::ObjectFile *obj_file) {
+  return new SymbolFileNativePDB(obj_file);
+}
+
+SymbolFileNativePDB::SymbolFileNativePDB(lldb_private::ObjectFile *object_file)
+    : SymbolFile(object_file) {}
+
+SymbolFileNativePDB::~SymbolFileNativePDB() {}
+
+uint32_t SymbolFileNativePDB::CalculateAbilities() {
+  uint32_t abilities = 0;
+  if (!m_obj_file)
+    return 0;
+
+  if (!m_index) {
+    // Lazily load and match the PDB file, but only do this once.
+    std::unique_ptr<PDBFile> file_up =
+        loadMatchingPDBFile(m_obj_file->GetFileSpec().GetPath(), m_allocator);
+
+    if (!file_up) {
+      auto module_sp = m_obj_file->GetModule();
+      if (!module_sp)
+        return 0;
+      // See if any symbol file is specified through `--symfile` option.
+      FileSpec symfile = module_sp->GetSymbolFileFileSpec();
+      if (!symfile)
+        return 0;
+      file_up = loadPDBFile(symfile.GetPath(), m_allocator);
+    }
+
+    if (!file_up)
+      return 0;
+
+    auto expected_index = PdbIndex::create(std::move(file_up));
+    if (!expected_index) {
+      llvm::consumeError(expected_index.takeError());
+      return 0;
+    }
+    m_index = std::move(*expected_index);
+  }
+  if (!m_index)
+    return 0;
+
+  // We don't especially have to be precise here.  We only distinguish between
+  // stripped and not stripped.
+  abilities = kAllAbilities;
+
+  if (m_index->dbi().isStripped())
+    abilities &= ~(Blocks | LocalVariables);
+  return abilities;
+}
+
+void SymbolFileNativePDB::InitializeObject() {
+  m_obj_load_address = m_obj_file->GetFileOffset();
+  m_index->SetLoadAddress(m_obj_load_address);
+  m_index->ParseSectionContribs();
+}
+
+uint32_t SymbolFileNativePDB::GetNumCompileUnits() {
+  const DbiModuleList &modules = m_index->dbi().modules();
+  uint32_t count = modules.getModuleCount();
+  if (count == 0)
+    return count;
+
+  // The linker can inject an additional "dummy" compilation unit into the
+  // PDB. Ignore this special compile unit for our purposes, if it is there.
+  // It is always the last one.
+  DbiModuleDescriptor last = modules.getModuleDescriptor(count - 1);
+  if (last.getModuleName() == "* Linker *")
+    --count;
+  return count;
+}
+
+lldb::FunctionSP SymbolFileNativePDB::CreateFunction(PdbSymUid func_uid,
+                                                     const SymbolContext &sc) {
+  lldbassert(func_uid.tag() == PDB_SymType::Function);
+
+  PdbSymUid cuid = PdbSymUid::makeCompilandId(func_uid.asCuSym().modi);
+
+  const CompilandIndexItem *cci = m_index->compilands().GetCompiland(cuid);
+  lldbassert(cci);
+  CVSymbol sym_record =
+      cci->m_debug_stream.readSymbolAtOffset(func_uid.asCuSym().offset);
+
+  lldbassert(sym_record.kind() == S_LPROC32 || sym_record.kind() == S_GPROC32);
+  SegmentOffsetLength sol = GetSegmentOffsetAndLength(sym_record);
+
+  auto file_vm_addr = m_index->MakeVirtualAddress(sol.so);
+  if (file_vm_addr == LLDB_INVALID_ADDRESS || file_vm_addr == 0)
+    return nullptr;
+
+  AddressRange func_range(file_vm_addr, sol.length,
+                          sc.module_sp->GetSectionList());
+  if (!func_range.GetBaseAddress().IsValid())
+    return nullptr;
+
+  lldb_private::Type *func_type = nullptr;
+
+  // FIXME: Resolve types and mangled names.
+  PdbSymUid sig_uid =
+      PdbSymUid::makeTypeSymId(PDB_SymType::FunctionSig, TypeIndex{0}, false);
+  Mangled mangled(getSymbolName(sym_record));
+
+  FunctionSP func_sp = std::make_shared<Function>(
+      sc.comp_unit, func_uid.toOpaqueId(), sig_uid.toOpaqueId(), mangled,
+      func_type, func_range);
+
+  sc.comp_unit->AddFunction(func_sp);
+  return func_sp;
+}
+
+CompUnitSP
+SymbolFileNativePDB::CreateCompileUnit(const CompilandIndexItem &cci) {
+  lldb::LanguageType lang =
+      cci.m_compile_opts ? TranslateLanguage(cci.m_compile_opts->getLanguage())
+                         : lldb::eLanguageTypeUnknown;
+
+  LazyBool optimized = eLazyBoolNo;
+  if (cci.m_compile_opts && cci.m_compile_opts->hasOptimizations())
+    optimized = eLazyBoolYes;
+
+  llvm::StringRef source_file_name =
+      m_index->compilands().GetMainSourceFile(cci);
+  lldb_private::FileSpec fs(source_file_name, false);
+
+  CompUnitSP cu_sp =
+      std::make_shared<CompileUnit>(m_obj_file->GetModule(), nullptr, fs,
+                                    cci.m_uid.toOpaqueId(), lang, optimized);
+
+  const PdbCompilandId &cuid = cci.m_uid.asCompiland();
+  m_obj_file->GetModule()->GetSymbolVendor()->SetCompileUnitAtIndex(cuid.modi,
+                                                                    cu_sp);
+  return cu_sp;
+}
+
+FunctionSP SymbolFileNativePDB::GetOrCreateFunction(PdbSymUid func_uid,
+                                                    const SymbolContext &sc) {
+  lldbassert(func_uid.tag() == PDB_SymType::Function);
+  auto emplace_result = m_functions.try_emplace(func_uid.toOpaqueId(), nullptr);
+  if (emplace_result.second)
+    emplace_result.first->second = CreateFunction(func_uid, sc);
+
+  lldbassert(emplace_result.first->second);
+  return emplace_result.first->second;
+}
+
+CompUnitSP
+SymbolFileNativePDB::GetOrCreateCompileUnit(const CompilandIndexItem &cci) {
+  auto emplace_result =
+      m_compilands.try_emplace(cci.m_uid.toOpaqueId(), nullptr);
+  if (emplace_result.second)
+    emplace_result.first->second = CreateCompileUnit(cci);
+
+  lldbassert(emplace_result.first->second);
+  return emplace_result.first->second;
+}
+
+lldb::CompUnitSP SymbolFileNativePDB::ParseCompileUnitAtIndex(uint32_t index) {
+  if (index >= GetNumCompileUnits())
+    return CompUnitSP();
+  lldbassert(index < UINT16_MAX);
+  if (index >= UINT16_MAX)
+    return nullptr;
+
+  CompilandIndexItem &item = m_index->compilands().GetOrCreateCompiland(index);
+
+  return GetOrCreateCompileUnit(item);
+}
+
+lldb::LanguageType SymbolFileNativePDB::ParseCompileUnitLanguage(
+    const lldb_private::SymbolContext &sc) {
+  // What fields should I expect to be filled out on the SymbolContext?  Is it
+  // safe to assume that `sc.comp_unit` is valid?
+  if (!sc.comp_unit)
+    return lldb::eLanguageTypeUnknown;
+  PdbSymUid uid = PdbSymUid::fromOpaqueId(sc.comp_unit->GetID());
+  lldbassert(uid.tag() == PDB_SymType::Compiland);
+
+  CompilandIndexItem *item = m_index->compilands().GetCompiland(uid);
+  lldbassert(item);
+  if (!item->m_compile_opts)
+    return lldb::eLanguageTypeUnknown;
+
+  return TranslateLanguage(item->m_compile_opts->getLanguage());
+}
+
+size_t SymbolFileNativePDB::ParseCompileUnitFunctions(
+    const lldb_private::SymbolContext &sc) {
+  lldbassert(sc.comp_unit);
+  return false;
+}
+
+static bool NeedsResolvedCompileUnit(uint32_t resolve_scope) {
+  // If any of these flags are set, we need to resolve the compile unit.
+  uint32_t flags = eSymbolContextCompUnit;
+  flags |= eSymbolContextVariable;
+  flags |= eSymbolContextFunction;
+  flags |= eSymbolContextBlock;
+  flags |= eSymbolContextLineEntry;
+  return (resolve_scope & flags) != 0;
+}
+
+uint32_t
+SymbolFileNativePDB::ResolveSymbolContext(const lldb_private::Address &addr,
+                                          uint32_t resolve_scope,
+                                          lldb_private::SymbolContext &sc) {
+  uint32_t resolved_flags = 0;
+  lldb::addr_t file_addr = addr.GetFileAddress();
+
+  if (NeedsResolvedCompileUnit(resolve_scope)) {
+    llvm::Optional<uint16_t> modi = m_index->GetModuleIndexForVa(file_addr);
+    if (!modi)
+      return 0;
+    PdbSymUid cuid = PdbSymUid::makeCompilandId(*modi);
+    CompilandIndexItem *cci = m_index->compilands().GetCompiland(cuid);
+    if (!cci)
+      return 0;
+
+    sc.comp_unit = GetOrCreateCompileUnit(*cci).get();
+    resolved_flags |= eSymbolContextCompUnit;
+  }
+
+  if (resolve_scope & eSymbolContextFunction) {
+    lldbassert(sc.comp_unit);
+    std::vector<SymbolAndUid> matches = m_index->FindSymbolsByVa(file_addr);
+    for (const auto &match : matches) {
+      if (match.uid.tag() != PDB_SymType::Function)
+        continue;
+      sc.function = GetOrCreateFunction(match.uid, sc).get();
+    }
+    resolved_flags |= eSymbolContextFunction;
+  }
+
+  if (resolve_scope & eSymbolContextLineEntry) {
+    lldbassert(sc.comp_unit);
+    if (auto *line_table = sc.comp_unit->GetLineTable()) {
+      if (line_table->FindLineEntryByAddress(addr, sc.line_entry))
+        resolved_flags |= eSymbolContextLineEntry;
+    }
+  }
+
+  return resolved_flags;
+}
+
+static void AppendLineEntryToSequence(LineTable &table, LineSequence &sequence,
+                                      const CompilandIndexItem &cci,
+                                      lldb::addr_t base_addr,
+                                      uint32_t file_number,
+                                      const LineFragmentHeader &block,
+                                      const LineNumberEntry &cur) {
+  LineInfo cur_info(cur.Flags);
+
+  if (cur_info.isAlwaysStepInto() || cur_info.isNeverStepInto())
+    return;
+
+  uint64_t addr = base_addr + cur.Offset;
+
+  bool is_statement = cur_info.isStatement();
+  bool is_prologue = IsFunctionPrologue(cci, addr);
+  bool is_epilogue = IsFunctionEpilogue(cci, addr);
+
+  uint32_t lno = cur_info.getStartLine();
+
+  table.AppendLineEntryToSequence(&sequence, addr, lno, 0, file_number,
+                                  is_statement, false, is_prologue, is_epilogue,
+                                  false);
+}
+
+static void TerminateLineSequence(LineTable &table,
+                                  const LineFragmentHeader &block,
+                                  lldb::addr_t base_addr, uint32_t file_number,
+                                  uint32_t last_line,
+                                  std::unique_ptr<LineSequence> seq) {
+  // The end is always a terminal entry, so insert it regardless.
+  table.AppendLineEntryToSequence(seq.get(), base_addr + block.CodeSize,
+                                  last_line, 0, file_number, false, false,
+                                  false, false, true);
+  table.InsertSequence(seq.release());
+}
+
+bool SymbolFileNativePDB::ParseCompileUnitLineTable(
+    const lldb_private::SymbolContext &sc) {
+  // Unfortunately LLDB is set up to parse the entire compile unit line table
+  // all at once, even if all it really needs is line info for a specific
+  // function.  In the future it would be nice if it could set the sc.m_function
+  // member, and we could only get the line info for the function in question.
+  lldbassert(sc.comp_unit);
+  PdbSymUid cu_id = PdbSymUid::fromOpaqueId(sc.comp_unit->GetID());
+  lldbassert(cu_id.isCompiland());
+  CompilandIndexItem *cci = m_index->compilands().GetCompiland(cu_id);
+  lldbassert(cci);
+  auto line_table = llvm::make_unique<LineTable>(sc.comp_unit);
+
+  // This is basically a copy of the .debug$S subsections from all original COFF
+  // object files merged together with address relocations applied.  We are
+  // looking for all DEBUG_S_LINES subsections.
+  for (const DebugSubsectionRecord &dssr :
+       cci->m_debug_stream.getSubsectionsArray()) {
+    if (dssr.kind() != DebugSubsectionKind::Lines)
+      continue;
+
+    DebugLinesSubsectionRef lines;
+    llvm::BinaryStreamReader reader(dssr.getRecordData());
+    if (auto EC = lines.initialize(reader)) {
+      llvm::consumeError(std::move(EC));
+      return false;
+    }
+
+    const LineFragmentHeader *lfh = lines.header();
+    uint64_t virtual_addr =
+        m_index->MakeVirtualAddress(lfh->RelocSegment, lfh->RelocOffset);
+
+    const auto &checksums = cci->m_strings.checksums().getArray();
+    const auto &strings = cci->m_strings.strings();
+    for (const LineColumnEntry &group : lines) {
+      // Indices in this structure are actually offsets of records in the
+      // DEBUG_S_FILECHECKSUMS subsection.  Those entries then have an index
+      // into the global PDB string table.
+      auto iter = checksums.at(group.NameIndex);
+      if (iter == checksums.end())
+        continue;
+
+      llvm::Expected<llvm::StringRef> efn =
+          strings.getString(iter->FileNameOffset);
+      if (!efn) {
+        llvm::consumeError(efn.takeError());
+        continue;
+      }
+
+      // LLDB wants the index of the file in the list of support files.
+      llvm::StringRef file_name = *efn;
+      auto fn_iter = llvm::find(cci->m_file_list, *efn);
+      lldbassert(fn_iter != cci->m_file_list.end());
+      uint32_t file_index = std::distance(cci->m_file_list.begin(), fn_iter);
+
+      std::unique_ptr<LineSequence> sequence(
+          line_table->CreateLineSequenceContainer());
+      lldbassert(!group.LineNumbers.empty());
+
+      for (const LineNumberEntry &entry : group.LineNumbers) {
+        AppendLineEntryToSequence(*line_table, *sequence, *cci, virtual_addr,
+                                  file_index, *lfh, entry);
+      }
+      LineInfo last_line(group.LineNumbers.back().Flags);
+      TerminateLineSequence(*line_table, *lfh, virtual_addr, file_index,
+                            last_line.getEndLine(), std::move(sequence));
+    }
+  }
+
+  if (line_table->GetSize() == 0)
+    return false;
+
+  sc.comp_unit->SetLineTable(line_table.release());
+  return true;
+}
+
+bool SymbolFileNativePDB::ParseCompileUnitDebugMacros(
+    const lldb_private::SymbolContext &sc) {
+  // PDB doesn't contain information about macros
+  return false;
+}
+
+bool SymbolFileNativePDB::ParseCompileUnitSupportFiles(
+    const lldb_private::SymbolContext &sc,
+    lldb_private::FileSpecList &support_files) {
+  lldbassert(sc.comp_unit);
+
+  PdbSymUid comp_uid = PdbSymUid::fromOpaqueId(sc.comp_unit->GetID());
+  lldbassert(comp_uid.tag() == PDB_SymType::Compiland);
+
+  const CompilandIndexItem *cci = m_index->compilands().GetCompiland(comp_uid);
+  lldbassert(cci);
+
+  for (llvm::StringRef f : cci->m_file_list) {
+    FileSpec spec(f, false, FileSpec::Style::windows);
+    support_files.Append(spec);
+  }
+
+  return true;
+}
+
+bool SymbolFileNativePDB::ParseImportedModules(
+    const lldb_private::SymbolContext &sc,
+    std::vector<lldb_private::ConstString> &imported_modules) {
+  // PDB does not yet support module debug info
+  return false;
+}
+
+size_t SymbolFileNativePDB::ParseFunctionBlocks(
+    const lldb_private::SymbolContext &sc) {
+  lldbassert(sc.comp_unit && sc.function);
+  return 0;
+}
+
+uint32_t SymbolFileNativePDB::FindFunctions(
+    const lldb_private::ConstString &name,
+    const lldb_private::CompilerDeclContext *parent_decl_ctx,
+    uint32_t name_type_mask, bool include_inlines, bool append,
+    lldb_private::SymbolContextList &sc_list) {
+  // For now we only support lookup by method name.
+  if (!(name_type_mask & eFunctionNameTypeMethod))
+    return 0;
+
+  using SymbolAndOffset = std::pair<uint32_t, llvm::codeview::CVSymbol>;
+
+  std::vector<SymbolAndOffset> matches = m_index->globals().findRecordsByName(
+      name.GetStringRef(), m_index->symrecords());
+  for (const SymbolAndOffset &match : matches) {
+    if (match.second.kind() != S_PROCREF && match.second.kind() != S_LPROCREF)
+      continue;
+    ProcRefSym proc(match.second.kind());
+    cantFail(SymbolDeserializer::deserializeAs<ProcRefSym>(match.second, proc));
+
+    if (!IsValidRecord(proc))
+      continue;
+
+    PdbSymUid cuid = PdbSymUid::makeCompilandId(proc);
+    CompilandIndexItem &cci = m_index->compilands().GetOrCreateCompiland(cuid);
+    lldb_private::SymbolContext sc;
+
+    sc.comp_unit = GetOrCreateCompileUnit(cci).get();
+    sc.module_sp = sc.comp_unit->GetModule();
+    PdbSymUid func_uid = PdbSymUid::makeCuSymId(proc);
+    sc.function = GetOrCreateFunction(func_uid, sc).get();
+
+    sc_list.Append(sc);
+  }
+
+  return sc_list.GetSize();
+}
+
+uint32_t
+SymbolFileNativePDB::FindFunctions(const lldb_private::RegularExpression &regex,
+                                   bool include_inlines, bool append,
+                                   lldb_private::SymbolContextList &sc_list) {
+  return 0;
+}
+
+lldb_private::CompilerDeclContext SymbolFileNativePDB::FindNamespace(
+    const lldb_private::SymbolContext &sc,
+    const lldb_private::ConstString &name,
+    const lldb_private::CompilerDeclContext *parent_decl_ctx) {
+  return {};
+}
+
+lldb_private::TypeSystem *
+SymbolFileNativePDB::GetTypeSystemForLanguage(lldb::LanguageType language) {
+  auto type_system =
+      m_obj_file->GetModule()->GetTypeSystemForLanguage(language);
+  if (type_system)
+    type_system->SetSymbolFile(this);
+  return type_system;
+}
+
+lldb_private::ConstString SymbolFileNativePDB::GetPluginName() {
+  static ConstString g_name("pdb");
+  return g_name;
+}
+
+uint32_t SymbolFileNativePDB::GetPluginVersion() { return 1; }

lldb/trunk/source/Plugins/SymbolFile/PDB/CMakeLists.txt

 add_lldb_library(lldbPluginSymbolFilePDB PLUGIN
   PDBASTParser.cpp
   PDBLocationToDWARFExpression.cpp
   SymbolFilePDB.cpp
 
   LINK_LIBS
     clangAST
     clangLex
     lldbCore
     lldbSymbol
   lldbUtility
+  lldbPluginSymbolFileNativePDB
   LINK_COMPONENTS
     DebugInfoPDB
     Support
   )

lldb/trunk/source/Plugins/SymbolFile/PDB/SymbolFilePDB.cpp

 #include "llvm/DebugInfo/PDB/PDBSymbolFuncDebugEnd.h"
 #include "llvm/DebugInfo/PDB/PDBSymbolFuncDebugStart.h"
 #include "llvm/DebugInfo/PDB/PDBSymbolPublicSymbol.h"
 #include "llvm/DebugInfo/PDB/PDBSymbolTypeEnum.h"
 #include "llvm/DebugInfo/PDB/PDBSymbolTypeTypedef.h"
 #include "llvm/DebugInfo/PDB/PDBSymbolTypeUDT.h"
 
 #include "Plugins/Language/CPlusPlus/CPlusPlusLanguage.h" // For IsCPPMangledName
+#include "Plugins/SymbolFile/NativePDB/SymbolFileNativePDB.h"
 #include "Plugins/SymbolFile/PDB/PDBASTParser.h"
 #include "Plugins/SymbolFile/PDB/PDBLocationToDWARFExpression.h"
 
 #include <regex>
 
 using namespace lldb;
 using namespace lldb_private;
 using namespace llvm::pdb;

 
 bool ShouldAddLine(uint32_t requested_line, uint32_t actual_line,
                    uint32_t addr_length) {
   return ((requested_line == 0 || actual_line == requested_line) &&
           addr_length > 0);
 }
 } // namespace
 
+static bool ShouldUseNativeReader() {
+#if !defined(_WIN32)
+  return true;
+#endif
+  llvm::StringRef use_native = ::getenv("LLDB_USE_NATIVE_PDB_READER");
+  return use_native.equals_lower("on") || use_native.equals_lower("yes") ||
+         use_native.equals_lower("1") || use_native.equals_lower("true");
+}
+
 void SymbolFilePDB::Initialize() {
+  if (ShouldUseNativeReader()) {
+    npdb::SymbolFileNativePDB::Initialize();
+  } else {
     PluginManager::RegisterPlugin(GetPluginNameStatic(),
                                   GetPluginDescriptionStatic(), CreateInstance,
                                   DebuggerInitialize);
   }
+}
 
 void SymbolFilePDB::Terminate() {
+  if (ShouldUseNativeReader()) {
+    npdb::SymbolFileNativePDB::Terminate();
+  } else {
     PluginManager::UnregisterPlugin(CreateInstance);
   }
+}
 
 void SymbolFilePDB::DebuggerInitialize(lldb_private::Debugger &debugger) {}
 
 lldb_private::ConstString SymbolFilePDB::GetPluginNameStatic() {
   static ConstString g_name("pdb");
   return g_name;
 }