Add a generic build script for building test inferiors

[zturner]

Summary

This is an alternative approach to D54731. Instead of allow the user to invoke arbitrary python code directly from inside of a test, this patch adds a python script called build.py which we can use the existing substitution system for. For example, we could write something like:

RUN: %build --arch=32 --source=%p/Inputs/foo.cpp --output=%t.exe --mode=compile-and-link

Currently, the build script requires you to pass an explicit compiler, so you would need to pass something like --compiler=path/to/clang++, and it will fail without this. So this isn't quite ready for immediate use. We need to figure out how to get that compiler from CMake into this script. Maybe it means bringing back LLDB_TEST_COMPILER (sigh). Maybe it means auto-detecting the most appropriate compiler by looking in the tools dir, perhaps with something like an additional command line --find-toolchain=msvc or --find-toolchain=clang.

We have some tests right now that require MSVC, so while we can always just keep using a cl.exe command line, it would be nice to be able to port these over and say something like

RUN: %build --find-toolchain=msvc --arch=32 --source=%p/Inputs/foo.cpp --output=%t.exe --find-toolchain=msvc --mode=compile-and-link

For now though, I've tested this on the command line with an explicit --compiler option, and it works with clang-cl + lld-link as well as cl.exe + link.exe, using both Python 2 and Python 3.

I have not implemented the GCC/clang builder yet, although I plan to do that in followups.

Note that at least for Windows, this new builder is significantly better than our existing method of using run lines, because it has advanced toolchain detection logic. Our existing scripts fail when being run from inside of Visual Studio and they also don't allow building an explicit architecture for technical reasons, but with this method we can support both.

[JDevlieghere]

I personally prefer this approach, if only for the fact that the substitutions are more familiar and that this change would be local to lldb.

[zturner]

It turned out to not be super complicated, so I went ahead and made it support toolchain auto-detection. This allows the substitution to bake in a command line parameter of --compiler=any, so the user doesn't have to specify anything when writing the test. But since subsequent arguments take precedence, they can pass --compiler=clang or something later and it will override the default value. I also made the arch default to 64 bit. Finally, I updated a test to actually use this to prove that it works.

[zturner]

I went ahead and fixed up the rest of the native pdb tests. This exposed a couple of minor issues with the build script which I also fixed in the process.

@stella.stamenova would you mind giving this a try? I think it will fix all your problems of running tests from inside of Visual Studio.

@aleksandr.urakov Would you also mind giving this a try? I think it will mean that you can re-write the SymbolFile/PDB tests to not be dependent on running from a VS command prompt. You should be able to pass --arch=32 or --arch=64 no matter which command prompt you are running from, and it should still work.

[aleksandr.urakov]

In D54914#1308919, @zturner wrote:

@aleksandr.urakov Would you also mind giving this a try? I think it will mean that you can re-write the SymbolFile/PDB tests to not be dependent on running from a VS command prompt. You should be able to pass --arch=32 or --arch=64 no matter which command prompt you are running from, and it should still work.

Sure, thank you for the new build system! I've created a review here: D54942. There are some problems left, I've explained them in the review description.

[labath]

I didn't look at the code in detail, as most of it deals with windows stuff, and I don't know much about those anyway. However, the interesting question for me would be how to make this useful for cross-compiling. Right now that sort of works for NativePDB tests because --compiler=clang-cl implies windows, but that won't help if I want to compile say a linux arm64 binary. I think that instead --arch, we should have a --triple argument, which specifies the exact target you want to build for. That can default to "host", and we can have special pseudo-triples like "host32" and "host64", if we want to be able to say "I want to build for a 32-bit flavour of the host arch". That could also make gcc detection easier, since you could just search for $triple-gcc.

Another route to take might be to say that this script only supports building for the host, and tests that want to target a specific architecture can just invoke the appropriate compiler directly -- since they know the target arch, they also know the correct compiler option syntax. Plus, these kinds of tests are the ones that are most likely to need fancy compiler command line switches that might be hard to represent generically. In this world, the NativePDB tests would continue to invoke %clang-cl (which would point to the build dir without any fancy logic), and build.py would be used by scripts that just want to build an executable for the host, and don't care much about the details of how this is accomplished (the stop-hook tests are a good example of this).

(Among the cosmetic things, I'd suggest to make --source a positional argument and enable spelling of --output as -o, just to make things shorter.)

[aprantl]

I would like to ask a general question that I (indirectly) also asked in D54731: Why do we want to implement support for building inferiors in LIT-based tests? IMHO, if we need to support for dealing with specific compilers, we should implement that once in Makefile.rules (which is in a declarative domain-specific-language for describing build logic) and write a dotest.py-style test instead. I'm assuming here that we need the support in Makefile.rules anyway to support the bulk of the tests. Having this support in two places increases the maintenance effort and cognitive load.

[zturner]

In D54914#1309901, @aprantl wrote:

I would like to ask a general question that I (indirectly) also asked in D54731: Why do we want to implement support for building inferiors in LIT-based tests?

I think this is not much different than asking "Why do we want LIT-based tests?". If you can't build an inferior, then you can't really do anything with LLDB at all, in which case there's nothing to test. So without support for building inferiors, there is no such thing as lldb lit tests.

IMHO, if we need to support for dealing with specific compilers, we should implement that once in Makefile.rules (which is in a declarative domain-specific-language for describing build logic) and write a dotest.py-style test instead. I'm assuming here that we need the support in Makefile.rules anyway to support the bulk of the tests. Having this support in two places increases the maintenance effort and cognitive load.

At least for the time being, I think of dotest.py style tests as being synonymous with "SB API test". I think there is a need for a testing framework that is more familiar to LLVM developers with a lower barrier to entry as it encourages people to write more tests, which is always a good thing. FWIW, I think the results so far have been positive. I did a search over a 3-month period on the repository. During that time, 37 new lit tests have been added, and 32 dotest tests have been added while 9 have been removed.

Your point about having the support in two places increasing the maintenance effort is reasonable, but if the build script actually does support everything that the Makefile does, then perhaps we could port the existing dotest tests over to use the build script. Make is notoriously finnicky and many people don't like it or understand it well and it's very easy to run into portability problems when using it. So if we can eventually do that, that would actually *decrease* the cognitive load and maintenance burden. I'm not proposing that at this point and I think the benefits of a system such as the one in this patch stand on their own regardless of whether we port dotest builders over to use this script, but it's something to think about regardless.

Note that the Makefiles do not currently support clang-cl since it uses a totally different command line syntax.

[zturner]

Updated with suggestions. Currently all NativePDB tests pass with this version of the script, and all other tests are unaffected since they don't use the new substitution.

Subsequent followups will focus on getting other tests using this, which should increase our test coverage across multiple platforms.

[zturner]

In D54914#1309700, @labath wrote:

I didn't look at the code in detail, as most of it deals with windows stuff, and I don't know much about those anyway. However, the interesting question for me would be how to make this useful for cross-compiling. Right now that sort of works for NativePDB tests because --compiler=clang-cl implies windows, but that won't help if I want to compile say a linux arm64 binary. I think that instead --arch, we should have a --triple argument, which specifies the exact target you want to build for. That can default to "host", and we can have special pseudo-triples like "host32" and "host64", if we want to be able to say "I want to build for a 32-bit flavour of the host arch". That could also make gcc detection easier, since you could just search for $triple-gcc.

A triple is one way. I don't know much about gcc, does it support the same triple format as clang? clang-cl certainly doesn't support triples since it implies one. Also, are there any existing use cases for specifying a triple in the lit test suite? I do agree we will need the ability to support triple specification, but it seems better to do this as a followup. I don't think it should be too hard as long as we can make some assumptions such as "specification of --triple limits the possible supported compilers", etc.

Another route to take might be to say that this script only supports building for the host, and tests that want to target a specific architecture can just invoke the appropriate compiler directly -- since they know the target arch, they also know the correct compiler option syntax. Plus, these kinds of tests are the ones that are most likely to need fancy compiler command line switches that might be hard to represent generically. In this world, the NativePDB tests would continue to invoke %clang-cl (which would point to the build dir without any fancy logic), and build.py would be used by scripts that just want to build an executable for the host, and don't care much about the details of how this is accomplished (the stop-hook tests are a good example of this).

That's another good possibility.

(Among the cosmetic things, I'd suggest to make --source a positional argument and enable spelling of --output as -o, just to make things shorter.)

I did both of these in the latest update. I need to make it possible to specify multiple inputs, but I'll do that in a followup as there's no immediate need for it in any of the tests I converted.

[labath]

In D54914#1310337, @zturner wrote:

In D54914#1309700, @labath wrote:

I didn't look at the code in detail, as most of it deals with windows stuff, and I don't know much about those anyway. However, the interesting question for me would be how to make this useful for cross-compiling. Right now that sort of works for NativePDB tests because --compiler=clang-cl implies windows, but that won't help if I want to compile say a linux arm64 binary. I think that instead --arch, we should have a --triple argument, which specifies the exact target you want to build for. That can default to "host", and we can have special pseudo-triples like "host32" and "host64", if we want to be able to say "I want to build for a 32-bit flavour of the host arch". That could also make gcc detection easier, since you could just search for $triple-gcc.

A triple is one way. I don't know much about gcc, does it support the same triple format as clang?

Gcc is different that clang in that the target triple is baked into the binary. For example, on my machine, I have a binary called x86_64-pc-linux-gnu-gcc (gcc is just a symlink to that), which always produces binaries for x86_64 linux. If I had a arm64 cross-gcc installed, it would be called something like aarch64-linux-gnu-gcc and so on. So the actual mechanism to search for the compiler and invoke it will be different, but a triple is still a good starting point.

Also, are there any existing use cases for specifying a triple in the lit test suite?

You can take a look at the tests in lit/SymbolFile/DWARF. Their purpose is very similar to the NativePDB tests -- check the operation of the dwarf parser, regardless of the host platform. Since mac uses a somewhat different flavour of dwarf than other platforms, the sources are usually compiled twice, targetting linux and mac.

I do agree we will need the ability to support triple specification, but it seems better to do this as a followup. I don't think it should be too hard as long as we can make some assumptions such as "specification of --triple limits the possible supported compilers", etc.

I don't think there needs to be much code initially, but I think it would be good to decide on the direction up front. For example, if we decide to go the triple way then we could avoid the churn in the tests by making all of them use --triple=x86_64-pc-windows instead of the current --arch=64, even if the actual implementation simply aborts in the more complicated cases.

[labath]

In D54914#1309901, @aprantl wrote:

I would like to ask a general question that I (indirectly) also asked in D54731: Why do we want to implement support for building inferiors in LIT-based tests? IMHO, if we need to support for dealing with specific compilers, we should implement that once in Makefile.rules (which is in a declarative domain-specific-language for describing build logic) and write a dotest.py-style test instead. I'm assuming here that we need the support in Makefile.rules anyway to support the bulk of the tests. Having this support in two places increases the maintenance effort and cognitive load.

While I agree that having two ways to do something is bad, I have to say that the current Makefile.rules system is far from perfect. First of, it is geared towards creating a fully working executables (the kind that dotest tests expect), and if you want to do something more exotic, things get a lot harder. Also writing more complex logic in makefiles is hard, particularly when you need to be portable across various shells and make implementations. In fact, the biggest feature of make, automatic dependency tracking, is unused now that we build tests out of tree and clean by nuking the build dir.

[zturner]

In D54914#1311137, @labath wrote:

In D54914#1310337, @zturner wrote:

In D54914#1309700, @labath wrote:

I didn't look at the code in detail, as most of it deals with windows stuff, and I don't know much about those anyway. However, the interesting question for me would be how to make this useful for cross-compiling. Right now that sort of works for NativePDB tests because --compiler=clang-cl implies windows, but that won't help if I want to compile say a linux arm64 binary. I think that instead --arch, we should have a --triple argument, which specifies the exact target you want to build for. That can default to "host", and we can have special pseudo-triples like "host32" and "host64", if we want to be able to say "I want to build for a 32-bit flavour of the host arch". That could also make gcc detection easier, since you could just search for $triple-gcc.

A triple is one way. I don't know much about gcc, does it support the same triple format as clang?

Gcc is different that clang in that the target triple is baked into the binary. For example, on my machine, I have a binary called x86_64-pc-linux-gnu-gcc (gcc is just a symlink to that), which always produces binaries for x86_64 linux. If I had a arm64 cross-gcc installed, it would be called something like aarch64-linux-gnu-gcc and so on. So the actual mechanism to search for the compiler and invoke it will be different, but a triple is still a good starting point.

Also, are there any existing use cases for specifying a triple in the lit test suite?

You can take a look at the tests in lit/SymbolFile/DWARF. Their purpose is very similar to the NativePDB tests -- check the operation of the dwarf parser, regardless of the host platform. Since mac uses a somewhat different flavour of dwarf than other platforms, the sources are usually compiled twice, targetting linux and mac.

I do agree we will need the ability to support triple specification, but it seems better to do this as a followup. I don't think it should be too hard as long as we can make some assumptions such as "specification of --triple limits the possible supported compilers", etc.

I don't think there needs to be much code initially, but I think it would be good to decide on the direction up front. For example, if we decide to go the triple way then we could avoid the churn in the tests by making all of them use --triple=x86_64-pc-windows instead of the current --arch=64, even if the actual implementation simply aborts in the more complicated cases.

I think it would be good to use the way dotest works as a starting point. You can specify --arch, and then you can run the test on multiple arches this way by running dotest several times in succession, each with different arch flags.

I think a --triple option could be useful in limited scenarios, but I think that most of the use cases will not need it. Most tests will probably want to specify nothing at all and let the lit configuration pass the correct information through. Actually, I think this is the same with the --arch flag though. Because as soon as you specify something, then it limits the ability of the test to run over and over with different parameters.

Perhaps we could support *both* a --triple and a --arch flag. Maybe we can make the script error if they're both specified. This would give each test the capability to be written in the simplest way possible.

[stella.stamenova]

I ran the tests with this change and there are about a dozen new failures:

2018-11-28T20:25:29.2437909Z          ********************
2018-11-28T20:25:29.2581909Z          FAIL: LLDB :: SymbolFile/NativePDB/function-types-builtins.cpp (16962 of 45532)
2018-11-28T20:25:29.2593650Z          ******************** TEST 'LLDB :: SymbolFile/NativePDB/function-types-builtins.cpp' FAILED ********************
2018-11-28T20:25:29.2627299Z          Script:
2018-11-28T20:25:29.2627541Z          --
2018-11-28T20:25:29.2628080Z          : 'RUN: at line 4';   C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\python.exe E:\_work\80\s\llvm\tools\lldb\lit\helper\build.py --compiler=any --arch=64 --tools-dir=E:/_work/80/b/LLVMBuild/Release/bin --compiler=clang-cl --nodefaultlib -o E:\_work\80\b\LLVMBuild\tools\lldb\lit\SymbolFile\NativePDB\Output\function-types-builtins.cpp.tmp.exe -- E:\_work\80\s\llvm\tools\lldb\lit\SymbolFile\NativePDB\function-types-builtins.cpp
2018-11-28T20:25:29.2629494Z          : 'RUN: at line 5';   env LLDB_USE_NATIVE_PDB_READER=1 E:\_work\80\b\LLVMBuild\Release\bin\lldb.EXE -S E:/_work/80/s/llvm/tools/lldb/lit\lit-lldb-init -f E:\_work\80\b\LLVMBuild\tools\lldb\lit\SymbolFile\NativePDB\Output\function-types-builtins.cpp.tmp.exe -s      E:\_work\80\s\llvm\tools\lldb\lit\SymbolFile\NativePDB/Inputs/function-types-builtins.lldbinit | E:\_work\80\b\LLVMBuild\Release\bin\FileCheck.EXE E:\_work\80\s\llvm\tools\lldb\lit\SymbolFile\NativePDB\function-types-builtins.cpp
2018-11-28T20:25:29.2629805Z          --
2018-11-28T20:25:29.2629901Z          Exit Code: 127
2018-11-28T20:25:29.2630016Z          
2018-11-28T20:25:29.2630140Z          Command Output (stdout):
2018-11-28T20:25:29.2630258Z          --
2018-11-28T20:25:29.2630427Z          $ ":" "RUN: at line 4"
2018-11-28T20:25:29.2630915Z          $ "C:\Program" "Files" "(x86)\Microsoft" "Visual" "Studio\Shared\Python36_64\python.exe" "E:\_work\80\s\llvm\tools\lldb\lit\helper\build.py" "--compiler=any" "--arch=64" "--tools-dir=E:/_work/80/b/LLVMBuild/Release/bin" "--compiler=clang-cl" "--nodefaultlib" "-o" "E:\_work\80\b\LLVMBuild\tools\lldb\lit\SymbolFile\NativePDB\Output\function-types-builtins.cpp.tmp.exe" "--" "E:\_work\80\s\llvm\tools\lldb\lit\SymbolFile\NativePDB\function-types-builtins.cpp"
2018-11-28T20:25:29.2631273Z          # command stderr:
2018-11-28T20:25:29.2631382Z          'C:\\Program': command not found

It looks like the script doesn't handle paths that have spaces correctly. I think the issue is on line 45 where we add sys.executable as the command - we need quotes around that. I'm going to rerun the tests with that change to see the result.

lldb/lit/helper/toolchain.py

45 ↗

(On Diff #175561)

I think this is where we need quotes around sys.executable, so that if the path contains spaces it is handled correctly

[stella.stamenova]

It works for me when you make the command:

command='"%s"' % (sys.executable),

I haven't tested it on Linux yet, but the tests work fine on Windows.

[labath]

In D54914#1311492, @zturner wrote:

I think it would be good to use the way dotest works as a starting point. You can specify --arch, and then you can run the test on multiple arches this way by running dotest several times in succession, each with different arch flags.

I think a --triple option could be useful in limited scenarios, but I think that most of the use cases will not need it. Most tests will probably want to specify nothing at all and let the lit configuration pass the correct information through. Actually, I think this is the same with the --arch flag though. Because as soon as you specify something, then it limits the ability of the test to run over and over with different parameters.

I am not sure about their relative ratio, but yes, I agree that we have (or want to have) two kinds of tests. One would specify the target and the other would not.

Perhaps we could support *both* a --triple and a --arch flag. Maybe we can make the script error if they're both specified. This would give each test the capability to be written in the simplest way possible.

Yes, I suppose that would work

[zturner]

If there are no further comments I'd like to get this in later today.

lit/SymbolFile/NativePDB/ast-reconstruction.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test various interesting cases for AST reconstruction.
-// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/ast-reconstruction.lldbinit 2>&1 | FileCheck %s
 
 // Test trivial versions of each tag type.
 class TrivialC {};
 struct TrivialS {};
 union TrivialU {};
 enum TrivialE {TE_A};

lit/SymbolFile/NativePDB/bitfields.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test various interesting cases for AST reconstruction.
-// RUN: %clang_cl /Z7 /GS- /GR- /std:c++latest -Xclang -fkeep-static-consts /c /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/bitfields.lldbinit 2>&1 | FileCheck %s
 
 // Test trivial versions of each tag type.
 struct Struct {
   int A : 5 = 6;
   int B : 7 = 8;
   unsigned C : 3 = 2;

lit/SymbolFile/NativePDB/disassembly.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can show disassembly and source.
-// RUN: %clang_cl -m64 /Z7 /GS- /GR- /c /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/disassembly.lldbinit | FileCheck %s
 
 // Some context lines before
 // the function.
 
 int foo() { return 42; }
 
 int main(int argc, char **argv) {
   foo();
   return 0;
 }
 
 
 // CHECK:      (lldb) disassemble --flavor=intel -m -n main
-// CHECK:         13   int foo() { return 42; }
-// CHECK-NEXT:    14
-// CHECK-NEXT: ** 15   int main(int argc, char **argv) {
+// CHECK:         12   int foo() { return 42; }
+// CHECK-NEXT:    13
+// CHECK-NEXT: ** 14   int main(int argc, char **argv) {
 // CHECK:      disassembly.cpp.tmp.exe`main:
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+0>:  sub    rsp, 0x38
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+4>:  mov    dword ptr [rsp + 0x34], 0x0
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+12>: mov    qword ptr [rsp + 0x28], rdx
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+17>: mov    dword ptr [rsp + 0x24], ecx
-// CHECK:      ** 16     foo();
-// CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+21>: call   {{.*}}               ; foo at disassembly.cpp:13
+// CHECK:      ** 15     foo();
+// CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+21>: call   {{.*}}               ; foo at disassembly.cpp:12
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+26>: xor    ecx, ecx
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+28>: mov    dword ptr [rsp + 0x20], eax
-// CHECK:      ** 17     return 0;
-// CHECK-NEXT:    18   }
-// CHECK-NEXT:    19
+// CHECK:      ** 16     return 0;
+// CHECK-NEXT:    17   }
+// CHECK-NEXT:    18
 // CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+32>: mov    eax, ecx
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+34>: add    rsp, 0x38
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+38>: ret

lit/SymbolFile/NativePDB/function-types-builtins.cpp

 // clang-format off
 // REQUIRES: lld
 
-// RUN: %clang_cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/function-types-builtins.lldbinit | FileCheck %s
 
 // Test that we can display function signatures with simple builtin
 // and pointer types.  We do this by using `target variable` in lldb
 // with global variables of type ptr-to-function or reference-to-function.
 // This technique in general allows us to exercise most of LLDB's type
 // system without a running process.

lit/SymbolFile/NativePDB/function-types-calling-conv.cpp

 // clang-format off
 // REQUIRES: lld
 
-// RUN: %clang_cl -m32 /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --arch=32 --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/function-types-calling-conv.lldbinit | FileCheck %s
 
 
 void __stdcall StdcallFn() {}
 void __fastcall FastcallFn() {}
 void __thiscall ThiscallFn() {}
 void __cdecl CdeclFn() {}

lit/SymbolFile/NativePDB/function-types-classes.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display function signatures with class types.
-// RUN: %clang_cl /Z7 /GS- /GR- /c -fstandalone-debug -Xclang -fkeep-static-consts /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/function-types-classes.lldbinit | FileCheck %s
 
 // This is just some unimportant helpers needed so that we can get reference and
 // rvalue-reference types into return values.
 template<typename T>
 struct MakeResult {
   static T result() {

lit/SymbolFile/NativePDB/global-classes.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display tag types.
-// RUN: %clang_cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/globals-classes.lldbinit | FileCheck %s
 
 enum class EnumType : unsigned {
   A = 1,
   B = 2
 };
 

lit/SymbolFile/NativePDB/globals-bss.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Make sure we can read variables from BSS
-// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: llvm-readobj -s %t.exe | FileCheck --check-prefix=BSS %s
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/globals-bss.lldbinit 2>&1 | FileCheck %s
 
 int GlobalVariable = 0;
 
 int main(int argc, char **argv) {
   return 0;

lit/SymbolFile/NativePDB/globals-fundamental.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display tag types.
-// RUN: %clang_cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/globals-fundamental.lldbinit | FileCheck %s
 
 
 // Fundamental data types
 bool BFalse = false;
 // CHECK: (lldb) target variable BFalse
 // CHECK-NEXT: (bool) BFalse = false

lit/SymbolFile/NativePDB/nested-types.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test various interesting cases for AST reconstruction.
-// RUN: %clang_cl /Z7 /GS- /GR- -Xclang -fkeep-static-consts /c /Fo%t.obj -- %s
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/nested-types.lldbinit 2>&1 | FileCheck %s
 
 struct S {
   struct NestedStruct {
     int A = 0;
     int B = 1;
   };

lit/SymbolFile/NativePDB/s_constant.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display S_CONSTANT records.
 
 // RUN: llvm-mc -filetype=obj -triple=x86_64-pc-win32 %p/Inputs/s_constant.s > %t.obj
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib --mode=link -o %t.exe -- %t.obj
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/s_constant.lldbinit | FileCheck %s
 
 // clang-cl cannot generate S_CONSTANT records, but we need to test that we can
 // handle them for compatibility with MSVC, which does emit them.  This test
 // case was generated by compiling this file with MSVC and copying the bytes
 // that they emit for S_CONSTANT records.  Then we compile the same code with
 // clang to get a .s file, and replace all S_LDATA32 records with the bytes from

lit/SymbolFile/NativePDB/simple-breakpoints.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can set simple breakpoints using PDB on any platform.
-// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/breakpoints.lldbinit | FileCheck %s
 
 // Use different indentation style for each overload so that the starting
 // line is in a different place.
 int OvlGlobalFn(int X) {
   return X + 42;
 }
int main(int argc, char **argv) {
   return Result;
 }
 
 
 // CHECK:      (lldb) target create "{{.*}}simple-breakpoints.cpp.tmp.exe"
 // CHECK:      Current executable set to '{{.*}}simple-breakpoints.cpp.tmp.exe'
 // CHECK:      (lldb) break set -n main
 // CHECK:      Breakpoint 1: where = simple-breakpoints.cpp.tmp.exe`main + {{[0-9]+}}
-// CHECK-SAME:               at simple-breakpoints.cpp:31
+// CHECK-SAME:               at simple-breakpoints.cpp:30
 // CHECK:      (lldb) break set -n OvlGlobalFn
 // CHECK:      Breakpoint 2: 3 locations.
 // CHECK:      (lldb) break set -n StaticFn
 // CHECK:      Breakpoint 3: where = simple-breakpoints.cpp.tmp.exe`StaticFn + {{[0-9]+}}
-// CHECK-SAME:               at simple-breakpoints.cpp:24
+// CHECK-SAME:               at simple-breakpoints.cpp:23
 // CHECK:      (lldb) break set -n DoesntExist
 // CHECK:      Breakpoint 4: no locations (pending).
 // CHECK:      (lldb) break list
 // CHECK:      Current breakpoints:
 // CHECK:      1: name = 'main', locations = 1
 // CHECK:        1.1: where = simple-breakpoints.cpp.tmp.exe`main + {{[0-9]+}}
-// CHECK-SAME:                at simple-breakpoints.cpp:31
+// CHECK-SAME:                at simple-breakpoints.cpp:30
 // CHECK:      2: name = 'OvlGlobalFn', locations = 3
 // CHECK:        2.1: where = simple-breakpoints.cpp.tmp.exe`OvlGlobalFn + {{[0-9]+}}
-// CHECK-SAME:                at simple-breakpoints.cpp:13
+// CHECK-SAME:                at simple-breakpoints.cpp:12
 // CHECK:        2.2: where = simple-breakpoints.cpp.tmp.exe`OvlGlobalFn
-// CHECK-SAME:                at simple-breakpoints.cpp:16
+// CHECK-SAME:                at simple-breakpoints.cpp:15
 // CHECK:        2.3: where = simple-breakpoints.cpp.tmp.exe`OvlGlobalFn + {{[0-9]+}}
-// CHECK-SAME:                at simple-breakpoints.cpp:20
+// CHECK-SAME:                at simple-breakpoints.cpp:19
 // CHECK:      3: name = 'StaticFn', locations = 1
 // CHECK:        3.1: where = simple-breakpoints.cpp.tmp.exe`StaticFn + {{[0-9]+}}
-// CHECK-SAME:                at simple-breakpoints.cpp:24
+// CHECK-SAME:                at simple-breakpoints.cpp:23
 // CHECK:      4: name = 'DoesntExist', locations = 0 (pending)

lit/SymbolFile/NativePDB/source-list.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can set display source of functions.
-// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/source-list.lldbinit | FileCheck %s
 
 
 
 // Some context lines before
 // the function.
 
 
 int main(int argc, char **argv) {
   // Here are some comments.
   // That we should print when listing source.
   return 0;
 }
 
 // Some context lines after
 // the function.
 
 // check lines go at the end so that line numbers stay stable when
 // changing this file.
 
 // CHECK: (lldb) source list -n main
 // CHECK: File: {{.*}}source-list.cpp
-// CHECK:    11
-// CHECK:    12    // Some context lines before
-// CHECK:    13   // the function.
+// CHECK:    10
+// CHECK:    11    // Some context lines before
+// CHECK:    12   // the function.
+// CHECK:    13
 // CHECK:    14
-// CHECK:    15
-// CHECK:    16   int main(int argc, char **argv) {
-// CHECK:    17     // Here are some comments.
-// CHECK:    18     // That we should print when listing source.
-// CHECK:    19     return 0;
-// CHECK:    20   }
-// CHECK:    21
-// CHECK:    22   // Some context lines after
-// CHECK:    23   // the function.
-// CHECK:    24
+// CHECK:    15   int main(int argc, char **argv) {
+// CHECK:    16     // Here are some comments.
+// CHECK:    17     // That we should print when listing source.
+// CHECK:    18     return 0;
+// CHECK:    19   }
+// CHECK:    20
+// CHECK:    21   // Some context lines after
+// CHECK:    22   // the function.
+// CHECK:    23

lit/SymbolFile/NativePDB/tag-types.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display tag types.
-// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
-// RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
+// RUN: %build --compiler=clang-cl --nodefaultlib -o %t.exe -- %s 
 // RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/tag-types.lldbinit | FileCheck %s
 
 // Test struct
 struct Struct {
   // Test builtin types, which are represented by special CodeView type indices.
   bool                B;
   char                C;

lit/helper/build.py

+from __future__ import print_function
+
+import argparse
+import os
+import signal
+import subprocess
+import sys
+
+if sys.platform == 'win32':
+    # This module was renamed in Python 3.  Make sure to import it using a
+    # consistent name regardless of python version.
+    try:
+        import winreg
+    except:
+        import _winreg as winreg
+
+if __name__ != "__main__":
+    raise RuntimeError("Do not import this script, run it instead")
+
+
+parser = argparse.ArgumentParser(description='LLDB compilation wrapper')
+parser.add_argument('--arch',
+                    metavar='arch',
+                    dest='arch',
+                    required=True,
+                    help='Specify the architecture to target.  Valid values=[32,64]')
+
+parser.add_argument('--compiler',
+                    metavar='compiler',
+                    dest='compiler',
+                    required=True,
+                    help='Path to a compiler executable, or one of the values [any, msvc, clang-cl, gcc, clang]')
+
+parser.add_argument('--tools-dir',
+                    metavar='directory',
+                    dest='tools_dir',
+                    required=False,
+                    action='append',
+                    help='If specified, a path to search in addition to PATH when --compiler is not an exact path')
+
+if sys.platform == 'darwin':
+    parser.add_argument('--apple-sdk',
+                        metavar='apple_sdk',
+                        dest='apple_sdk',
+                        default="macosx",
+                        help='Specify the name of the Apple SDK (macosx, macosx.internal, iphoneos, iphoneos.internal, or path to SDK) and use the appropriate tools from that SDK\'s toolchain.')
+
+parser.add_argument('--output', '-o',
+                    dest='output',
+                    metavar='file',
+                    required=True,
+                    help='Path to output file')
+
+parser.add_argument('--nodefaultlib',
+                    dest='nodefaultlib',
+                    action='store_true',
+                    default=False,
+                    help='When specified, the resulting image should not link against system libraries or include system headers.  Useful when writing cross-targeting tests.')
+
+parser.add_argument('--opt',
+                    dest='opt',
+                    default='none',
+                    choices=['none', 'basic', 'lto'],
+                    help='Optimization level')
+
+parser.add_argument('--mode',
+                    dest='mode',
+                    default='compile-and-link',
+                    choices=['compile', 'link', 'compile-and-link'],
+                    help='Specifies whether to compile, link, or both')
+
+parser.add_argument('--noclean',
+                    dest='clean',
+                    action='store_false',
+                    default=True,
+                    help='Dont clean output file before building')
+
+parser.add_argument('--verbose',
+                    dest='verbose',
+                    action='store_true',
+                    default=False,
+                    help='Print verbose output')
+
+parser.add_argument('input',
+                    metavar='file',
+                    help='Source file to compile / object file to link')
+
+
+args = parser.parse_args(args=sys.argv[1:])
+
+
+def to_string(b):
+    """Return the parameter as type 'str', possibly encoding it.
+
+    In Python2, the 'str' type is the same as 'bytes'. In Python3, the
+    'str' type is (essentially) Python2's 'unicode' type, and 'bytes' is
+    distinct.
+
+    This function is copied from llvm/utils/lit/lit/util.py
+    """
+    if isinstance(b, str):
+        # In Python2, this branch is taken for types 'str' and 'bytes'.
+        # In Python3, this branch is taken only for 'str'.
+        return b
+    if isinstance(b, bytes):
+        # In Python2, this branch is never taken ('bytes' is handled as 'str').
+        # In Python3, this is true only for 'bytes'.
+        try:
+            return b.decode('utf-8')
+        except UnicodeDecodeError:
+            # If the value is not valid Unicode, return the default
+            # repr-line encoding.
+            return str(b)
+
+    # By this point, here's what we *don't* have:
+    #
+    #  - In Python2:
+    #    - 'str' or 'bytes' (1st branch above)
+    #  - In Python3:
+    #    - 'str' (1st branch above)
+    #    - 'bytes' (2nd branch above)
+    #
+    # The last type we might expect is the Python2 'unicode' type. There is no
+    # 'unicode' type in Python3 (all the Python3 cases were already handled). In
+    # order to get a 'str' object, we need to encode the 'unicode' object.
+    try:
+        return b.encode('utf-8')
+    except AttributeError:
+        raise TypeError('not sure how to convert %s to %s' % (type(b), str))
+
+def print_environment(env):
+    for e in env:
+        value = env[e]
+        split = value.split(os.pathsep)
+        print('    {0} = {1}'.format(e, split[0]))
+        prefix_width = 3 + len(e)
+        for next in split[1:]:
+            print('    {0}{1}'.format(' ' * prefix_width, next))
+
+def find_executable(binary_name, search_paths):
+    if sys.platform == 'win32':
+        binary_name = binary_name + '.exe'
+
+    search_paths = os.pathsep.join(search_paths)
+    paths = search_paths + os.pathsep + os.environ.get('PATH', '')
+    for path in paths.split(os.pathsep):
+        p = os.path.join(path, binary_name)
+        if os.path.exists(p) and not os.path.isdir(p):
+            return os.path.normpath(p)
+    return None
+
+def find_toolchain(compiler, tools_dir):
+    if compiler == 'msvc':
+        return ('msvc', find_executable('cl', tools_dir))
+    if compiler == 'clang-cl':
+        return ('clang-cl', find_executable('clang-cl', tools_dir))
+    if compiler == 'gcc':
+        return ('gcc', find_executable('g++', tools_dir))
+    if compiler == 'clang':
+        return ('clang', find_executable('clang++', tools_dir))
+    if compiler == 'any':
+        priorities = []
+        if sys.platform == 'win32':
+            priorities = ['clang-cl', 'msvc', 'clang', 'gcc']
+        else:
+            priorities = ['clang', 'gcc', 'clang-cl']
+        for toolchain in priorities:
+            (type, dir) = find_toolchain(toolchain, tools_dir)
+            if type and dir:
+                return (type, dir)
+        # Could not find any toolchain.
+        return (None, None)
+
+    # From here on, assume that |compiler| is a path to a file.
+    file = os.path.basename(compiler)
+    name, ext = os.path.splitext(file)
+    if file.lower() == 'cl.exe':
+        return 'msvc'
+    if name == 'clang-cl':
+        return 'clang-cl'
+    if name.startswith('clang'):
+        return 'clang'
+    if name.startswith('gcc') or name.startswith('g++'):
+        return 'gcc'
+    if name == 'cc' or name == 'c++':
+        return 'generic'
+    return 'unknown'
+
+class Builder(object):
+    def __init__(self, toolchain_type, args):
+        self.toolchain_type = toolchain_type
+        self.input = args.input
+        self.arch = args.arch
+        self.opt = args.opt
+        self.compiler = args.compiler
+        self.clean = args.clean
+        self.output = args.output
+        self.mode = args.mode
+        self.nodefaultlib = args.nodefaultlib
+        self.verbose = args.verbose
+
+class MsvcBuilder(Builder):
+    def __init__(self, toolchain_type, args):
+        Builder.__init__(self, toolchain_type, args)
+
+        self.msvc_arch_str = 'x86' if self.arch == '32' else 'x64'
+
+        if toolchain_type == 'msvc':
+            # Make sure we're using the appropriate toolchain for the desired
+            # target type.
+            compiler_parent_dir = os.path.dirname(self.compiler)
+            selected_target_version = os.path.basename(compiler_parent_dir)
+            if selected_target_version != self.msvc_arch_str:
+                host_dir = os.path.dirname(compiler_parent_dir)
+                self.compiler = os.path.join(host_dir, self.msvc_arch_str, 'cl.exe')
+                if self.verbose:
+                    print('Using alternate compiler "{0}" to match selected target.'.format(self.compiler))
+
+        if self.mode == 'link' or self.mode == 'compile-and-link':
+            self.linker = self._find_linker('link') if toolchain_type == 'msvc' else self._find_linker('lld-link')
+            if not self.linker:
+                raise ValueError('Unable to find an appropriate linker.')
+
+        self.compile_env, self.link_env = self._get_visual_studio_environment()
+
+    def _find_linker(self, name):
+        if sys.platform == 'win32':
+            name = name + '.exe'
+        compiler_dir = os.path.dirname(self.compiler)
+        linker_path = os.path.join(compiler_dir, name)
+        if not os.path.exists(linker_path):
+            raise ValueError('Could not find \'{}\''.format(linker_path))
+        return linker_path
+
+    def _get_vc_install_dir(self):
+        dir = os.getenv('VCINSTALLDIR', None)
+        if dir:
+            if self.verbose:
+                print('Using %VCINSTALLDIR% {}'.format(dir))
+            return dir
+
+        dir = os.getenv('VSINSTALLDIR', None)
+        if dir:
+            if self.verbose:
+                print('Using %VSINSTALLDIR% {}'.format(dir))
+            return os.path.join(dir, 'VC')
+
+        dir = os.getenv('VS2019INSTALLDIR', None)
+        if dir:
+            if self.verbose:
+                print('Using %VS2019INSTALLDIR% {}'.format(dir))
+            return os.path.join(dir, 'VC')
+
+        dir = os.getenv('VS2017INSTALLDIR', None)
+        if dir:
+            if self.verbose:
+                print('Using %VS2017INSTALLDIR% {}'.format(dir))
+            return os.path.join(dir, 'VC')
+
+        dir = os.getenv('VS2015INSTALLDIR', None)
+        if dir:
+            if self.verbose:
+                print('Using %VS2015INSTALLDIR% {}'.format(dir))
+            return os.path.join(dir, 'VC')
+        return None
+
+    def _get_vctools_version(self):
+        ver = os.getenv('VCToolsVersion', None)
+        if ver:
+            if self.verbose:
+                print('Using %VCToolsVersion% {}'.format(ver))
+            return ver
+
+        vcinstalldir = self._get_vc_install_dir()
+        vcinstalldir = os.path.join(vcinstalldir, 'Tools', 'MSVC')
+        subdirs = next(os.walk(vcinstalldir))[1]
+        if not subdirs:
+            return None
+
+        from distutils.version import StrictVersion
+        subdirs.sort(key=lambda x : StrictVersion(x))
+
+        if self.verbose:
+            full_path = os.path.join(vcinstalldir, subdirs[-1])
+            print('Using VC tools version directory {0} found by directory walk.'.format(full_path))
+        return subdirs[-1]
+
+    def _get_vctools_install_dir(self):
+        dir = os.getenv('VCToolsInstallDir', None)
+        if dir:
+            if self.verbose:
+                print('Using %VCToolsInstallDir% {}'.format(dir))
+            return dir
+
+        vcinstalldir = self._get_vc_install_dir()
+        if not vcinstalldir:
+            return None
+        vctoolsver = self._get_vctools_version()
+        if not vctoolsver:
+            return None
+        result = os.path.join(vcinstalldir, 'Tools', 'MSVC', vctoolsver)
+        if not os.path.exists(result):
+            return None
+        if self.verbose:
+            print('Using VC tools install dir {} found by directory walk'.format(result))
+        return result
+
+    def _find_windows_sdk_in_registry_view(self, view):
+        products_key = None
+        roots_key = None
+        installed_options_keys = []
+        try:
+            sam = view | winreg.KEY_READ
+            products_key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE,
+                                          r'Software\Microsoft\Windows Kits\Installed Products',
+                                          0,
+                                          sam)
+
+            # This is the GUID for the desktop component.  If this is present
+            # then the components required for the Desktop SDK are installed.
+            # If not it will throw an exception.
+            winreg.QueryValueEx(products_key, '{5A3D81EC-D870-9ECF-D997-24BDA6644752}')
+
+            roots_key = winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE,
+                                       r'Software\Microsoft\Windows Kits\Installed Roots',
+                                       0,
+                                       sam)
+            root_dir = winreg.QueryValueEx(roots_key, 'KitsRoot10')
+            root_dir = to_string(root_dir[0])
+            sdk_versions = []
+            index = 0
+            while True:
+                # Installed SDK versions are stored as sub-keys of the
+                # 'Installed Roots' key.  Find all of their names, then sort
+                # them by version
+                try:
+                    ver_key = winreg.EnumKey(roots_key, index)
+                    sdk_versions.append(ver_key)
+                    index = index + 1
+                except WindowsError:
+                    break
+            if not sdk_versions:
+                return (None, None)
+
+            # Windows SDK version numbers consist of 4 dotted components, so we
+            # have to use LooseVersion, as StrictVersion supports 3 or fewer.
+            from distutils.version import LooseVersion
+            sdk_versions.sort(key=lambda x : LooseVersion(x), reverse=True)
+            option_value_name = 'OptionId.DesktopCPP' + self.msvc_arch_str
+            for v in sdk_versions:
+                try:
+                    version_subkey = v + r'\Installed Options'
+                    key = winreg.OpenKey(roots_key, version_subkey)
+                    installed_options_keys.append(key)
+                    (value, value_type) = winreg.QueryValueEx(key, option_value_name)
+                    if value == 1:
+                        # The proper architecture is installed.  Return the
+                        # associated paths.
+                        if self.verbose:
+                            print('Found Installed Windows SDK v{0} at {1}'.format(v, root_dir))
+                        return (root_dir, v)
+                except:
+                    continue
+        except:
+            return (None, None)
+        finally:
+            del products_key
+            del roots_key
+            for k in installed_options_keys:
+                del k
+        return (None, None)
+
+    def _find_windows_sdk_in_registry(self):
+        # This could be a clang-cl cross-compile.  If so, there's no registry
+        # so just exit.
+        if sys.platform != 'win32':
+            return (None, None)
+        if self.verbose:
+            print('Looking for Windows SDK in 64-bit registry.')
+        dir, ver = self._find_windows_sdk_in_registry_view(winreg.KEY_WOW64_64KEY)
+        if not dir or not ver:
+            if self.verbose:
+                print('Looking for Windows SDK in 32-bit registry.')
+            dir, ver = self._find_windows_sdk_in_registry_view(winreg.KEY_WOW64_32KEY)
+
+        return (dir, ver)
+
+    def _get_winsdk_dir(self):
+        # If a Windows SDK is specified in the environment, use that.  Otherwise
+        # try to find one in the Windows registry.
+        dir = os.getenv('WindowsSdkDir', None)
+        if not dir or not os.path.exists(dir):
+            return self._find_windows_sdk_in_registry()
+        ver = os.getenv('WindowsSDKLibVersion', None)
+        if not ver:
+            return self._find_windows_sdk_in_registry()
+
+        ver = ver.rstrip('\\')
+        if self.verbose:
+            print('Using %WindowsSdkDir% {}'.format(dir))
+            print('Using %WindowsSDKLibVersion% {}'.format(ver))
+        return (dir, ver)
+
+    def _get_msvc_native_toolchain_dir(self):
+        assert self.toolchain_type == 'msvc'
+        compiler_dir = os.path.dirname(self.compiler)
+        target_dir = os.path.dirname(compiler_dir)
+        host_name = os.path.basename(target_dir)
+        host_name = host_name[4:].lower()
+        return os.path.join(target_dir, host_name)
+
+    def _get_visual_studio_environment(self):
+        vctools = self._get_vctools_install_dir()
+        winsdk, winsdkver = self._get_winsdk_dir()
+
+        if not vctools and self.verbose:
+            print('Unable to find VC tools installation directory.')
+        if (not winsdk or not winsdkver) and self.verbose:
+            print('Unable to find Windows SDK directory.')
+
+        vcincludes = []
+        vclibs = []
+        sdkincludes = []
+        sdklibs = []
+        if vctools is not None:
+            includes = [['ATLMFC', 'include'], ['include']]
+            libs = [['ATLMFC', 'lib'], ['lib']]
+            vcincludes = [os.path.join(vctools, *y) for y in includes]
+            vclibs = [os.path.join(vctools, *y) for y in libs]
+        if winsdk is not None:
+            includes = [['include', winsdkver, 'ucrt'],
+                        ['include', winsdkver, 'shared'],
+                        ['include', winsdkver, 'um'],
+                        ['include', winsdkver, 'winrt'],
+                        ['include', winsdkver, 'cppwinrt']]
+            libs = [['lib', winsdkver, 'ucrt'],
+                    ['lib', winsdkver, 'um']]
+            sdkincludes = [os.path.join(winsdk, *y) for y in includes]
+            sdklibs = [os.path.join(winsdk, *y) for y in libs]
+
+        includes = vcincludes + sdkincludes
+        libs = vclibs + sdklibs
+        libs = [os.path.join(x, self.msvc_arch_str) for x in libs]
+        compileenv = None
+        linkenv = None
+        defaultenv = {}
+        if sys.platform == 'win32':
+            defaultenv = { x : os.environ[x] for x in
+                          ['SystemDrive', 'SystemRoot', 'TMP', 'TEMP'] }
+            # The directory to mspdbcore.dll needs to be in PATH, but this is
+            # always in the native toolchain path, not the cross-toolchain
+            # path.  So, for example, if we're using HostX64\x86 then we need
+            # to add HostX64\x64 to the path, and if we're using HostX86\x64
+            # then we need to add HostX86\x86 to the path.
+            if self.toolchain_type == 'msvc':
+                defaultenv['PATH'] = self._get_msvc_native_toolchain_dir()
+
+        if includes:
+            compileenv = {}
+            compileenv['INCLUDE'] = os.pathsep.join(includes)
+            compileenv.update(defaultenv)
+        if libs:
+            linkenv = {}
+            linkenv['LIB'] = os.pathsep.join(libs)
+            linkenv.update(defaultenv)
+        return (compileenv, linkenv)
+
+    def _ilk_file_name(self):
+        if self.mode == 'link':
+            return None
+        return os.path.splitext(self.output)[0] + '.ilk'
+
+    def _obj_file_name(self):
+        if self.mode == 'compile':
+            return self.output
+        return os.path.splitext(self.output)[0] + '.obj'
+
+    def _pdb_file_name(self):
+        if self.mode == 'compile':
+            return None
+        return os.path.splitext(self.output)[0] + '.pdb'
+
+    def _exe_file_name(self):
+        if self.mode == 'compile':
+            return None
+        return self.output
+
+    def _get_compilation_command(self):
+        args = []
+
+        args.append(self.compiler)
+        if self.toolchain_type == 'clang-cl':
+            args.append('-m' + self.arch)
+
+        if self.opt == 'none':
+            args.append('/Od')
+        elif self.opt == 'basic':
+            args.append('/O2')
+        elif self.opt == 'lto':
+            if self.toolchain_type == 'msvc':
+                args.append('/GL')
+                args.append('/Gw')
+            else:
+                args.append('-flto=thin')
+        if self.nodefaultlib:
+            args.append('/GS-')
+            args.append('/GR-')
+        args.append('/Z7')
+        if self.toolchain_type == 'clang-cl':
+            args.append('-Xclang')
+            args.append('-fkeep-static-consts')
+        args.append('/c')
+
+        args.append('/Fo' + self._obj_file_name())
+        args.append(self.input)
+        input = os.path.basename(self.input)
+        output = os.path.basename(self._obj_file_name())
+        return ('compiling {0} -> {1}'.format(input, output),
+                self.compile_env,
+                args)
+
+    def _get_link_command(self):
+        args = []
+        args.append(self.linker)
+        args.append('/DEBUG:FULL')
+        args.append('/INCREMENTAL:NO')
+        if self.nodefaultlib:
+            args.append('/nodefaultlib')
+            args.append('/entry:main')
+        args.append('/PDB:' + self._pdb_file_name())
+        args.append('/OUT:' + self._exe_file_name())
+        args.append(self._obj_file_name())
+
+        input = os.path.basename(self._obj_file_name())
+        output = os.path.basename(self._exe_file_name())
+        return ('linking {0} -> {1}'.format(input, output),
+                self.link_env,
+                args)
+
+    def build_commands(self):
+        commands = []
+        if self.mode == 'compile' or self.mode == 'compile-and-link':
+            commands.append(self._get_compilation_command())
+        if self.mode == 'link' or self.mode == 'compile-and-link':
+            commands.append(self._get_link_command())
+        return commands
+
+    def output_files(self):
+        outdir = os.path.dirname(self.output)
+        file = os.path.basename(self.output)
+        name, ext = os.path.splitext(file)
+
+        outputs = []
+        if self.mode == 'compile' or self.mode == 'compile-and-link':
+            outputs.append(self._ilk_file_name())
+            outputs.append(self._obj_file_name())
+        if self.mode == 'link' or self.mode == 'compile-and-link':
+            outputs.append(self._pdb_file_name())
+            outputs.append(self._exe_file_name())
+
+        return [x for x in outputs if x is not None]
+
+class GccBuilder(Builder):
+    def __init__(self, toolchain_type, args):
+        Builder.__init__(self, toolchain_type, args)
+
+    def build_commands(self):
+        pass
+
+    def output_files(self):
+        pass
+
+def indent(text, spaces):
+    def prefixed_lines():
+        prefix = ' ' * spaces
+        for line in text.splitlines(True):
+            yield prefix + line
+    return ''.join(prefixed_lines())
+
+def build(commands):
+    global args
+    for (status, env, child_args) in commands:
+        print('\n\n')
+        print(status)
+        if args.verbose:
+            print('  Command Line: ' + ' '.join(child_args))
+            print('  Env:')
+            print_environment(env)
+        popen = subprocess.Popen(child_args,
+                                 stdout=subprocess.PIPE,
+                                 stderr=subprocess.PIPE,
+                                 env=env,
+                                 universal_newlines=True)
+        stdout, stderr = popen.communicate()
+        res = popen.wait()
+        if res == -signal.SIGINT:
+            raise KeyboardInterrupt
+        print('  STDOUT:')
+        print(indent(stdout, 4))
+        if res != 0:
+            print('  STDERR:')
+            print(indent(stderr, 4))
+            sys.exit(res)
+
+def clean(files):
+    global args
+    for o in files:
+        file = o if args.verbose else os.path.basename(o)
+        print('Cleaning {0}'.format(file))
+        try:
+            if os.path.exists(o):
+                os.remove(o)
+                if args.verbose:
+                    print('  The file was successfully cleaned.')
+            elif args.verbose:
+                print('  The file does not exist.')
+        except:
+            if args.verbose:
+                print('  The file could not be removed.')
+
+(toolchain_type, toolchain_path) = find_toolchain(args.compiler, args.tools_dir)
+if not toolchain_path or not toolchain_type:
+    print('Unable to find toolchain {0}'.format(args.compiler))
+    sys.exit(1)
+
+if args.verbose:
+    print("Script Environment:")
+    print_environment(os.environ)
+
+args.compiler = toolchain_path
+if not os.path.exists(args.compiler):
+    raise ValueError('The toolchain {} does not exist.'.format(args.compiler))
+
+if toolchain_type == 'msvc' or toolchain_type=='clang-cl':
+    builder = MsvcBuilder(toolchain_type, args)
+else:
+    builder = GccBuilder(toolchain_type, args)
+
+if args.clean:
+    clean(builder.output_files())
+
+cmds = builder.build_commands()
+
+build(cmds)

lit/helper/toolchain.py

 import os
+import itertools
 import platform
 import subprocess
 import sys
 
 import lit.util
 from lit.llvm import llvm_config
 from lit.llvm.subst import FindTool
 from lit.llvm.subst import ToolSubst
 
 def use_lldb_substitutions(config):
     # Set up substitutions for primary tools.  These tools must come from config.lldb_tools_dir
     # which is basically the build output directory.  We do not want to find these in path or
     # anywhere else, since they are specifically the programs which are actually being tested.
 
     dsname = 'debugserver' if platform.system() in ['Darwin'] else 'lldb-server'
     dsargs = [] if platform.system() in ['Darwin'] else ['gdbserver']
     lldbmi = ToolSubst('%lldbmi',
                        command=FindTool('lldb-mi'),
                        extra_args=['--synchronous'],
                        unresolved='ignore')
+    build_script = os.path.dirname(__file__)
+    build_script = os.path.join(build_script, 'build.py')
+    build_script_args = [build_script, 
+                        '--compiler=any', # Default to best compiler
+                        '--arch=64'] # Default to 64-bit, user can override
+    if config.lldb_lit_tools_dir:
+        build_script_args.append('--tools-dir={0}'.format(config.lldb_lit_tools_dir))
+    if config.lldb_tools_dir:
+        build_script_args.append('--tools-dir={0}'.format(config.lldb_tools_dir))
     primary_tools = [
         ToolSubst('%lldb',
                   command=FindTool('lldb'),
                   extra_args=['-S',
                               os.path.join(config.test_source_root,
                                            'lit-lldb-init')]),
         lldbmi,
         ToolSubst('%debugserver',
                   command=FindTool(dsname),
                   extra_args=dsargs,
                   unresolved='ignore'),
-        'lldb-test'
+        'lldb-test',
+        ToolSubst('%build',
+                  command="'" + sys.executable + "'",
+                  extra_args=build_script_args)
         ]
 
     llvm_config.add_tool_substitutions(primary_tools,
                                        [config.lldb_tools_dir])
     if lldbmi.was_resolved:
         config.available_features.add('lldb-mi')
 
 def _use_msvc_substitutions(config):