Fix LLDB's lit files

[zturner]

Summary

Recently I tried to port LLDB's lit configuration files over to use a lot of the common lit infrastructure down in LLVM. This appeared to work on the surface, but broke some cases that weren't broken before and also exposed some additional problems with the old approach that we were just getting lucky with.

This is a big patch, so I'll try to give a high level overview of the problem as best I can to make reviewing easier.

When we set up a lit environment, the goal is to make it as hermetic as possible. We should not be relying on PATH and enabling the use of arbitrary shell commands. Instead, only whitelisted commands should be allowed. These are, generally speaking, the lit builtins such as echo, cd, etc, as well as anything for which substitutions have been explicitly set up for. These substitutions should map to the build output directory, but in some cases it's useful to be able to override this (for example to point to an installed tools directory).

This is, of course, how it's supposed to work. What was actually happening is that we were bringing in PATH and LD_LIBRARY_PATH and then just running the given run line as a shell command. This led to problems such as finding the wrong version of clang-cl on PATH since it wasn't even a substitution, and flakiness / non-determinism since the environment the tests were running in would change per-machine. On the other hand, it also made other things possible. For example, we had some tests that were explicitly running cl.exe and link.exe instead of clang-cl and lld-link and the only reason it worked at all is because it was finding them on PATH. Unfortunately we can't entirely get rid of these tests, because they support a few things in debug info that clang-cl and lld-link don't (notably, the LF_UDT_MOD_SRC_LINE record which makes some of the tests fail.

This patch attempts to fix all of this. It's a large patch, but I'll try to summarize the changes:

1. Removal of functionality - The lit test suite no longer respects LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER. This means there is no more support for gcc, but nobody was using this anyway (note: The functionality is still there for the dotest suite, just not the lit test suite). There is no longer a single substitution %cxx and %cc which maps to <arbitrary-compiler>, you now explicitly specify the compiler with a substitution like %clang or %clangxx or %clang_cl. We can revisit this in the future when someone needs gcc.

2. Introduction of the LLDB_LIT_TOOLS_DIR directory. This does in spirit what LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER used to do, but now more friendly. If this is not specified, all tools are expected to be the just-built tools. If it is specified, the tools which are not themselves being tested but are being used to construct and run checks (e.g. clang, FileCheck, llvm-mc, etc) will be searched for in this directory first, then the build output directory.

3. Changes to core llvm lit files. The use_lld() and use_clang() functions were introduced long ago in anticipation of using them in lldb, but since they were never actually used anywhere but their respective problems, there were some issues to be resolved regarding generality and ability to use them outside their project.

4. Changes to .test files - These are all just replacing things like clang-cl with %clang_cl and %cxx with %clangxx, etc.

5. Changes to lit.cfg.py - Previously we would load up some system environment variables and then add some new things to them. Then do a bunch of work building out our own substitutions. First, we delete the system environment variable code, making the environment hermetic. Then, we refactor the substitution logic into two separate helper functions, one which sets up substitutions for the tools we want to test (which must come from the build output directory), and another which sets up substitutions for support tools (like compilers, etc).

6. New substitutions for MSVC -- Previously we relied on location of MSVC by bringing in the entire parent's PATH and letting subprocess.Popen just run the command line. Now we set up real substitutions that should have the same effect. We use PATH to find them, and then look for INCLUDE and LIB to construct a substitution command line with appropriate /I and /LIBPATH: arguments. The nice thing about this is that it opens the door to having separate %msvc-cl32 and %msvc-cl64 substitutions, rather than only requiring the user to run vcvars first. Because we can deduce the path to 32-bit libraries from 64-bit library directories, and vice versa. Without these substitutions this would have been impossible.

Stella and Aleksandr, can you apply this patch locally and see if it fixes the issues you wer encountering?

[aprantl]

Removal of functionality - The lit test suite no longer respects LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER. This means there is no more support for gcc, but nobody was using this anyway (note: The functionality is still there for the dotest suite, just not the lit test suite).

The "nobody is using this part" of that statement is just not true.

On green dragon, we have two bots that use LLDB_TEST_C_COMPILER / LLDB_TEST_CXX_COMPILER that run the LLDB testsuite against older versions of clang:
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-6.0.1/

[stella.stamenova]

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

[aprantl]

There is also this bot that does something similar with even more compilers, including gcc:

http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31242

Do you happen to know who maintains it?

[aprantl]

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

[stella.stamenova]

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

[zturner]

In D54567#1299982, @aprantl wrote:

Removal of functionality - The lit test suite no longer respects LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER. This means there is no more support for gcc, but nobody was using this anyway (note: The functionality is still there for the dotest suite, just not the lit test suite).

The "nobody is using this part" of that statement is just not true.

On green dragon, we have two bots that use LLDB_TEST_C_COMPILER / LLDB_TEST_CXX_COMPILER that run the LLDB testsuite against older versions of clang:
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-6.0.1/

It's possible I didn't make this part clear enough. I didn't mean that nobody is using LLDB_TEST_C_COMPILER, I meant that nobody is using it in order to compile inferiors with gcc. There is also a dichotomy between what happens for the dotest suite and the lit suite. For the dotest suite, LLDB_TEST_C(XX)_COMPILER are still respected, this patch hasn't changed that. It has only changed the behavior of running tests in lldb/lit/{Breakpoint/Expr/Modules/Quit/Settings/SymbolFile/tools}. Even for those tests, it is still possible to use a not-just-built clang, just that instead of specifying LLDB_TEST_C(XX)_COMPILER, you now specify LLDB_LIT_TOOLS_DIR.

[aprantl]

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

Perhaps I'm misunderstanding something. My primary point is that we need a way to run configure which C and C++ compiler is used to compile the tests in LLDB testsuite. As long as this is still possible, I'm happy.

We discussed previously that we might want to separate tests that exercise different compilers from tests that exercise core functionality of LLDB and perhaps the lit/ vs. the packages/ subdirectories is where we want to draw this line, and only keep the configurability of the compiler for the tests in the packages/ directory. But it isn't clear to me that that is where this is going either.

[zturner]

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

[zturner]

In D54567#1299998, @aprantl wrote:

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

Perhaps I'm misunderstanding something. My primary point is that we need a way to run configure which C and C++ compiler is used to compile the tests in LLDB testsuite. As long as this is still possible, I'm happy.

We discussed previously that we might want to separate tests that exercise different compilers from tests that exercise core functionality of LLDB and perhaps the lit/ vs. the packages/ subdirectories is where we want to draw this line, and only keep the configurability of the compiler for the tests in the packages/ directory. But it isn't clear to me that that is where this is going either.

I always have trouble making sense of the Greendragon bots. Starting from this page:

http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/

What do I need to click on to get the equivalent of this: http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31244/steps/test6/logs/stdio

[aprantl]

In D54567#1299997, @zturner wrote:

It's possible I didn't make this part clear enough. I didn't mean that nobody is using LLDB_TEST_C_COMPILER, I meant that nobody is using it in order to compile inferiors with gcc. There is also a dichotomy between what happens for the dotest suite and the lit suite. For the dotest suite, LLDB_TEST_C(XX)_COMPILER are still respected, this patch hasn't changed that. It has only changed the behavior of running tests in lldb/lit/{Breakpoint/Expr/Modules/Quit/Settings/SymbolFile/tools}. Even for those tests, it is still possible to use a not-just-built clang, just that instead of specifying LLDB_TEST_C(XX)_COMPILER, you now specify LLDB_LIT_TOOLS_DIR.

Thanks, that makes more sense to me! (I'm still not sure that it is correct though: cf. http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31242)

In D54567#1299999, @zturner wrote:

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

Do we document the fact that the two testsuites behave differently in this respect? I'm worried that developers that aren't aware of that difference will test new functionality only with a LIT test out of convenience (because they are more familiar with this style from LLVM) and that that will erode our test coverage on other or older compilers over time.

[stella.stamenova]

In D54567#1299999, @zturner wrote:

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

I think this is actually confusing - there are two ways to specify compilers for the lldb test suite at cmake time:

1. Via LLDB_TEST_USE_CUSTOM_C_COMPILER and friends
2. Via LLDB_TEST_USER_ARGS

As far as I can tell, the ubuntu 14 bot that @aprantl pointed to uses the LLDB_TEST_USER_ARGS path. I *think* the green dragon bots also use the LLDB_TEST_USER_ARGS and AFAIK it's the only way gcc is ever specified for the tests. If the LLDB_TEST_USE_CUSTOM_C_COMPILER and friends are not used for the lit tests, I don't think we need them for the suite tests.

[aprantl]

In D54567#1300000, @zturner wrote:

http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/

What do I need to click on to get the equivalent of this: http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31244/steps/test6/logs/stdio

Click on a recent build and then on "Console Output". Here is one from yesterday (we are upgrading the machines right now and temporarily broke the build):
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/1356/consoleFull

[stella.stamenova]

In D54567#1300030, @aprantl wrote:

In D54567#1300000, @zturner wrote:

http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/

What do I need to click on to get the equivalent of this: http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31244/steps/test6/logs/stdio

Click on a recent build and then on "Console Output". Here is one from yesterday (we are upgrading the machines right now and temporarily broke the build):
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/1356/consoleFull

Ok, now I see it:

python /opt/llvm/zorg/zorg/jenkins//build.py --assertions --cmake-type=Release --cmake-flag=-DLLVM_TARGETS_TO_BUILD=X86 --cmake-flag=-DLLDB_TEST_USE_CUSTOM_C_COMPILER=On --cmake-flag=-DLLDB_TEST_USE_CUSTOM_CXX_COMPILER=On --cmake-flag=-DLLDB_TEST_C_COMPILER=/Users/buildslave/jenkins/workspace/lldb-cmake-clang-5.0.2/lldb-build/history/clang-5.0.2/bin/clang --cmake-flag=-DLLDB_TEST_CXX_COMPILER=/Users/buildslave/jenkins/workspace/lldb-cmake-clang-5.0.2/lldb-build/history/clang-5.0.2/bin/clang++ --dotest-flag=--skip-category --dotest-flag=gmodules lldb-cmake

[zturner]

In D54567#1300028, @aprantl wrote:

In D54567#1299997, @zturner wrote:

It's possible I didn't make this part clear enough. I didn't mean that nobody is using LLDB_TEST_C_COMPILER, I meant that nobody is using it in order to compile inferiors with gcc. There is also a dichotomy between what happens for the dotest suite and the lit suite. For the dotest suite, LLDB_TEST_C(XX)_COMPILER are still respected, this patch hasn't changed that. It has only changed the behavior of running tests in lldb/lit/{Breakpoint/Expr/Modules/Quit/Settings/SymbolFile/tools}. Even for those tests, it is still possible to use a not-just-built clang, just that instead of specifying LLDB_TEST_C(XX)_COMPILER, you now specify LLDB_LIT_TOOLS_DIR.

Thanks, that makes more sense to me! (I'm still not sure that it is correct though: cf. http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31242)

FWIW this bot appears unmaintained. I can try to contact the owner though. That aside, it looks like this bot directly runs dotest.py and constructs the command line itself (http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31242/steps/test1/logs/stdio). The first few lines of output in the stdio look like this:

++ dotest_args=()
++ dotest_args+=(-A "$arch" -C "$compiler")
++ dotest_args+=(-v -s "logs-$cc_log-$arch")
++ dotest_args+=(-u CXXFLAGS -u CFLAGS)
++ dotest_args+=(--env ARCHIVER=ar --env OBJCOPY=objcopy)
++ appendCommonArgs
++ dotest_args+=(--executable "$buildDir/bin/lldb")
++ dotest_args+=(--filecheck "$buildDir/bin/FileCheck")
++ for c in '"gdb-remote packets"' '"lldb all"'
++ dotest_args+=(--channel "$c")
++ for c in '"gdb-remote packets"' '"lldb all"'
++ dotest_args+=(--channel "$c")
++ for c in '"${categories[@]}"'
++ case "$c" in
++ dotest_args+=(--skip-category "${c#-}")
++ for c in '"${categories[@]}"'
++ case "$c" in

In that case, I don't think it relies on LLDB_TEST_C_COMPILER etc. It definitely does compile inferiors with gcc, but it seems to do it use its own custom scripts to make all this work.

In D54567#1299999, @zturner wrote:

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

Do we document the fact that the two testsuites behave differently in this respect? I'm worried that developers that aren't aware of that difference will test new functionality only with a LIT test out of convenience (because they are more familiar with this style from LLVM) and that that will erode our test coverage on other or older compilers over time.

For older versions of clang, I think it's still just as easy to run the test suite with that compiler as opposed to the just built compiler. Just stick your clang binaries in a folder and pass -DLLDB_LIT_TOOLS_DIR (we can change the variable names if anyone has ideas for better / clearer names). So any bots which are currently running tests with old versions of clang can pass this flag and that will still provide coverage with older compiler versions.

For other compilers entirely, it would be nice to come up with a solution. I'd like to eventually get rid of the check-lldb target (which constructs a dotest command line inside of CMake and just runs it directly, bypassing all of this) and if that ever happens, we could get rid of the parallel variables, but it would also necessitate solving the other-compilers problem.

[zturner]

In D54567#1300031, @stella.stamenova wrote:

In D54567#1300030, @aprantl wrote:

In D54567#1300000, @zturner wrote:

http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/

What do I need to click on to get the equivalent of this: http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31244/steps/test6/logs/stdio

Click on a recent build and then on "Console Output". Here is one from yesterday (we are upgrading the machines right now and temporarily broke the build):
http://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-clang-5.0.2/1356/consoleFull

Ok, now I see it:

python /opt/llvm/zorg/zorg/jenkins//build.py --assertions --cmake-type=Release --cmake-flag=-DLLVM_TARGETS_TO_BUILD=X86 --cmake-flag=-DLLDB_TEST_USE_CUSTOM_C_COMPILER=On --cmake-flag=-DLLDB_TEST_USE_CUSTOM_CXX_COMPILER=On --cmake-flag=-DLLDB_TEST_C_COMPILER=/Users/buildslave/jenkins/workspace/lldb-cmake-clang-5.0.2/lldb-build/history/clang-5.0.2/bin/clang --cmake-flag=-DLLDB_TEST_CXX_COMPILER=/Users/buildslave/jenkins/workspace/lldb-cmake-clang-5.0.2/lldb-build/history/clang-5.0.2/bin/clang++ --dotest-flag=--skip-category --dotest-flag=gmodules lldb-cmake

The equivalent command line after this patch would be to keep all of the -DLLDB_TEST_XXX variables, but to add one more that says -DLLDB_LIT_TOOLS_DIR=/Users/buildslave/jenkins/workspace/lldb-cmake-clang-5.0.2/lldb-build/history/clang-5.0.2/bin.

It will search this path first when looking for clang and clang++ for all of the LLDB tests, but still use the _CXX_COMPILER and _C_COMPILER paths for the LLDB-Suite tests.

At least, that's the intention.

[zturner]

In D54567#1300029, @stella.stamenova wrote:

In D54567#1299999, @zturner wrote:

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

I think this is actually confusing - there are two ways to specify compilers for the lldb test suite at cmake time:

1. Via LLDB_TEST_USE_CUSTOM_C_COMPILER and friends
2. Via LLDB_TEST_USER_ARGS

As far as I can tell, the ubuntu 14 bot that @aprantl pointed to uses the LLDB_TEST_USER_ARGS path. I *think* the green dragon bots also use the LLDB_TEST_USER_ARGS and AFAIK it's the only way gcc is ever specified for the tests. If the LLDB_TEST_USE_CUSTOM_C_COMPILER and friends are not used for the lit tests, I don't think we need them for the suite tests.

The ubuntu bot doesn't use the cmake configuration at all. It runs the test in 6 different configurations, but CMake gives you exactly 1 configuration, so it can't do things that way. AFAICT the Ubuntu bot adrian linked has 6 different shell scripts that manually construct the proper dotest command lines.

[aprantl]

In D54567#1300035, @zturner wrote:

In D54567#1300028, @aprantl wrote:

In D54567#1299997, @zturner wrote:

It's possible I didn't make this part clear enough. I didn't mean that nobody is using LLDB_TEST_C_COMPILER, I meant that nobody is using it in order to compile inferiors with gcc. There is also a dichotomy between what happens for the dotest suite and the lit suite. For the dotest suite, LLDB_TEST_C(XX)_COMPILER are still respected, this patch hasn't changed that. It has only changed the behavior of running tests in lldb/lit/{Breakpoint/Expr/Modules/Quit/Settings/SymbolFile/tools}. Even for those tests, it is still possible to use a not-just-built clang, just that instead of specifying LLDB_TEST_C(XX)_COMPILER, you now specify LLDB_LIT_TOOLS_DIR.

Thanks, that makes more sense to me! (I'm still not sure that it is correct though: cf. http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31242)

FWIW this bot appears unmaintained. I can try to contact the owner though. That aside, it looks like this bot directly runs dotest.py and constructs the command line itself (http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/31242/steps/test1/logs/stdio). The first few lines of output in the stdio look like this:

Thanks for checking! It would be unfortunate if it is unmaintained, because bots like that one are a very valuable sanity-check against symmetric bugs, where, e.g., someone misinterprets the DWARF spec and then implements the same wrong behavior in both clang and LLDB.

[zturner]

In D54567#1300029, @stella.stamenova wrote:

In D54567#1299999, @zturner wrote:

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

I think this is actually confusing - there are two ways to specify compilers for the lldb test suite at cmake time:

1. Via LLDB_TEST_USE_CUSTOM_C_COMPILER and friends
2. Via LLDB_TEST_USER_ARGS

As far as I can tell, the ubuntu 14 bot that @aprantl pointed to uses the LLDB_TEST_USER_ARGS path. I *think* the green dragon bots also use the LLDB_TEST_USER_ARGS and AFAIK it's the only way gcc is ever specified for the tests. If the LLDB_TEST_USE_CUSTOM_C_COMPILER and friends are not used for the lit tests, I don't think we need them for the suite tests.

You're right that it's confusing, but I don't have any good ideas here. Some ideas (which are not necessarily good):

• We could change the names of LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER to LLDB_DOTEST_C_COMPILER and LLDB_DOTEST_CXX_COMPILER.
• We could restructure the directories under lldb/lit so that it looks like this:

lldb
\- lit
   |- tests
   |- unittests
   \- dotest

• We could get rid of the lit folder entirely, and put everythign in the tests folder, like this:

lldb
\- test
   |- lit
   |- unittests
   \- dotest

Maybe some combinatino of these (or other ideas that others have) can make things clearer. But I don't want to change too much all at once.

[stella.stamenova]

In D54567#1300064, @zturner wrote:

In D54567#1300029, @stella.stamenova wrote:

In D54567#1299999, @zturner wrote:

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

I think this is actually confusing - there are two ways to specify compilers for the lldb test suite at cmake time:

1. Via LLDB_TEST_USE_CUSTOM_C_COMPILER and friends
2. Via LLDB_TEST_USER_ARGS

As far as I can tell, the ubuntu 14 bot that @aprantl pointed to uses the LLDB_TEST_USER_ARGS path. I *think* the green dragon bots also use the LLDB_TEST_USER_ARGS and AFAIK it's the only way gcc is ever specified for the tests. If the LLDB_TEST_USE_CUSTOM_C_COMPILER and friends are not used for the lit tests, I don't think we need them for the suite tests.

You're right that it's confusing, but I don't have any good ideas here. Some ideas (which are not necessarily good):

• We could change the names of LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER to LLDB_DOTEST_C_COMPILER and LLDB_DOTEST_CXX_COMPILER.
• We could restructure the directories under lldb/lit so that it looks like this:

lldb
\- lit
   |- tests
   |- unittests
   \- dotest

• We could get rid of the lit folder entirely, and put everythign in the tests folder, like this:

lldb
\- test
   |- lit
   |- unittests
   \- dotest

Maybe some combinatino of these (or other ideas that others have) can make things clearer. But I don't want to change too much all at once.

I actually think we should remove it entirely and rely on LLDB_USER_TEST_ARGS because it allows for the user to specify exactly which compiler to use for the lldb suite and it only gives the user one way to do it (instead of two which magically get merged for the lldb suite).

In the cmake for the tests, this is what we do:

list(APPEND LLDB_TEST_COMMON_ARGS
  --executable ${LLDB_TEST_EXECUTABLE}
  --dsymutil ${LLDB_TEST_DSYMUTIL}
  --filecheck ${LLDB_TEST_FILECHECK}
  -C ${LLDB_TEST_C_COMPILER}
  )

so setting the LLDB_TEST_C_COMPILER is the equivalent of passing `-c \path\to\compiler``` as part of the LLDB_TEST_USER_ARGS. Also, we don't actually use the LLDB_TEST_CXX_COMPILER for the lldb suite at all.

[zturner]

In D54567#1300083, @stella.stamenova wrote:

In D54567#1300064, @zturner wrote:

In D54567#1300029, @stella.stamenova wrote:

In D54567#1299999, @zturner wrote:

In D54567#1299993, @stella.stamenova wrote:

In D54567#1299992, @aprantl wrote:

In D54567#1299989, @stella.stamenova wrote:

We should also remove LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER from the cmake files along with this change, otherwise, people will still expect them to work.

That would not be a good idea. There are several bots that are using these flags.

The change that Zachary is making is removing their usage, so after his change they would not do anything. If he ends up committing this change, these two properties (along with LLDB_DEFAULT_TEST_C_COMPILER and LLDB_TEST_USE_CUSTOM_C_COMPILER, etc.) should also go.

The flags are still needed for (and used by) the dotest suite, I didn't change that part. Normally you run that suite by doing ninja check-lldb, in which case it never goes through these lit files to begin with. But they will also run as part of ninja check-lldb-lit, but that lit configuration file totally overrides everything in the parent one, so nothing in this patch should have any effect on that.

I think this is actually confusing - there are two ways to specify compilers for the lldb test suite at cmake time:

1. Via LLDB_TEST_USE_CUSTOM_C_COMPILER and friends
2. Via LLDB_TEST_USER_ARGS

As far as I can tell, the ubuntu 14 bot that @aprantl pointed to uses the LLDB_TEST_USER_ARGS path. I *think* the green dragon bots also use the LLDB_TEST_USER_ARGS and AFAIK it's the only way gcc is ever specified for the tests. If the LLDB_TEST_USE_CUSTOM_C_COMPILER and friends are not used for the lit tests, I don't think we need them for the suite tests.

You're right that it's confusing, but I don't have any good ideas here. Some ideas (which are not necessarily good):

• We could change the names of LLDB_TEST_C_COMPILER and LLDB_TEST_CXX_COMPILER to LLDB_DOTEST_C_COMPILER and LLDB_DOTEST_CXX_COMPILER.
• We could restructure the directories under lldb/lit so that it looks like this:

lldb
\- lit
   |- tests
   |- unittests
   \- dotest

• We could get rid of the lit folder entirely, and put everythign in the tests folder, like this:

lldb
\- test
   |- lit
   |- unittests
   \- dotest

Maybe some combinatino of these (or other ideas that others have) can make things clearer. But I don't want to change too much all at once.

I actually think we should remove it entirely and rely on LLDB_USER_TEST_ARGS because it allows for the user to specify exactly which compiler to use for the lldb suite and it only gives the user one way to do it (instead of two which magically get merged for the lldb suite).

In the cmake for the tests, this is what we do:

list(APPEND LLDB_TEST_COMMON_ARGS
  --executable ${LLDB_TEST_EXECUTABLE}
  --dsymutil ${LLDB_TEST_DSYMUTIL}
  --filecheck ${LLDB_TEST_FILECHECK}
  -C ${LLDB_TEST_C_COMPILER}
  )

so setting the LLDB_TEST_C_COMPILER is the equivalent of passing `-c \path\to\compiler``` as part of the LLDB_TEST_USER_ARGS. Also, we don't actually use the LLDB_TEST_CXX_COMPILER for the lldb suite at all.

That could work. Part of the existing confusion and complexity comes from the fact that CMake is static up-front configuration but dotest has a lot of args that control the way things run, including multiple compilers and architectures. Trying to expose these through independent CMake variables and building up a command line probably introduces more complexity than it alleviates.

That said, in the interest of not changing too much all at once, it still seems like something that's perhaps better done in a future patch. WDYT?

[JDevlieghere]

Personally I don't think we should differentiate between the lit and dotest suite when it comes to using a custom compiler. The separation between the two suits is mostly the result of what framework is easier to write your test is (or which one you're more familiar with). How hard would it be to have one variable that works for both?

[stella.stamenova]

That said, in the interest of not changing too much all at once, it still seems like something that's perhaps better done in a future patch. WDYT?

I actually think it would be better now - the people who use the properties for compile the lit tests will have to make a change now anyway and those are most likely the same people that use the properties to compile the lldb suite tests, so they could make a change once and have both working rather than making it a two-step process.

[zturner]

In D54567#1300183, @JDevlieghere wrote:

Personally I don't think we should differentiate between the lit and dotest suite when it comes to using a custom compiler. The separation between the two suits is mostly the result of what framework is easier to write your test is (or which one you're more familiar with). How hard would it be to have one variable that works for both?

I'm not sure how hard it would be.

One problem is that dotest supports not just choosing a compiler, but choosing multiple compilers, as well as multiple architectures and it runs the test suite over the cross product of all of these. That's hard to express in CMake. This is why, for example, people end up subverting it entirely for productionizing test suite runs, such as what you see on the ubuntu bot linked earlier. It doesn't even use the CMake variables for running the dotest suite, it just has scripts that build command lines and runs them.

There is another issue I'm aware of, which is that some people's compilers have version numbers embedded in the binary name. Right now the code that uses LLDB_LIT_TOOLS_DIR to find the binaries won't handle these cases, because it looks specifically for clang and clang++, but not, for example, clang-7.0 or clang++-hexagon-7.0.

I do think an iterative approach is better though. This is already a big change and as this thread (and the previous patch which is what I'm trying to fix) shows, people use things in a lot of different ways so changing something has potential for lots of breakage in subtle ways. So I still kind of prefer doing things incrementally, letting people tell me what's broken, and then working on a solution.

We could try to converge on the single LLDB_LIT_TOOLS_DIR approach for both dotest as well as the lit suite, because having one variable with simple semantics is nice. But I think we should worry first about getting to a known good baseline and then working incrementally to make simplifications.

[JDevlieghere]

In D54567#1300459, @zturner wrote:

I'm not sure how hard it would be.

One problem is that dotest supports not just choosing a compiler, but choosing multiple compilers, as well as multiple architectures and it runs the test suite over the cross product of all of these. That's hard to express in CMake. This is why, for example, people end up subverting it entirely for productionizing test suite runs, such as what you see on the ubuntu bot linked earlier. It doesn't even use the CMake variables for running the dotest suite, it just has scripts that build command lines and runs them.

Makes sense. Just to be clear, I'm not advocating running a product for the lit suite, just having one option that controls both dotest and lit.

There is another issue I'm aware of, which is that some people's compilers have version numbers embedded in the binary name. Right now the code that uses LLDB_LIT_TOOLS_DIR to find the binaries won't handle these cases, because it looks specifically for clang and clang++, but not, for example, clang-7.0 or clang++-hexagon-7.0.

How is this handled today? Do we have tests that do something like that?

I do think an iterative approach is better though. This is already a big change and as this thread (and the previous patch which is what I'm trying to fix) shows, people use things in a lot of different ways so changing something has potential for lots of breakage in subtle ways. So I still kind of prefer doing things incrementally, letting people tell me what's broken, and then working on a solution.

I'm all for iteration! We just wanna make sure we share the same "end goal".

We could try to converge on the single LLDB_LIT_TOOLS_DIR approach for both dotest as well as the lit suite, because having one variable with simple semantics is nice. But I think we should worry first about getting to a known good baseline and then working incrementally to make simplifications.

I'm worried that the directory approach is incompatible with settings a specific compiler (like gcc).

[zturner]

In D54567#1300641, @JDevlieghere wrote:

In D54567#1300459, @zturner wrote:

I'm not sure how hard it would be.

One problem is that dotest supports not just choosing a compiler, but choosing multiple compilers, as well as multiple architectures and it runs the test suite over the cross product of all of these. That's hard to express in CMake. This is why, for example, people end up subverting it entirely for productionizing test suite runs, such as what you see on the ubuntu bot linked earlier. It doesn't even use the CMake variables for running the dotest suite, it just has scripts that build command lines and runs them.

Makes sense. Just to be clear, I'm not advocating running a product for the lit suite, just having one option that controls both dotest and lit.

There is another issue I'm aware of, which is that some people's compilers have version numbers embedded in the binary name. Right now the code that uses LLDB_LIT_TOOLS_DIR to find the binaries won't handle these cases, because it looks specifically for clang and clang++, but not, for example, clang-7.0 or clang++-hexagon-7.0.

How is this handled today? Do we have tests that do something like that?

The tests themselves don't try to do it, usually our tests try to be compiler agnostic, but then people will run the tests with one or more different compilers. I think people use dotest in this manner, by specifying a path to a compiler with a version number in it, but I think it may be all downstream stuff, with no bot coverage. Mostly they do it by specifying LLDB_TEST_C_COMPILER=/path/to/clang-7.0 or something, and dotest is smart enough to figure this out. We could certainly have this same logic in the lit files though once we get to that point.

I do think an iterative approach is better though. This is already a big change and as this thread (and the previous patch which is what I'm trying to fix) shows, people use things in a lot of different ways so changing something has potential for lots of breakage in subtle ways. So I still kind of prefer doing things incrementally, letting people tell me what's broken, and then working on a solution.

I'm all for iteration! We just wanna make sure we share the same "end goal".

We could try to converge on the single LLDB_LIT_TOOLS_DIR approach for both dotest as well as the lit suite, because having one variable with simple semantics is nice. But I think we should worry first about getting to a known good baseline and then working incrementally to make simplifications.

I'm worried that the directory approach is incompatible with settings a specific compiler (like gcc).

I don't think the directory approach is fundamentally incompatible with using non-clang compilers. But it's important to differentiate what the test suite itself supports and what the test suite supports when invoked via a CMake-generated target. That is to say, we don't have to expose all the power of the underlying test suite through CMake. I actually think trying to do so is a mistake, because you're often going to be circumventing it anyway to run the test suite in some special way that wasn't how you configured CMake. dotest.py, for example, takes tons of different command line arguments, many of which are only available if you invoke it directly, as the CMake-generated command line will never use those arguments.

So I think this is similar. You can pass arbitrary parameters to lit when you point it to a test directory, so there's fundamental limitation in terms of what's possible this way. Of course, we don't currently recognize any, and yes it means you can't specify a specific compiler binary (such as GCC anymore), but I'm not entirely sure how useful that is in the first place, because we're talking about a static CMake configuration.

So I think as far as what we expose through CMake, it should be as simple as possible and support the developer use case and the most common buildbot use cases, but more advanced use cases are still supported by directly invoking llvm-lit.py with some special arguments (which we would still need to teach the LLDB lit configuration files to understand).

Hopefully this all makes sense.

[JDevlieghere]

In D54567#1300670, @zturner wrote:

In D54567#1300641, @JDevlieghere wrote:

Makes sense. Just to be clear, I'm not advocating running a product for the lit suite, just having one option that controls both dotest and lit.

How is this handled today? Do we have tests that do something like that?

The tests themselves don't try to do it, usually our tests try to be compiler agnostic, but then people will run the tests with one or more different compilers. I think people use dotest in this manner, by specifying a path to a compiler with a version number in it, but I think it may be all downstream stuff, with no bot coverage. Mostly they do it by specifying LLDB_TEST_C_COMPILER=/path/to/clang-7.0 or something, and dotest is smart enough to figure this out. We could certainly have this same logic in the lit files though once we get to that point.

I was aware of the dotest stuff (like Adrian mentioned our bots are using that). My question was specific to lit, but I guess nobody is using that functionality there (yet).

I'm all for iteration! We just wanna make sure we share the same "end goal".

I'm worried that the directory approach is incompatible with settings a specific compiler (like gcc).

I don't think the directory approach is fundamentally incompatible with using non-clang compilers. But it's important to differentiate what the test suite itself supports and what the test suite supports when invoked via a CMake-generated target. That is to say, we don't have to expose all the power of the underlying test suite through CMake. I actually think trying to do so is a mistake, because you're often going to be circumventing it anyway to run the test suite in some special way that wasn't how you configured CMake. dotest.py, for example, takes tons of different command line arguments, many of which are only available if you invoke it directly, as the CMake-generated command line will never use those arguments.

I agree. However I believe that *if* we expose it through CMake (for the convenience of dotest and the lldb-dotest wrapper) it should apply to CMake as well, with the notable exception of arguments we pass through the LLDB_DOTEST_ARGS variable.

So I think this is similar. You can pass arbitrary parameters to lit when you point it to a test directory, so there's fundamental limitation in terms of what's possible this way. Of course, we don't currently recognize any, and yes it means you can't specify a specific compiler binary (such as GCC anymore), but I'm not entirely sure how useful that is in the first place, because we're talking about a static CMake configuration.

Yup, I think we're on the same page here, both dotest and lit accept parameters. We already have a mechanism to forward args to dotest, so if we wanted we could hook things up to make a lit argument work with either or both.

So I think as far as what we expose through CMake, it should be as simple as possible and support the developer use case and the most common buildbot use cases, but more advanced use cases are still supported by directly invoking llvm-lit.py with some special arguments (which we would still need to teach the LLDB lit configuration files to understand).

No objections to this.

Hopefully this all makes sense.

It does, thanks!

[zturner]

• Added an __init__.py to the helper directory. This is required on Python 2 to make the import work.
• Changed a bunch of %cc substitutions to %clang.
• Changed the %clang_cl command lines to always pass /c. This fixes an issue where we are unable to automatically locate the installed Visual Studio when running from inside of VS.

After this patch, the test run is clean on OSX and awaiting word on whether it's clean on Windows (it is for me).

Thanks to @stella.stamenova for the help.

[stella.stamenova]

Feel free to check this in. At this point this will only improve the results on Windows and it works correctly on Linux.

lldb/lit/SymbolFile/PDB/class-layout.test

1

Why only msvc and not system-windows like the rest of the tests?

[zturner]

Well msvc implies system-windows, but i can have both

clang/test/lit.cfg.py

 
 # test_exec_root: The root path where tests should be run.
 config.test_exec_root = os.path.join(config.clang_obj_root, 'test')
 
 llvm_config.use_default_substitutions()
 
 llvm_config.use_clang()
 
+config.substitutions.append(
+    ('%src_include_dir', config.clang_src_dir + '/include'))
+
+
 # Propagate path to symbolizer for ASan/MSan.
 llvm_config.with_system_environment(
     ['ASAN_SYMBOLIZER_PATH', 'MSAN_SYMBOLIZER_PATH'])
 
 config.substitutions.append(('%PATH%', config.environment['PATH']))
 
 
 # For each occurrence of a clang tool name, replace it with the full path to

lldb/lit/Breakpoint/case-insensitive.test

 # REQUIRES: system-windows
 # XFAIL: system-windows
 # -> llvm.org/pr24528
 #
-# RUN: %cc %p/Inputs/case-sensitive.c -g -o %t
+# RUN: %clang %p/Inputs/case-sensitive.c -g -o %t
 # RUN: lldb-test breakpoints %t %s | FileCheck %s
 
 breakpoint set -f case-sensitive.c -l 3
 # CHECK-LABEL: breakpoint set -f case-sensitive.c -l 3
 # CHECK: At least one location.
 
 breakpoint set -f %p/Inputs/case-sensitive.c -l 3
 # CHECK-LABEL: breakpoint set -f {{.*}}/Inputs/case-sensitive.c -l 3

lldb/lit/Breakpoint/case-sensitive.test

 # REQUIRES: nowindows
 #
-# RUN: %cc %p/Inputs/case-sensitive.c -g -o %t
+# RUN: %clang %p/Inputs/case-sensitive.c -g -o %t
 # RUN: lldb-test breakpoints %t %s | FileCheck %s
 
 breakpoint set -f case-sensitive.c -l 3
 # CHECK-LABEL: breakpoint set -f case-sensitive.c -l 3
 # CHECK: At least one location.
 
 breakpoint set -f %p/Inputs/case-sensitive.c -l 3
 # CHECK-LABEL: breakpoint set -f {{.*}}/Inputs/case-sensitive.c -l 3

lldb/lit/Expr/TestIRMemoryMap.test

 # UNSUPPORTED: windows
 
-# RUN: %cxx %p/Inputs/call-function.cpp -g -o %t
+# RUN: %clangxx %p/Inputs/call-function.cpp -g -o %t
 
 # RUN: lldb-test ir-memory-map %t %S/Inputs/ir-memory-map-basic
 # RUN: lldb-test ir-memory-map -host-only %t %S/Inputs/ir-memory-map-basic
 
 # RUN: lldb-test ir-memory-map %t %S/Inputs/ir-memory-map-overlap1
 # RUN: lldb-test ir-memory-map -host-only %t %S/Inputs/ir-memory-map-overlap1
 
 # RUN: lldb-test ir-memory-map %t %S/Inputs/ir-memory-map-mix-malloc-free
 # RUN: lldb-test ir-memory-map -host-only %t %S/Inputs/ir-memory-map-mix-malloc-free

lldb/lit/Expr/TestIRMemoryMapWindows.test

 # REQUIRES: system-windows
 
-# RUN: clang-cl /Zi %p/Inputs/call-function.cpp -o %t
+# RUN: %clang_cl /Zi %p/Inputs/call-function.cpp -o %t
 
 # RUN: lldb-test ir-memory-map %t %S/Inputs/ir-memory-map-basic
 # RUN: lldb-test ir-memory-map -host-only %t %S/Inputs/ir-memory-map-basic
 
 # RUN: lldb-test ir-memory-map %t %S/Inputs/ir-memory-map-overlap1
 # RUN: lldb-test ir-memory-map -host-only %t %S/Inputs/ir-memory-map-overlap1
 
 # RUN: lldb-test ir-memory-map %t %S/Inputs/ir-memory-map-mix-malloc-free
 # RUN: lldb-test ir-memory-map -host-only %t %S/Inputs/ir-memory-map-mix-malloc-free

lldb/lit/Quit/TestQuitExitCode-30.test

 # UNSUPPORTED: windows
-# RUN: python %S/expect_exit_code.py 226 %lldb -b -s %s
+# RUN: %python %S/expect_exit_code.py 226 %lldb -b -s %s
 q -30

lldb/lit/Quit/TestQuitExitCode30.test

 # UNSUPPORTED: windows
-# RUN: python %S/expect_exit_code.py 30  %lldb -b -s %s
+# RUN: %python %S/expect_exit_code.py 30  %lldb -b -s %s
 q 30

lldb/lit/Quit/TestQuitExitCodeHexA.test

 # UNSUPPORTED: windows
-# RUN: python %S/expect_exit_code.py 10 %lldb -b -s %s
+# RUN: %python %S/expect_exit_code.py 10 %lldb -b -s %s
 q 0xA

lldb/lit/SymbolFile/DWARF/apple-index-is-used.cpp

 // Test that we use the apple indexes.
-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols %t | FileCheck %s
 
 // CHECK: .apple_names index present
 // CHECK: .apple_types index present
 
 int foo;

lldb/lit/SymbolFile/DWARF/debug-names-compressed.cpp

 // Test for a bug where we crashed while processing a compressed debug_names
 // section (use after free).
 
 // REQUIRES: lld, zlib
 
-// RUN: clang -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf %s
+// RUN: %clang -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf %s
 // RUN: ld.lld %t.o -o %t --compress-debug-sections=zlib
 // RUN: lldb-test symbols --find=variable --name=foo %t | FileCheck %s
 
 // CHECK: Found 1 variables:
 int foo;
 // ONE-DAG: name = "foo", type = {{.*}} (int), {{.*}} decl = debug-names-compressed.cpp:[[@LINE-1]]
 
 extern "C" void _start() {}

lldb/lit/SymbolFile/DWARF/dwarf5-index-is-used.cpp

 // Test that we use the DWARF v5 name indexes.
 
 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf -gpubnames
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf -gpubnames
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols %t | FileCheck %s
 
 // CHECK: Name Index
 // CHECK: String: 0x{{.*}} "_start"
 // CHECK: Tag: DW_TAG_subprogram
 
 extern "C" void _start() {}

lldb/lit/SymbolFile/DWARF/dwarf5-partial-index.cpp

 // Test that we return complete results when only a part of the binary is built
 // with an index.
 
 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t-1.o --target=x86_64-pc-linux -DONE -mllvm -accel-tables=Dwarf
-// RUN: clang %s -g -c -o %t-2.o --target=x86_64-pc-linux -DTWO -mllvm -accel-tables=Dwarf
+// RUN: %clang %s -g -c -o %t-1.o --target=x86_64-pc-linux -DONE -mllvm -accel-tables=Dwarf
+// RUN: %clang %s -g -c -o %t-2.o --target=x86_64-pc-linux -DTWO -mllvm -accel-tables=Dwarf
 // RUN: ld.lld %t-1.o %t-2.o -o %t
 // RUN: lldb-test symbols --find=variable --name=foo  %t | FileCheck %s
 
 // CHECK: Found 2 variables:
 #ifdef ONE
 namespace one {
 int foo;
 // CHECK-DAG: name = "foo", {{.*}} decl = dwarf5-partial-index.cpp:[[@LINE-1]]
 } // namespace one
 extern "C" void _start() {}
 #else
 namespace two {
 int foo;
 // CHECK-DAG: name = "foo", {{.*}} decl = dwarf5-partial-index.cpp:[[@LINE-1]]
 } // namespace two
 #endif

lldb/lit/SymbolFile/DWARF/find-basic-function.cpp

 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=base %t | \
 // RUN:   FileCheck --check-prefix=BASE %s
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=method %t | \
 // RUN:   FileCheck --check-prefix=METHOD %s
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=full %t | \
 // RUN:   FileCheck --check-prefix=FULL %s
 // RUN: lldb-test symbols --name=_Z3fooi --find=function --function-flags=full %t | \
 // RUN:   FileCheck --check-prefix=FULL-MANGLED %s
 // RUN: lldb-test symbols --name=foo --context=context --find=function --function-flags=base %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=function %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 //
-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=base %t | \
 // RUN:   FileCheck --check-prefix=BASE %s
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=method %t | \
 // RUN:   FileCheck --check-prefix=METHOD %s
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=full %t | \
 // RUN:   FileCheck --check-prefix=FULL %s
 // RUN: lldb-test symbols --name=_Z3fooi --find=function --function-flags=full %t | \
 // RUN:   FileCheck --check-prefix=FULL-MANGLED %s
 // RUN: lldb-test symbols --name=foo --context=context --find=function --function-flags=base %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=function %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=base %t | \
 // RUN:   FileCheck --check-prefix=BASE %s
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=method %t | \
 // RUN:   FileCheck --check-prefix=METHOD %s
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=full %t | \
 // RUN:   FileCheck --check-prefix=FULL %s
 // RUN: lldb-test symbols --name=_Z3fooi --find=function --function-flags=full %t | \

lldb/lit/SymbolFile/DWARF/find-basic-namespace.cpp

 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=namespace %t | \
 // RUN:   FileCheck --check-prefix=FOO %s
 // RUN: lldb-test symbols --name=foo --find=namespace --context=context %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=namespace %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 //
-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=foo --find=namespace %t | \
 // RUN:   FileCheck --check-prefix=FOO %s
 // RUN: lldb-test symbols --name=foo --find=namespace --context=context %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=namespace %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=namespace %t | \
 // RUN:   FileCheck --check-prefix=FOO %s
 // RUN: lldb-test symbols --name=foo --find=namespace --context=context %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=namespace %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 

lldb/lit/SymbolFile/DWARF/find-basic-type.cpp

 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=type %t | \
 // RUN:   FileCheck --check-prefix=NAME %s
 // RUN: lldb-test symbols --name=foo --context=context --find=type %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=type %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 //
-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=foo --find=type %t | \
 // RUN:   FileCheck --check-prefix=NAME %s
 // RUN: lldb-test symbols --name=foo --context=context --find=type %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=type %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=type %t | \
 // RUN:   FileCheck --check-prefix=NAME %s
 // RUN: lldb-test symbols --name=foo --context=context --find=type %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=not_there --find=type %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 

lldb/lit/SymbolFile/DWARF/find-basic-variable.cpp

 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=variable --context=context %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=foo --find=variable %t | \
 // RUN:   FileCheck --check-prefix=NAME %s
 // RUN: lldb-test symbols --regex --name=foo --find=variable %t | \
 // RUN:   FileCheck --check-prefix=REGEX %s
 // RUN: lldb-test symbols --name=not_there --find=variable %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 //
-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=foo --find=variable --context=context %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=foo --find=variable %t | \
 // RUN:   FileCheck --check-prefix=NAME %s
 // RUN: lldb-test symbols --regex --name=foo --find=variable %t | \
 // RUN:   FileCheck --check-prefix=REGEX %s
 // RUN: lldb-test symbols --name=not_there --find=variable %t | \
 // RUN:   FileCheck --check-prefix=EMPTY %s
 //
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=variable --context=context %t | \
 // RUN:   FileCheck --check-prefix=CONTEXT %s
 // RUN: lldb-test symbols --name=foo --find=variable %t | \
 // RUN:   FileCheck --check-prefix=NAME %s
 // RUN: lldb-test symbols --regex --name=foo --find=variable %t | \
 // RUN:   FileCheck --check-prefix=REGEX %s
 // RUN: lldb-test symbols --name=not_there --find=variable %t | \

lldb/lit/SymbolFile/DWARF/find-function-regex.cpp

 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=f.o --regex --find=function %t | FileCheck %s
 //
-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=f.o --regex --find=function %t | FileCheck %s
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=f.o --regex --find=function %t | FileCheck %s
 
 // CHECK: Found 3 functions:
 // CHECK-DAG: name = "foo()", mangled = "_Z3foov"
 // CHECK-DAG: name = "ffo()", mangled = "_Z3ffov"
 // CHECK-DAG: name = "bar::foo()", mangled = "_ZN3bar3fooEv"
 
 void foo() {}
 void ffo() {}
 namespace bar {
 void foo() {}
 } // namespace bar
 void fof() {}
 
 extern "C" void _start() {}

lldb/lit/SymbolFile/DWARF/find-method-local-struct.cpp

-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=method %t | \
 // RUN:   FileCheck %s
 
 // CHECK-DAG: name = "sbar::foo()", mangled = "_ZN4sbar3fooEv"
 // CHECK-DAG: name = "ffbar()::sbar::foo()", mangled = "_ZZ5ffbarvEN4sbar3fooEv"
 
 struct sbar {
   void foo();

lldb/lit/SymbolFile/DWARF/find-method.cpp

 // REQUIRES: lld
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=method %t | \
 // RUN:   FileCheck %s
 //
-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=foo --find=function --function-flags=method %t | \
 // RUN:   FileCheck %s
 
 // CHECK-DAG: name = "A::foo()", mangled = "_ZN1A3fooEv"
 // CHECK-DAG: name = "B::foo()", mangled = "_ZN1B3fooEv"
 // CHECK-DAG: name = "C::foo()", mangled = "_ZN1C3fooEv"
 
 struct A {

lldb/lit/SymbolFile/DWARF/find-qualified-variable.cpp

-// RUN: clang %s -g -c -o %t --target=x86_64-apple-macosx
+// RUN: %clang %s -g -c -o %t --target=x86_64-apple-macosx
 // RUN: lldb-test symbols --name=A::foo --find=variable %t | FileCheck %s
 
 // CHECK: Found 1 variables:
 
 struct A {
   static int foo;
 };
 int A::foo;
 // NAME-DAG: name = "foo", {{.*}} decl = find-qualified-variable.cpp:[[@LINE-1]]
 
 struct B {
   static int foo;
 };
 int B::foo;

lldb/lit/SymbolFile/DWARF/find-type-in-function.cpp

 // REQUIRES: lld
 
 // XFAIL: *
 
-// RUN: clang %s -g -c -o %t.o --target=x86_64-pc-linux
+// RUN: %clang %s -g -c -o %t.o --target=x86_64-pc-linux
 // RUN: ld.lld %t.o -o %t
 // RUN: lldb-test symbols --name=foo --find=type %t | \
 // RUN:   FileCheck --check-prefix=NAME %s
 
 // Lookup for "foo" should find either both "struct foo" types or just the
 // global one. Right now, it finds the definition inside bar(), which is
 // definitely wrong.
 

lldb/lit/SymbolFile/DWARF/find-variable-dwo.cpp

 // REQUIRES: lld
 
-// RUN: clang %s -g -gsplit-dwarf -c -emit-llvm -o - --target=x86_64-pc-linux -DONE | \
+// RUN: %clang %s -g -gsplit-dwarf -c -emit-llvm -o - --target=x86_64-pc-linux -DONE | \
 // RUN:   llc -accel-tables=Dwarf -filetype=obj -split-dwarf-file=%t-1.dwo -o %t-1.o
 // RUN: llvm-objcopy --split-dwo=%t-1.dwo %t-1.o
-// RUN: clang %s -g -gsplit-dwarf -c -emit-llvm -o - --target=x86_64-pc-linux -DTWO | \
+// RUN: %clang %s -g -gsplit-dwarf -c -emit-llvm -o - --target=x86_64-pc-linux -DTWO | \
 // RUN:   llc -accel-tables=Dwarf -filetype=obj -split-dwarf-file=%t-2.dwo -o %t-2.o
 // RUN: llvm-objcopy --split-dwo=%t-2.dwo %t-2.o
 // RUN: ld.lld %t-1.o %t-2.o -o %t
 // RUN: lldb-test symbols --name=foo --find=variable %t | FileCheck %s
 
 // CHECK: Found 2 variables:
 #ifdef ONE
 namespace one {

lldb/lit/SymbolFile/DWARF/find-variable-file.cpp

 // REQUIRES: lld
 
-// RUN: clang -g -c -o %t-1.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable %s
-// RUN: clang -g -c -o %t-2.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable %S/Inputs/find-variable-file-2.cpp
+// RUN: %clang -g -c -o %t-1.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable %s
+// RUN: %clang -g -c -o %t-2.o --target=x86_64-pc-linux -mllvm -accel-tables=Disable %S/Inputs/find-variable-file-2.cpp
 // RUN: ld.lld %t-1.o %t-2.o -o %t
 // RUN: lldb-test symbols --file=find-variable-file.cpp --find=variable %t | \
 // RUN:   FileCheck --check-prefix=ONE %s
 // RUN: lldb-test symbols --file=find-variable-file-2.cpp --find=variable %t | \
 // RUN:   FileCheck --check-prefix=TWO %s
 
-// RUN: clang -g -c -o %t-1.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf %s
-// RUN: clang -g -c -o %t-2.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf %S/Inputs/find-variable-file-2.cpp
+// RUN: %clang -g -c -o %t-1.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf %s
+// RUN: %clang -g -c -o %t-2.o --target=x86_64-pc-linux -mllvm -accel-tables=Dwarf %S/Inputs/find-variable-file-2.cpp
 // RUN: ld.lld %t-1.o %t-2.o -o %t
 // RUN: lldb-test symbols --file=find-variable-file.cpp --find=variable %t | \
 // RUN:   FileCheck --check-prefix=ONE %s
 // RUN: lldb-test symbols --file=find-variable-file-2.cpp --find=variable %t | \
 // RUN:   FileCheck --check-prefix=TWO %s
 
 // ONE: Found 1 variables:
 namespace one {
 int foo;
 // ONE-DAG: name = "foo", type = {{.*}} (int), {{.*}} decl = find-variable-file.cpp:[[@LINE-1]]
 } // namespace one
 
 extern "C" void _start() {}
 
 // TWO: Found 1 variables:
 // TWO-DAG: name = "foo", {{.*}} decl = find-variable-file-2.cpp:2

lldb/lit/SymbolFile/NativePDB/ast-reconstruction.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test various interesting cases for AST reconstruction.
-// RUN: clang-cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/ast-reconstruction.lldbinit 2>&1 | FileCheck %s
 
 // Test trivial versions of each tag type.
 class TrivialC {};
 struct TrivialS {};
 union TrivialU {};
 enum TrivialE {TE_A};
 

lldb/lit/SymbolFile/NativePDB/bitfields.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test various interesting cases for AST reconstruction.
-// RUN: clang-cl /Z7 /GS- /GR- /std:c++latest -Xclang -fkeep-static-consts /c /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- /std:c++latest -Xclang -fkeep-static-consts /c /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/bitfields.lldbinit 2>&1 | FileCheck %s
 
 // Test trivial versions of each tag type.
 struct Struct {
   int A : 5 = 6;
   int B : 7 = 8;
   unsigned C : 3 = 2;
   unsigned D : 15 = 12345;

lldb/lit/SymbolFile/NativePDB/disassembly.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can show disassembly and source.
-// RUN: clang-cl -m64 /Z7 /GS- /GR- /c /Fo%t.obj -- %s
+// RUN: %clang_cl -m64 /Z7 /GS- /GR- /c /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/disassembly.lldbinit | FileCheck %s
 
 // Some context lines before
 // the function.
 
 int foo() { return 42; }
 
 int main(int argc, char **argv) {

lldb/lit/SymbolFile/NativePDB/function-types-builtins.cpp

 // clang-format off
 // REQUIRES: lld
 
-// RUN: clang-cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/function-types-builtins.lldbinit | FileCheck %s
 
 // Test that we can display function signatures with simple builtin
 // and pointer types.  We do this by using `target variable` in lldb
 // with global variables of type ptr-to-function or reference-to-function.
 // This technique in general allows us to exercise most of LLDB's type
 // system without a running process.
 

lldb/lit/SymbolFile/NativePDB/function-types-calling-conv.cpp

 // clang-format off
 // REQUIRES: lld
 
-// RUN: clang-cl -m32 /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
+// RUN: %clang_cl -m32 /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/function-types-calling-conv.lldbinit | FileCheck %s
 
 
 void __stdcall StdcallFn() {}
 void __fastcall FastcallFn() {}
 void __thiscall ThiscallFn() {}
 void __cdecl CdeclFn() {}
 void __vectorcall VectorcallFn() {}

lldb/lit/SymbolFile/NativePDB/function-types-classes.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display function signatures with class types.
-// RUN: clang-cl /Z7 /GS- /GR- /c -fstandalone-debug -Xclang -fkeep-static-consts /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- /c -fstandalone-debug -Xclang -fkeep-static-consts /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/function-types-classes.lldbinit | FileCheck %s
 
 // This is just some unimportant helpers needed so that we can get reference and
 // rvalue-reference types into return values.
 template<typename T>
 struct MakeResult {
   static T result() {
     return T{};

lldb/lit/SymbolFile/NativePDB/global-classes.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display tag types.
-// RUN: clang-cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/globals-classes.lldbinit | FileCheck %s
 
 enum class EnumType : unsigned {
   A = 1,
   B = 2
 };
 
 class ClassNoPadding {

lldb/lit/SymbolFile/NativePDB/globals-bss.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Make sure we can read variables from BSS
-// RUN: clang-cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
 // RUN: llvm-readobj -s %t.exe | FileCheck --check-prefix=BSS %s
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/globals-bss.lldbinit 2>&1 | FileCheck %s
 
 int GlobalVariable = 0;
 
 int main(int argc, char **argv) {
   return 0;
 }
 

lldb/lit/SymbolFile/NativePDB/globals-fundamental.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display tag types.
-// RUN: clang-cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- /c -Xclang -fkeep-static-consts /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/globals-fundamental.lldbinit | FileCheck %s
 
 
 // Fundamental data types
 bool BFalse = false;
 // CHECK: (lldb) target variable BFalse
 // CHECK-NEXT: (bool) BFalse = false
 bool BTrue = true;

lldb/lit/SymbolFile/NativePDB/nested-types.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test various interesting cases for AST reconstruction.
-// RUN: clang-cl /Z7 /GS- /GR- -Xclang -fkeep-static-consts /c /Fo%t.obj -- %s
+// RUN: %clang_cl /Z7 /GS- /GR- -Xclang -fkeep-static-consts /c /Fo%t.obj -- %s
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/nested-types.lldbinit 2>&1 | FileCheck %s
 
 struct S {
   struct NestedStruct {
     int A = 0;
     int B = 1;
   };
   int C = 2;

lldb/lit/SymbolFile/NativePDB/s_constant.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display S_CONSTANT records.
 
 // RUN: llvm-mc -filetype=obj -triple=x86_64-pc-win32 %p/Inputs/s_constant.s > %t.obj
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/s_constant.lldbinit | FileCheck %s
 
 // clang-cl cannot generate S_CONSTANT records, but we need to test that we can
 // handle them for compatibility with MSVC, which does emit them.  This test
 // case was generated by compiling this file with MSVC and copying the bytes
 // that they emit for S_CONSTANT records.  Then we compile the same code with
 // clang to get a .s file, and replace all S_LDATA32 records with the bytes from
 // the S_CONSTANT records.  This way we end up with a .s file that contains

lldb/lit/SymbolFile/NativePDB/simple-breakpoints.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can set simple breakpoints using PDB on any platform.
-// RUN: clang-cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
+// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/breakpoints.lldbinit | FileCheck %s
 
 // Use different indentation style for each overload so that the starting
 // line is in a different place.
 int OvlGlobalFn(int X) {
   return X + 42;
 }
 

lldb/lit/SymbolFile/NativePDB/source-list.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can set display source of functions.
-// RUN: clang-cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
+// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/source-list.lldbinit | FileCheck %s
 
 
 
 // Some context lines before
 // the function.
 
 

lldb/lit/SymbolFile/NativePDB/tag-types.cpp

 // clang-format off
 // REQUIRES: lld
 
 // Test that we can display tag types.
-// RUN: clang-cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
+// RUN: %clang_cl /Z7 /GS- /GR- /c /Fo%t.obj -- %s 
 // RUN: lld-link /DEBUG /nodefaultlib /entry:main /OUT:%t.exe /PDB:%t.pdb -- %t.obj
-// RUN: env LLDB_USE_NATIVE_PDB_READER=1 lldb -f %t.exe -s \
+// RUN: env LLDB_USE_NATIVE_PDB_READER=1 %lldb -f %t.exe -s \
 // RUN:     %p/Inputs/tag-types.lldbinit | FileCheck %s
 
 // Test struct
 struct Struct {
   // Test builtin types, which are represented by special CodeView type indices.
   bool                B;
   char                C;
   signed char         SC;

lldb/lit/SymbolFile/PDB/ast-restore.test

 REQUIRES: system-windows
-RUN: cl /Zi /GS- /c %S/Inputs/AstRestoreTest.cpp /Fo%t.obj
-RUN: link /debug:full /nodefaultlib /entry:main %t.obj /out:%t.exe
+RUN: %msvc_cl /Zi /GS- /c %S/Inputs/AstRestoreTest.cpp /Fo%t.obj
+RUN: %msvc_link /debug:full /nodefaultlib /entry:main %t.obj /out:%t.exe
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=ENUM %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=GLOBAL %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=BASE %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=CLASS %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=INNER %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=TEMPLATE %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=FOO %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=MAIN %s

lldb/lit/SymbolFile/PDB/calling-conventions.test

 REQUIRES: system-windows, lld
-RUN: clang-cl -m32 /Zi /GS- /c %S/Inputs/CallingConventionsTest.cpp /o %t.obj
+RUN: %clang_cl -m32 /Zi /GS- /c %S/Inputs/CallingConventionsTest.cpp /o %t.obj
 RUN: lld-link /debug:full /nodefaultlib /entry:main %t.obj /out:%t.exe
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck %s
 
 CHECK: Module: {{.*}}
 CHECK-DAG: int (*FuncCCallPtr)();
 CHECK-DAG: int (*FuncStdCallPtr)() __attribute__((stdcall));
 CHECK-DAG: int (*FuncFastCallPtr)() __attribute__((fastcall));
 CHECK-DAG: int (*FuncVectorCallPtr)() __attribute__((vectorcall));
 CHECK-DAG: int (S::*FuncThisCallPtr)() __attribute__((thiscall));

lldb/lit/SymbolFile/PDB/class-layout.test

-REQUIRES: system-windows
-RUN: clang-cl -m32 /Z7 /c /GS- %S/Inputs/ClassLayoutTest.cpp /o %T/ClassLayoutTest.cpp.obj
-RUN: link %T/ClassLayoutTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/ClassLayoutTest.cpp.exe
+REQUIRES: msvc
+RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/ClassLayoutTest.cpp /o %T/ClassLayoutTest.cpp.obj
+RUN: %msvc_link %T/ClassLayoutTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/ClassLayoutTest.cpp.exe
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=ENUM %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=UNION %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=STRUCT %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=COMPLEX %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=LIST %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=UNNAMED-STRUCT %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=BASE %s

lldb/lit/SymbolFile/PDB/compilands.test

-REQUIRES: system-windows
-RUN: clang-cl /Z7 %S/Inputs/CompilandsTest.cpp /o %T/CompilandsTest.cpp.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl /Z7 /c %S/Inputs/CompilandsTest.cpp /o %T/CompilandsTest.cpp.obj
+RUN: %msvc_link /debug:full /nodefaultlib /entry:main %T/CompilandsTest.cpp.obj /out:%T/CompilandsTest.cpp.exe
 RUN: lldb-test symbols %T/CompilandsTest.cpp.exe | FileCheck %s
 
 ; Link default libraries
 
 CHECK: Module [[CU:.*]]
 CHECK: {{^[0-9A-F]+}}: SymbolVendor ([[CU]])
 CHECK: {{^[0-9A-F]+}}:   CompileUnit{{[{]0x[0-9a-f]+[}]}}, language = "c++", file = '{{.*}}\CompilandsTest.cpp'

lldb/lit/SymbolFile/PDB/enums-layout.test

-REQUIRES: system-windows
-RUN: clang-cl -m32 /Z7 /c /GS- %S/Inputs/SimpleTypesTest.cpp /o %T/SimpleTypesTest.cpp.enums.obj
-RUN: link %T/SimpleTypesTest.cpp.enums.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/SimpleTypesTest.cpp.enums.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/SimpleTypesTest.cpp /o %T/SimpleTypesTest.cpp.enums.obj
+RUN: %msvc_link %T/SimpleTypesTest.cpp.enums.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/SimpleTypesTest.cpp.enums.exe
 RUN: lldb-test symbols %T/SimpleTypesTest.cpp.enums.exe | FileCheck %s
 
 ; FIXME: PDB does not have information about scoped enumeration (Enum class) so the  
 ; compiler type used is the same as the one for unscoped enumeration.
 
 CHECK: Module [[CU:.*]]
 CHECK-DAG: {{^[0-9A-F]+}}: SymbolVendor ([[CU]])
 CHECK:      Type{{.*}} , name = "Enum", size = 4, decl = SimpleTypesTest.cpp:19, compiler_type = {{.*}} enum Enum {

lldb/lit/SymbolFile/PDB/func-symbols.test

-REQUIRES: system-windows
-RUN: clang-cl -m32 /Z7 /c /GS- %S/Inputs/FuncSymbolsTestMain.cpp /o %T/FuncSymbolsTestMain.cpp.obj
-RUN: clang-cl -m32 /Z7 /c /GS- %S/Inputs/FuncSymbols.cpp /o %T/FuncSymbols.cpp.obj
-RUN: link %T/FuncSymbolsTestMain.cpp.obj %T/FuncSymbols.cpp.obj /DEBUG /nodefaultlib /Entry:main /OUT:%T/FuncSymbolsTest.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/FuncSymbolsTestMain.cpp /o %T/FuncSymbolsTestMain.cpp.obj
+RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/FuncSymbols.cpp /o %T/FuncSymbols.cpp.obj
+RUN: %msvc_link %T/FuncSymbolsTestMain.cpp.obj %T/FuncSymbols.cpp.obj /DEBUG /nodefaultlib /Entry:main /OUT:%T/FuncSymbolsTest.exe
 RUN: lldb-test symbols %T/FuncSymbolsTest.exe | FileCheck --check-prefix=CHECK-ONE %s
 RUN: lldb-test symbols %T/FuncSymbolsTest.exe | FileCheck --check-prefix=CHECK-TWO %s
 
 ; Link multiple objects
 ; In this test, We don't check demangled name of a mangled function.
 
 CHECK-ONE: Module [[MD:.*]]
 CHECK-ONE-DAG: {{.*}}: SymbolVendor ([[MD]])

lldb/lit/SymbolFile/PDB/function-level-linking.test

 REQUIRES: system-windows, lld
-RUN: clang-cl /c /Zi /Gy %S/Inputs/FunctionLevelLinkingTest.cpp /o %t.obj
+RUN: %clang_cl /c /Zi /Gy %S/Inputs/FunctionLevelLinkingTest.cpp /o %t.obj
 RUN: lld-link /debug:full /nodefaultlib /entry:main /order:@%S/Inputs/FunctionLevelLinkingTest.ord %t.obj /out:%t.exe
 RUN: lldb-test symbols -verify %t.exe

lldb/lit/SymbolFile/PDB/function-nested-block.test

 REQUIRES: system-windows, lld
-RUN: clang-cl /c /Zi %S/Inputs/FunctionNestedBlockTest.cpp /o %t.obj
+RUN: %clang_cl /c /Zi %S/Inputs/FunctionNestedBlockTest.cpp /o %t.obj
 RUN: lld-link /debug:full /nodefaultlib /entry:main %t.obj /out:%t.exe
 RUN: lldb-test symbols -find=function -file FunctionNestedBlockTest.cpp -line 4 %t.exe | FileCheck --check-prefix=CHECK-FUNCTION %s
 RUN: lldb-test symbols -find=block -file FunctionNestedBlockTest.cpp -line 4 %t.exe | FileCheck --check-prefix=CHECK-BLOCK %s
 
 CHECK-FUNCTION: Found 1 functions:
 CHECK-FUNCTION: name = "{{.*}}", mangled = "{{_?}}main"
 
 CHECK-BLOCK: Found 1 blocks:
 CHECK-BLOCK: Blocks: id = {{.*}}, range = {{.*}}
 CHECK-BLOCK:         id = {{.*}}, range = {{.*}}

lldb/lit/SymbolFile/PDB/pointers.test

-REQUIRES: system-windows
-RUN: clang-cl -m32 /Z7 /c /GS- %S/Inputs/PointerTypeTest.cpp /o %T/PointerTypeTest.cpp.obj
-RUN: link %T/PointerTypeTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/PointerTypeTest.cpp.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/PointerTypeTest.cpp /o %T/PointerTypeTest.cpp.obj
+RUN: %msvc_link %T/PointerTypeTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/PointerTypeTest.cpp.exe
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=MAIN-ST-F %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=MAIN-ST %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=MAIN %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=F %s
 
 CHECK: Module [[MOD:.*]]
 CHECK: {{^[0-9A-F]+}}:   CompileUnit{{[{]0x[0-9a-f]+[}]}}, language = "c++", file = '{{.*}}\PointerTypeTest.cpp'

lldb/lit/SymbolFile/PDB/type-quals.test

-REQUIRES: system-windows
-RUN: clang-cl -m32 /Z7 /c /GS- %S/Inputs/TypeQualsTest.cpp /o %T/TypeQualsTest.cpp.obj
-RUN: link %T/TypeQualsTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/TypeQualsTest.cpp.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/TypeQualsTest.cpp /o %T/TypeQualsTest.cpp.obj
+RUN: %msvc_link %T/TypeQualsTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/TypeQualsTest.cpp.exe
 RUN: lldb-test symbols %T/TypeQualsTest.cpp.exe | FileCheck %s
 
 CHECK: Module [[MOD:.*]]
 CHECK-DAG: {{^[0-9A-F]+}}: SymbolVendor ([[MOD]])
 CHECK:      Type{{.*}} , name = "const int", size = 4, compiler_type = {{.*}} const int
 CHECK:      Type{{.*}} , size = 4, compiler_type = {{.*}} const int *
 CHECK:      Type{{.*}} , size = 4, compiler_type = {{.*}} const int **const
 CHECK:      Type{{.*}} , size = 4, compiler_type = {{.*}} const int *const

lldb/lit/SymbolFile/PDB/typedefs.test

-REQUIRES: system-windows
-RUN: clang-cl -m32 /Z7 /c /GS- %S/Inputs/SimpleTypesTest.cpp /o %T/SimpleTypesTest.cpp.typedefs.obj
-RUN: link %T/SimpleTypesTest.cpp.typedefs.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/SimpleTypesTest.cpp.typedefs.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/SimpleTypesTest.cpp /o %T/SimpleTypesTest.cpp.typedefs.obj
+RUN: %msvc_link %T/SimpleTypesTest.cpp.typedefs.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/SimpleTypesTest.cpp.typedefs.exe
 RUN: lldb-test symbols %T/SimpleTypesTest.cpp.typedefs.exe | FileCheck %s
 
 ; Generate 32-bit target
 
 ; FIXME: PDB does not have line information for typedef statements so source
 ; and line information for them is not tested.
 
 ; Note, types `long double` and `double` have same bit size in MSVC and there

lldb/lit/SymbolFile/PDB/udt-layout.test

-REQUIRES: system-windows
-RUN: clang-cl /Zi %S/Inputs/UdtLayoutTest.cpp /o %t.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl /Zi %S/Inputs/UdtLayoutTest.cpp /c /o %t.obj
+RUN: %msvc_link /DEBUG:FULL /out:%t.exe %t.obj 
 RUN: %lldb -b -s %S/Inputs/UdtLayoutTest.script -- %t.exe | FileCheck %s
 
 CHECK:(int) int C::abc = 123
 CHECK:(List [16]) ls = {
 CHECK:  [15] = {
 CHECK:    Prev = 0x00000000
 CHECK:    Next = 0x00000000
 CHECK:    Value = {

lldb/lit/SymbolFile/PDB/variables-locations.test

-REQUIRES: system-windows
-RUN: clang-cl /Zi %S/Inputs/VariablesLocationsTest.cpp /o %t.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl /Zi %S/Inputs/VariablesLocationsTest.cpp /c /o %t.obj
+RUN: %msvc_link /debug:full %t.obj /out:%t.exe
 RUN: %lldb -b -s %S/Inputs/VariablesLocationsTest.script -- %t.exe | FileCheck %s
 
 CHECK: g_var = 2222
 
 CHECK: arg_0 = 1111
 CHECK: arg_1 = 0.123
 
 CHECK: loc_0 = 'x'
 CHECK: loc_1 = 0.567
 
 CHECK: loc_0 = true
 CHECK: loc_1 = 3333

lldb/lit/SymbolFile/PDB/variables.test

-REQUIRES: system-windows
-RUN: clang-cl -m64 /Z7 /c /GS- %S/Inputs/VariablesTest.cpp /o %T/VariablesTest.cpp.obj
-RUN: link %T/VariablesTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/VariablesTest.cpp.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl -m64 /Z7 /c /GS- %S/Inputs/VariablesTest.cpp /o %T/VariablesTest.cpp.obj
+RUN: %msvc_link %T/VariablesTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/VariablesTest.cpp.exe
 RUN: lldb-test symbols %T/VariablesTest.cpp.exe | FileCheck %s
 
 CHECK: Module [[MOD:.*]]
 CHECK: SymbolVendor ([[MOD]])
 CHECK:     CompileUnit{{.*}}, language = "c++", file = '{{.*}}\VariablesTest.cpp'
 CHECK-DAG:   Variable{{.*}}, name = "g_IntVar"
 CHECK-SAME:  scope = global, location = {{.*}}, external
 CHECK-DAG:   Variable{{.*}}, name = "m_StaticClassMember"

 CHECK-DAG:     Variable{{.*}}, name = "a"
 CHECK-SAME:    scope = parameter
 
 CHECK-DAG:   Function{[[FID4:.*]]}, mangled = ?Func@Class@@QEAAXXZ
 CHECK-NEXT:  Block{[[FID4]]}
 CHECK-DAG:     Variable{{.*}}, name = "this"
 CHECK-SAME:    scope = parameter
 CHECK-SAME:    artificial
 No newline at end of file

lldb/lit/SymbolFile/PDB/vbases.test

-REQUIRES: system-windows
-RUN: clang-cl /Zi %S/Inputs/VBases.cpp /o %t.exe
+REQUIRES: system-windows, msvc
+RUN: %clang_cl /Zi %S/Inputs/VBases.cpp /c /o %t.obj
+RUN: %msvc_link /debug:full %t.obj /out:%t.exe
 RUN: %lldb -b -s %S/Inputs/VBases.script -- %t.exe | FileCheck %s
 
 CHECK: {
 CHECK:   A = (a = '\x01')
 CHECK:   B = (b = 2)
 CHECK:   c = 3
 CHECK: }
 
 CHECK: {
 CHECK:   A = (a = '\x01')
 CHECK:   B = (b = 2)
 CHECK:   c = 3
 CHECK: }

lldb/lit/helper/toolchain.py

+import os
+import platform
+import subprocess
+import sys
+
+import lit.util
+from lit.llvm import llvm_config
+from lit.llvm.subst import FindTool
+from lit.llvm.subst import ToolSubst
+
+def use_lldb_substitutions(config):
+    # Set up substitutions for primary tools.  These tools must come from config.lldb_tools_dir
+    # which is basically the build output directory.  We do not want to find these in path or
+    # anywhere else, since they are specifically the programs which are actually being tested.
+
+    dsname = 'debugserver' if platform.system() in ['Darwin'] else 'lldb-server'
+    dsargs = [] if platform.system() in ['Darwin'] else ['gdbserver']
+    lldbmi = ToolSubst('%lldbmi',
+                       command=FindTool('lldb-mi'),
+                       extra_args=['--synchronous'],
+                       unresolved='ignore')
+    primary_tools = [
+        ToolSubst('%lldb',
+                  command=FindTool('lldb'),
+                  extra_args=['-S',
+                              os.path.join(config.test_source_root,
+                                           'lit-lldb-init')]),
+        lldbmi,
+        ToolSubst('%debugserver',
+                  command=FindTool(dsname),
+                  extra_args=dsargs,
+                  unresolved='ignore'),
+        'lldb-test'
+        ]
+
+    llvm_config.add_tool_substitutions(primary_tools,
+                                       [config.lldb_tools_dir])
+    if lldbmi.was_resolved:
+        config.available_features.add('lldb-mi')
+
+def _use_msvc_substitutions(config):
+    # If running from a Visual Studio Command prompt (e.g. vcvars), this will
+    # detect the include and lib paths, and find cl.exe and link.exe and create
+    # substitutions for each of them that explicitly specify /I and /L paths
+    cl = '"' + lit.util.which('cl') + '"'
+    link = '"' + lit.util.which('link') + '"'
+
+    if not cl or not link:
+        return
+
+    includes = os.getenv('INCLUDE', '').split(';')
+    libs = os.getenv('LIB', '').split(';')
+
+    config.available_features.add('msvc')
+    compiler_flags = ['"/I{}"'.format(x) for x in includes if os.path.exists(x)]
+    linker_flags = ['"/LIBPATH:{}"'.format(x) for x in libs if os.path.exists(x)]
+
+    tools = [
+        ToolSubst('%msvc_cl', command=cl, extra_args=compiler_flags),
+        ToolSubst('%msvc_link', command=link, extra_args=linker_flags)]
+    llvm_config.add_tool_substitutions(tools)
+    return
+
+def use_support_substitutions(config):
+    # Set up substitutions for support tools.  These tools can be overridden at the CMake
+    # level (by specifying -DLLDB_LIT_TOOLS_DIR), installed, or as a last resort, we can use
+    # the just-built version.
+    flags = []
+    if platform.system() in ['Darwin']:
+        try:
+            out = subprocess.check_output(['xcrun', '--show-sdk-path']).strip()
+            res = 0
+        except OSError:
+            res = -1
+        if res == 0 and out:
+            sdk_path = lit.util.to_string(out)
+            lit_config.note('using SDKROOT: %r' % sdk_path)
+            flags = ['-isysroot', sdk_path]
+    elif platform.system() in ['OpenBSD']:
+        flags = ['-pthread']
+
+
+    additional_tool_dirs=[]
+    if config.lldb_lit_tools_dir:
+        additional_tool_dirs.append(config.lldb_lit_tools_dir)
+
+    llvm_config.use_clang(additional_flags=flags,
+                          additional_tool_dirs=additional_tool_dirs,
+                          required=True)
+
+    if sys.platform == 'win32':
+        _use_msvc_substitutions(config)
+
+    have_lld = llvm_config.use_lld(additional_tool_dirs=additional_tool_dirs,
+                                   required=False)
+    if have_lld:
+        config.available_features.add('lld')
+
+
+    support_tools = ['yaml2obj', 'obj2yaml', 'llvm-pdbutil',
+                     'llvm-mc', 'llvm-readobj', 'llvm-objdump',
+                     'llvm-objcopy']
+    additional_tool_dirs += [config.lldb_tools_dir, config.llvm_tools_dir]
+    llvm_config.add_tool_substitutions(support_tools, additional_tool_dirs)

lldb/lit/lit.cfg.py

 # -*- Python -*-
 
 import os
-import sys
 import re
-import platform
 import shutil
-import subprocess
+import site
+import sys
 
-import lit.util
 import lit.formats
 from lit.llvm import llvm_config
 from lit.llvm.subst import FindTool
 from lit.llvm.subst import ToolSubst
 
+from helper import toolchain
+
 # name: The name of this test suite.
 config.name = 'LLDB'
 
 # testFormat: The test format to use to interpret tests.
 config.test_format = lit.formats.ShTest(not llvm_config.use_lit_shell)
 
 # suffixes: A list of file extensions to treat as test files. This is overriden
 # by individual lit.local.cfg files in the test subdirectories.
 config.suffixes = ['.test', '.cpp', '.s']
 
 # excludes: A list of directories to exclude from the testsuite. The 'Inputs'
 # subdirectories contain auxiliary inputs for various tests in their parent
 # directories.
 config.excludes = ['Inputs', 'CMakeLists.txt', 'README.txt', 'LICENSE.txt']
 
 # test_source_root: The root path where tests are located.
 config.test_source_root = os.path.dirname(__file__)
 
 # test_exec_root: The root path where tests should be run.
 config.test_exec_root = os.path.join(config.lldb_obj_root, 'lit')
 
-# Tweak the PATH to include the tools dir.
-llvm_config.with_system_environment('PATH')
-llvm_config.with_environment('PATH', config.lldb_tools_dir, append_path=True)
-llvm_config.with_environment('PATH', config.llvm_tools_dir, append_path=True)
-
-llvm_config.with_environment('LD_LIBRARY_PATH', config.lldb_libs_dir, append_path=True)
-llvm_config.with_environment('LD_LIBRARY_PATH', config.llvm_libs_dir, append_path=True)
-llvm_config.with_system_environment('LD_LIBRARY_PATH', append_path=True)
-
 
 llvm_config.use_default_substitutions()
 
-if platform.system() in ['Darwin']:
-    debugserver = lit.util.which('debugserver', config.lldb_tools_dir)
-else:
-    debugserver = lit.util.which('lldb-server', config.lldb_tools_dir)
-lldb = "%s -S %s/lit-lldb-init" % (lit.util.which('lldb', config.lldb_tools_dir),
-                               config.test_source_root)
-
-lldbmi = lit.util.which('lldb-mi', config.lldb_tools_dir)
-if lldbmi:
-    config.available_features.add('lldb-mi')
-
-config.cc = llvm_config.use_llvm_tool(config.cc, required=True)
-config.cxx = llvm_config.use_llvm_tool(config.cxx, required=True)
-
-if platform.system() in ['Darwin']:
-    try:
-        out = subprocess.check_output(['xcrun', '--show-sdk-path']).strip()
-        res = 0
-    except OSError:
-        res = -1
-    if res == 0 and out:
-        sdk_path = lit.util.to_string(out)
-        lit_config.note('using SDKROOT: %r' % sdk_path)
-        config.cc += " -isysroot %s" % sdk_path
-        config.cxx += " -isysroot %s" % sdk_path
-
-if platform.system() in ['OpenBSD']:
-    config.cc += " -pthread"
-    config.cxx += " -pthread"
-
-config.substitutions.append(('%cc', config.cc))
-config.substitutions.append(('%cxx', config.cxx))
-
-if lldbmi:
-  config.substitutions.append(('%lldbmi', lldbmi + " --synchronous"))
-config.substitutions.append(('%lldb', lldb))
-
-if debugserver is not None:
-    if platform.system() in ['Darwin']:
-        config.substitutions.append(('%debugserver', debugserver))
-    else:
-        config.substitutions.append(('%debugserver', debugserver + ' gdbserver'))
+toolchain.use_lldb_substitutions(config)
+
+toolchain.use_support_substitutions(config)
 
-tools = ['lldb-test', 'yaml2obj', 'obj2yaml', 'llvm-pdbutil']
-llvm_config.add_tool_substitutions(tools, [config.llvm_tools_dir, config.lldb_tools_dir])
 
 if re.match(r'^arm(hf.*-linux)|(.*-linux-gnuabihf)', config.target_triple):
     config.available_features.add("armhf-linux")
 
-print("config.cc = {}".format(config.cc))
-if re.match(r'icc', config.cc):
-    config.available_features.add("compiler-icc")
-elif re.match(r'clang', config.cc):
-    config.available_features.add("compiler-clang")
-elif re.match(r'gcc', config.cc):
-    config.available_features.add("compiler-gcc")
-elif re.match(r'cl', config.cc):
-    config.available_features.add("compiler-msvc")
-
-if config.have_lld:
-  config.available_features.add("lld")
-
 def calculate_arch_features(arch_string):
     # This will add a feature such as x86, arm, mips, etc for each built
     # target
     features = []
     for arch in arch_string.split():
         features.append(arch.lower())
     return features
 

lldb/lit/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 config.llvm_src_root = "@LLVM_SOURCE_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
 config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
+config.llvm_shlib_dir = "@SHLIBDIR@"
 config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
 config.lldb_obj_root = "@LLDB_BINARY_DIR@"
 config.lldb_libs_dir = "@LLDB_LIBS_DIR@"
 config.lldb_tools_dir = "@LLDB_TOOLS_DIR@"
+# Since it comes from the command line, it may have backslashes which
+# should not need to be escaped.
+config.lldb_lit_tools_dir = r"@LLDB_LIT_TOOLS_DIR@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.python_executable = "@PYTHON_EXECUTABLE@"
-config.cc = "@LLDB_TEST_C_COMPILER@"
-config.cxx = "@LLDB_TEST_CXX_COMPILER@"
 config.have_zlib = @LLVM_ENABLE_ZLIB@
-config.have_lld = @LLDB_HAVE_LLD@
+config.host_triple = "@LLVM_HOST_TRIPLE@"
 
 # Support substitution of the tools and libs dirs with user parameters. This is
 # used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
+    config.llvm_shlib_dir = config.llvm_shlib_dir % lit_config.params
     config.lldb_libs_dir = config.lldb_libs_dir % lit_config.params
     config.lldb_tools_dir = config.lldb_tools_dir % lit_config.params
-    config.cc = config.cc % lit_config.params
-    config.cxx = config.cxx % lit_config.params
+    config.lldb_lit_tools_dir = config.lldb_lit_tools_dir % lit_config.params
+
 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@LLDB_SOURCE_DIR@/lit/lit.cfg.py")

lldb/lit/tools/lldb-mi/breakpoint/break-insert-enable-pending.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/break-insert-pending.c -g
+# RUN: %clang -o %t %p/inputs/break-insert-pending.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test for enabling pending breakpoints globally
 
 -break-insert printf
 # CHECK: ^error,msg="Command 'break-insert'. Breakpoint location 'printf' not found
 
 -gdb-set breakpoint pending on

lldb/lit/tools/lldb-mi/breakpoint/break-insert.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o a.exe %p/inputs/break-insert.c -g
+# RUN: %clang -o a.exe %p/inputs/break-insert.c -g
 # RUN: %lldbmi < %s | FileCheck %s
 
 # Test that a breakpoint can be inserted before creating a target.
 
 -break-insert breakpoint
 # CHECK: ^done,bkpt={number="1"
 
 -file-exec-and-symbols a.exe
 # CHECK: ^done
 
 -exec-run
 # CHECK: ^running
 # CHECK: *stopped,reason="breakpoint-hit"
 

lldb/lit/tools/lldb-mi/data/data-info-line.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/data-info-line.c -g
+# RUN: %clang -o %t %p/inputs/data-info-line.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -data-info-line command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main

lldb/lit/tools/lldb-mi/exec/exec-continue.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -exec-continue command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main
 # CHECK: ^done,bkpt={number="1"
 
 -exec-run
 # CHECK: ^running
 # CHECK: *stopped,reason="breakpoint-hit"
 
 -exec-continue
 # CHECK: ^running

lldb/lit/tools/lldb-mi/exec/exec-finish.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -exec-finish command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main

lldb/lit/tools/lldb-mi/exec/exec-interrupt.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -exec-interrupt command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main
 # CHECK: ^done,bkpt={number="1"
 
 -exec-run
 # CHECK: ^running
 # CHECK: *stopped,reason="breakpoint-hit"
 
 -exec-interrupt
 # CHECK: ^error,msg="Process is not running."

lldb/lit/tools/lldb-mi/exec/exec-next-instruction.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -exec-next-instruction command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main

lldb/lit/tools/lldb-mi/exec/exec-next.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -exec-next command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main

lldb/lit/tools/lldb-mi/exec/exec-step-instruction.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -exec-step-instruction command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main

lldb/lit/tools/lldb-mi/exec/exec-step.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -exec-step command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -break-insert main

lldb/lit/tools/lldb-mi/symbol/symbol-list-lines.test

 # XFAIL: system-windows
 # -> llvm.org/pr24452
 #
-# RUN: %cc -o %t %p/inputs/main.c %p/inputs/symbol-list-lines.c %p/inputs/list-lines-helper.c -g
+# RUN: %clang -o %t %p/inputs/main.c %p/inputs/symbol-list-lines.c %p/inputs/list-lines-helper.c -g
 # RUN: %lldbmi %t < %s | FileCheck %s
 
 # Test lldb-mi -symbol-list-lines command.
 
 # Check that we have a valid target created via '%lldbmi %t'.
 # CHECK: ^done
 
 -symbol-list-lines invalid_file.c
 # Check a case of invalid file name.
 # CHECK: ^error,msg="File Handler. Invalid file name path"
 
 -symbol-list-lines symbol-list-lines.c
 # CHECK: ^done,lines=[{pc="0x{{[0-9A-Fa-f]+}}",line="3"},{pc="0x{{[0-9A-Fa-f]+}}",line="4"},{pc="0x{{[0-9A-Fa-f]+}}",line="5"}{{.*}}]
 
 -symbol-list-lines list-lines-helper.c
 # CHECK: ^done,lines=[{pc="0x{{[0-9A-Fa-f]+}}",line="1"},{pc="0x{{[0-9A-Fa-f]+}}",line="2"},{pc="0x{{[0-9A-Fa-f]+}}",line="3"}{{.*}}]

lldb/lit/tools/lldb-mi/target/target-select-so-path.test

 # UNSUPPORTED: windows, darwin
 #
-# RUN: %cc -o %t %p/inputs/main.c -g
+# RUN: %clang -o %t %p/inputs/main.c -g
 # RUN: %python %p/inputs/target-select-so-path.py "%debugserver" "%lldbmi %t" %s
 
 # Test that -target-select command can hook up a path
 # added by gdb-set solib-search-path.
 
 # Check that we have a valid target created via file-exec-and-symbols.
 # CHECK: ^done
 

lldb/test/CMakeLists.txt

if(LLDB_BUILD_FRAMEWORK)
   list(APPEND LLDB_TEST_COMMON_ARGS --framework ${LLDB_FRAMEWORK_DIR}/LLDB.framework)
 endif()
 
 if (NOT ${CMAKE_SYSTEM_NAME} MATCHES "Windows|Darwin")
   list(APPEND LLDB_TEST_COMMON_ARGS
     --env ARCHIVER=${CMAKE_AR} --env OBJCOPY=${CMAKE_OBJCOPY})
 endif()
 
+if (NOT "${LLDB_LIT_TOOLS_DIR}" STREQUAL "")
+  if (NOT EXISTS "${LLDB_LIT_TOOLS_DIR}")
+    message(WARNING "LLDB_LIT_TOOLS_DIR ${LLDB_LIT_TOOLS_DIR} does not exist.")
+  endif()
+endif()
+
 if(CMAKE_HOST_APPLE)
   list(APPEND LLDB_TEST_COMMON_ARGS --server ${DEBUGSERVER_PATH})
 endif()
 
 if(SKIP_DEBUGSERVER)
   list(APPEND LLDB_TEST_COMMON_ARGS --out-of-tree-debugserver)
 endif()
 

llvm/utils/lit/lit/TestingConfig.py

def load_from_path(self, path, litConfig):
             litConfig.fatal('unable to load config file: %r' % (path,))
         f.close()
 
         # Execute the config script to initialize the object.
         cfg_globals = dict(globals())
         cfg_globals['config'] = self
         cfg_globals['lit_config'] = litConfig
         cfg_globals['__file__'] = path
+        original_sys_path = list(sys.path)
         try:
+            sys.path.insert(0, os.path.dirname(path))
             exec(compile(data, path, 'exec'), cfg_globals, None)
             if litConfig.debug:
                 litConfig.note('... loaded config %r' % path)
         except SystemExit:
             e = sys.exc_info()[1]
             # We allow normal system exit inside a config file to just
             # return control without error.
             if e.args:
                 raise
         except:
             import traceback
             litConfig.fatal(
                 'unable to parse config file %r, traceback: %s' % (
                     path, traceback.format_exc()))
-
+        sys.path = original_sys_path
         self.finish(litConfig)
 
     def __init__(self, parent, name, suffixes, test_format,
                  environment, substitutions, unsupported,
                  test_exec_root, test_source_root, excludes,
                  available_features, pipefail, limit_to_features = [],
                  is_early = False, parallelism_group = ""):
         self.parent = parent

llvm/utils/lit/lit/llvm/config.py

def use_llvm_tool(self, name, search_env=None, required=False, quiet=False):
             self.lit_config.fatal(message)
 
         if tool:
             tool = os.path.normpath(tool)
             if not self.lit_config.quiet and not quiet:
                 self.lit_config.note('using {}: {}'.format(name, tool))
         return tool
 
-    def use_clang(self, required=True):
+    def use_clang(self, additional_tool_dirs=[], additional_flags=[], required=True):
         """Configure the test suite to be able to invoke clang.
 
         Sets up some environment variables important to clang, locates a
         just-built or installed clang, and add a set of standard
         substitutions useful to any test suite that makes use of clang.
 
         """
         # Clear some environment variables that might affect Clang.
def use_clang(self, additional_tool_dirs=[], additional_flags=[], required=True):
         # Clang/Win32 may refer to %INCLUDE%. vsvarsall.bat sets it.
         if platform.system() != 'Windows':
             possibly_dangerous_env_vars.append('INCLUDE')
 
         self.clear_environment(possibly_dangerous_env_vars)
 
         # Tweak the PATH to include the tools dir and the scripts dir.
         # Put Clang first to avoid LLVM from overriding out-of-tree clang builds.
-        possible_paths = ['clang_tools_dir', 'llvm_tools_dir']
-        paths = [getattr(self.config, pp) for pp in possible_paths
+        exe_dir_props = [self.config.name.lower() + '_tools_dir', 'clang_tools_dir', 'llvm_tools_dir']
+        paths = [getattr(self.config, pp) for pp in exe_dir_props
                  if getattr(self.config, pp, None)]
+        paths = additional_tool_dirs + paths
         self.with_environment('PATH', paths, append_path=True)
 
-        paths = [self.config.llvm_shlib_dir, self.config.llvm_libs_dir]
+        lib_dir_props = [self.config.name.lower() + '_libs_dir', 'clang_libs_dir', 'llvm_shlib_dir', 'llvm_libs_dir']
+        paths = [getattr(self.config, pp) for pp in lib_dir_props
+                 if getattr(self.config, pp, None)]
+
         self.with_environment('LD_LIBRARY_PATH', paths, append_path=True)
 
         # Discover the 'clang' and 'clangcc' to use.
 
         self.config.clang = self.use_llvm_tool(
             'clang', search_env='CLANG', required=required)
 
-        self.config.substitutions.append(
-            ('%llvmshlibdir', self.config.llvm_shlib_dir))
-        self.config.substitutions.append(
-            ('%pluginext', self.config.llvm_plugin_ext))
+        shl = getattr(self.config, 'llvm_shlib_dir', None)
+        pext = getattr(self.config, 'llvm_plugin_ext', None)
+        if shl:
+            self.config.substitutions.append(('%llvmshlibdir', shl))
+        if pext:
+            self.config.substitutions.append(('%pluginext', pext))
 
         builtin_include_dir = self.get_clang_builtin_include_dir(self.config.clang)
         tool_substitutions = [
-            ToolSubst('%clang', command=self.config.clang),
-            ToolSubst('%clang_analyze_cc1', command='%clang_cc1', extra_args=['-analyze', '%analyze']),
-            ToolSubst('%clang_cc1', command=self.config.clang, extra_args=['-cc1', '-internal-isystem', builtin_include_dir, '-nostdsysteminc']),
-            ToolSubst('%clang_cpp', command=self.config.clang, extra_args=['--driver-mode=cpp']),
-            ToolSubst('%clang_cl', command=self.config.clang, extra_args=['--driver-mode=cl']),
-            ToolSubst('%clangxx', command=self.config.clang, extra_args=['--driver-mode=g++']),
+            ToolSubst('%clang', command=self.config.clang, extra_args=additional_flags),
+            ToolSubst('%clang_analyze_cc1', command='%clang_cc1', extra_args=['-analyze', '%analyze']+additional_flags),
+            ToolSubst('%clang_cc1', command=self.config.clang, extra_args=['-cc1', '-internal-isystem', builtin_include_dir, '-nostdsysteminc']+additional_flags),
+            ToolSubst('%clang_cpp', command=self.config.clang, extra_args=['--driver-mode=cpp']+additional_flags),
+            ToolSubst('%clang_cl', command=self.config.clang, extra_args=['--driver-mode=cl']+additional_flags),
+            ToolSubst('%clangxx', command=self.config.clang, extra_args=['--driver-mode=g++']+additional_flags),
             ]
         self.add_tool_substitutions(tool_substitutions)
 
         self.config.substitutions.append(('%itanium_abi_triple',
                                           self.make_itanium_abi_triple(self.config.target_triple)))
         self.config.substitutions.append(('%ms_abi_triple',
                                           self.make_msabi_triple(self.config.target_triple)))
         self.config.substitutions.append(
             ('%resource_dir', builtin_include_dir))
 
         # The host triple might not be set, at least if we're compiling clang from
         # an already installed llvm.
         if self.config.host_triple and self.config.host_triple != '@LLVM_HOST_TRIPLE@':
             self.config.substitutions.append(('%target_itanium_abi_host_triple',
                                               '--target=%s' % self.make_itanium_abi_triple(self.config.host_triple)))
         else:
             self.config.substitutions.append(
                 ('%target_itanium_abi_host_triple', ''))
 
-        self.config.substitutions.append(
-            ('%src_include_dir', self.config.clang_src_dir + '/include'))
-
         # FIXME: Find nicer way to prohibit this.
         self.config.substitutions.append(
-            (' clang ', """*** Do not use 'clang' in tests, use '%clang'. ***"""))
+            (' clang ', """\"*** Do not use 'clang' in tests, use '%clang'. ***\""""))
         self.config.substitutions.append(
-            (' clang\+\+ ', """*** Do not use 'clang++' in tests, use '%clangxx'. ***"""))
+            (' clang\+\+ ', """\"*** Do not use 'clang++' in tests, use '%clangxx'. ***\""""))
         self.config.substitutions.append(
             (' clang-cc ',
-             """*** Do not use 'clang-cc' in tests, use '%clang_cc1'. ***"""))
+             """\"*** Do not use 'clang-cc' in tests, use '%clang_cc1'. ***\""""))
+        self.config.substitutions.append(
+            (' clang-cl ',
+             """\"*** Do not use 'clang-cl' in tests, use '%clang_cl'. ***\""""))
         self.config.substitutions.append(
             (' clang -cc1 -analyze ',
-             """*** Do not use 'clang -cc1 -analyze' in tests, use '%clang_analyze_cc1'. ***"""))
+             """\"*** Do not use 'clang -cc1 -analyze' in tests, use '%clang_analyze_cc1'. ***\""""))
         self.config.substitutions.append(
             (' clang -cc1 ',
-             """*** Do not use 'clang -cc1' in tests, use '%clang_cc1'. ***"""))
+             """\"*** Do not use 'clang -cc1' in tests, use '%clang_cc1'. ***\""""))
         self.config.substitutions.append(
             (' %clang-cc1 ',
-             """*** invalid substitution, use '%clang_cc1'. ***"""))
+             """\"*** invalid substitution, use '%clang_cc1'. ***\""""))
         self.config.substitutions.append(
             (' %clang-cpp ',
-             """*** invalid substitution, use '%clang_cpp'. ***"""))
+             """\"*** invalid substitution, use '%clang_cpp'. ***\""""))
         self.config.substitutions.append(
             (' %clang-cl ',
-             """*** invalid substitution, use '%clang_cl'. ***"""))
+             """\"*** invalid substitution, use '%clang_cl'. ***\""""))
 
-    def use_lld(self, required=True):
+    def use_lld(self, additional_tool_dirs=[], required=True):
         """Configure the test suite to be able to invoke lld.
 
         Sets up some environment variables important to lld, locates a
         just-built or installed lld, and add a set of standard
         substitutions useful to any test suite that makes use of lld.
 
         """
-        # Tweak the PATH to include the tools dir
-        tool_dirs = [self.config.llvm_tools_dir]
-        lib_dirs = [self.config.llvm_libs_dir]
-        lld_tools_dir = getattr(self.config, 'lld_tools_dir', None)
-        lld_libs_dir = getattr(self.config, 'lld_libs_dir', None)
-
-        if lld_tools_dir:
-            tool_dirs = tool_dirs + [lld_tools_dir]
-        if lld_libs_dir:
-            lib_dirs = lib_dirs + [lld_libs_dir]
 
-        self.with_environment('PATH', tool_dirs, append_path=True)
-        self.with_environment('LD_LIBRARY_PATH', lib_dirs, append_path=True)
+        # Tweak the PATH to include the tools dir and the scripts dir.
+        exe_dir_props = [self.config.name.lower() + '_tools_dir', 'lld_tools_dir', 'llvm_tools_dir']
+        paths = [getattr(self.config, pp) for pp in exe_dir_props
+                 if getattr(self.config, pp, None)]
+        paths = additional_tool_dirs + paths
+        self.with_environment('PATH', paths, append_path=True)
+
+        lib_dir_props = [self.config.name.lower() + '_libs_dir', 'lld_libs_dir', 'llvm_libs_dir']
+        paths = [getattr(self.config, pp) for pp in lib_dir_props
+                 if getattr(self.config, pp, None)]
+
+        self.with_environment('LD_LIBRARY_PATH', paths, append_path=True)
 
-        tool_patterns = ['lld', 'ld.lld', 'lld-link', 'ld64.lld', 'wasm-ld']
+        # Discover the 'clang' and 'clangcc' to use.
 
-        self.add_tool_substitutions(tool_patterns, tool_dirs)
+        ld_lld = self.use_llvm_tool('ld.lld', required=required)
+        lld_link = self.use_llvm_tool('lld-link', required=required)
+        ld64_lld = self.use_llvm_tool('ld64.lld', required=required)
+        wasm_ld = self.use_llvm_tool('wasm-ld', required=required)
+
+        was_found = ld_lld and lld_link and ld64_lld and wasm_ld
+        tool_substitutions = []
+        if ld_lld:
+            tool_substitutions.append(ToolSubst('ld.lld', command=ld_lld))
+        if lld_link:
+            tool_substitutions.append(ToolSubst('lld-link', command=lld_link))
+        if ld64_lld:
+            tool_substitutions.append(ToolSubst('ld64.lld', command=ld64_lld))
+        if wasm_ld:
+            tool_substitutions.append(ToolSubst('wasm-ld', command=wasm_ld))
+        self.add_tool_substitutions(tool_substitutions)
+        return was_found
 No newline at end of file

llvm/utils/lit/lit/llvm/subst.py

def __init__(self, key, command=None, pre=r'.-^/\<', post='-.', verbatim=False,
         explicit_path: If specified, the exact path will be used as a substitution.
         Otherwise, the tool will be searched for as if by calling which(tool)
 
         """
         self.unresolved = unresolved
         self.extra_args = extra_args
         self.key = key
         self.command = command if command is not None else FindTool(key)
+        self.was_resolved = False
         if verbatim:
             self.regex = key
             return
 
         def not_in(chars, where=''):
             if not chars:
                 return ''
             pattern_str = '|'.join(re.escape(x) for x in chars)
def resolve(self, config, search_dirs):
                 # caller treats this as a failure.
                 pass
             elif self.unresolved == 'ignore':
                 # By returning None, the caller just assumes there was no
                 # match in the first place.
                 return None
             else:
                 raise 'Unexpected value for ToolSubst.unresolved'
-
+        if command_str:
+            self.was_resolved = True
         return (self.regex, tool_pipe, command_str)