[lit] Fix testing of standalone clang and lld builds

[jhenderson]

Summary

In such cases, the executables are not in the llvm_tools_dir directory, so we need to look in the other search locations. Previously, they were found via the PATH, but this was disabled by default in commit rGa1e6565.

Depends on D103154.

[teemperor]

Depends on D101354.

Typo?

[jhenderson]

In D103156#2781804, @teemperor wrote:

Depends on D101354.

Typo?

Thanks. Fixed.

[tstellar]

D103154 seems to be making a lot of the same changes as this patch.

[jhenderson]

In D103156#2783447, @tstellar wrote:

D103154 seems to be making a lot of the same changes as this patch.

Sorry, I stuffed up the diff in that patch - it shouldn't include the config.py changes. Will fix.

[jhenderson]

Remove unrelated parts of diff.

[jhenderson]

Upload correct patch.

[thopre]

LGTM

[jhenderson]

Rebase. I'll try landing this once the pre-merge checks complete, if there are no objections.

FYI: I'm off next week, so if this doesn't fix things, and the issue is an issue that needs resolving faster, I'd recommend reverting the original change.

[kwk]

This comment was removed by kwk.

[kwk]

I've applied this patch but for me clang still fails to build. I've build python-lit and clang in revision 1546c52d971292ed4145b6d41aaca0d02229ebff. And applied this patch on top.

[340/341] cd /builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu && /usr/bin/python3.9 /usr/bin/lit -sv --param USE_Z3_SOLVER=0 /builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/tools/extra/test /builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/tools/extra/clangd/test/../unittests /builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/tools/extra/clangd/test /builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/test
lit: /usr/lib/python3.9/site-packages/lit/llvm/config.py:431: fatal: couldn't find 'clang' program, try setting CLANG in your environment
FAILED: CMakeFiles/check-all

These are the clang files in my mock environment:

$ find / -name "clang"
/builddir/build/BUILDROOT/clang-13.0.0~pre20210527.g1546c52d971292-2.fc35.x86_64/usr/lib64/cmake/clang
/builddir/build/BUILDROOT/clang-13.0.0~pre20210527.g1546c52d971292-2.fc35.x86_64/usr/lib64/clang
/builddir/build/BUILDROOT/clang-13.0.0~pre20210527.g1546c52d971292-2.fc35.x86_64/usr/lib/python3.9/site-packages/clang
/builddir/build/BUILDROOT/clang-13.0.0~pre20210527.g1546c52d971292-2.fc35.x86_64/usr/bin/clang
/builddir/build/BUILDROOT/clang-13.0.0~pre20210527.g1546c52d971292-2.fc35.x86_64/usr/include/clang
/builddir/build/BUILDROOT/clang-13.0.0~pre20210527.g1546c52d971292-2.fc35.x86_64/usr/share/doc/clang
/builddir/build/BUILDROOT/clang-13.0.0~pre20210527.g1546c52d971292-2.fc35.x86_64/usr/share/clang
/builddir/build/BUILD/clang-13.0.0.src/test/Tooling/Inputs/mock-libcxx/bin/clang
/builddir/build/BUILD/clang-13.0.0.src/test/Driver/Inputs/mips_mti_linux/lib/clang
/builddir/build/BUILD/clang-13.0.0.src/include/clang
/builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/lib64/cmake/clang
/builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/lib64/clang
/builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/cmake/modules/CMakeFiles/Export/lib64/cmake/clang
/builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/bin/clang
/builddir/build/BUILD/clang-13.0.0.src/x86_64-redhat-linux-gnu/include/clang
/builddir/build/BUILD/clang-13.0.0.src/bindings/python/clang
/builddir/build/BUILD/clang-tools-extra-13.0.0.src/test/clang-tidy/checkers/Inputs/Headers/clang
/builddir/build/BUILD/clang-tools-extra-13.0.0.src/test/clang-tidy/infrastructure/Inputs/mock-libcxx/bin/clang

[jhenderson]

Ah, I see I missed that paths is reused in use_clang, before the call to use_llvm_tool. I'll fix that now.

[jhenderson]

Use the right set of paths in use_clang. This should fix the issue. If it doesn't I'll need someone who can reproduce to dig more into why it doesn't.

[jhenderson]

@kwk, could you try again please with the latest version of the patch? The code now looks at the same paths as are added to the PATH environment variable, so it should work.

[thopre]

New change LGTM, provided @kwk confirms it works for him.

[kwk]

@jhenderson @thopre it works for me now! Thank you!

[jhenderson]

In D103156#2787777, @kwk wrote:

@jhenderson @thopre it works for me now! Thank you!

Great, thanks for letting me know. I'm not going to be at a computer to be able to land this for the next week, so someone else can on my behalf if they wish. Otherwise it will be in a bit over a week's time before I do.

llvm/utils/lit/lit/llvm/config.py

def use_default_substitutions(self):
         self.config.substitutions.append(('%python', '"%s"' % (sys.executable)))
 
         self.add_tool_substitutions(
             tool_patterns, [self.config.llvm_tools_dir])
 
         self.add_err_msg_substitutions()
 
     def use_llvm_tool(self, name, search_env=None, required=False, quiet=False,
-                      use_installed=False):
+                      search_paths=None, use_installed=False):
         """Find the executable program 'name', optionally using the specified
         environment variable as an override before searching the build directory
         and then optionally the configuration's PATH."""
         # If the override is specified in the environment, use it without
         # validation.
         tool = None
         if search_env:
             tool = self.config.environment.get(search_env)
 
         if not tool:
-            # Use the build directory version.
-            tool = lit.util.which(name, self.config.llvm_tools_dir)
+            if search_paths is None:
+                search_paths = [self.config.llvm_tools_dir]
+            # Use the specified search paths.
+            path = os.pathsep.join(search_paths)
+            tool = lit.util.which(name, path)
 
         if not tool and use_installed:
             # Otherwise look in the path, if enabled.
             tool = lit.util.which(name, self.config.environment['PATH'])
 
         if required and not tool:
             message = "couldn't find '{}' program".format(name)
             if search_env:
def use_clang(self, additional_tool_dirs=[], additional_flags=[],
         if shl:
             self.config.substitutions.append(('%llvmshlibdir', shl))
         if pext:
             self.config.substitutions.append(('%pluginext', pext))
 
         # Discover the 'clang' and 'clangcc' to use.
         self.config.clang = self.use_llvm_tool(
             'clang', search_env='CLANG', required=required,
-            use_installed=use_installed)
+            search_paths=paths, use_installed=use_installed)
         if self.config.clang:
           self.config.available_features.add('clang')
           builtin_include_dir = self.get_clang_builtin_include_dir(
               self.config.clang)
           tool_substitutions = [
               ToolSubst('%clang', command=self.config.clang,
                         extra_args=additional_flags),
               ToolSubst('%clang_analyze_cc1', command='%clang_cc1',
def use_lld(self, additional_tool_dirs=[], required=True,
                          'lld_tools_dir', 'llvm_tools_dir']
         paths = [getattr(self.config, pp) for pp in exe_dir_props
                  if getattr(self.config, pp, None)]
         paths = additional_tool_dirs + paths
         self.with_environment('PATH', paths, append_path=True)
 
         lib_dir_props = [self.config.name.lower() + '_libs_dir',
                          'lld_libs_dir', 'llvm_libs_dir']
-        paths = [getattr(self.config, pp) for pp in lib_dir_props
+        lib_paths = [getattr(self.config, pp) for pp in lib_dir_props
                      if getattr(self.config, pp, None)]
 
-        self.with_environment('LD_LIBRARY_PATH', paths, append_path=True)
+        self.with_environment('LD_LIBRARY_PATH', lib_paths, append_path=True)
 
         # Discover the LLD executables to use.
 
         ld_lld = self.use_llvm_tool('ld.lld', required=required,
+                                    search_paths=paths,
                                     use_installed=use_installed)
         lld_link = self.use_llvm_tool('lld-link', required=required,
+                                      search_paths=paths,
                                       use_installed=use_installed)
         ld64_lld = self.use_llvm_tool('ld64.lld', required=required,
+                                      search_paths=paths,
                                       use_installed=use_installed)
         wasm_ld = self.use_llvm_tool('wasm-ld', required=required,
+                                     search_paths=paths,
                                      use_installed=use_installed)
 
         was_found = ld_lld and lld_link and ld64_lld and wasm_ld
         tool_substitutions = []
         if ld_lld:
             tool_substitutions.append(ToolSubst(r'ld\.lld', command=ld_lld))
             self.config.available_features.add('ld.lld')
         if lld_link:

llvm/utils/lit/tests/Inputs/use-llvm-tool/lit.cfg

 lit.llvm.llvm_config.use_llvm_tool('case1', search_env='CASE1')
 lit.llvm.llvm_config.use_llvm_tool('case2', search_env='CASE2')
 lit.llvm.llvm_config.use_llvm_tool('case3')
 lit.llvm.llvm_config.use_llvm_tool('case4', use_installed=True)
 lit.llvm.llvm_config.use_llvm_tool('case5')
 lit.llvm.llvm_config.use_llvm_tool('case6', search_env='CASE6', use_installed=True)
 lit.llvm.llvm_config.use_llvm_tool('case7', use_installed=True)
 lit.llvm.llvm_config.use_llvm_tool('case8', use_installed=True)
+paths = [os.path.join(this_dir, 'search1'), os.path.join(this_dir, 'search2'), os.path.join(this_dir, 'search3')]
+lit.llvm.llvm_config.use_llvm_tool('case9', search_paths=paths)
+lit.llvm.llvm_config.use_llvm_tool('case10', search_paths=paths, use_installed=True)

llvm/utils/lit/tests/use-llvm-tool.py

 ##   1  |  /  |     X     |  N/S | <- Can be found via env
 ##   2  |  X  |     /     |  N/S | <- Can be found via build dir if env specified
 ##   3  | N/S |     /     |  N/S | <- Can be found via build dir
 ##   4  | N/S |     X     |   /  | <- Can be found via PATH, if requested
 ##   5  | N/S |     X     |  N/S | <- Cannot be found via PATH, if not requested
 ##   6  |  /  |     /     |   /  | <- Env is preferred over build, PATH
 ##   7  | N/S |     /     |   /  | <- Build dir is preferred over PATH
 ##   8  |  X  |     X     |   X  | <- Say nothing if cannot be found if not required
+##   9  | N/S |  override |  N/S | <- Use specified search directory, instead of default directory
+##  10  | N/S |  override |   /  | <- Use PATH if not in search directory
 
 ## Check the exact path reported for the first case, but don't bother for the
 ## others.
 # CHECK:      note: using case1: [[DIR]]{{[\\/]}}Inputs{{[\\/]}}use-llvm-tool{{[\\/]}}env-case1
 # CHECK-NEXT: note: using case2: {{.*}}build{{[\\/]}}case2
 # CHECK-NEXT: note: using case3: {{.*}}build{{[\\/]}}case3
 # CHECK-NEXT: note: using case4: {{.*}}path{{[\\/]}}case4
 # CHECK-NOT:  case5
 # CHECK-NEXT: note: using case6: {{.*}}env-case6
 # CHECK-NEXT: note: using case7: {{.*}}build{{[\\/]}}case7
 # CHECK-NOT:  case8
+# CHECK-NEXT: note: using case9: {{.*}}search2{{[\\/]}}case9
+# CHECK-NEXT: note: using case10: {{.*}}path{{[\\/]}}case10
 
 ## Test that if required is True, lit errors if the tool is not found.
 # RUN: not %{lit} %{inputs}/use-llvm-tool-required 2>&1 | \
 # RUN:   FileCheck %s --check-prefix=ERROR
 # ERROR:      note: using found: {{.*}}found
 # ERROR-NEXT: fatal: couldn't find 'not-found' program