[lld] [test] Use LLD-specific binary&library dirs when building stand-alone

[mgorny]

Summary

Use both LLD- and LLVM-specific binary&library directories when LLD is
being built stand-alone. This ensures that the freshly built tools and
libraries are found and used correctly.

Without this patch, the test suite uses LLVM_TOOLS_DIR and LLVM_LIBS_DIR
to locate lld, and set PATH and LD_LIBRARY_PATH. When doing
a stand-alone builds, these variables represent the installed LLVM.
As a result, tests either fail due to missing lld executables/libraries
or use an earlier installed LLD version rather than the one being built.

To solve this, an additional LLD_TOOLS_DIR and LLD_LIBS_DIR variables
are added that are populated using LLVM_*_OUTPUT_INTDIR. Those variables
are populated with directories used to output built executables
and libraries. In stand-alone builds, they represent the directories
used by LLD. In integrated builds, they have the same values as
LLVM_*_DIR and therefore using them does not harm.

The new variables are prepended to PATH and LD_LIBRARY_PATH to ensure
that freshly built binaries are preferred over potentially earlier
installed ones. Furthermore, the resulting PATH is used to locate tools
for substitutions since the search includes both tools built as part of
LLD and of LLVM.

[mgorny]

A gentle ping.

[rnk]

lgtm

test/lit.cfg

183 ↗

(On Diff #86450)

I guess there is a risk here that if llvm_tools_dir is missing an LLVM tool, we might accidentally pick up an old verison from PATH, but... I don't think that's a big deal.

lld/trunk/test/lit.cfg

if lld_obj_root is not None:
     config.test_exec_root = os.path.join(lld_obj_root, 'test')
 
 # Set llvm_{src,obj}_root for use by others.
 config.llvm_src_root = getattr(config, 'llvm_src_root', None)
 config.llvm_obj_root = getattr(config, 'llvm_obj_root', None)
 
 # Tweak the PATH to include the tools dir and the scripts dir.
 if lld_obj_root is not None:
+    lld_tools_dir = getattr(config, 'lld_tools_dir', None)
+    if not lld_tools_dir:
+        lit_config.fatal('No LLD tools dir set!')
     llvm_tools_dir = getattr(config, 'llvm_tools_dir', None)
     if not llvm_tools_dir:
         lit_config.fatal('No LLVM tools dir set!')
-    path = os.path.pathsep.join((llvm_tools_dir, config.environment['PATH']))
+    path = os.path.pathsep.join((lld_tools_dir, llvm_tools_dir, config.environment['PATH']))
     path = os.path.pathsep.join((os.path.join(getattr(config, 'llvm_src_root', None),'test','Scripts'),path))
 
     config.environment['PATH'] = path
 
+    lld_libs_dir = getattr(config, 'lld_libs_dir', None)
+    if not lld_libs_dir:
+        lit_config.fatal('No LLD libs dir set!')
     llvm_libs_dir = getattr(config, 'llvm_libs_dir', None)
     if not llvm_libs_dir:
         lit_config.fatal('No LLVM libs dir set!')
-    path = os.path.pathsep.join((llvm_libs_dir,
+    path = os.path.pathsep.join((lld_libs_dir, llvm_libs_dir,
                                  config.environment.get('LD_LIBRARY_PATH','')))
     config.environment['LD_LIBRARY_PATH'] = path
 
     # Propagate LLVM_SRC_ROOT into the environment.
     config.environment['LLVM_SRC_ROOT'] = getattr(config, 'llvm_src_root', '')
 
     # Propagate PYTHON_EXECUTABLE into the environment
     config.environment['PYTHON_EXECUTABLE'] = getattr(config, 'python_executable',
for pattern in tool_patterns:
     # Extract the tool name from the pattern.  This relies on the tool
     # name being surrounded by \b word match operators.  If the
     # pattern starts with "| ", include it in the string to be
     # substituted.
     tool_match = re.match(r"^(\\)?((\| )?)\W+b([0-9A-Za-z-_\.]+)\\b\W*$",
                           pattern)
     tool_pipe = tool_match.group(2)
     tool_name = tool_match.group(4)
-    tool_path = lit.util.which(tool_name, llvm_tools_dir)
+    tool_path = lit.util.which(tool_name, config.environment['PATH'])
     if not tool_path:
         # Warn, but still provide a substitution.
-        lit_config.note('Did not find ' + tool_name + ' in ' + llvm_tools_dir)
+        lit_config.note('Did not find ' + tool_name + ' in ' + path)
         tool_path = llvm_tools_dir + '/' + tool_name
     config.substitutions.append((pattern, tool_pipe + tool_path))
 
 # Add site-specific substitutions.
 config.substitutions.append( ('%python', config.python_executable) )
 
 ###
 

lld/trunk/test/lit.site.cfg.in

 @LIT_SITE_CFG_IN_HEADER@
 
 config.llvm_src_root = "@LLVM_SOURCE_DIR@"
 config.llvm_obj_root = "@LLVM_BINARY_DIR@"
 config.llvm_tools_dir = "@LLVM_TOOLS_DIR@"
 config.llvm_libs_dir = "@LLVM_LIBS_DIR@"
 config.lit_tools_dir = "@LLVM_LIT_TOOLS_DIR@"
 config.lld_obj_root = "@LLD_BINARY_DIR@"
+config.lld_libs_dir = "@LLVM_LIBRARY_OUTPUT_INTDIR@"
+config.lld_tools_dir = "@LLVM_RUNTIME_OUTPUT_INTDIR@"
 config.target_triple = "@TARGET_TRIPLE@"
 config.python_executable = "@PYTHON_EXECUTABLE@"
 config.have_zlib = "@HAVE_LIBZ@"
 
 # Support substitution of the tools and libs dirs with user parameters. This is
 # used when we can't determine the tool dir at configuration time.
 try:
     config.llvm_tools_dir = config.llvm_tools_dir % lit_config.params
     config.llvm_libs_dir = config.llvm_libs_dir % lit_config.params
 except KeyError as e:
     key, = e.args
     lit_config.fatal("unable to find %r parameter, use '--param=%s=VALUE'" % (key,key))
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@LLD_SOURCE_DIR@/test/lit.cfg")