[clang] Search runtimes build tree for openmp runtime

[JonChesterfield]

Summary

D96248 added support for finding the openmp devicertl bitcode in the clang
lib directory where it is installed, or in the LIBRARY_PATH environment.

When testing openmp from a build using ENABLE_RUNTIMES, the corresponding
bitcode may be found under runtimes/runtimes-bins. This patch adds that to
the list of places clang looks, at lower priority then the others.

[JonChesterfield]

I think the lit test suite is intended to set environment variables such that this is not necessary when running the tests.

edit: This patch resulted from some confusion on my part. Given a list test that fails, copy&paste the failing RUN line into a shell fails to find the device runtime, because the environment setup done by lit is not reflected in the printed RUN line.

That seems to be a drawback from using environment variables to tie the test cases together. We could emit LIBRARY_PATH= lines in front of RUN, but that is probably a cross-OS hazard.

[tianshilei1992]

From my perspective, this is not a good direction. No need to bother Clang driver for one case of testing. LIBRARY_PATH needs to be set correctly by lit instead of hardcoded the logic in the driver.

[JonChesterfield]

More compelling than the testing case is that openmp is presently unusable without setting ld_library_path. This is part of fixing that.

Edit: got this confused with one of the other patches. This one is only valuable if the other patches to drop library path land.

[tianshilei1992]

Unlike others, such as /usr/local/lib or /usr/local/cuda, it is not a standard place. It doesn't make too much sense to hardcode it in the driver.

[JonChesterfield]

I'm not so sure about that. With the other stuff in flight, it would let us build openmp apps from the clang build directory. That's useful for quicker build/debug cycles, as well as for lit tests that can be run by copying into a shell. At zero runtime cost when the library is found elsewhere, and trivial extra code in the driver.

[JonChesterfield]

At present libomptarget_amdgcn_bc_path and nvptx need to be a path to a file. If we relax that to accept a path to a directory in which the corresponding file is found, then we can use that argument in place of LIBRARY_PATH from the test scripts. That will let the tests run, gives developers absolute control over which deviceRTL library is linked in, stops clang from picking up the wrong file from an unrelated toolchain on LIBRARY_PATH. I consider that strictly better than the above patch. Thoughts?

[ye-luo]

It seems that this path is baked in to clang executable even after make install. I'm not convinced this is the right direction.

[JonChesterfield]

Yeah, we probably can't leave the installed program with a search path only used for testing. I'm planning to revisit this to use libomptarget_amdgcn_bc_path instead of LIBRARY_PATH for testing, then drop the LIBRARY_PATH override.

clang/lib/Driver/ToolChains/CommonArgs.cpp

for (StringRef Path : Frags)
       LibraryPaths.emplace_back(Path.trim());
   }
 
   // Add path to lib / lib64 folder.
   SmallString<256> DefaultLibPath = llvm::sys::path::parent_path(D.Dir);
   llvm::sys::path::append(DefaultLibPath, Twine("lib") + CLANG_LIBDIR_SUFFIX);
   LibraryPaths.emplace_back(DefaultLibPath.c_str());
 
+  // Add path to runtimes binary folder, used by ENABLE_RUNTIMES build
+  SmallString<256> RuntimesBinPath = llvm::sys::path::parent_path(D.Dir);
+  llvm::sys::path::append(RuntimesBinPath,
+                          "runtimes/runtimes-bins/openmp/libomptarget");
+  LibraryPaths.emplace_back(RuntimesBinPath.c_str());
+
   OptSpecifier LibomptargetBCPathOpt =
       Triple.isAMDGCN() ? options::OPT_libomptarget_amdgcn_bc_path_EQ
                         : options::OPT_libomptarget_nvptx_bc_path_EQ;
 
   StringRef ArchPrefix = Triple.isAMDGCN() ? "amdgcn" : "nvptx";
   // First check whether user specifies bc library
   if (const Arg *A = DriverArgs.getLastArg(LibomptargetBCPathOpt)) {
     std::string LibOmpTargetName(A->getValue());