[OPENMP] Adds <install_path>/lib to rpath to avoid need to set LD_LIBRARY_PATH to find plugins.

[gregrodgers]

Summary

Currently, The user of an OpenMP application needs to set LD_LIBRARY_PATH to the directory
that contains the offload plugins. This change adds the rpath for OpenMP applications.

[jhuber6]

Using -rpath by default with OpenMP was removed after a long conversation in https://reviews.llvm.org/D143306. The way forward is most likely to have AOMP provide this in a resource file configuration. I think @ronlieb has a working version of that.

[JonChesterfield]

Sorry Greg, we can't have this back in trunk. It's now value add for the ROCm toolchain, and some llvm releases had this feature, but no longer.

The consensus in the OpenMP weekly meeting was to require users to hack around with linker flags or environment variables to get a working program. This makes Fedora's packaging scripts happy.

I do not like this but was comprehensively outvoted.

[gregrodgers]

Its ugly, but to avoid requirement to set LD_LIBRARY_PATH for end-users who may need LD_LIBRARY_PATH for their own application, we will modify the compiler installation with these bash commands where _INSTALL_DIR is the installation directory.

echo "-Wl,-rpath=${_INSTALL_DIR}/lib" > ${_INSTALL_DIR}/bin/rpath.cfg
echo "-L${_INSTALL_DIR}/lib" >> ${_INSTALL_DIR}/bin/rpath.cfg
ln -sf rpath.cfg ${_INSTALL_DIR}/bin/clang++.cfg
ln -sf rpath.cfg ${_INSTALL_DIR}/bin/clang.cfg
ln -sf rpath.cfg ${_INSTALL_DIR}/bin/flang.cfg
ln -sf rpath.cfg ${_INSTALL_DIR}/bin/flang-new.cfg

My apologies to linux distro packaging teams.

clang/lib/Driver/ToolChains/CommonArgs.cpp

 //===--- CommonArgs.cpp - Args handling for multiple toolchains -*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
void tools::addOpenMPRuntimeLibraryPath(const ToolChain &TC,
   llvm::sys::path::append(DefaultLibPath, CLANG_INSTALL_LIBDIR_BASENAME);
   CmdArgs.push_back(Args.MakeArgString("-L" + DefaultLibPath));
 }
 
 void tools::addArchSpecificRPath(const ToolChain &TC, const ArgList &Args,
                                  ArgStringList &CmdArgs) {
   // Enable -frtlib-add-rpath by default for the case of VE.
   const bool IsVE = TC.getTriple().isVE();
-  bool DefaultValue = IsVE;
+  const bool IsOpenMP = (TC.getDriver().getOpenMPRuntime(Args)
+      != Driver::OMPRT_Unknown);
+  bool DefaultValue = IsVE || IsOpenMP ;
   if (!Args.hasFlag(options::OPT_frtlib_add_rpath,
                     options::OPT_fno_rtlib_add_rpath, DefaultValue))
     return;
 
   std::string CandidateRPath = TC.getArchSpecificLibPath();
   if (TC.getVFS().exists(CandidateRPath)) {
     CmdArgs.push_back("-rpath");
     CmdArgs.push_back(Args.MakeArgString(CandidateRPath));
+  } else {
+    if (IsOpenMP) {
+      SmallString<256> TopLibPath =
+        llvm::sys::path::parent_path(TC.getDriver().Dir);
+      llvm::sys::path::append(TopLibPath, "lib");
+      if (TC.getVFS().exists(TopLibPath)) {
+        CmdArgs.push_back("-rpath");
+        CmdArgs.push_back(Args.MakeArgString(TopLibPath));
+      }
+    }
   }
 }
 
 bool tools::addOpenMPRuntime(ArgStringList &CmdArgs, const ToolChain &TC,
                              const ArgList &Args, bool ForceStaticHostRuntime,
                              bool IsOffloadingHost, bool GompNeedsRT) {
   if (!Args.hasFlag(options::OPT_fopenmp, options::OPT_fopenmp_EQ,
                     options::OPT_fno_openmp, false))