[Windows] Autolink with basenames and add libdir to libpath

[rnk]

Summary

Prior to this change, for a few compiler-rt libraries such as ubsan and
the profile library, Clang would embed "-defaultlib:path/to/rt-arch.lib"
into the .drective section of every object compiled with
-finstr-profile-generate or -fsanitize=ubsan as appropriate.

These paths assume that the link step will run from the same working
directory as the compile step. There is also evidence that sometimes the
paths become absolute, such as when clang is run from a different drive
letter from the current working directory. This is fragile, and I'd like
to get away from having paths embedded in the object if possible. Long
ago it was suggested that we use this for ASan, and apparently I felt
the same way back then:
https://reviews.llvm.org/D4428#56536

This is also consistent with how all other autolinking usage works for
PS4, Mac, and Windows: they all use basenames, not paths.

To keep things working for people using the standard GCC driver
workflow, the driver now adds the resource directory to the linker
library search path when it calls the linker. This is enough to make
check-ubsan pass, and seems like a generally good thing.

Users that invoke the linker directly (most clang-cl users) will have to
add clang's resource library directory to their linker search path in
their build system. I'm not sure where I can document this. Ideally I'd
also do it in the MSBuild files, but I can't figure out where they go.
I'd like to start with this for now.

[pcc]

The idea behind embedding the path was that these objects are usually not distributed, so things usually just work.

Is there some other way that we could arrange for the linker to search the sanitizer library directory automatically? Maybe we could add the path to one of the files in llvm/tools/msbuild? I guess that wouldn't solve the problem for command-line users but maybe that's enough.

[rnk]

I'll look into that. I also noticed that check-ubsan fails. I think we should also change clang's driver to add this libpath when it invokes the linker, so that this works transparently when using the GCC-style driver.

[thakis]

I kind of like the current behavior since it's consistent with how resource dirs are passed to cc1, and how FILE expands if you give a relative work to clang, and how debug info paths look if you pass relative paths to clang.

[rnk]

I happened to notice that I felt the same way about this in 2014 that I do today:
https://reviews.llvm.org/D4428#56536
That makes me feel like I should keep pushing for this change, even though so far people don't seem enthusiastic about it. :)

I changed the driver to add the library path to the link line when the driver is used to link. This fixes the vanilla GCC-style driver usage in the ubsan test suite.

Obviously, most users of clang-cl don't use clang-cl to link, so they won't get this behavior, but in the absence of documentation, it shows how things are intended to work. Chromium already happens to put clang's library directory on the link line, so everything should work for them.

Does this seem like reasonable behavior for now? I'd modify the msbuild integration myself to make this stuff work out of the box, but I honestly don't know how to test it or change it. The other thing worth checking is the clang PGO self-host on Windows. This has the potential to break that, and the fix would be to add a linker flag in LLVM's cmake.

[thakis]

If you strongly feel that this is the right direction, go for it, you're code owner here :)

To me, this feels like a regression. This change has no benefit that I can see (at least none that anybody's asked for that I'm aware of – then again I read llvm's bugzilla less than you do), and the drawback that folks have to explicitly pass /libpath:\path\to\clang\lib\clang\$changing_version\lib\windows to the linker, which to me is a pretty poor experience. I'd prefer if we used this qualified path for all runtime libs, so that users would never have to use this libpath. (We currently use a qualified path for libprofile but not asan and the like iirc.)

[russell.gallop]

The other thing worth checking is the clang PGO self-host on Windows.
This has the potential to break that, and the fix would be to add a linker flag in LLVM's cmake.

This does indeed break PGO self-host with lld-link (applied on top of r373200):

<...>\bin\lld-link.exe /nologo utils\not\CMakeFiles\not.dir\not.cpp.obj utils\not\CMakeFiles\not.dir\__\__\resources\windows_version_resource.rc.res /out:bin\not.exe /implib:lib\not.lib /pdb:bin\not.pdb /version:0.0 /machine:x64 -fuse-ld=lld /STACK:10000000 /INCREMENTAL:NO /subsystem:console lib\LLVMSupport.lib psapi.lib shell32.lib ole32.lib uuid.lib advapi32.lib delayimp.lib -delayload:shell32.dll -delayload:ole32.dll lib\LLVMDemangle.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:bin\not.exe.manifest
lld-link: warning: ignoring unknown argument '-fuse-ld=lld'
lld-link: error: could not open 'clang_rt.profile-x86_64.lib': no such file or directory

So at the very least this change would need that path adding.

I agree with Nico that having to add a path dependent on the LLVM version sounds like a pain.

Is it possible for the compiler to embed a /libpath as well as the dependent lib? That goes back to having a path embedded, though you could override it if required so could be an improvement over things at the moment.

It sounds to me like:

...folks have to explicitly pass /libpath:\path\to\clang\lib\clang\$changing_version\lib\windows to the linker, which to me is a pretty poor experience

and

I'd like to get away from having paths embedded in the object if possible.

are hard to reconcile.

It may be possible to have the Windows installer add the path to LIB environment variable but that would rely on having run the installer, and could cause problems if you have multiple versions of LLVM around.

[pcc]

Another option would be to let the behaviour be controlled by /Brepro, so that the default behaviour would be to embed the path, while /Brepro builds would only embed the basename.

[thakis]

/Brepro seems orthogonal to me. If you only pass relative paths and -no-canonical-prefixes then the embedded path is relative and doesn't impeded determinism.

[mstorsjo]

Another slightly related thread, regarding libs from the clang resource dir and how they are specified to the linker (regarding the builtins library): https://reviews.llvm.org/D51440

[rnk]

• Fix PGO build

[rnk]

In D65543#1691819, @mstorsjo wrote:

Another slightly related thread, regarding libs from the clang resource dir and how they are specified to the linker (regarding the builtins library): https://reviews.llvm.org/D51440

I support that effort. :)

In D65543#1687974, @russell.gallop wrote:

This does indeed break PGO self-host with lld-link (applied on top of r373200):

<...>\bin\lld-link.exe /nologo utils\not\CMakeFiles\not.dir\not.cpp.obj utils\not\CMakeFiles\not.dir\__\__\resources\windows_version_resource.rc.res /out:bin\not.exe /implib:lib\not.lib /pdb:bin\not.pdb /version:0.0 /machine:x64 -fuse-ld=lld /STACK:10000000 /INCREMENTAL:NO /subsystem:console lib\LLVMSupport.lib psapi.lib shell32.lib ole32.lib uuid.lib advapi32.lib delayimp.lib -delayload:shell32.dll -delayload:ole32.dll lib\LLVMDemangle.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST /MANIFESTFILE:bin\not.exe.manifest
lld-link: warning: ignoring unknown argument '-fuse-ld=lld'
lld-link: error: could not open 'clang_rt.profile-x86_64.lib': no such file or directory

So at the very least this change would need that path adding.

I added a fragment to HandleLLVMOptions.cmake to ask clang for its resource directory and add it to the search path when compiling with instrumentation or sanitizers. I think it's not so bad, and we don't have to write down the version.

I agree with Nico that having to add a path dependent on the LLVM version sounds like a pain.

Is it possible for the compiler to embed a /libpath as well as the dependent lib? That goes back to having a path embedded, though you could override it if required so could be an improvement over things at the moment.

Nope, I tried it, got this:

t.obj : warning LNK4229: invalid directive '/libpath:foo' encountered; ignored

It sounds to me like:

...folks have to explicitly pass /libpath:\path\to\clang\lib\clang\$changing_version\lib\windows to the linker, which to me is a pretty poor experience

and

I'd like to get away from having paths embedded in the object if possible.

are hard to reconcile.

True.

It may be possible to have the Windows installer add the path to LIB environment variable but that would rely on having run the installer, and could cause problems if you have multiple versions of LLVM around.

Well, and I wouldn't want to follow Microsoft down the route of increasing dependence on the environment.

In D65543#1687338, @thakis wrote:

If you strongly feel that this is the right direction, go for it, you're code owner here :)

I think I do feel strongly about it, but I don't want to be too pushy. :)

To me, this feels like a regression. This change has no benefit that I can see (at least none that anybody's asked for that I'm aware of – then again I read llvm's bugzilla less than you do), and the drawback that folks have to explicitly pass /libpath:\path\to\clang\lib\clang\$changing_version\lib\windows to the linker, which to me is a pretty poor experience. I'd prefer if we used this qualified path for all runtime libs, so that users would never have to use this libpath. (We currently use a qualified path for libprofile but not asan and the like iirc.)

libgcc follows a similar model, FWIW, it lives in a compiler-version-dependent path. The difference just happens to be that, on this particular platform, there's no compiler driver running the link to paper over this detail for the user.

LLVM has a lot of runtime libraries these days:

• profile (pgo & coverage)
• asan, ubsan, msan, tsan, sancov
• libfuzzer
• openmp
• libc++
• parallel stl

It doesn't seem so bad to document that, if you want to use LLVM instrumentation tools, you have to add one library search path. So, speaking of documentation, we don't really have good holistic instructions for how to use clang-cl. Where is the most obvious place to put this? The main one I know about and haven't done yet is the release notes.

[rnk]

One last idea is that we could teach LLD to automatically add this directory to library search path, but then users who link with Visual C++ (not many anymore) will run into this as a corner case issue.

[rnk]

• rebase

[rnk]

I ran into this issue again, and would like to move forward with this. I'll rebase it.

[rnk]

• rebase, ping

[rnk]

I believe folks are in agreement about this direction, but I'm waiting for someone to mark the revision accepted. Let me know if that's not the case.

[hans]

I agree with the sentiment that it's annoying when clang doesn't just work out of the box. But it will still do, except for asan and profile info etc, for which I think it's reasonable to add to the documentation what the user needs to do. I think the place to do that is https://clang.llvm.org/docs/UsersManual.html#clang-cl (and a release note). It would be great if the doc includes the error message one would get when the link fails, to make it easy to find through web search.

One last idea is that we could teach LLD to automatically add this directory to library search path, but then users who link with Visual C++ (not many anymore) will run into this as a corner case issue.

I thought about that too, and I think it's a good idea. Having the linker in a toolchain know about the runtime libs seems pretty reasonable I think?

clang/docs/ReleaseNotes.rst

74

Could this be expanded to a more explain-like-I'm-five style example?

[rnk]

• Lots of documentation words

[hans]

The docs look great, thanks!

[dmajor]

After this change, PGO builds of clang on Windows can no longer find clang_rt.profile-x86_64.lib. Is there some cmake magic that could potentially make this seamless? Or is it expected that the burden is on the user to set up the paths manually during the LLVM_BUILD_INSTRUMENTED phase?

[dmajor]

In D65543#2122942, @dmajor wrote:

After this change, PGO builds of clang on Windows can no longer find clang_rt.profile-x86_64.lib. Is there some cmake magic that could potentially make this seamless? Or is it expected that the burden is on the user to set up the paths manually during the LLVM_BUILD_INSTRUMENTED phase?

Err, it looks like this commit already has exactly such cmake magic!

In that case, more specifically, the problem I'm seeing is that these flags are not making it through to a small handful of files: LLVMHello.dll, PrintFunctionNames.dll, AnnotateFunctions.dll, Attribute.dll. Let me dig in a little deeper...

[dmajor]

In D65543#2123200, @dmajor wrote:

In that case, more specifically, the problem I'm seeing is that these flags are not making it through to a small handful of files: LLVMHello.dll, PrintFunctionNames.dll, AnnotateFunctions.dll, Attribute.dll. Let me dig in a little deeper...

Patch in D82888

clang/docs/ReleaseNotes.rst

- -Wbitwise-conditional-parentheses will warn on operator precedence issues
   conditional operator (?:).
 
 Non-comprehensive list of changes in this release
 -------------------------------------------------
 
 * In both C and C++ (C17 ``6.5.6p8``, C++ ``[expr.add]``), pointer arithmetic is
   only permitted within arrays. In particular, the behavior of a program is not
   defined if it adds a non-zero offset (or in C, any offset) to a null pointer,
   or if it forms a null pointer by subtracting an integer from a non-null
   pointer, and the LLVM optimizer now uses those guarantees for transformations.
   This may lead to unintended behavior in code that performs these operations.
   The Undefined Behavior Sanitizer ``-fsanitize=pointer-overflow`` check has
   been extended to detect these cases, so that code relying on them can be
   detected and fixed.
 
 * The Implicit Conversion Sanitizer (``-fsanitize=implicit-conversion``) has
   learned to sanitize pre/post increment/decrement of types with bit width
   smaller than ``int``.
 
 - For X86 target, -march=skylake-avx512, -march=icelake-client,
   -march=icelake-server, -march=cascadelake, -march=cooperlake will default to
   not using 512-bit zmm registers in vectorized code unless 512-bit intrinsics
   are used in the source code. 512-bit operations are known to cause the CPUs
   to run at a lower frequency which can impact performance. This behavior can be
   changed by passing -mprefer-vector-width=512 on the command line.
+- Users of clang-cl now need to add clang's runtime library path to library
+  search path when using UBSan, PGO, or coverage on Windows. Previously,
+  clang-cl embedded paths to these libraries in object files, but now the
+  compiler no longer embeds a full path.
 
 New Compiler Flags
 ------------------
 
 - The -fgnuc-version= flag now controls the value of ``__GNUC__`` and related
   macros. This flag does not enable or disable any GCC extensions implemented in
   Clang. Setting the version to zero causes Clang to leave ``__GNUC__`` and
   other GNU-namespaced macros, such as ``__GXX_WEAK__``, undefined.

clang/include/clang/Driver/ToolChain.h

public:
   virtual std::string getCompilerRT(const llvm::opt::ArgList &Args,
                                     StringRef Component,
                                     FileType Type = ToolChain::FT_Static) const;
 
   const char *
   getCompilerRTArgString(const llvm::opt::ArgList &Args, StringRef Component,
                          FileType Type = ToolChain::FT_Static) const;
 
+  std::string getCompilerRTBasename(const llvm::opt::ArgList &Args,
+                                    StringRef Component,
+                                    FileType Type = ToolChain::FT_Static,
+                                    bool AddArch = true) const;
+
   // Returns target specific runtime path if it exists.
   virtual Optional<std::string> getRuntimePath() const;
 
   // Returns target specific C++ library path if it exists.
   virtual Optional<std::string> getCXXStdlibPath() const;
 
   // Returns <ResourceDir>/lib/<OSName>/<arch>.  This is used by runtimes (such
   // as OpenMP) to find arch-specific libraries.

clang/lib/Driver/SanitizerArgs.cpp

void SanitizerArgs::addArgs(const ToolChain &TC, const llvm::opt::ArgList &Args,
   for (auto F : CoverageFlags) {
     if (CoverageFeatures & F.first)
       CmdArgs.push_back(F.second);
   }
 
   if (TC.getTriple().isOSWindows() && needsUbsanRt()) {
     // Instruct the code generator to embed linker directives in the object file
     // that cause the required runtime libraries to be linked.
-    CmdArgs.push_back(Args.MakeArgString(
-        "--dependent-lib=" + TC.getCompilerRT(Args, "ubsan_standalone")));
+    CmdArgs.push_back(
+        Args.MakeArgString("--dependent-lib=" +
+                           TC.getCompilerRTBasename(Args, "ubsan_standalone")));
     if (types::isCXX(InputType))
       CmdArgs.push_back(Args.MakeArgString(
-          "--dependent-lib=" + TC.getCompilerRT(Args, "ubsan_standalone_cxx")));
+          "--dependent-lib=" +
+          TC.getCompilerRTBasename(Args, "ubsan_standalone_cxx")));
   }
   if (TC.getTriple().isOSWindows() && needsStatsRt()) {
-    CmdArgs.push_back(Args.MakeArgString("--dependent-lib=" +
-                                         TC.getCompilerRT(Args, "stats_client")));
+    CmdArgs.push_back(Args.MakeArgString(
+        "--dependent-lib=" + TC.getCompilerRTBasename(Args, "stats_client")));
 
     // The main executable must export the stats runtime.
     // FIXME: Only exporting from the main executable (e.g. based on whether the
     // translation unit defines main()) would save a little space, but having
     // multiple copies of the runtime shouldn't hurt.
-    CmdArgs.push_back(Args.MakeArgString("--dependent-lib=" +
-                                         TC.getCompilerRT(Args, "stats")));
+    CmdArgs.push_back(Args.MakeArgString(
+        "--dependent-lib=" + TC.getCompilerRTBasename(Args, "stats")));
     addIncludeLinkerOption(TC, Args, CmdArgs, "__sanitizer_stats_register");
   }
 
   if (Sanitizers.empty())
     return;
   CmdArgs.push_back(Args.MakeArgString("-fsanitize=" + toString(Sanitizers)));
 
   if (!RecoverableSanitizers.empty())

clang/lib/Driver/ToolChain.cpp

std::string ToolChain::getCompilerRTPath() const {
   if (Triple.isOSUnknown()) {
     llvm::sys::path::append(Path, "lib");
   } else {
     llvm::sys::path::append(Path, "lib", getOSLibName());
   }
   return Path.str();
 }
 
-std::string ToolChain::getCompilerRT(const ArgList &Args, StringRef Component,
-                                     FileType Type) const {
+std::string ToolChain::getCompilerRTBasename(const ArgList &Args,
+                                             StringRef Component, FileType Type,
+                                             bool AddArch) const {
   const llvm::Triple &TT = getTriple();
   bool IsITANMSVCWindows =
       TT.isWindowsMSVCEnvironment() || TT.isWindowsItaniumEnvironment();
 
   const char *Prefix =
       IsITANMSVCWindows || Type == ToolChain::FT_Object ? "" : "lib";
   const char *Suffix;
   switch (Type) {
   case ToolChain::FT_Object:
     Suffix = IsITANMSVCWindows ? ".obj" : ".o";
     break;
   case ToolChain::FT_Static:
     Suffix = IsITANMSVCWindows ? ".lib" : ".a";
     break;
   case ToolChain::FT_Shared:
     Suffix = Triple.isOSWindows()
                  ? (Triple.isWindowsGNUEnvironment() ? ".dll.a" : ".lib")
                  : ".so";
     break;
   }
 
+  std::string ArchAndEnv;
+  if (AddArch) {
+    StringRef Arch = getArchNameForCompilerRTLib(*this, Args);
+    const char *Env = TT.isAndroid() ? "-android" : "";
+    ArchAndEnv = ("-" + Arch + Env).str();
+  }
+  return (Prefix + Twine("clang_rt.") + Component + ArchAndEnv + Suffix).str();
+}
+
+std::string ToolChain::getCompilerRT(const ArgList &Args, StringRef Component,
+                                     FileType Type) const {
+  // Check for runtime files in the new layout without the architecture first.
+  std::string CRTBasename =
+      getCompilerRTBasename(Args, Component, Type, /*AddArch=*/false);
   for (const auto &LibPath : getLibraryPaths()) {
     SmallString<128> P(LibPath);
-    llvm::sys::path::append(P, Prefix + Twine("clang_rt.") + Component + Suffix);
+    llvm::sys::path::append(P, CRTBasename);
     if (getVFS().exists(P))
       return P.str();
   }
 
-  StringRef Arch = getArchNameForCompilerRTLib(*this, Args);
-  const char *Env = TT.isAndroid() ? "-android" : "";
+  // Fall back to the old expected compiler-rt name if the new one does not
+  // exist.
+  CRTBasename = getCompilerRTBasename(Args, Component, Type, /*AddArch=*/true);
   SmallString<128> Path(getCompilerRTPath());
-  llvm::sys::path::append(Path, Prefix + Twine("clang_rt.") + Component + "-" +
-                                    Arch + Env + Suffix);
+  llvm::sys::path::append(Path, CRTBasename);
   return Path.str();
 }
 
 const char *ToolChain::getCompilerRTArgString(const llvm::opt::ArgList &Args,
                                               StringRef Component,
                                               FileType Type) const {
   return Args.MakeArgString(getCompilerRT(Args, Component, Type));
 }

clang/lib/Driver/ToolChains/Clang.cpp

if (ProfileGenerateArg) {
     if (ProfileGenerateArg->getOption().matches(
             options::OPT_fprofile_instr_generate_EQ))
       CmdArgs.push_back(Args.MakeArgString(Twine("-fprofile-instrument-path=") +
                                            ProfileGenerateArg->getValue()));
     // The default is to use Clang Instrumentation.
     CmdArgs.push_back("-fprofile-instrument=clang");
     if (TC.getTriple().isWindowsMSVCEnvironment()) {
       // Add dependent lib for clang_rt.profile
-      CmdArgs.push_back(Args.MakeArgString("--dependent-lib=" +
-                                           TC.getCompilerRT(Args, "profile")));
+      CmdArgs.push_back(Args.MakeArgString(
+          "--dependent-lib=" + TC.getCompilerRTBasename(Args, "profile")));
     }
   }
 
   Arg *PGOGenArg = nullptr;
   if (PGOGenerateArg) {
     assert(!CSPGOGenerateArg);
     PGOGenArg = PGOGenerateArg;
     CmdArgs.push_back("-fprofile-instrument=llvm");
   }
   if (CSPGOGenerateArg) {
     assert(!PGOGenerateArg);
     PGOGenArg = CSPGOGenerateArg;
     CmdArgs.push_back("-fprofile-instrument=csllvm");
   }
   if (PGOGenArg) {
     if (TC.getTriple().isWindowsMSVCEnvironment()) {
-      CmdArgs.push_back(Args.MakeArgString("--dependent-lib=" +
-                                           TC.getCompilerRT(Args, "profile")));
+      // Add dependent lib for clang_rt.profile
+      CmdArgs.push_back(Args.MakeArgString(
+          "--dependent-lib=" + TC.getCompilerRTBasename(Args, "profile")));
     }
     if (PGOGenArg->getOption().matches(
             PGOGenerateArg ? options::OPT_fprofile_generate_EQ
                            : options::OPT_fcs_profile_generate_EQ)) {
       SmallString<128> Path(PGOGenArg->getValue());
       llvm::sys::path::append(Path, "default_%m.profraw");
       CmdArgs.push_back(
           Args.MakeArgString(Twine("-fprofile-instrument-path=") + Path));

clang/lib/Driver/ToolChains/MSVC.cpp

if (!llvm::sys::Process::GetEnv("LIB")) {
     }
 
     std::string WindowsSdkLibPath;
     if (TC.getWindowsSDKLibraryPath(WindowsSdkLibPath))
       CmdArgs.push_back(
           Args.MakeArgString(std::string("-libpath:") + WindowsSdkLibPath));
   }
 
+  // Add the compiler-rt library directories to libpath if they exist to help
+  // the linker find the various sanitizer, builtin, and profiling runtimes.
+  for (const auto &LibPath : TC.getLibraryPaths()) {
+    if (TC.getVFS().exists(LibPath))
+      CmdArgs.push_back(Args.MakeArgString("-libpath:" + LibPath));
+  }
+  auto CRTPath = TC.getCompilerRTPath();
+  if (TC.getVFS().exists(CRTPath))
+    CmdArgs.push_back(Args.MakeArgString("-libpath:" + CRTPath));
+
   if (!C.getDriver().IsCLMode() && Args.hasArg(options::OPT_L))
     for (const auto &LibPath : Args.getAllArgValues(options::OPT_L))
       CmdArgs.push_back(Args.MakeArgString("-libpath:" + LibPath));
 
   CmdArgs.push_back("-nologo");
 
   if (Args.hasArg(options::OPT_g_Group, options::OPT__SLASH_Z7,
                   options::OPT__SLASH_Zd))

clang/test/Driver/cl-options.c

 // RUN: %clang_cl /Z7 -gno-column-info -### -- %s 2>&1 | FileCheck -check-prefix=gnocolumn %s
 // gnocolumn-NOT: -dwarf-column-info
 
 // RUN: %clang_cl /Z7 -### -- %s 2>&1 | FileCheck -check-prefix=gdefcolumn %s
 // gdefcolumn-NOT: -dwarf-column-info
 
 // RUN: %clang_cl -### /FA -fprofile-instr-generate -- %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-INSTR-GENERATE %s
 // RUN: %clang_cl -### /FA -fprofile-instr-generate=/tmp/somefile.profraw -- %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-INSTR-GENERATE-FILE %s
-// CHECK-PROFILE-INSTR-GENERATE: "-fprofile-instrument=clang" "--dependent-lib={{[^"]*}}clang_rt.profile-{{[^"]*}}.lib"
+// CHECK-PROFILE-INSTR-GENERATE: "-fprofile-instrument=clang" "--dependent-lib=clang_rt.profile-{{[^"]*}}.lib"
 // CHECK-PROFILE-INSTR-GENERATE-FILE: "-fprofile-instrument-path=/tmp/somefile.profraw"
 
 // RUN: %clang_cl -### /FA -fprofile-generate -- %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-GENERATE %s
-// CHECK-PROFILE-GENERATE: "-fprofile-instrument=llvm" "--dependent-lib={{[^"]*}}clang_rt.profile-{{[^"]*}}.lib"
+// CHECK-PROFILE-GENERATE: "-fprofile-instrument=llvm" "--dependent-lib=clang_rt.profile-{{[^"]*}}.lib"
 
 // RUN: %clang_cl -### /FA -fprofile-instr-generate -fprofile-instr-use -- %s 2>&1 | FileCheck -check-prefix=CHECK-NO-MIX-GEN-USE %s
 // RUN: %clang_cl -### /FA -fprofile-instr-generate -fprofile-instr-use=file -- %s 2>&1 | FileCheck -check-prefix=CHECK-NO-MIX-GEN-USE %s
 // CHECK-NO-MIX-GEN-USE: '{{[a-z=-]*}}' not allowed with '{{[a-z=-]*}}'
 
 // RUN: %clang_cl -### /FA -fprofile-instr-use -- %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE %s
 // RUN: %clang_cl -### /FA -fprofile-instr-use=/tmp/somefile.prof -- %s 2>&1 | FileCheck -check-prefix=CHECK-PROFILE-USE-FILE %s
 // CHECK-PROFILE-USE: "-fprofile-instrument-use-path=default.profdata"

clang/test/Driver/sanitizer-ld.c

 // CHECK-CFI-STATS-DARWIN: "{{.*}}ld{{(.exe)?}}"
 // CHECK-CFI-STATS-DARWIN: "{{[^"]*}}libclang_rt.stats_client_osx.a"
 // CHECK-CFI-STATS-DARWIN: "{{[^"]*}}libclang_rt.stats_osx_dynamic.dylib"
 
 // RUN: %clang -fsanitize=cfi -fsanitize-stats %s -### -o %t.o 2>&1 \
 // RUN:     -target x86_64-pc-windows \
 // RUN:     --sysroot=%S/Inputs/basic_linux_tree \
 // RUN:   | FileCheck --check-prefix=CHECK-CFI-STATS-WIN64 %s
-// CHECK-CFI-STATS-WIN64: "--dependent-lib={{[^"]*}}clang_rt.stats_client-x86_64.lib"
-// CHECK-CFI-STATS-WIN64: "--dependent-lib={{[^"]*}}clang_rt.stats-x86_64.lib"
+// CHECK-CFI-STATS-WIN64: "--dependent-lib=clang_rt.stats_client-x86_64.lib"
+// CHECK-CFI-STATS-WIN64: "--dependent-lib=clang_rt.stats-x86_64.lib"
 // CHECK-CFI-STATS-WIN64: "--linker-option=/include:__sanitizer_stats_register"
 
 // RUN: %clang -fsanitize=cfi -fsanitize-stats %s -### -o %t.o 2>&1 \
 // RUN:     -target i686-pc-windows \
 // RUN:     --sysroot=%S/Inputs/basic_linux_tree \
 // RUN:   | FileCheck --check-prefix=CHECK-CFI-STATS-WIN32 %s
-// CHECK-CFI-STATS-WIN32: "--dependent-lib={{[^"]*}}clang_rt.stats_client-i386.lib"
-// CHECK-CFI-STATS-WIN32: "--dependent-lib={{[^"]*}}clang_rt.stats-i386.lib"
+// CHECK-CFI-STATS-WIN32: "--dependent-lib=clang_rt.stats_client-i386.lib"
+// CHECK-CFI-STATS-WIN32: "--dependent-lib=clang_rt.stats-i386.lib"
 // CHECK-CFI-STATS-WIN32: "--linker-option=/include:___sanitizer_stats_register"
 
 // RUN: %clang -no-canonical-prefixes %s -### -o %t.o 2>&1 \
 // RUN:     -target arm-linux-androideabi -fuse-ld=ld -fsanitize=safe-stack \
 // RUN:     --sysroot=%S/Inputs/basic_android_tree \
 // RUN:   | FileCheck --check-prefix=CHECK-SAFESTACK-ANDROID-ARM %s
 //
 // CHECK-SAFESTACK-ANDROID-ARM: "{{(.*[^-.0-9A-Z_a-z])?}}ld{{(.exe)?}}"

llvm/cmake/modules/HandleLLVMOptions.cmake

else()
     if(NOT LINKER_IS_LLD_LINK)
       append("-fprofile-instr-generate=\"${LLVM_PROFILE_FILE_PATTERN}\""
         CMAKE_EXE_LINKER_FLAGS
         CMAKE_SHARED_LINKER_FLAGS)
     endif()
   endif()
 endif()
 
+# When using clang-cl with an instrumentation-based tool, add clang's library
+# resource directory to the library search path. Because cmake invokes the
+# linker directly, it isn't sufficient to pass -fsanitize=* to the linker.
+if (CLANG_CL AND (LLVM_BUILD_INSTRUMENTED OR LLVM_USE_SANITIZER))
+  execute_process(
+    COMMAND ${CMAKE_CXX_COMPILER} /clang:-print-resource-dir
+    OUTPUT_VARIABLE clang_resource_dir
+    ERROR_VARIABLE clang_cl_stderr
+    OUTPUT_STRIP_TRAILING_WHITESPACE
+    ERROR_STRIP_TRAILING_WHITESPACE
+    RESULT_VARIABLE clang_cl_exit_code)
+  if (NOT "${clang_cl_exit_code}" STREQUAL "0")
+    message(FATAL_ERROR
+      "Unable to invoke clang-cl to find resource dir: ${clang_cl_stderr}")
+  endif()
+  file(TO_CMAKE_PATH "${clang_resource_dir}" clang_resource_dir)
+  append("/libpath:${clang_resource_dir}/lib/windows"
+    CMAKE_EXE_LINKER_FLAGS
+    CMAKE_SHARED_LINKER_FLAGS)
+endif()
+
 if(LLVM_PROFDATA_FILE AND EXISTS ${LLVM_PROFDATA_FILE})
   if ("${CMAKE_CXX_COMPILER_ID}" MATCHES "Clang" )
     append("-fprofile-instr-use=\"${LLVM_PROFDATA_FILE}\""
       CMAKE_CXX_FLAGS
       CMAKE_C_FLAGS)
     if(NOT LINKER_IS_LLD_LINK)
       append("-fprofile-instr-use=\"${LLVM_PROFDATA_FILE}\""
         CMAKE_EXE_LINKER_FLAGS