[llvm-lit] copy CRT/STL DLLs into the output directory

[strega-nil]

Summary

It turns out that the STL requires you to place the DLLs into the
directory of the executable; else, msvcp140.dll will be taken from
C:\Windows\System32 (which is unsupported).

This was discovered because of the recent constexpr mutex change;
see [microsoft/STL#3824][]. Especially the fuzzer unit tests fail
completely with the mutex changes.

[microsoft/STL#3824]: https://github.com/microsoft/STL/issues/3824

[strega-nil]

llvm/utils/lit/lit/TestRunner.py

2079

This doesn't work with clang's version of lit, only with our msvc version of lit, since the output directory is shared; I'm going to need to figure out a better way to do this, that works with either.

[strega-nil]

Fix the code to work with built-in lit/main.py

The code in the first commit was designed to work with MSVC's internal
lit/msvc_main.py; the new code works with either.

[strega-nil]

oops, missed a commit (ignore)

[rnk]

I'm not that familiar with the typical compatibility requirements for DLLs in the Windows system folder, but it seems pretty disruptive to ship a DLL in the system folder, and then ship ABI incompatible changes without increasing the version number. Wouldn't this all work if the STL increased the version number of msvcp140.dll instead?

llvm/utils/lit/lit/run.py

73

This is typically quite a lot of directories, so this doesn't seem like a great solution. I looked at the DLL search path docs, and it looks like the system directory (C:\Windows\System32) precedes PATH, which is unfortunate. Is there no other way to inject a DLL search path *before* the system directory? SxS manifests seem like the most promising option, but that doesn't sound like fun either.

[strega-nil]

It turns out I misunderstood how this release process works; it is _only_ important inside our internal test harness, not inside the LLVM public test harness; I apologize for opening this non-useful PR.

llvm/utils/lit/lit/TestRunner.py

 from __future__ import absolute_import
 import errno
 import io
 import itertools
 import getopt
 import os, signal, subprocess, sys
 import re
 import stat
 import pathlib
 import platform
 import shutil
 import tempfile
 import threading
-from pathlib import Path
 
 import io
 
 try:
     from StringIO import StringIO
 except ImportError:
     from io import StringIO
 
def runOnce(execdir):
         else:
             if timeoutInfo is None:
                 status = Test.FAIL
             else:
                 status = Test.TIMEOUT
         return out, err, exitCode, timeoutInfo, status
 
     # Create the output directory if it does not already exist.
-    outputDir = os.path.dirname(tmpBase)
-    lit.util.mkdir_p(outputDir)
-
-    # On Windows, copy the required DLLs from PATH into the test directory
-    # This avoids the loader finding DLLs in C:\Windows\System32
-    if litConfig.isWindows:
-        toolsetDirectory = Path(lit.util.which("cl.exe")).parent
-        for dllToCopy in itertools.chain(\
-                toolsetDirectory.glob('msvcp*.dll'),\
-                toolsetDirectory.glob('vcruntime*.dll'),\
-                toolsetDirectory.glob('ucrtbase*.dll')):
-            shutil.copyfile(dllToCopy, os.path.join(outputDir, dllToCopy.name))
+    lit.util.mkdir_p(os.path.dirname(tmpBase))
 
     # Re-run failed tests up to test.allowed_retries times.
     execdir = os.path.dirname(test.getExecPath())
     attempts = test.allowed_retries + 1
     for i in range(attempts):
         res = runOnce(execdir)
         if isinstance(res, lit.Test.Result):
             return res
 
         out, err, exitCode, timeoutInfo, status = res
         if status != Test.FAIL:
             break
 
     # If we had to run the test more than once, count it as a flaky pass. These
     # will be printed separately in the test summary.
     if i > 0 and status == Test.PASS:
         status = Test.FLAKYPASS
 

llvm/utils/lit/lit/run.py

+import itertools
 import multiprocessing
 import os
+import pathlib
+import shutil
 import time
 
 import lit.Test
 import lit.util
 import lit.worker
 
 
 class MaxFailuresError(Exception):
def execute(self):
         self.failures = 0
 
         # Larger timeouts (one year, positive infinity) don't work on Windows.
         one_week = 7 * 24 * 60 * 60  # days * hours * minutes * seconds
         timeout = self.timeout or one_week
         deadline = time.time() + timeout
 
         try:
+            # On Windows, copy the required DLLs from PATH into the test directory
+            # This avoids the loader finding DLLs in C:\Windows\System32
+            if self.lit_config.isWindows:
+                def outputPath(path):
+                    # this handles both the unit test C:\...\TestCases\Windows\foo.cpp,
+                    # and also the regular test C:\...\X86_64WindowsDynamicConfig\Asan-x86_64-inline-Dynamic-Test.exe\16\18
+                    path = pathlib.Path(path)
+                    parent = path.parent
+                    while path.suffix == '':
+                        path, parent = parent, parent.parent
+                        assert(path != path.parent)
+                    return parent / 'Output'
+                outputDirectories = {outputPath(test.getExecPath()) for test in self.tests}
+                toolsetDirectory = pathlib.Path(lit.util.which("cl.exe")).parent
+                for outputDirectory in outputDirectories:
+                    lit.util.mkdir_p(outputDirectory)
+                    for dllToCopy in itertools.chain(\
+                            toolsetDirectory.glob('msvcp*.dll'),\
+                            toolsetDirectory.glob('vcruntime*.dll'),\
+                            toolsetDirectory.glob('ucrtbase*.dll')):
+                        shutil.copyfile(dllToCopy, outputDirectory / dllToCopy.name)
+
             self._execute(deadline)
         finally:
             skipped = lit.Test.Result(lit.Test.SKIPPED)
             for test in self.tests:
                 if test.result is None:
                     test.setResult(skipped)
 
     def _execute(self, deadline):