lit.py: Allow configs and local configs to have a setup_script entry

[jordan_rose]

Summary

If set, the contents of this field are run as a Python script before any tests within that directory are executed. This happens once per execution directory, so tests in subdirectories should take care. The execution of the setup script gets a global environment like lit.local.cfg, but with shared_output as an additional global. This is a path that can be shared by all files within a particular directory, located alongside the %t paths within the Output/ directories. It's also available in tests as %shared_output.

This is useful for families of tests that have some expensive setup, which would be bad to duplicate across tests.

Either this or D35396 will need to be rebased on top of the other.

[zturner]

Why not just have lit.setup-folder.cfg similar to how we have lit.local.cfg?

Also, what about folder-level teardown? Would it instead be better to have the setup script provide an init and teardown function that get run at the beginning an end?

utils/lit/tests/Inputs/setup-script/subdir-custom/lit.local.cfg

2

Can you make this not be a shell script? This is essentially saying "This test doesn't work on Windows" when there's no particular reason why it shouldn't

[zturner]

utils/lit/tests/Inputs/setup-script/subdir-custom/lit.local.cfg

2

Actually I take that back. This can't be a shell script, because it's saying "this feature doesn't work on Windows". So I think this has to be a Python script, and you just execfile it. If you want to run a shell script, you can still do that by having a Python script which runs a shell script. But the lit infrastructure should only deal with Python scripts.

[zturner]

Actually, I guess I'm a bit confused. What is this solving that you can't already do in a local config? Couldn't you just embed the stuff you want to execute in the local config?

[jordan_rose]

Actually, I guess I'm a bit confused. What is this solving that you can't already do in a local config? Couldn't you just embed the stuff you want to execute in the local config?

I felt uncomfortable making the local config files actually have side effects, but really I want to follow up with this by moving the setup tasks into the parallel processing as actual dependencies. This is just the first iteration of the feature. If you think that's not a good direction, though, we could go with one of the other options.

(This also allows for the inheritance of setup logic into subdirectories, but that's not a critical feature for my use case, and indeed it's likely unexpected in many cases.)

utils/lit/tests/Inputs/setup-script/subdir-custom/lit.local.cfg

2

I'm happy to make the tests use Python—good point—but I don't see why the feature wouldn't work on Windows. Windows can already run Python scripts from the command line, right?

[zturner]

utils/lit/tests/Inputs/setup-script/subdir-custom/lit.local.cfg

2

No, you need to specify the path to the python executable. You can't just run foo.py, you need to run C:\Python27\python.exe foo.py. People can make this work by modifying some environment variables or doing some other tweaking, but that won't work here because it's a feature of the shell, not the OS. And it could end up pointing to a different python executable than what they ran the test suite is. If everything is Python you can address this by using sys.executable.

That aside though, given that lit is intended to be cross-platform, it seems like it should actively encourage people to write only cross-platform tests. If you make it possible to execute shell scripts, people are going to make use of that functionality and then if they ever need to add portability down the line, it's going to be much harder.

[jordan_rose]

utils/lit/tests/Inputs/setup-script/subdir-custom/lit.local.cfg

2

Hm, I was treating the docs at https://docs.python.org/2/using/windows.html?highlight=windows#executing-scripts as saying that wasn't necessary, but you'd know better than me. The point about picking a different Python is taken, though.

That aside though, given that lit is intended to be cross-platform, it seems like it should actively encourage people to write only cross-platform tests.

This part seems silly to me. Plenty of test content isn't cross-platform, and the most common kind of lit test are RUN-based shell-like tests that can invoke anything in PATH. I don't think it's a goal of lit to keep people from writing non-portable tests.

That said, I'm fine with making this Python-only.

[jordan_rose]

Make setup_script refer to a Python script rather than an arbitrary executable, as suggested by @zturner.

[zturner]

In D38010#875411, @jordan_rose wrote:

Actually, I guess I'm a bit confused. What is this solving that you can't already do in a local config? Couldn't you just embed the stuff you want to execute in the local config?

I felt uncomfortable making the local config files actually have side effects, but really I want to follow up with this by moving the setup tasks into the parallel processing as actual dependencies. This is just the first iteration of the feature. If you think that's not a good direction, though, we could go with one of the other options.

(This also allows for the inheritance of setup logic into subdirectories, but that's not a critical feature for my use case, and indeed it's likely unexpected in many cases.)

This version is better, thanks for making it work with Python scripts.

I'm interested in getting Daniel's input, but I still find it a little strange that we have 2 features that essentially do the same thing. For example, let's imagine this setup script file contains a block of code X. Couldn't we just paste X into the lit.local.cfg and have it be functionally equivalent but with one less file for the user to deal with?

You mentioned you want to make this work with the multiprocessing pool. That worries me a little bit because different processes are going to be using the same shared output dir (which isn't created up front by the infrastructure), if I understand correctly. Like if you have A and A/B and A/C, then we will run the setup script 3 times in parallel all using the same shared output folder. Is this going to be a source of flakiness?

Still though, let's say we decide that's ok. In that case, I'm still curious how this would be functionally different from just pasting all the code into a lit.local.cfg. Does that not get run for each subdirectory?

[jordan_rose]

You mentioned you want to make this work with the multiprocessing pool. That worries me a little bit because different processes are going to be using the same shared output dir (which isn't created up front by the infrastructure), if I understand correctly. Like if you have A and A/B and A/C, then we will run the setup script 3 times in parallel all using the same shared output folder. Is this going to be a source of flakiness?

Ah, that's not what happens. In the scenario you described, the script gets run for A/Output/Shared, A/B/Output/Shared, and A/C/Output/Shared. There's no feature (currently) for having a common %shared_output across multiple test directories because I didn't want to deal with synchronization. (Something like that would mean bringing back %T, though perhaps under a different name.)

[zturner]

In D38010#875713, @jordan_rose wrote:

You mentioned you want to make this work with the multiprocessing pool. That worries me a little bit because different processes are going to be using the same shared output dir (which isn't created up front by the infrastructure), if I understand correctly. Like if you have A and A/B and A/C, then we will run the setup script 3 times in parallel all using the same shared output folder. Is this going to be a source of flakiness?

Ah, that's not what happens. In the scenario you described, the script gets run for A/Output/Shared, A/B/Output/Shared, and A/C/Output/Shared. There's no feature (currently) for having a common %shared_output across multiple test directories because I didn't want to deal with synchronization. (Something like that would mean bringing back %T, though perhaps under a different name.)

I'm just brainstorming here, but what if a local config could say something like:

# lit.local.cfg
config.use_shared_output_folder('foo')

The implementation of this function would just find the output directory using the site config (which should already be created), and create this folder under there.

We only ever run the local config once, although we pass the resulting TestingConfig object to each subtest. So there would be no need to worry about parallelization.

This really only supports the single use case of a shared output directory though, and not running arbitrarily complex scripts. But if all you need is a shared output folder, maybe this is sufficient?

[jordan_rose]

Oh, no, I definitely need an arbitrarily complex script. Swift wants to prebuild mock libraries shared by several test cases. Today we're either doing it once per test case or using a terrible hack mode that can parse source on the fly.

[zturner]

Alright then, if you need it you need it :)

Let's see if Daniel has any thoughts. I mildly prefer finding a way to keep everything in the lit.local.cfg, but if someone else feels strongly that it's the way to go, I don't think it's the end of the world.

One other idea would be something like

def setup_subdir():
  pass

config.setup_func = setup_subdir

[jordan_rose]

Yep, that's not bad either. I'm not super happy with the idea of lit.local.cfg parsing relying on running in the same process as lit itself, but maybe that ship has sailed.

[jordan_rose]

Move setup code from execute_tests_in_pool to execute_tests.

This is always what I meant to do; I messed up when rebasing at one point. As mentioned in the discussion, I'm hoping to move these into being actual multiprocessing tasks in a follow-up commit.

[zturner]

utils/lit/lit/run.py

49

Maybe move this into a function such as

lit.util.run_inline_script(config, litConfig, setup_script, 
                           {'shared_output' : shared_output})

and update TestingConfig.load_from_path to call this same function?

[jordan_rose]

@ddunbar convinced me in person that this is either overkill or doesn't go far enough: as long as there's a shared directory, either there can just be a common setup task at the start of each test that does its own ad hoc filesystem-based synchronization, or we might as well go all the way to full-on inter-test dependencies. I'm going to split out the shared directory stuff into a separate patch and go play with the idea of dependencies for a bit.

docs/CommandGuide/lit.rst

 
  **pipefail** Normally a test using a shell pipe fails if any of the commands
  on the pipe fail. If this is not desired, setting this variable to false
  makes the test fail only if the last command in the pipe fails.
 
  **available_features** A set of features that can be used in `XFAIL`,
  `REQUIRES`, and `UNSUPPORTED` directives.
 
+ **setup_script** A Python script to run before any of the tests in this suite
+ are executed, or None. May be relative to the directory containing the
+ configuration file. Caution: this script will be run for this directory and
+ *once for every subdirectory.*
+
 TEST DISCOVERY
 ~~~~~~~~~~~~~~
 
 Once test suites are located, :program:`lit` recursively traverses the source
 directory (following *test_source_root*) looking for tests.  When :program:`lit`
 enters a sub-directory, it first checks to see if a nested test suite is
 defined in that directory.  If so, it loads that test suite recursively,
 otherwise it instantiates a local test config for the directory (see

 suffixes which identify test files.
 
 PRE-DEFINED SUBSTITUTIONS
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 :program:`lit` provides various patterns that can be used with the RUN command.
 These are defined in TestRunner.py. The base set of substitutions are:
 
- ========== ==============
-  Macro      Substitution
- ========== ==============
- %s         source path (path to the file currently being run)
- %S         source dir (directory of the file currently being run)
- %p         same as %S
- %{pathsep} path separator
- %t         temporary file name unique to the test
- %T         temporary directory unique to the test
- %%         %
- ========== ==============
+ ============== ==============
+  Macro         Substitution
+ ============== ==============
+ %s             source path (path to the file currently being run)
+ %S             source dir (directory of the file currently being run)
+ %p             same as %S
+ %{pathsep}     path separator
+ %t             temporary file name unique to the test
+ %T             temporary directory unique to the test
+ %shared_output temporary directory shared by all tests in the same directory
+ %%             %
+ ============== ==============
 
 Other substitutions are provided that are variations on this base set and
 further substitution patterns can be defined by each test module. See the
 modules :ref:`local-configuration-files`.
 
 More detailed information on substitutions can be found in the
 :doc:`../TestingGuide`.
 

docs/TestingGuide.rst

 actually participate in the test besides holding the ``RUN:`` lines.
 
 .. note::
 
   Some existing tests use ``RUN: true`` in extra files instead of just
   putting the extra files in an ``Inputs/`` directory. This pattern is
   deprecated.
 
+
+Setup scripts
+-------------
+
+Some tests may have common shared setup that would be redundant to perform in
+each test. In this case you can use a local configuration file
+(``lit.local.cfg``) to provide a setup script, using the following bit of
+Python:
+
+.. code-block:: python
+
+  config.setup_script = 'Inputs/base.py'
+
+The path may be absolute or relative to the configuration file. This script
+will be run before any tests in the directory are executed, with the following
+globals defined:
+
+- ``config``: The testing configuration, like in a ``lit.local.cfg`` file
+- ``lit_config``: Shared configuration for this lit execution
+- ``shared_output``: The value of ``%shared_output`` for this directory
+
+Note that like other configuration, ``setup_script`` is inherited for tests in
+subdirectories. This means the same script may be executed more than once, with
+values of ``shared_output``.
+
+
 Fragile tests
 -------------
 
 It is easy to write a fragile test that would fail spuriously if the tool being
 tested outputs a full path to the input file.  For example, :program:`opt` by
 default outputs a ``ModuleID``:
 
 .. code-block:: console
``%t``
 
    Example: ``/home/user/llvm.build/test/MC/ELF/Output/foo_test.s.tmp``
 
 ``%T``
    Directory of ``%t``.
 
    Example: ``/home/user/llvm.build/test/MC/ELF/Output``
 
+``%shared_output``
+   Path to a directory that can be shared across all test cases in the same
+   source directory. (lit will not create this directory for you, but the
+   containing directory will exist.)
+
 ``%{pathsep}``
 
    Expands to the path separator, i.e. ``:`` (or ``;`` on Windows).
 
 ``%/s, %/S, %/t, %/T:``
 
   Act like the corresponding substitution above but replace any ``\``
   character with a ``/``. This is useful to normalize path separators.

utils/lit/lit/TestRunner.py

if normalize_slashes:
         tmpBase = tmpBase.replace('\\', '/')
 
     # We use #_MARKER_# to hide %% while we do the other substitutions.
     substitutions = []
     substitutions.extend([('%%', '#_MARKER_#')])
     substitutions.extend(test.config.substitutions)
     tmpName = tmpBase + '.tmp'
     baseName = os.path.basename(tmpBase)
-    substitutions.extend([('%s', sourcepath),
+    sharedDir = os.path.join(os.path.dirname(tmpName), 'Shared')
+    substitutions.extend([('%shared_output', sharedDir),
+                          ('%s', sourcepath),
                           ('%S', sourcedir),
                           ('%p', sourcedir),
                           ('%{pathsep}', os.pathsep),
                           ('%t', tmpName),
                           ('%basename_t', baseName),
                           ('%T', tmpDir),
                           ('#_MARKER_#', '%')])
 
     # "%/[STpst]" should be normalized.
     substitutions.extend([
+            ('%/shared_output', sharedDir.replace('\\', '/')),
             ('%/s', sourcepath.replace('\\', '/')),
             ('%/S', sourcedir.replace('\\', '/')),
             ('%/p', sourcedir.replace('\\', '/')),
             ('%/t', tmpBase.replace('\\', '/') + '.tmp'),
             ('%/T', tmpDir.replace('\\', '/')),
             ])
 
     # "%:[STpst]" are paths without colons.
     if kIsWindows:
         substitutions.extend([
+                ('%:shared_output', re.sub(r'^(.):', r'\1', sharedDir)),
                 ('%:s', re.sub(r'^(.):', r'\1', sourcepath)),
                 ('%:S', re.sub(r'^(.):', r'\1', sourcedir)),
                 ('%:p', re.sub(r'^(.):', r'\1', sourcedir)),
                 ('%:t', re.sub(r'^(.):', r'\1', tmpBase) + '.tmp'),
                 ('%:T', re.sub(r'^(.):', r'\1', tmpDir)),
                 ])
     else:
         substitutions.extend([
+                ('%:shared_output', sharedDir),
                 ('%:s', sourcepath),
                 ('%:S', sourcedir),
                 ('%:p', sourcedir),
                 ('%:t', tmpBase + '.tmp'),
                 ('%:T', tmpDir),
                 ])
     return substitutions
 

utils/lit/lit/TestingConfig.py

def load_from_path(self, path, litConfig):
             if e.args:
                 raise
         except:
             import traceback
             litConfig.fatal(
                 'unable to parse config file %r, traceback: %s' % (
                     path, traceback.format_exc()))
 
-        self.finish(litConfig)
+        self.finish(litConfig, path)
 
     def __init__(self, parent, name, suffixes, test_format,
                  environment, substitutions, unsupported,
                  test_exec_root, test_source_root, excludes,
                  available_features, pipefail, limit_to_features = [],
-                 is_early = False, parallelism_group = ""):
+                 is_early = False, parallelism_group = "", setup_script = None):
         self.parent = parent
         self.name = str(name)
         self.suffixes = set(suffixes)
         self.test_format = test_format
         self.environment = dict(environment)
         self.substitutions = list(substitutions)
         self.unsupported = unsupported
         self.test_exec_root = test_exec_root
         self.test_source_root = test_source_root
         self.excludes = set(excludes)
         self.available_features = set(available_features)
         self.pipefail = pipefail
         # This list is used by TestRunner.py to restrict running only tests that
         # require one of the features in this list if this list is non-empty.
         # Configurations can set this list to restrict the set of tests to run.
         self.limit_to_features = set(limit_to_features)
         # Whether the suite should be tested early in a given run.
         self.is_early = bool(is_early)
         self.parallelism_group = parallelism_group
+        self.setup_script = setup_script
 
-    def finish(self, litConfig):
+    def finish(self, litConfig, path):
         """finish() - Finish this config object, after loading is complete."""
 
         self.name = str(self.name)
         self.suffixes = set(self.suffixes)
         self.environment = dict(self.environment)
         self.substitutions = list(self.substitutions)
         if self.test_exec_root is not None:
             # FIXME: This should really only be suite in test suite config
             # files. Should we distinguish them?
             self.test_exec_root = str(self.test_exec_root)
         if self.test_source_root is not None:
             # FIXME: This should really only be suite in test suite config
             # files. Should we distinguish them?
             self.test_source_root = str(self.test_source_root)
         self.excludes = set(self.excludes)
+        if self.setup_script is not None:
+            self.setup_script = os.path.join(os.path.dirname(path),
+                                             self.setup_script)
 
     @property
     def root(self):
         """root attribute - The root configuration for the test suite."""
         if self.parent is None:
             return self
         else:
             return self.parent.root
 

utils/lit/lit/run.py

def __init__(self, display, provider, maxFailures):
         self.maxFailures = maxFailures or object()
         self.failedCount = 0
     def update(self, test):
         self.display.update(test)
         self.failedCount += (test.result.code == lit.Test.FAIL)
         if self.failedCount == self.maxFailures:
             self.provider.cancel()
 
+def _runSetup(test, litConfig):
+    if not test.config.setup_script:
+        raise
+    setup_script = test.config.setup_script
+
+    # FIXME: This should be kept in sync with '%shared_output' in the default
+    # substitutions.
+    shared_output = os.path.join(os.path.dirname(test.getExecPath()),
+                                 'Output', 'Shared')
+
+    # FIXME: Copied from TestingConfig.load_from_path
+    data = None
+    f = open(setup_script)
+    try:
+        data = f.read()
+    except:
+        litConfig.fatal('unable to load config file: %r' % (setup_script,))
+    f.close()
+
+    setup_script_globals = dict(globals())
+    setup_script_globals['config'] = test.config
+    setup_script_globals['lit_config'] = litConfig
+    setup_script_globals['shared_output'] = shared_output
+    setup_script_globals['__file__'] = setup_script
+    try:
+        exec(compile(data, setup_script, 'exec'), setup_script_globals, None)
+    except SystemExit:
+        e = sys.exc_info()[1]
+        # We allow normal system exit inside a config file to just
+        # return control without error.
+        if e.args:
+            raise
+    except:
+        import traceback
+        litConfig.fatal(
+            'unable to parse setup file %r, traceback: %s' % (
+                setup_script, traceback.format_exc()))
+
 class Run(object):
     """
     This class represents a concrete, configured testing run.
     """
 
     def __init__(self, lit_config, tests):
         self.lit_config = lit_config
         self.tests = tests
class Run(object):
 
     def execute_tests_in_pool(self, jobs, max_time):
         # We need to issue many wait calls, so compute the final deadline and
         # subtract time.time() from that as we go along.
         deadline = None
         if max_time:
             deadline = time.time() + max_time
 
+        # Handle any setup necessary for these tests. Set up outer directories
+        # before inner ones, but promise nothing else.
+        # FIXME: Parallelize this too!
+        seenDirs = set()
+        testsWithSetup = []
+        for test in self.tests:
+            testDir = os.path.dirname(test.getExecPath())
+            if testDir in seenDirs:
+                continue
+            seenDirs.add(testDir)
+            if not test.config.setup_script:
+                continue
+            testsWithSetup.append((test, testDir))
+        testsWithSetup.sort(key=lambda pair: pair[1].count(os.sep))
+        for test, _ in testsWithSetup:
+            _runSetup(test, self.lit_config)
+
         # Start a process pool. Copy over the data shared between all test runs.
         # FIXME: Find a way to capture the worker process stderr. If the user
         # interrupts the workers before we make it into our task callback, they
         # will each raise a KeyboardInterrupt exception and print to stderr at
         # the same time.
         pool = multiprocessing.Pool(jobs, worker_initializer,
                                     (self.lit_config,
                                      self.parallelism_semaphores))

utils/lit/tests/Inputs/setup-script/Inputs/base.py

+import os
+
+def touch(fname, times=None):
+    with open(fname, 'a'):
+        os.utime(fname, times)
+
+# mkdir -p $shared_output, from https://stackoverflow.com/a/600612
+try:
+    os.makedirs(shared_output)
+except OSError as e:
+    if exc.errno == errno.EEXIST and os.path.isdir(path):
+        pass
+    else:
+        raise
+
+touch(os.path.join(shared_output, 'base.txt'))
+with open(os.environ['SHARED_LOG_FILE'], 'a') as shared_log_file:
+    shared_log_file.write(os.path.dirname(os.path.dirname(shared_output)))
+    shared_log_file.write('\n')
+

utils/lit/tests/Inputs/setup-script/lit.cfg

+import lit.formats
+import os
+
+config.name = 'shtest-shell'
+config.suffixes = ['.txt']
+config.test_format = lit.formats.ShTest()
+config.test_source_root = None
+config.test_exec_root = None
+config.setup_script = 'Inputs/base.py'
+config.environment['SHARED_LOG_FILE'] = os.environ.get('SHARED_LOG_FILE')

utils/lit/tests/Inputs/setup-script/subdir-custom/Inputs/custom.py

+import os
+
+base = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))),
+                    'Inputs', 'base.py')
+with open(base) as base_file:
+    exec(compile(base_file.read(), base, 'exec'))
+
+touch(os.path.join(shared_output, 'custom.txt'))

utils/lit/tests/Inputs/setup-script/subdir-custom/lit.local.cfg

+config.setup_script = 'Inputs/custom.py'

utils/lit/tests/Inputs/setup-script/subdir-custom/test.txt

+RUN: test -f %shared_output/base.txt
+RUN: test -f %shared_output/custom.txt

utils/lit/tests/Inputs/setup-script/subdir-inherited/lit.local.cfg

+# No extra configuration, but we should still get the setup script run.

utils/lit/tests/Inputs/setup-script/subdir-inherited/test.txt

+RUN: test -f %shared_output/base.txt

utils/lit/tests/Inputs/setup-script/subdir-no-local/test.txt

+RUN: test -f %shared_output/base.txt

utils/lit/tests/Inputs/setup-script/subdir-no-setup/lit.local.cfg

+config.setup_script = None

utils/lit/tests/Inputs/setup-script/subdir-no-setup/test.txt

+RUN: not test -d %shared_output
 No newline at end of file

utils/lit/tests/Inputs/setup-script/test.txt

+RUN: test -f %shared_output/base.txt
 No newline at end of file

utils/lit/tests/setup-script.py

+# Test the 'setup_script' feature.
+
+# RUN: rm -rf %t %t.log.txt
+# RUN: cp -r %{inputs}/setup-script %t
+# RUN: env SHARED_LOG_FILE=%t.log.txt %{lit} -j 1 -v %t
+# RUN: FileCheck %s < %t.log.txt
+# RUN: FileCheck -check-prefix=NEGATIVE %s < %t.log.txt
+
+# RUN: rm -rf %t-shuffle %t-shuffle.log.txt
+# RUN: cp -r %{inputs}/setup-script %t-shuffle
+# RUN: env SHARED_LOG_FILE=%t-shuffle.log.txt %{lit} -j 1 -v --shuffle %t-shuffle
+# RUN: FileCheck %s < %t-shuffle.log.txt
+# RUN: FileCheck -check-prefix=NEGATIVE %s < %t-shuffle.log.txt
+
+# RUN: rm -rf %t-specific %t-specific.log.txt
+# RUN: cp -r %{inputs}/setup-script %t-specific
+# RUN: env SHARED_LOG_FILE=%t-specific.log.txt %{lit} -j 1 -v %t-specific/subdir-inherited
+# RUN: FileCheck -check-prefix=CHECK-SPECIFIC %s < %t-specific.log.txt
+# RUN: FileCheck -check-prefix=CHECK-SPECIFIC-NEGATIVE %s < %t-specific.log.txt
+
+# CHECK: setup-script.py.tmp{{(-shuffle)?}}{{$}}
+# CHECK-DAG: setup-script.py.tmp{{(-shuffle)?}}/subdir-custom{{$}}
+# CHECK-DAG: setup-script.py.tmp{{(-shuffle)?}}/subdir-inherited{{$}}
+# CHECK-DAG: setup-script.py.tmp{{(-shuffle)?}}/subdir-no-local{{$}}
+# NEGATIVE-NOT: subdir-no-setup
+
+# CHECK-SPECIFIC: setup-script.py.tmp-specific/subdir-inherited{{$}}
+# CHECK-SPECIFIC-NEGATIVE-NOT: setup-script.py.tmp-specific{{$}}
+# CHECK-SPECIFIC-NEGATIVE-NOT: subdir-custom
+# CHECK-SPECIFIC-NEGATIVE-NOT: subdir-no-local
+# CHECK-SPECIFIC-NEGATIVE-NOT: subdir-no-setup