[OpenMP][NFC] lit: Allow setting default environment variables for test

[t-msn]

Summary

Add CHECK_OPENMP_ENV environment variable which will be passed to environment
variables for test (make check-* target). This provides a handy way to
exercise various openmp code with different settings during development.

For example, to change default barrier pattern:
$ env CHECK_OPENMP_ENV="KMP_FORKJOIN_BARRIER_PATTERN=hier,hier \

KMP_PLAIN_BARRIER_PATTERN=hier,hier \
KMP_REDUCTION_BARRIER_PATTERN=hier,hier" \
ninja check-openmp

Even with this, each test can set appropriate envionment variables if needed
as before.

Also, this commit adds missing documention about how to run tests in README.

[protze.joachim]

I think, the name of the variable is confusing. The tests triggered and influenced by your example command are not limited to libomp. Currently the LIBOMP_ prefix is mainly used for cmake variables. The make target to run the tests is called check-* rather than test-*.

My suggestion for the variable name would be CHECK_OPENMP_ENV.

[t-msn]

Thanks for clarifications, I'm ok with CHECK_OPENMP_ENV.
If anyone has a different suggestion, please let me know.

[t-msn]

Update name to use CHECK_OPENMP_ENV and also update README

I realized that check-openmp runs tools tests too, so update
tool's lit.cfg as well

[t-msn]

While testing this, I found some minor problems in hier and dist barrier. Here are the patches: D123193, D123194, D123195

[protze.joachim]

I think, in addition to manually selecting the barrier implementation, we should add additional run lines to specific tests that trigger the different barrier implementations. I added a related, OMPT-specific comment to D123193.

[t-msn]

I think, in addition to manually selecting the barrier implementation, we should add additional run lines to specific tests that trigger the different barrier implementations. I added a related, OMPT-specific comment to D123193.

I agree. The problem is I'm not so sure which tests are needed to run when changing barrier especially for dist which affects thread handling code in kmp_runtime.cpp.
At least, for OMPT part, I believe reduction is only affected tests for different barrier implementation. So, I will update or add patch as suggested.

[protze.joachim]

I checked the runtime tests and found that openmp/runtime/test/barrier/omp_barrier.c already triggers different combinations of barriers. Do the combinations cover all configurations you tried manually?
Regarding reductions, I think it would make sense to add similar run lines to openmp/runtime/test/worksharing/for/omp_for_reduction.c in order to cover different code paths in the runtime.

Probably @AndreyChurbanov has suggestions regarding tests that could be augmented to improve test coverage of the different barrier implementations.

[protze.joachim]

This patch LGTM.

[t-msn]

I checked the runtime tests and found that openmp/runtime/test/barrier/omp_barrier.c already triggers different combinations of barriers. Do the combinations cover all configurations you tried manually?

Not really.
There are currently 5 barriers in libomp: linear, tree, hyper(default), hier, dist.
With manually changing barriers with this patch, I don't see any problems with linear, tree and hyper while found some in hier and dist (incl. the problems addressed by other patches I submitted).

openmp/runtime/test/barrier/omp_barrier.c does not test linear and tree, so it may better to be added for completeness.
(Also, omp_barrier.c checks barrier with KMP_BLOCKTIME=infinit. I tried changing barrier with this option and It seems KMP_BLOCKTIME=infinit makes problem in env/omp_wait_policy.c test regardless of barrier. I will try to see the reason
EDIT: OMP_WAIT_POLICY=passive simply does not work with KMP_BLOCKTIME=infinit. there is no worry for that)

Regarding reductions, I think it would make sense to add similar run lines to openmp/runtime/test/worksharing/for/omp_for_reduction.c in order to cover different code paths in the runtime.

Ok, I will look that too.

Thanks for the review and comments.

openmp/README.rst

- Build the stubs library
 
   .. code-blocks:: console
 
     $ cmake -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DLIBOMP_LIB_TYPE=stubs ..
 
 **Footnotes**
 
 .. [*] Other names and brands may be claimed as the property of others.
+
+How to Run Tests
+================
+
+There are following check-* make targets for tests.
+
+- ``check-ompt`` (ompt tests under runtime/test/ompt)
+- ``check-ompt-multiplex`` (ompt multiplex tests under tools/multiplex/tests)
+- ``check-libarcher`` (libarcher tests under tools/archer/tests)
+- ``check-libomp`` (libomp tests under runtime/test. This includes check-ompt tests too)
+- ``check-libomptarget-*`` (libomptarget tests for specific target under libomptarget/test)
+- ``check-libomptarget`` (all check-libomptarget-* tests)
+- ``check-openmp`` (combination of all above tests excluding duplicates)
+
+For example, to run all available tests, use ``make check-openmp``.
+
+Options for Tests
+------------------
+Tests use lit framework.
+See `lit documentation <https://llvm.org/docs/CommandGuide/lit.html>`_ for lit options.
+
+**CHECK_OPENMP_ENV** = ``""``
+  Default environment variables which test process uses for ``check-openmp``
+  separated by space.  This can be used for individual targets (``check-ompt``,
+  ``check-ompt-multiplex``, ``check-libarcher``, ``check-libomp`` and
+  ``check-libomptarget-*``) too.  Note that each test still overrides
+  environment variables if needed.  For example, to change barrier pattern to be
+  used from default hyper barrier to hierarchical barrier, run:
+
+.. code-block:: console
+
+  $ CHECK_OPENMP_ENV="KMP_PLAIN_BARRIER_PATTERN=hier,hier KMP_FORKJOIN_BARRIER_PATTERN=hier,hier KMP_REDUCTION_BARRIER_PATTERN=hier,hier" make check-openmp

openmp/libomptarget/test/lit.cfg

if 'LIBOMPTARGET_NEXTGEN_PLUGINS' in os.environ:
     config.environment['LIBOMPTARGET_NEXTGEN_PLUGINS'] = os.environ['LIBOMPTARGET_NEXTGEN_PLUGINS']
 
 if 'LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS' in os.environ:
     config.environment['LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS'] = os.environ['LIBOMPTARGET_LOCK_MAPPED_HOST_BUFFERS']
 
 if 'OMP_TARGET_OFFLOAD' in os.environ:
     config.environment['OMP_TARGET_OFFLOAD'] = os.environ['OMP_TARGET_OFFLOAD']
 
+# set default environment variables for test
+if 'CHECK_OPENMP_ENV' in os.environ:
+    test_env = os.environ['CHECK_OPENMP_ENV'].split()
+    for env in test_env:
+        name = env.split('=')[0]
+        value = env.split('=')[1]
+        config.environment[name] = value
+
 def append_dynamic_library_path(name, value, sep):
     if name in config.environment:
         config.environment[name] = value + sep + config.environment[name]
     else:
         config.environment[name] = value
 
 # Evalute the environment variable which is a string boolean value.
 def evaluate_bool_env(env):

openmp/runtime/test/lit.cfg

if multiprocessing.cpu_count() > 1:
         config.available_features.add('multicpu')
 except NotImplementedError:
     pass
 
 # to run with icc INTEL_LICENSE_FILE must be set
 if 'INTEL_LICENSE_FILE' in os.environ:
     config.environment['INTEL_LICENSE_FILE'] = os.environ['INTEL_LICENSE_FILE']
 
+# set default environment variables for test
+if 'CHECK_OPENMP_ENV' in os.environ:
+    test_env = os.environ['CHECK_OPENMP_ENV'].split()
+    for env in test_env:
+        name = env.split('=')[0]
+        value = env.split('=')[1]
+        config.environment[name] = value
+
 # substitutions
 config.substitutions.append(("%libomp-compile-and-run", \
     "%libomp-compile && %libomp-run"))
 config.substitutions.append(("%libomp-irbuilder-compile-and-run", \
     "%libomp-irbuilder-compile && %libomp-run"))
 config.substitutions.append(("%libomp-c99-compile-and-run", \
     "%libomp-c99-compile && %libomp-run"))
 config.substitutions.append(("%libomp-cxx-compile-and-run", \

openmp/tools/archer/tests/lit.cfg

 
 if config.has_tsan == True:
     config.available_features.add("tsan")
 
 # to run with icc INTEL_LICENSE_FILE must be set
 if 'INTEL_LICENSE_FILE' in os.environ:
     config.environment['INTEL_LICENSE_FILE'] = os.environ['INTEL_LICENSE_FILE']
 
+# set default environment variables for test
+if 'CHECK_OPENMP_ENV' in os.environ:
+    test_env = os.environ['CHECK_OPENMP_ENV'].split()
+    for env in test_env:
+        name = env.split('=')[0]
+        value = env.split('=')[1]
+        config.environment[name] = value
+
 config.environment['ARCHER_OPTIONS'] = "report_data_leak=1"
 
 # Race Tests
 config.substitutions.append(("%libarcher-compile-and-run-race-noserial", \
     "%%libarcher-compile && env ARCHER_OPTIONS=\"ignore_serial=1 %s\" %%libarcher-run-race" \
     % config.environment['ARCHER_OPTIONS']))
 config.substitutions.append(("%libarcher-compile-and-run-race", \
     "%libarcher-compile && %libarcher-run-race"))

openmp/tools/multiplex/tests/lit.cfg

config.test_flags = " -I " + config.test_source_root + "/.."\
     " -Wl,-rpath," + config.omp_library_dir + \
     " " + config.test_openmp_flags + \
     " " + config.test_extra_flags
 
 # to run with icc INTEL_LICENSE_FILE must be set
 if 'INTEL_LICENSE_FILE' in os.environ:
     config.environment['INTEL_LICENSE_FILE'] = os.environ['INTEL_LICENSE_FILE']
 
+# set default environment variables for test
+if 'CHECK_OPENMP_ENV' in os.environ:
+    test_env = os.environ['CHECK_OPENMP_ENV'].split()
+    for env in test_env:
+        name = env.split('=')[0]
+        value = env.split('=')[1]
+        config.environment[name] = value
+
 # Allow XFAIL to work
 config.target_triple = [ ]
 for feature in config.test_compiler_features:
     config.available_features.add(feature)
 
 # Setup environment to find dynamic library at runtime
 append_dynamic_library_path(config.omp_library_dir)
 append_dynamic_library_path(config.test_obj_root+"/..")