[OpenMP] Introduce kernel environment

[tianshilei1992]

Summary

This patch introduces per kernel environment. Previously, flags such as execution mode are set through global variables with name like __kernel_name_exec_mode. They are accessible on the host by reading the corresponding global variable, but not from the device. Besides, some assumptions, such as no nested parallelism, are not per kernel basis, preventing us applying per kernel optimization in the device runtime.

This is a combination and refinement of patch series D116908, D116909, and D116910.

[arsenm]

Description not clear

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

3936–3938

ConstantExpr::getAddrSpaceCast

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

3525–3532

Optimizations should be a separate change

[tianshilei1992]

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

3525–3532

That's gonna break the tests because the interfaces between compiler and runtime are changed.

[jdoerfert]

Link the old review too. Pre-commit, w/o review, the file rename and include changes.

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

321

Do we need the offset parts? I think we should only work with field numbers not offsets.

openmp/libomptarget/DeviceRTL/include/Configuration.h

19 ↗

(On Diff #492197)

Precommit.

24 ↗

(On Diff #492197)

Unused for now.

openmp/libomptarget/DeviceRTL/src/Configuration.cpp

26

Keep it for now. Makes the transition easier. Use an || below.

[tianshilei1992]

rebase, add more code, fix comments

[tianshilei1992]

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

2643

This will be removed properly later.

[jdoerfert]

openmp/libomptarget/DeviceRTL/src/Configuration.cpp

57

This looks wrong. We want either flag to allow us to remove nested parallelism handling. So
__omp_rtl_assume_no_nested_parallelism is good enough, and
!state::getKernelEnvironment().Configuration.MayUseNestedParallelism is good enough.

[tianshilei1992]

openmp/libomptarget/DeviceRTL/src/Configuration.cpp

57

Yeah, since I already updated OpenMPOpt.cpp, __omp_rtl_assume_no_nested_parallelism will not be generated.

[tianshilei1992]

rebase, fix comments, added more code.

features are finished now. time to move to tests.

[tianshilei1992]

fix bugs and add plugin support

[tianshilei1992]

rebase and fix problems

[tianshilei1992]

rebase and fix bugs

[tianshilei1992]

rebase and fix bugs

openmp/libomptarget/plugins/cuda/src/rtl.cpp

26

These changes in old plugins will be gone because the old plugins will be removed.

[tianshilei1992]

rebase

[jdoerfert]

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

3919

That is not the kernel, at least not when clang emits a debug wrapper as part of -g, I think.
We have one workaround for this problem already but we need to provide a generic way. I'll add something.

3931

We can't have it constant as a whole. We need a constant part and a non-constant part. Let's add them like that from the very beginning.
I'd suggest we have a third struct type that is references from the kernelenv. It contains for now only the Level value, but we can/will add more things. It's an independent global such that we can make the kernelenv constant.

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

212

Maybe close the namespace here.

3748

Shouldn't we always have one here?

3755

This should happen above in the initializer. We should assume no-nested-parallelism and go back on that assumption if need be.

4473

And here you need to check nested-parallelism then too.

openmp/libomptarget/DeviceRTL/src/Configuration.cpp

57

__omp_rtl_assume_no_nested_parallelism is generated by clang, the value depends on -fopenmp-assume-no-nested-parallelism

[tianshilei1992]

rebase and fix comments

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

3923

This doesn't work as expected. Need to figure it out.

[tianshilei1992]

fix the WA, rebase, and update some tests

[jdoerfert]

LG

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

3917

Typo: tt

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

596

We used to default to SPMD w/o a _exec_mode. Unsure if this will cause problems. I am inclined to allow missing _kernel_env as well and use SPMD then.

[tianshilei1992]

rebase, update tests, fix comments

[tianshilei1992]

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

596

Now we need the global variable to call target_init at runtime, so missing it sounds like a bug. I'm not sure if we would optimize the symbol out, but I kinda doubt it.

[tianshilei1992]

Pass two benchmarks: XSBench and RSBench.
The patch will break the old plugins as well. I'm wondering we might want to land it after D142820.

[jdoerfert]

openmp/libomptarget/plugins-nextgen/common/PluginInterface/PluginInterface.cpp

596

Missing used to mean, and does, that it's not a "target region" kernel. So it might be a con/de-structor or a kernel coming from elsewhere, e.g., CUDA. We should continue that behavior I think.

[tianshilei1992]

rebase and fix comments

[tianshilei1992]

update tests in LLVM

[tianshilei1992]

Only llvm/test/Transforms/OpenMP/spmdization_constant_prop.ll is left.