[OpenMP] Introduce the OpenMP-IR-Builder

[jdoerfert]

Summary

This is the initial patch for the OpenMP-IR-Builder, as discussed on the
mailing list ([1] and later) and at the US Dev Meeting'19.

The design is similar to D61953 but:

• placed in llvm/lib/IR/ next to IRBuilder, for lack of a better location.
• in a non-WIP status, with proper documentation and working.
• using a OpenMPKinds.def file to manage lists of directives, runtime functions, types, ..., similar to the current Clang implementation.
• restricted to handle only (simple) barriers, to implement most #pragma omp barrier directives and most implicit barriers.
• properly hooked into Clang to be used if possible.
• compatible with the remaining code generation.

The plan is to have multiple people working on moving logic from Clang
here once the initial scaffolding (=this patch) landed.

[1] http://lists.flang-compiler.org/pipermail/flang-dev_lists.flang-compiler.org/2019-May/000197.html

[fpetrogalli]

llvm/lib/IR/OpenMPIRBuilder.cpp

140 ↗

(On Diff #227647)

const method?

168 ↗

(On Diff #227647)

const method?

[jdoerfert]

In D69785#1732951, @rogfer01 wrote:

I made a small experiment lowering a taskwait (which is even simpler than barrier in "lowering" complexity). It was pretty straightforward after all.

Nice, can you share the code? Was it based on the OpenMPIRBuilder (this one or the old one) or somehow separate?

I have a few questions just to reassure I fully understand your vision here:

Sure thing.

• the parameter Loc is currently unused but my understanding from the comment in OpenMPIRBuilder::getSrcLocStr eventually we will convert a clang::SourceLocation to a llvm::Constant* that represents the location as used by KMP, right?

It is not unused everywhere (emitOMPBarrier uses it) but you are right that some uses are missing. The idea is that it will combine information about the location in LLVM-IR (which it does already) and the location in the source. The latter is in Clang a clang::SourceLocation but I want a frontend agnostic interface that takes only the values we need. That is, we add something along the lines of StringRef FileName; unsigned LineNo, ColumnNo; to the struct and use these to create the string that goes into KMP.

• emitting a taskwait however has this

if (auto *Region = dyn_cast_or_null<CGOpenMPRegionInfo>(CGF.CapturedStmtInfo))
  Region->emitUntiedSwitch(CGF);

but my understanding is that we can apply the same scheme to these hooks as well, recursively, i.e.: progressively convert them to calls to OpenMPIRBuilder (if needed under the flag suggested above by Alexey). Does this make sense?

Yes. Eventually, the logic to create "untied switches" should move, then the logic surrounding it, ...
Though, I doubt we should start there as it will cause too much interaction with the internals of Clang.
Instead, we port task created (tied ones only), then add code that keeps track of untied tasks, then add the switches, ...

Does that makes sense?

llvm/include/llvm/IR/OpenMPIRBuilder.h

51 ↗

(On Diff #227647)

Agreed.

72 ↗

(On Diff #227647)

Fine with me.

llvm/lib/IR/OpenMPIRBuilder.cpp

24 ↗

(On Diff #227647)

Some methods could be made const (and I will) but most of them not (they modify maps and the builder).
Making them const has only little benefit but I'll add it where appropriate anyway. There is little benefit because we basically do not hand out a const OpenMPIRBuilder to anyone anyway.

29–48 ↗

(On Diff #227647)

because this way we can define attributes easily for a subset of RT declarations separate from the type definition. Arguably, we can merge everything into a big macro that defines name, type and attributes but I figured this is easier to maintain. (The cost is not important as we pay it once per function at most and the switches should be eliminated). Can be changed later if we determine attributes will become part of all runtime function declarations.

177 ↗

(On Diff #227647)

(No need to repeat such comments, I will change the type name everywhere once I update ;))

[rogfer01]

In D69785#1733233, @jdoerfert wrote:

In D69785#1732951, @rogfer01 wrote:

I made a small experiment lowering a taskwait (which is even simpler than barrier in "lowering" complexity). It was pretty straightforward after all.

Nice, can you share the code? Was it based on the OpenMPIRBuilder (this one or the old one) or somehow separate?

I posted the WIP D69828.

Kind regards,

[jdoerfert]

Split D69853 out and changed according to (most) comments

[jdoerfert]

Be consistent wrt. enum classes

[ABataev]

clang/include/clang/Driver/Options.td

1643 ↗

(On Diff #227895)

Maybe just -fopenmp-experimental?

llvm/include/llvm/IR/OpenMPIRBuilder.h

25 ↗

(On Diff #227895)

class?

29 ↗

(On Diff #227895)

Do we need a new Builder here or we can reuse the one from clang CodeGenFunction?

52 ↗

(On Diff #227895)

Do you really need to spell it as createOMPBarrier? Maybe, just createBarrier since it is already a member of OMPBuilder.

62 ↗

(On Diff #227895)

StringRef?

llvm/lib/IR/OpenMPIRBuilder.cpp

178 ↗

(On Diff #227895)

Function * instead of auto

[jdoerfert]

Minor changes according to comments

clang/include/clang/Driver/Options.td

1643 ↗

(On Diff #227895)

I would prefer the option to be self explanatory but I'm not married to the current name.

llvm/include/llvm/IR/OpenMPIRBuilder.h

29 ↗

(On Diff #227895)

If you have a "simple" way to do it, we can think about it but I am still unsure if that is actually useful. The clang (=frontend) builder is used for callbacks so user code is build with it either way. We could set up ours here differently if we wish to and I'm a little afraid we would generate some unwanted interactions.

That being said, I tried to reuse the one in clang but struggled *a long time* to make it work. The problem is that it is a templated class with Clang specific template parameters. We would need to make this a template class as well (I think) and that comes with a long tail of problems.

52 ↗

(On Diff #227895)

Do you really need to spell it as createOMPBarrier? Maybe, just createBarrier since it is already a member of OMPBuilder.

fair point. I will rename if no one objects.

[jdoerfert]

make it a class (NFC)

[ABataev]

Also, I think it would better to split LLVM part and clang part into separate patches.

llvm/include/llvm/IR/OpenMPIRBuilder.h

29 ↗

(On Diff #227895)

You can make this class a template and instantiate it with the type of the CodeGenFunction IRBuilder and pass it by reference in the constructor. But only if it really worth it.

[jdoerfert]

In D69785#1734205, @ABataev wrote:

Also, I think it would better to split LLVM part and clang part into separate patches.

What do you mean exactly and why?

llvm/include/llvm/IR/OpenMPIRBuilder.h

29 ↗

(On Diff #227895)

That doesn't work as easily because the implementation is not part of this header, so we need an extern template and we'll open up a link nightmare that I would like to avoid.

[jdoerfert]

FWIW, *I will enable the new pass in some tests before this goes in*

[ABataev]

In D69785#1734292, @jdoerfert wrote:

In D69785#1734205, @ABataev wrote:

Also, I think it would better to split LLVM part and clang part into separate patches.

What do you mean exactly and why?

I just think it would be better to split this patch into 2 parts, one for LLVM builder (without tests, probably, or just with some kind of unittests) + another one for clang-related changes. It will reduce the size of the patches and all that stuff related to keep patches smaller.

[JonChesterfield]

clang/include/clang/Driver/Options.td

1643 ↗

(On Diff #227895)

enable-openmpirbuilder?

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

Sharing constants between the compiler and the runtime is an interesting subproblem. I think the cleanest solution is the one used by libc, where the compiler generates header files containing the constants which the runtime library includes.

[jdoerfert]

clang/include/clang/Driver/Options.td

1643 ↗

(On Diff #227895)

Sure, I'll change it.

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

I'd prefer not to tackle this right now but get this done first and revisit the issue later. OK?

[Meinersbur]

clang/include/clang/Basic/LangOptions.def

215 ↗

(On Diff #227902)

Shouldn't this rather be a CODEGENOPT (CodeGenOptions.def)?

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

I don't think this is a good solution. It means that libomp cannot built built anymore without also building clang. Moreover, the values cannot be changed anyway since it would break the ABI.

I'd go the other route: The libomp defines what it's ABI is, the compiler generates code for it.

[jdoerfert]

LLVM stuff only + unit test

[ABataev]

llvm/lib/IR/OpenMPConstants.cpp

1 ↗

(On Diff #227986)

Constants?

llvm/lib/IR/OpenMPIRBuilder.cpp

120 ↗

(On Diff #227986)

auto *?

[jdoerfert]

clang/include/clang/Basic/LangOptions.def

215 ↗

(On Diff #227902)

This can/should be changed for all relevant fopenmp options above in a dedicated patch. My option mimics other that exist.

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

This patch doesn't change what we do, just where. The numbers are hard coded in clang now. Let's start a discussion on the list and if we come up with a different scheme we do it after this landed.

llvm/lib/IR/OpenMPIRBuilder.cpp

120 ↗

(On Diff #227986)

No benefit and less clear, I'll stick with the type.

[jdoerfert]

LLVM code only, added unit test and fix issues

[jdoerfert]

Improve type generation & handling, provide examples for function types and
more attributes

[lebedev.ri]

As far as i can tell the builder does not add any debug info.
Should it?

[jdoerfert]

Use source location information to build the ident string and debug info.

[jdoerfert]

In D69785#1737581, @lebedev.ri wrote:

As far as i can tell the builder does not add any debug info.
Should it?

It does now.

[jdoerfert]

Adjust RTL attributes

[jdoerfert]

Remove accidental newline change

[jdoerfert]

Do not eagerly create function declarations

[ABataev]

llvm/lib/IR/OpenMPIRBuilder.cpp

153–159 ↗

(On Diff #228364)

Do you really need this if you have a deduplication pass?

[jdoerfert]

llvm/lib/IR/OpenMPIRBuilder.cpp

153–159 ↗

(On Diff #228364)

Not once the pass is in. I'll remove this with the pass.

[JonChesterfield]

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

Revisit later sounds good.

@Meinersbur Do you know of an example of a non-llvm compiler using this libomp?

The usual order is build a compiler, then use it to build the runtime libraries, then the whole package can build other stuff. Provided the compiler doesn't need any of the runtime libraries (compiler-rt, maths libraries, libomp etc) itself the system bootstraps cleanly. Especially important when cross compiling and I suspect the gpu targets in openmp have similarly strict requirements on the first compiler.

Closely related to that, I tend to assume that the runtime libraries can be rewritten to best serve their only client - the associated compiler - so if libomp is used by out of tree compilers I'd like to know who we are at risk of breaking.

[jdoerfert]

Make attributes opt-in

[ABataev]

llvm/lib/IR/OpenMPIRBuilder.cpp

49–51 ↗

(On Diff #228754)

Do you need the default: here?

[ABataev]

llvm/include/llvm/IR/OpenMPConstants.h

57 ↗

(On Diff #228754)

Why do you use 0x7FFFFFFF as the largest value?

[jdoerfert]

llvm/include/llvm/IR/OpenMPConstants.h

57 ↗

(On Diff #228754)

Because it is a valid upper bound. We don't have all values in OMP_IDENT_FLAG but if you want to come up with a tighter bound, please feel free.

llvm/lib/IR/OpenMPIRBuilder.cpp

49–51 ↗

(On Diff #228754)

If we want attributes to be optional, then yes. If we say they are mandatory, then not.

[Meinersbur]

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

Do you know of an example of a non-llvm compiler using this libomp?

gcc (using libomp's gomp compatibility layer), icc (as libomp was initially donated by Intel).

I don't understand why it even matters if there are other compilers using libomp. Every LLVM runtime library can be built stand-alone.
With constant values being determined during compiler bootstrapping, programs built on one computer would be potentially ABI-incompatible with a runtime library on another. Think about updating your compiler-rt/libomp/libc++ on you computer causing all existing binaries on the system to crash because constants changed in the updated compiler's bootstrapping process.

The only use case I know that does this is are operating system's syscall tables. Linux's reference is unistd.h which is platform-specific and Windows generates the table during its build process. Therefore on Windows, system calls can only be done through ntdll. Even on Linux one should use the system's libc instead of directly invoking a system call.

[jdoerfert]

Add cancel_barrier functionality + test, move everything to "Frontend"

[JonChesterfield]

Uncertainty over the handling of constant data between clang and libopenmp not withstanding, I think this is good to go.

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

Thanks. GCC and ICC would presumably be happier with the magic numbers stored with openmp then (though with the move to a monorepo that's a little less persuasive).

When constants that affect the ABI change, the result won't work with existing software regardless of whether the compiler or the library contains the change. Either the new compiler builds things that don't work with the old library, or the new library doesn't work with things built by the old compiler. The two have to agree on the ABI.

At present, openmp does the moral equivalent of #include OpenMPKinds.def from clang. Moving the constants to libomp means clang will do the equivalent of #include OpenMPKinds.def from openmp. Breaking that dependency means making a new subproject that just holds/generates the constants, that both depend on, which seems more hassle than it's worth.

I'd like to generate this header as part of the clang build (though ultimately don't care that much if it's generated as part of the openmp build) because it's going to become increasingly challenging to read as non-nvptx architectures are introduced. Likewise it would be useful to generate the interface.h for deviceRTL (or equivalently a set of unit tests checking the function types) from the same source to ensure it matches and that's not economically feasible within the C preprocessor.

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

179 ↗

(On Diff #228792)

auto *

228 ↗

(On Diff #228792)

Maybe add an assert when the cancellation version is requested but the cancellation block is not set? Instead of the generating simple version of barrier.

[jdoerfert]

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

I am unsure how this conversation evolved and what you want me to do now.

I repeat what I said before:

This does neither change the constants, our usage of them, nor the fact that we have them defined in multiple places, just one of the places (now llvm, before clang) changed.

llvm/lib/Frontend/OpenMPIRBuilder.cpp

179 ↗

(On Diff #228792)

Sure.

228 ↗

(On Diff #228792)

The interface doesn't work that way as we do not know here if the cancellation was requested except if the block was set. That is basically the flag (and I expect it to continue to be that way).

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

Maybe instead of ForceSimpleBarrier add a flag EmitCancelBarrier and if it set to true, always emit cancel barrier, otherwise always emit simple barrier? And add an assertion for non-set cancellation block or even accept it as a parameter here.

Also, what if we have inner exception handling in the region? Will you handle the cleanup correctly in case of the cancelation barrier?

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

Maybe instead of ForceSimpleBarrier add a flag EmitCancelBarrier and if it set to true, always emit cancel barrier, otherwise always emit simple barrier? And add an assertion for non-set cancellation block or even accept it as a parameter here.

What is the difference in moving some of the boolean logic to the caller? Also, we have test to verify we get cancellation barriers if we need them, both unit tests and clang lit tests.

Also, what if we have inner exception handling in the region? Will you handle the cleanup correctly in case of the cancelation barrier?

I think so. Right now through the code in clang that does the set up of the cancellation block, later through callbacks but we only need that for regions where we actually go out of some scope, e.g., parallel.

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

1. I'm just thinking about future users of thus interface. It woild be good if we could provide safe interface for all the users, not only clang.
2. Exit out of the OpenMP region is not allowed. But you may have inner try...catch or just simple compound statement with local vars that require constructors/destructors. And the cancellation barrier may exit out of these regions. And you need to call all required destructors. You'd better to think about it now, not later.

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

2. [...] You'd better to think about it now, not later.

First, I do think about it now and I hope this was not an insinuation to suggest otherwise.

1. I'm just thinking about future users of thus interface. It woild be good if we could provide safe interface for all the users, not only clang.
2. Exit out of the OpenMP region is not allowed. But you may have inner try...catch or just simple compound statement with local vars that require constructors/destructors. And the cancellation barrier may exit out of these regions. And you need to call all required destructors.

Generally speaking, we shall not add features that we cannot use or test with the assumption we will use them in the future. This is suggested by the LLVM best practices. If you have specific changes in mind that are testable and better than what I suggested so far, please bring them forward. You can also bring forward suggestions on how it might look in the future but without a real use case now it is not practical to block a review based on that, given that we can change the interface once the time has come.

I said before, we will need callbacks for destructors, actual handling of cancellation blocks, and there are various other features missing right now. Nevertheless, we cannot build them into the current interface, or even try to prepare for all of them, while keeping the patches small and concise.

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

It won't work for clang, I'm afraid. You need a list of desructors here. But clang uses recursive codegen and it is very hard to walk over the call tree and gather all required destructors into a list. At least, it will require significant rework in clang frontend.
Instead of generating the branch to cancellation block in the builder, I would suggest to call a single callback function provided by the frontend, which will generate correct branch over a chain of the destructor blocks. In this case, you won't need this cancellation block at all. This is what I meant when said that you need to think about this problem right now. That current solution is not very suitable for the use in the frontend.

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

It won't work for clang,

It won't work in the future or it does not work now? If the latter, do you have a mwe to show the problem?

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

1. Both.
2. What is mwe? Sure, will simple test tomorrow.

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

both what?
A simple test is what I wanted, thx.

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

Both - it won't work now and in tbe future it is going to be very hard to adapt clang to this interface.

228 ↗

(On Diff #228792)

I mean, handling of the cleanups.

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

As an example, you can take a look at the code in clang/test/OpenMP/cancel_codegen_cleanup.cpp test. It is very simple. The simplest version of the same code will something like this:

struct Obj {
  int a;
  Obj();
  ~Obj();
};

void foo() {
      #pragma omp for
      for (int i=0; i<1000; i++) {
            if(i==100) {
                Obj obj;
                #pragma omp cancel for
            }
        }
}

The object obj won't be deleted correctly with your scheme.

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

How did you run/compare this to come to the conclusion it does not work?

I run it with the OpenMPIRBuilder for barriers enabled (D69922 + -fopenmp-enable-irbuilder) and without, here is the full diff:

-declare dso_local void @__kmpc_barrier(%struct.ident_t*, i32)
+declare void @__kmpc_barrier(%struct.ident_t*, i32)

I don't see what you mean by it doesn't work, looks fine to me.

---

The above notwithstanding, if you have examples that expose problems with this patch, please let me know.

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

Try this one:

struct Obj {
  int a;
  Obj();
  ~Obj();
};

void foo() {
      #pragma omp parallel
      for (int i=0; i<1000; i++) {
            if(i==100) {
                Obj obj;
                #pragma omp cancel parallel
                #pragma omp barrier
            }
        }
}

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

Same result, cancel semantic is unaffected. Are you trying these?

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

There must be different code for _kmpc_cancel_barrier call and further processing. Will try to check with your patch tomorrow.

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

200 ↗

(On Diff #228792)

Maybe split emission ща __kmpc_barrier and __kmpc_cancel_barrier functions into 2 independent functions fo the frontends? Rather than rely on the boolean flags?

228 ↗

(On Diff #228792)

Ok, I see, you're using the block that jumps through the cleanups. Ok, this seems good.

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

200 ↗

(On Diff #228792)

The frontend doesn't know necessarily when a barrier is issued if it is a cancellable, arguably it shouldn't need to know at all

I copied the flags from clang but I will look into removing them eventually (and to add a TODO in the meantime).

[jdoerfert]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

228 ↗

(On Diff #228792)

Yes. I also used the same logic in the generic solution (D70258) that will work with both the old code gen and the new one, e.g., D70109.

[jdoerfert]

ping

[ABataev]

llvm/lib/Frontend/OpenMPIRBuilder.cpp

152–155 ↗

(On Diff #228792)

better to use streams

169–174 ↗

(On Diff #228792)

Will it work with the late outlined parallel regions?

[jdoerfert]

Anything else?

llvm/lib/Frontend/OpenMPIRBuilder.cpp

152–155 ↗

(On Diff #228792)

Sure.

169–174 ↗

(On Diff #228792)

I'll remove the caching here. No need for it given that we will deduplicate anyway.

[JonChesterfield]

llvm/include/llvm/IR/OpenMPKinds.def

186 ↗

(On Diff #227902)

Apologies for the digression. There is no change to the status quo so considering how to improve matters later cannot be blocking.

[jdoerfert]

use streams, remove caching

[jdoerfert]

@ABataev @JonChesterfield anything else blocking this patch?

[JonChesterfield]

I'd very much like this to land soon. It's the prereq for a lot of other patches and the code looks good.

It's tricky to test the infra before the users are landed so the unit test is particularly appreciated.

[rogfer01]

llvm/include/llvm/Frontend/OpenMPKinds.def

165 ↗

(On Diff #230928)

As we migrate, we will end with a significant number of interfaces here.

@jdoerfert what do you think about adding a comment with their C prototype before each one like we do in clang/lib/CodeGen/CGOpenMPRuntime.cpp?

Something like this

// void __kmpc_barrier(ident_t *loc, kmp_int32 global_tid);
__OMP_RTL(__kmpc_barrier, false, Void, IdentPtr, Int32)
// kmp_int32 __kmpc_cancel_barrier(ident_t *loc, kmp_int32
// global_tid)
__OMP_RTL(__kmpc_cancel_barrier, false, Int32, IdentPtr, Int32)
...

[jdoerfert]

In D69785#1762438, @JonChesterfield wrote:

I'd very much like this to land soon. It's the prereq for a lot of other patches and the code looks good.

It's tricky to test the infra before the users are landed so the unit test is particularly appreciated.

I'm confused. Was this a review? I'm waiting for a decision here so we can move on and improve on this instead of me modifying it inp-lace two comments at a time.

llvm/include/llvm/Frontend/OpenMPKinds.def

165 ↗

(On Diff #230928)

I'm fine with this but I doubt it'll help much (compared to the lines we have that show name and types).

If you want this to happen you should create a patch do add comments for the ones we have here, and others can way in. If there is agreement to apply it, we will do so and continue that tradition from then on. Does that sound good?

[rogfer01]

llvm/include/llvm/Frontend/OpenMPKinds.def

165 ↗

(On Diff #230928)

Sounds reasonable to me. Thanks!

[JonChesterfield]

In D69785#1763317, @jdoerfert wrote:

I'm confused. Was this a review? I'm waiting for a decision here so we can move on and improve on this instead of me modifying it inp-lace two comments at a time.

Explicitly marked as accepted. Patch has looked good for a while and even has other people building on it.

[jdoerfert]

rebase

[merge_guards_bot]

clang-format: pass.

Build artifacts: console-log.txt, CMakeCache.txt, diff.json

[jdoerfert]

Adjust path

[ABataev]

llvm/include/llvm/Frontend/OpenMP/OMPConstants.h

74

Extra comments?

93

Here too.

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

175–178

Are you sure it will work correctly with late outlining?

187–189

Clang drops insert point after some code, like call of exit() etc. That's why we need to check it in the frontend, otherwise, the compiler crashes.

[merge_guards_bot]

Unit tests: fail. 60698 tests passed, 1 failed and 726 were skipped.

failed: LLVM-Unit.Frontend/_/LLVMFrontendTests/OpenMPIRBuilderTest.CreateCancelBarrier

clang-format: pass.

Build artifacts: console-log.txt, CMakeCache.txt, test-results.xml, diff.json

[jdoerfert]

llvm/include/llvm/Frontend/OpenMP/OMPConstants.h

74

I don't know what you want to tell me.

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

175–178

Yes.

[jdoerfert]

Remove call caching in anticipation of D69930

[ABataev]

llvm/include/llvm/Frontend/OpenMP/OMPConstants.h

74

I mean, you have this ///{ here. Do you need it?

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

175–178

Ah, the code fro threadid changed already, probably missed it.

[jdoerfert]

llvm/include/llvm/Frontend/OpenMP/OMPConstants.h

74

Yes, to help the doxygen documentation.

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

175–178

I removed the TODOs now, including the caching.

llvm/include/llvm/Frontend/OpenMP/OMPConstants.h

 //===- OMPConstants.h - OpenMP related constants and helpers ------ C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 /// \file
 ///
 /// This file defines constans and helpers used when dealing with OpenMP.
 ///
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_OPENMP_CONSTANTS_H
 #define LLVM_OPENMP_CONSTANTS_H
 
+#include "llvm/ADT/BitmaskEnum.h"
 #include "llvm/ADT/StringRef.h"
 
 namespace llvm {
+class Type;
+class Module;
+class StructType;
+class PointerType;
+class FunctionType;
 
 namespace omp {
+LLVM_ENABLE_BITMASK_ENUMS_IN_NAMESPACE();
 
 /// IDs for all OpenMP directives.
 enum class Directive {
 #define OMP_DIRECTIVE(Enum, ...) Enum,
 #include "llvm/Frontend/OpenMP/OMPKinds.def"
 };
 
 /// Make the enum values available in the llvm::omp namespace. This allows us to
 /// write something like OMPD_parallel if we have a `using namespace omp`. At
 /// the same time we do not loose the strong type guarantees of the enum class,
 /// that is we cannot pass an unsigned as Directive without an explicit cast.
 #define OMP_DIRECTIVE(Enum, ...) constexpr auto Enum = omp::Directive::Enum;
 #include "llvm/Frontend/OpenMP/OMPKinds.def"
 
+/// IDs for all omp runtime library (RTL) functions.
+enum class RuntimeFunction {
+#define OMP_RTL(Enum, ...) Enum,
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+};
+
+#define OMP_RTL(Enum, ...) constexpr auto Enum = omp::RuntimeFunction::Enum;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+
+/// IDs for all omp runtime library ident_t flag encodings (see
+/// their defintion in openmp/runtime/src/kmp.h).
+enum class IdentFlag {
+#define OMP_IDENT_FLAG(Enum, Str, Value) Enum = Value,
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+  LLVM_MARK_AS_BITMASK_ENUM(0x7FFFFFFF)
+};
+
+#define OMP_IDENT_FLAG(Enum, ...) constexpr auto Enum = omp::IdentFlag::Enum;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+
 /// Parse \p Str and return the directive it matches or OMPD_unknown if none.
 Directive getOpenMPDirectiveKind(StringRef Str);
 
 /// Return a textual representation of the directive \p D.
 StringRef getOpenMPDirectiveName(Directive D);
 
+/// Forward declarations for LLVM-IR types (simple, function and structure) are
+/// generated below. Their names are defined and used in OpenMPKinds.def. Here
+/// we provide the forward declarations, the initializeTypes function will
+/// provide the values.
+///
+///{
+namespace types {
+
+#define OMP_TYPE(VarName, InitValue) extern Type *VarName;
+#define OMP_FUNCTION_TYPE(VarName, IsVarArg, ReturnType, ...)                  \
+  extern FunctionType *VarName;                                                \
+  extern PointerType *VarName##Ptr;
+#define OMP_STRUCT_TYPE(VarName, StrName, ...)                                 \
+  extern StructType *VarName;                                                  \
+  extern PointerType *VarName##Ptr;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+
+/// Helper to initialize all types defined in OpenMPKinds.def.
+void initializeTypes(Module &M);
+
+/// Helper to uninitialize all types defined in OpenMPKinds.def.
+void uninitializeTypes();
+
+} // namespace types
+///}
+
 } // end namespace omp
 
 } // end namespace llvm
 
 #endif // LLVM_OPENMP_CONSTANTS_H

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

+//===- IR/OpenMPIRBuilder.h - OpenMP encoding builder for LLVM IR - C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines the OpenMPIRBuilder class and helpers used as a convenient
+// way to create LLVM instructions for OpenMP directives.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_OPENMP_IR_IRBUILDER_H
+#define LLVM_OPENMP_IR_IRBUILDER_H
+
+#include "llvm/IR/DebugLoc.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/Frontend/OpenMP/OMPConstants.h"
+
+namespace llvm {
+
+/// An interface to create LLVM-IR for OpenMP directives.
+///
+/// Each OpenMP directive has a corresponding public generator method.
+class OpenMPIRBuilder {
+public:
+  /// Create a new OpenMPIRBuilder operating on the given module \p M. This will
+  /// not have an effect on \p M (see initialize).
+  OpenMPIRBuilder(Module &M) : M(M), Builder(M.getContext()) {}
+
+  /// Initialize the internal state, this will put structures types and
+  /// potentially other helpers into the underlying module. Must be called
+  /// before any other method and only once!
+  void initialize();
+
+  /// Add attributes known for \p FnID to \p Fn.
+  void addAttributes(omp::RuntimeFunction FnID, Function &Fn);
+
+  /// Set the cancellation block to \p CBB.
+  void setCancellationBlock(BasicBlock *CBB) { CancellationBlock = CBB; }
+
+  /// Type used throughout for insertion points.
+  using InsertPointTy = IRBuilder<>::InsertPoint;
+
+  /// Description of a LLVM-IR insertion point (IP) and a debug/source location
+  /// (filename, line, column, ...).
+  struct LocationDescription {
+    template <typename T, typename U>
+    LocationDescription(const IRBuilder<T, U> &IRB)
+        : IP(IRB.saveIP()), DL(IRB.getCurrentDebugLocation()) {}
+    LocationDescription(const InsertPointTy &IP) : IP(IP) {}
+    LocationDescription(const InsertPointTy &IP, const DebugLoc &DL)
+        : IP(IP), DL(DL) {}
+    InsertPointTy IP;
+    DebugLoc DL;
+  };
+
+  /// Emitter methods for OpenMP directives.
+  ///
+  ///{
+
+  /// Generator for '#omp barrier'
+  ///
+  /// \param Loc The location where the barrier directive was encountered.
+  /// \param DK The kind of directive that caused the barrier.
+  /// \param ForceSimpleCall Flag to force a simple (=non-cancellation) barrier.
+  /// \param CheckCancelFlag Flag to indicate a cancel barrier return value
+  ///                        should be checked and acted upon.
+  ///
+  /// \returns The insertion point after the barrier.
+  InsertPointTy CreateBarrier(const LocationDescription &Loc, omp::Directive DK,
+                              bool ForceSimpleCall = false,
+                              bool CheckCancelFlag = true);
+
+  ///}
+
+private:
+  /// Update the internal location to \p Loc.
+  bool updateToLocation(const LocationDescription &Loc) {
+    Builder.restoreIP(Loc.IP);
+    Builder.SetCurrentDebugLocation(Loc.DL);
+    return Loc.IP.getBlock() != nullptr;
+  }
+
+  /// Return the function declaration for the runtime function with \p FnID.
+  Function *getOrCreateRuntimeFunction(omp::RuntimeFunction FnID);
+
+  /// Return the (LLVM-IR) string describing the source location \p LocStr.
+  Constant *getOrCreateSrcLocStr(StringRef LocStr);
+
+  /// Return the (LLVM-IR) string describing the default source location.
+  Constant *getOrCreateDefaultSrcLocStr();
+
+  /// Return the (LLVM-IR) string describing the source location \p Loc.
+  Constant *getOrCreateSrcLocStr(const LocationDescription &Loc);
+
+  /// Return an ident_t* encoding the source location \p SrcLocStr and \p Flags.
+  Value *getOrCreateIdent(Constant *SrcLocStr,
+                          omp::IdentFlag Flags = omp::IdentFlag(0));
+
+  /// Generate a barrier runtime call.
+  ///
+  /// \param Loc The location at which the request originated and is fulfilled.
+  /// \param DK The directive which caused the barrier
+  /// \param ForceSimpleCall Flag to force a simple (=non-cancellation) barrier.
+  /// \param CheckCancelFlag Flag to indicate a cancel barrier return value
+  ///                        should be checked and acted upon.
+  ///
+  /// \returns The insertion point after the barrier.
+  InsertPointTy emitBarrierImpl(const LocationDescription &Loc,
+                                omp::Directive DK, bool ForceSimpleCall,
+                                bool CheckCancelFlag);
+
+  /// Return the current thread ID.
+  ///
+  /// \param Ident The ident (ident_t*) describing the query origin.
+  Value *getOrCreateThreadID(Value *Ident);
+
+  /// The underlying LLVM-IR module
+  Module &M;
+
+  /// The LLVM-IR Builder used to create IR.
+  IRBuilder<> Builder;
+
+  /// TODO: Stub for a cancellation block stack.
+  BasicBlock *CancellationBlock = nullptr;
+
+  /// Map to remember source location strings
+  StringMap<Constant *> SrcLocStrMap;
+
+  /// Map to remember existing ident_t*.
+  DenseMap<std::pair<Constant *, uint64_t>, GlobalVariable *> IdentMap;
+};
+
+} // end namespace llvm
+
+#endif // LLVM_IR_IRBUILDER_H

llvm/include/llvm/Frontend/OpenMP/OMPKinds.def

 // Has to be the last because Clang implicitly expects it to be.
 __OMP_DIRECTIVE(unknown)
 
 #undef __OMP_DIRECTIVE_EXT
 #undef __OMP_DIRECTIVE
 #undef OMP_DIRECTIVE
 
 ///}
+
+/// Types used in runtime structs or runtime functions
+///
+///{
+
+#ifndef OMP_TYPE
+#define OMP_TYPE(VarName, InitValue)
+#endif
+
+#define __OMP_TYPE(VarName) OMP_TYPE(VarName, Type::get##VarName##Ty(Ctx))
+
+__OMP_TYPE(Void)
+__OMP_TYPE(Int8)
+__OMP_TYPE(Int32)
+__OMP_TYPE(Int8Ptr)
+__OMP_TYPE(Int32Ptr)
+
+#undef __OMP_TYPE
+#undef OMP_TYPE
+
+///}
+
+/// Struct and function types
+///
+///{
+
+#ifndef OMP_STRUCT_TYPE
+#define OMP_STRUCT_TYPE(VarName, StructName, ...)
+#endif
+
+#define __OMP_STRUCT_TYPE(VarName, Name, ...)                                  \
+  OMP_STRUCT_TYPE(VarName, "struct." #Name, __VA_ARGS__)
+
+__OMP_STRUCT_TYPE(Ident, ident_t, Int32, Int32, Int32, Int32, Int8Ptr)
+
+#undef __OMP_STRUCT_TYPE
+#undef OMP_STRUCT_TYPE
+
+#ifndef OMP_FUNCTION_TYPE
+#define OMP_FUNCTION_TYPE(VarName, IsVarArg, ReturnType, ...)
+#endif
+
+#define __OMP_FUNCTION_TYPE(VarName, IsVarArg, ReturnType, ...)                \
+  OMP_FUNCTION_TYPE(VarName, IsVarArg, ReturnType, __VA_ARGS__)
+
+__OMP_FUNCTION_TYPE(ParallelTask, true, Void, Int32Ptr, Int32Ptr)
+
+#undef __OMP_FUNCTION_TYPE
+#undef OMP_FUNCTION_TYPE
+
+///}
+
+/// Runtime library function (and their attributes)
+///
+///{
+
+#ifndef OMP_RTL
+#define OMP_RTL(Enum, Str, IsVarArg, ReturnType, ...)
+#endif
+
+#define __OMP_RTL(Name, IsVarArg, ReturnType, ...)                             \
+  OMP_RTL(OMPRTL_##Name, #Name, IsVarArg, ReturnType, __VA_ARGS__)
+
+__OMP_RTL(__kmpc_barrier, false, Void, IdentPtr, Int32)
+__OMP_RTL(__kmpc_cancel_barrier, false, Int32, IdentPtr, Int32)
+__OMP_RTL(__kmpc_global_thread_num, false, Int32, IdentPtr)
+__OMP_RTL(__kmpc_fork_call, true, Void, IdentPtr, Int32, ParallelTaskPtr)
+__OMP_RTL(omp_get_thread_num, false, Int32, )
+
+#undef __OMP_RTL
+#undef OMP_RTL
+
+#define EnumAttr(Kind) Attribute::get(Ctx, Attribute::AttrKind::Kind)
+#define AttributeSet(...)                                                      \
+  AttributeSet::get(Ctx, ArrayRef<Attribute>({__VA_ARGS__}))
+
+#ifndef OMP_ATTRS_SET
+#define OMP_ATTRS_SET(VarName, AttrSet)
+#endif
+
+#define __OMP_ATTRS_SET(VarName, AttrSet) OMP_ATTRS_SET(VarName, AttrSet)
+
+__OMP_ATTRS_SET(GetterAttrs,
+                OptimisticAttributes
+                    ? AttributeSet(EnumAttr(NoUnwind), EnumAttr(ReadOnly),
+                                   EnumAttr(NoSync), EnumAttr(NoFree))
+                    : AttributeSet(EnumAttr(NoUnwind)))
+
+#undef __OMP_ATTRS_SET
+#undef OMP_ATTRS_SET
+
+#ifndef OMP_RTL_ATTRS
+#define OMP_RTL_ATTRS(Enum, FnAttrSet, RetAttrSet, ArgAttrSets)
+#endif
+
+#define __OMP_RTL_ATTRS(Name, FnAttrSet, RetAttrSet, ArgAttrSets)              \
+  OMP_RTL_ATTRS(OMPRTL_##Name, FnAttrSet, RetAttrSet, ArgAttrSets)
+
+__OMP_RTL_ATTRS(__kmpc_global_thread_num, GetterAttrs, AttributeSet(), {})
+__OMP_RTL_ATTRS(omp_get_thread_num, GetterAttrs, AttributeSet(), {})
+
+#undef __OMP_RTL_ATTRS
+#undef OMP_RTL_ATTRS
+#undef AttributeSet
+#undef EnumAttr
+
+///}
+
+/// KMP ident_t bit flags
+///
+/// In accordance with the values in `openmp/runtime/src/kmp.h`.
+///
+///{
+
+#ifndef OMP_IDENT_FLAG
+#define OMP_IDENT_FLAG(Enum, Str, Value)
+#endif
+
+#define __OMP_IDENT_FLAG(Name, Value)                                          \
+  OMP_IDENT_FLAG(OMP_IDENT_FLAG_##Name, #Name, Value)
+
+__OMP_IDENT_FLAG(KMPC, 0x02)
+__OMP_IDENT_FLAG(BARRIER_EXPL, 0x20)
+__OMP_IDENT_FLAG(BARRIER_IMPL, 0x0040)
+__OMP_IDENT_FLAG(BARRIER_IMPL_MASK, 0x01C0)
+__OMP_IDENT_FLAG(BARRIER_IMPL_FOR, 0x0040)
+__OMP_IDENT_FLAG(BARRIER_IMPL_SECTIONS, 0x00C0)
+__OMP_IDENT_FLAG(BARRIER_IMPL_SINGLE, 0x0140)
+__OMP_IDENT_FLAG(BARRIER_IMPL_WORKSHARE, 0x01C0)
+
+#undef __OMP_IDENT_FLAG
+#undef OMP_IDENT_FLAG
+
+///}

llvm/lib/Frontend/OpenMP/CMakeLists.txt

 add_llvm_component_library(LLVMFrontendOpenMP
   OMPConstants.cpp
+  OMPIRBuilder.cpp
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/Frontend
-  ${LLVM_MAIN_INCLUDE_DIR}/llvm/Frontend/OpenMP
+  ${LLVM_MAIN_INCLUDE_DIR}/llvm/Frontend/OpenMP/OMP
 
   DEPENDS
   intrinsics_gen
   )

llvm/lib/Frontend/OpenMP/OMPConstants.cpp

 //===- OMPConstants.cpp - Helpers related to OpenMP code generation ---===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Frontend/OpenMP/OMPConstants.h"
 
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/StringSwitch.h"
+#include "llvm/IR/Module.h"
+#include "llvm/IR/Type.h"
 
 using namespace llvm;
 using namespace omp;
+using namespace types;
 
 Directive llvm::omp::getOpenMPDirectiveKind(StringRef Str) {
   return llvm::StringSwitch<Directive>(Str)
 #define OMP_DIRECTIVE(Enum, Str) .Case(Str, Enum)
 #include "llvm/Frontend/OpenMP/OMPKinds.def"
       .Default(OMPD_unknown);
 }
 
 StringRef llvm::omp::getOpenMPDirectiveName(Directive Kind) {
   switch (Kind) {
 #define OMP_DIRECTIVE(Enum, Str)                                               \
   case Enum:                                                                   \
     return Str;
 #include "llvm/Frontend/OpenMP/OMPKinds.def"
   }
   llvm_unreachable("Invalid OpenMP directive kind");
 }
+
+/// Declarations for LLVM-IR types (simple, function and structure) are
+/// generated below. Their names are defined and used in OpenMPKinds.def. Here
+/// we provide the declarations, the initializeTypes function will provide the
+/// values.
+///
+///{
+
+#define OMP_TYPE(VarName, InitValue) Type *llvm::omp::types::VarName = nullptr;
+#define OMP_FUNCTION_TYPE(VarName, IsVarArg, ReturnType, ...)                  \
+  FunctionType *llvm::omp::types::VarName = nullptr;                           \
+  PointerType *llvm::omp::types::VarName##Ptr = nullptr;
+#define OMP_STRUCT_TYPE(VarName, StrName, ...)                                 \
+  StructType *llvm::omp::types::VarName = nullptr;                             \
+  PointerType *llvm::omp::types::VarName##Ptr = nullptr;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+
+///}
+
+void llvm::omp::types::initializeTypes(Module &M) {
+  if (Void)
+    return;
+
+  LLVMContext &Ctx = M.getContext();
+  // Create all simple and struct types exposed by the runtime and remember
+  // the llvm::PointerTypes of them for easy access later.
+  StructType *T;
+#define OMP_TYPE(VarName, InitValue) VarName = InitValue;
+#define OMP_FUNCTION_TYPE(VarName, IsVarArg, ReturnType, ...)                  \
+  VarName = FunctionType::get(ReturnType, {__VA_ARGS__}, IsVarArg);            \
+  VarName##Ptr = PointerType::getUnqual(T);
+#define OMP_STRUCT_TYPE(VarName, StructName, ...)                              \
+  T = M.getTypeByName(StructName);                                             \
+  if (!T)                                                                      \
+    T = StructType::create(Ctx, {__VA_ARGS__}, StructName);                    \
+  VarName = T;                                                                 \
+  VarName##Ptr = PointerType::getUnqual(T);
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+}
+
+void llvm::omp::types::uninitializeTypes() {
+#define OMP_TYPE(VarName, InitValue) VarName = nullptr;
+#define OMP_FUNCTION_TYPE(VarName, IsVarArg, ReturnType, ...)                  \
+  VarName = nullptr;                                                           \
+  VarName##Ptr = nullptr;
+#define OMP_STRUCT_TYPE(VarName, StrName, ...)                                 \
+  VarName = nullptr;                                                           \
+  VarName##Ptr = nullptr;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+}

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

+//===- OpenMPIRBuilder.cpp - Builder for LLVM-IR for OpenMP directives ----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file
+///
+/// This file implements the OpenMPIRBuilder class, which is used as a
+/// convenient way to create LLVM instructions for OpenMP directives.
+///
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Frontend/OpenMP/OMPIRBuilder.h"
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSwitch.h"
+#include "llvm/IR/DebugInfo.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Transforms/Utils/BasicBlockUtils.h"
+
+#include <sstream>
+
+#define DEBUG_TYPE "openmp-ir-builder"
+
+using namespace llvm;
+using namespace omp;
+using namespace types;
+
+static cl::opt<bool>
+    OptimisticAttributes("openmp-ir-builder-optimistic-attributes", cl::Hidden,
+                         cl::desc("Use optimistic attributes describing "
+                                  "'as-if' properties of runtime calls."),
+                         cl::init(false));
+
+void OpenMPIRBuilder::addAttributes(omp::RuntimeFunction FnID, Function &Fn) {
+  LLVMContext &Ctx = Fn.getContext();
+
+#define OMP_ATTRS_SET(VarName, AttrSet) AttributeSet VarName = AttrSet;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+
+  // Add attributes to the new declaration.
+  switch (FnID) {
+#define OMP_RTL_ATTRS(Enum, FnAttrSet, RetAttrSet, ArgAttrSets)                \
+  case Enum:                                                                   \
+    Fn.setAttributes(                                                          \
+        AttributeList::get(Ctx, FnAttrSet, RetAttrSet, ArgAttrSets));          \
+    break;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+  default:
+    // Attributes are optional.
+    break;
+  }
+}
+
+Function *OpenMPIRBuilder::getOrCreateRuntimeFunction(RuntimeFunction FnID) {
+  Function *Fn = nullptr;
+
+  // Try to find the declation in the module first.
+  switch (FnID) {
+#define OMP_RTL(Enum, Str, IsVarArg, ReturnType, ...)                          \
+  case Enum:                                                                   \
+    Fn = M.getFunction(Str);                                                   \
+    break;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+  }
+
+  if (!Fn) {
+    // Create a new declaration if we need one.
+    switch (FnID) {
+#define OMP_RTL(Enum, Str, IsVarArg, ReturnType, ...)                          \
+  case Enum:                                                                   \
+    Fn = Function::Create(FunctionType::get(ReturnType,                        \
+                                            ArrayRef<Type *>{__VA_ARGS__},     \
+                                            IsVarArg),                         \
+                          GlobalValue::ExternalLinkage, Str, M);               \
+    break;
+#include "llvm/Frontend/OpenMP/OMPKinds.def"
+    }
+
+    addAttributes(FnID, *Fn);
+  }
+
+  assert(Fn && "Failed to create OpenMP runtime function");
+  return Fn;
+}
+
+void OpenMPIRBuilder::initialize() { initializeTypes(M); }
+
+Value *OpenMPIRBuilder::getOrCreateIdent(Constant *SrcLocStr,
+                                         IdentFlag LocFlags) {
+  // Enable "C-mode".
+  LocFlags |= OMP_IDENT_FLAG_KMPC;
+
+  GlobalVariable *&DefaultIdent = IdentMap[{SrcLocStr, uint64_t(LocFlags)}];
+  if (!DefaultIdent) {
+    Constant *I32Null = ConstantInt::getNullValue(Int32);
+    Constant *IdentData[] = {I32Null,
+                             ConstantInt::get(Int32, uint64_t(LocFlags)),
+                             I32Null, I32Null, SrcLocStr};
+    Constant *Initializer = ConstantStruct::get(
+        cast<StructType>(IdentPtr->getPointerElementType()), IdentData);
+
+    // Look for existing encoding of the location + flags, not needed but
+    // minimizes the difference to the existing solution while we transition.
+    for (GlobalVariable &GV : M.getGlobalList())
+      if (GV.getType() == IdentPtr && GV.hasInitializer())
+        if (GV.getInitializer() == Initializer)
+          return DefaultIdent = &GV;
+
+    DefaultIdent = new GlobalVariable(M, IdentPtr->getPointerElementType(),
+                                      /* isConstant = */ false,
+                                      GlobalValue::PrivateLinkage, Initializer);
+    DefaultIdent->setUnnamedAddr(GlobalValue::UnnamedAddr::Global);
+    DefaultIdent->setAlignment(Align(8));
+  }
+  return DefaultIdent;
+}
+
+Constant *OpenMPIRBuilder::getOrCreateSrcLocStr(StringRef LocStr) {
+  Constant *&SrcLocStr = SrcLocStrMap[LocStr];
+  if (!SrcLocStr) {
+    Constant *Initializer =
+        ConstantDataArray::getString(M.getContext(), LocStr);
+
+    // Look for existing encoding of the location, not needed but minimizes the
+    // difference to the existing solution while we transition.
+    for (GlobalVariable &GV : M.getGlobalList())
+      if (GV.isConstant() && GV.hasInitializer() &&
+          GV.getInitializer() == Initializer)
+        return SrcLocStr = ConstantExpr::getPointerCast(&GV, Int8Ptr);
+
+    SrcLocStr = Builder.CreateGlobalStringPtr(LocStr);
+  }
+  return SrcLocStr;
+}
+
+Constant *OpenMPIRBuilder::getOrCreateDefaultSrcLocStr() {
+  return getOrCreateSrcLocStr(";unknown;unknown;0;0;;");
+}
+
+Constant *
+OpenMPIRBuilder::getOrCreateSrcLocStr(const LocationDescription &Loc) {
+  DILocation *DIL = Loc.DL.get();
+  if (!DIL)
+    return getOrCreateDefaultSrcLocStr();
+  StringRef Filename =
+      !DIL->getFilename().empty() ? DIL->getFilename() : M.getName();
+  StringRef Function = DIL->getScope()->getSubprogram()->getName();
+  Function =
+      !Function.empty() ? Function : Loc.IP.getBlock()->getParent()->getName();
+  std::string LineStr = std::to_string(DIL->getLine());
+  std::string ColumnStr = std::to_string(DIL->getColumn());
+  std::stringstream SrcLocStr;
+  SrcLocStr << ";" << Filename.data() << ";" << Function.data() << ";"
+            << LineStr << ";" << ColumnStr << ";;";
+  return getOrCreateSrcLocStr(SrcLocStr.str());
+}
+
+Value *OpenMPIRBuilder::getOrCreateThreadID(Value *Ident) {
+  return Builder.CreateCall(
+      getOrCreateRuntimeFunction(OMPRTL___kmpc_global_thread_num), Ident,
+      "omp_global_thread_num");
+}
+
+OpenMPIRBuilder::InsertPointTy
+OpenMPIRBuilder::CreateBarrier(const LocationDescription &Loc, Directive DK,
+                               bool ForceSimpleCall, bool CheckCancelFlag) {
+  if (!updateToLocation(Loc))
+    return Loc.IP;
+  return emitBarrierImpl(Loc, DK, ForceSimpleCall, CheckCancelFlag);
+}
+
+OpenMPIRBuilder::InsertPointTy
+OpenMPIRBuilder::emitBarrierImpl(const LocationDescription &Loc, Directive Kind,
+                                 bool ForceSimpleCall, bool CheckCancelFlag) {
+  // Build call __kmpc_cancel_barrier(loc, thread_id) or
+  //            __kmpc_barrier(loc, thread_id);
+
+  IdentFlag BarrierLocFlags;
+  switch (Kind) {
+  case OMPD_for:
+    BarrierLocFlags = OMP_IDENT_FLAG_BARRIER_IMPL_FOR;
+    break;
+  case OMPD_sections:
+    BarrierLocFlags = OMP_IDENT_FLAG_BARRIER_IMPL_SECTIONS;
+    break;
+  case OMPD_single:
+    BarrierLocFlags = OMP_IDENT_FLAG_BARRIER_IMPL_SINGLE;
+    break;
+  case OMPD_barrier:
+    BarrierLocFlags = OMP_IDENT_FLAG_BARRIER_EXPL;
+    break;
+  default:
+    BarrierLocFlags = OMP_IDENT_FLAG_BARRIER_IMPL;
+    break;
+  }
+
+  Constant *SrcLocStr = getOrCreateSrcLocStr(Loc);
+  Value *Args[] = {getOrCreateIdent(SrcLocStr, BarrierLocFlags),
+                   getOrCreateThreadID(getOrCreateIdent(SrcLocStr))};
+
+  // If we are in a cancellable parallel region, barriers are cancellation
+  // points.
+  // TODO: Check why we would force simple calls or to ignore the cancel flag.
+  bool UseCancelBarrier = !ForceSimpleCall && CancellationBlock;
+
+  Value *Result = Builder.CreateCall(
+      getOrCreateRuntimeFunction(UseCancelBarrier ? OMPRTL___kmpc_cancel_barrier
+                                                  : OMPRTL___kmpc_barrier),
+      Args);
+
+  if (UseCancelBarrier && CheckCancelFlag) {
+    // For a cancel barrier we create two new blocks.
+    BasicBlock *BB = Builder.GetInsertBlock();
+    BasicBlock *NonCancellationBlock = BasicBlock::Create(
+        BB->getContext(), BB->getName() + ".cont", BB->getParent());
+
+    // Jump to them based on the return value.
+    Value *Cmp = Builder.CreateIsNull(Result);
+    Builder.CreateCondBr(Cmp, NonCancellationBlock, CancellationBlock,
+                         /* TODO weight */ nullptr, nullptr);
+
+    Builder.SetInsertPoint(NonCancellationBlock);
+    assert(CancellationBlock->getParent() == BB->getParent() &&
+           "Unexpected cancellation block parent!");
+
+    // TODO: This is a workaround for now, we always reset the cancellation
+    // block until we manage it ourselves here.
+    CancellationBlock = nullptr;
+  }
+
+  return Builder.saveIP();
+}

llvm/unittests/CMakeLists.txt

 add_subdirectory(AsmParser)
 add_subdirectory(BinaryFormat)
 add_subdirectory(Bitcode)
 add_subdirectory(Bitstream)
 add_subdirectory(CodeGen)
 add_subdirectory(DebugInfo)
 add_subdirectory(Demangle)
 add_subdirectory(ExecutionEngine)
+add_subdirectory(Frontend)
 add_subdirectory(FuzzMutate)
 add_subdirectory(IR)
 add_subdirectory(LineEditor)
 add_subdirectory(Linker)
 add_subdirectory(MC)
 add_subdirectory(MI)
 add_subdirectory(Object)
 add_subdirectory(ObjectYAML)

llvm/unittests/Frontend/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  Analysis
+  Core
+  FrontendOpenMP
+  Support
+  Passes
+  )
+
+add_llvm_unittest(LLVMFrontendTests
+  OpenMPIRBuilderTest.cpp
+  )
+
+target_link_libraries(LLVMFrontendTests PRIVATE LLVMTestingSupport)

llvm/unittests/Frontend/OpenMPIRBuilderTest.cpp

+//===- llvm/unittest/IR/OpenMPIRBuilderTest.cpp - OpenMPIRBuilder tests ---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Frontend/OpenMP/OMPIRBuilder.h"
+#include "llvm/IR/BasicBlock.h"
+#include "llvm/IR/DIBuilder.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Frontend/OpenMP/OMPConstants.h"
+#include "llvm/IR/Verifier.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+using namespace omp;
+using namespace types;
+
+namespace {
+
+class OpenMPIRBuilderTest : public testing::Test {
+protected:
+  void SetUp() override {
+    M.reset(new Module("MyModule", Ctx));
+    FunctionType *FTy =
+        FunctionType::get(Type::getVoidTy(Ctx), {Type::getInt32Ty(Ctx)},
+                          /*isVarArg=*/false);
+    F = Function::Create(FTy, Function::ExternalLinkage, "", M.get());
+    BB = BasicBlock::Create(Ctx, "", F);
+
+    DIBuilder DIB(*M);
+    auto File = DIB.createFile("test.dbg", "/");
+    auto CU =
+        DIB.createCompileUnit(dwarf::DW_LANG_C, File, "llvm-C", true, "", 0);
+    auto Type = DIB.createSubroutineType(DIB.getOrCreateTypeArray(None));
+    auto SP = DIB.createFunction(
+        CU, "foo", "", File, 1, Type, 1, DINode::FlagZero,
+        DISubprogram::SPFlagDefinition | DISubprogram::SPFlagOptimized);
+    F->setSubprogram(SP);
+    auto Scope = DIB.createLexicalBlockFile(SP, File, 0);
+    DIB.finalize();
+    DL = DebugLoc::get(3, 7, Scope);
+  }
+
+  void TearDown() override {
+    BB = nullptr;
+    M.reset();
+    uninitializeTypes();
+  }
+
+  LLVMContext Ctx;
+  std::unique_ptr<Module> M;
+  Function *F;
+  BasicBlock *BB;
+  DebugLoc DL;
+};
+
+TEST_F(OpenMPIRBuilderTest, CreateBarrier) {
+  OpenMPIRBuilder OMPBuilder(*M);
+  OMPBuilder.initialize();
+
+  IRBuilder<> Builder(BB);
+
+  OMPBuilder.CreateBarrier({IRBuilder<>::InsertPoint()}, OMPD_for);
+  EXPECT_TRUE(M->global_empty());
+  EXPECT_EQ(M->size(), 1U);
+  EXPECT_EQ(F->size(), 1U);
+  EXPECT_EQ(BB->size(), 0U);
+
+  OpenMPIRBuilder::LocationDescription Loc({Builder.saveIP()});
+  OMPBuilder.CreateBarrier(Loc, OMPD_for);
+  EXPECT_FALSE(M->global_empty());
+  EXPECT_EQ(M->size(), 3U);
+  EXPECT_EQ(F->size(), 1U);
+  EXPECT_EQ(BB->size(), 2U);
+
+  CallInst *GTID = dyn_cast<CallInst>(&BB->front());
+  EXPECT_NE(GTID, nullptr);
+  EXPECT_EQ(GTID->getNumArgOperands(), 1U);
+  EXPECT_EQ(GTID->getCalledFunction()->getName(), "__kmpc_global_thread_num");
+  EXPECT_FALSE(GTID->getCalledFunction()->doesNotAccessMemory());
+  EXPECT_FALSE(GTID->getCalledFunction()->doesNotFreeMemory());
+
+  CallInst *Barrier = dyn_cast<CallInst>(GTID->getNextNode());
+  EXPECT_NE(Barrier, nullptr);
+  EXPECT_EQ(Barrier->getNumArgOperands(), 2U);
+  EXPECT_EQ(Barrier->getCalledFunction()->getName(), "__kmpc_barrier");
+  EXPECT_FALSE(Barrier->getCalledFunction()->doesNotAccessMemory());
+  EXPECT_FALSE(Barrier->getCalledFunction()->doesNotFreeMemory());
+
+  EXPECT_EQ(cast<CallInst>(Barrier)->getArgOperand(1), GTID);
+
+  Builder.CreateUnreachable();
+  EXPECT_FALSE(verifyModule(*M));
+}
+
+TEST_F(OpenMPIRBuilderTest, CreateCancelBarrier) {
+  OpenMPIRBuilder OMPBuilder(*M);
+  OMPBuilder.initialize();
+
+  BasicBlock *CBB = BasicBlock::Create(Ctx, "", F);
+  new UnreachableInst(Ctx, CBB);
+  OMPBuilder.setCancellationBlock(CBB);
+
+  IRBuilder<> Builder(BB);
+
+  OpenMPIRBuilder::LocationDescription Loc({Builder.saveIP()});
+  auto NewIP = OMPBuilder.CreateBarrier(Loc, OMPD_for);
+  Builder.restoreIP(NewIP);
+  EXPECT_FALSE(M->global_empty());
+  EXPECT_EQ(M->size(), 3U);
+  EXPECT_EQ(F->size(), 3U);
+  EXPECT_EQ(BB->size(), 4U);
+
+  CallInst *GTID = dyn_cast<CallInst>(&BB->front());
+  EXPECT_NE(GTID, nullptr);
+  EXPECT_EQ(GTID->getNumArgOperands(), 1U);
+  EXPECT_EQ(GTID->getCalledFunction()->getName(), "__kmpc_global_thread_num");
+  EXPECT_FALSE(GTID->getCalledFunction()->doesNotAccessMemory());
+  EXPECT_FALSE(GTID->getCalledFunction()->doesNotFreeMemory());
+
+  CallInst *Barrier = dyn_cast<CallInst>(GTID->getNextNode());
+  EXPECT_NE(Barrier, nullptr);
+  EXPECT_EQ(Barrier->getNumArgOperands(), 2U);
+  EXPECT_EQ(Barrier->getCalledFunction()->getName(), "__kmpc_cancel_barrier");
+  EXPECT_FALSE(Barrier->getCalledFunction()->doesNotAccessMemory());
+  EXPECT_FALSE(Barrier->getCalledFunction()->doesNotFreeMemory());
+  EXPECT_EQ(Barrier->getNumUses(), 1U);
+  Instruction *BarrierBBTI = Barrier->getParent()->getTerminator();
+  EXPECT_EQ(BarrierBBTI->getNumSuccessors(), 2U);
+  EXPECT_EQ(BarrierBBTI->getSuccessor(0), NewIP.getBlock());
+  EXPECT_EQ(BarrierBBTI->getSuccessor(1), CBB);
+
+  EXPECT_EQ(cast<CallInst>(Barrier)->getArgOperand(1), GTID);
+
+  Builder.CreateUnreachable();
+  EXPECT_FALSE(verifyModule(*M));
+}
+
+TEST_F(OpenMPIRBuilderTest, DbgLoc) {
+  OpenMPIRBuilder OMPBuilder(*M);
+  OMPBuilder.initialize();
+  F->setName("func");
+
+  IRBuilder<> Builder(BB);
+
+  OpenMPIRBuilder::LocationDescription Loc({Builder.saveIP(), DL});
+  OMPBuilder.CreateBarrier(Loc, OMPD_for);
+  CallInst *GTID = dyn_cast<CallInst>(&BB->front());
+  CallInst *Barrier = dyn_cast<CallInst>(GTID->getNextNode());
+  EXPECT_EQ(GTID->getDebugLoc(), DL);
+  EXPECT_EQ(Barrier->getDebugLoc(), DL);
+  EXPECT_TRUE(isa<GlobalVariable>(Barrier->getOperand(0)));
+  if (!isa<GlobalVariable>(Barrier->getOperand(0)))
+    return;
+  GlobalVariable *Ident = cast<GlobalVariable>(Barrier->getOperand(0));
+  EXPECT_TRUE(Ident->hasInitializer());
+  if (!Ident->hasInitializer())
+    return;
+  Constant *Initializer = Ident->getInitializer();
+  EXPECT_TRUE(
+      isa<GlobalVariable>(Initializer->getOperand(4)->stripPointerCasts()));
+  GlobalVariable *SrcStrGlob =
+      cast<GlobalVariable>(Initializer->getOperand(4)->stripPointerCasts());
+  if (!SrcStrGlob)
+    return;
+  EXPECT_TRUE(isa<ConstantDataArray>(SrcStrGlob->getInitializer()));
+  ConstantDataArray *SrcSrc =
+      dyn_cast<ConstantDataArray>(SrcStrGlob->getInitializer());
+  if (!SrcSrc)
+    return;
+  EXPECT_EQ(SrcSrc->getAsCString(), ";test.dbg;foo;3;7;;");
+}
+} // namespace