[Attributor] Add AAExecutionDomainInfo interface to OpenMPOpt

[jhuber6]

Summary

Add the AAExecutionDomainInfo attributor instance to OpenMPOpt.
This will infer information relating to domain information that an
instruction might be expecting in. Right now this only includes a very
crude check for instructions that will be executed by the master thread
by comparing a thread-id function with a constant zero.

[jdoerfert]

Looks like a reasonable start, some comments:

Hook it up to the globalization to make it easier to test it. We also need negative tests and such, swapped conditions, swapped edges, ...

llvm/include/llvm/Transforms/IPO/Attributor.h

3851

Comments are copy&paste but I don't assume we need these functions at all.
The interface we want is below/

3866

Nit: take references

llvm/lib/Transforms/IPO/AttributorAttributes.cpp

4347

Not needed

8150

8177

-Master
+MainThreadOnly or FirstThreadOnly

---

I don't get the TrueBB part, you don't know which successor the block you are looking at is from the code below. Make it "SuccBB" and check the condition first, then check if this is the successor under which the condition is tid == 0.

---

Given that we have all the openmp specific functions in here we should move this into OpenMPOpt.

8185

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

1141

I assume this is only for testing, right?

[jhuber6]

Moving to OpenMPOpt.

[jdoerfert]

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

2277

return how many blocks are single threaded of how many, as string: "3/7 BBs thread 0 only"

2308

You could use a set, during initialize put all blocks in the set, remove one if you cannot argue it's single threaded. you know something changed if the size at the end of update is not the size at the beginning. That should make things a lot simpler, whenever you fail to show single threaded, remove it from the set right away, no need to communicate booleans all over the place and check the set all the time to update "changed".

2312

unreachable for all but function. See the other createForPosition

2329

also check if it is an equality, this would allow tid >= 0, I think.

2365

Move down to the usage, also IsSingleThreadOnly

2376–2378

[jhuber6]

Making suggested changes.

[jdoerfert]

LG, two minor nits.

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

2294

Here and elsewhere, if the type is trivial, use it. 4 vs 9 chars but better readability.
E is not a good variable name for a BB.

2335–2337

[jhuber6]

Fixing nits.

[sebastian-ne]

Hi,
The added test llvm/test/Transforms/OpenMP/single_threaded_execution.ll fails in release builds when assertions are disabled.
You probably need to add ; REQUIRES: asserts like it is done in llvm/test/CodeGen/Mips/brind-tailcall.ll.

[jhuber6]

In D101578#2735986, @sebastian-ne wrote:

Hi,
The added test llvm/test/Transforms/OpenMP/single_threaded_execution.ll fails in release builds when assertions are disabled.
You probably need to add ; REQUIRES: asserts like it is done in llvm/test/CodeGen/Mips/brind-tailcall.ll.

Thanks, seems like someone already fixed it in 05146fe5171076eaec3ef6a9eeeab4c8a33acf15.

llvm/include/llvm/Transforms/IPO/Attributor.h

struct AANoUndef
   static bool classof(const AbstractAttribute *AA) {
     return (AA->getIdAddr() == &ID);
   }
 
   /// Unique ID (due to the unique address)
   static const char ID;
 };
 
+struct AAExecutionDomain
+    : public StateWrapper<BooleanState, AbstractAttribute> {
+  using Base = StateWrapper<BooleanState, AbstractAttribute>;
+  AAExecutionDomain(const IRPosition &IRP, Attributor &A) : Base(IRP) {}
+
+  /// Returns true if HeapToStack conversion is assumed to be possible.
+  bool isAssumedSingleThreaded() const { return getAssumed(); }
+
+  /// Returns true if HeapToStack conversion is known to be possible.
+  bool isKnownSingleThreaded() const { return getKnown(); }
+
+  /// Create an abstract attribute view for the position \p IRP.
+  static AAExecutionDomain &createForPosition(const IRPosition &IRP,
+                                              Attributor &A);
+
+  /// See AbstractAttribute::getName().
+  const std::string getName() const override { return "AAExecutionDomain"; }
+
+  /// See AbstractAttribute::getIdAddr().
+  const char *getIdAddr() const override { return &ID; }
+
+  /// Check if an instruction is executed by a single thread.
+  virtual bool isSingleThreadExeuction(const Instruction *) = 0;
+
+  virtual bool isSingleThreadExeuction(const BasicBlock *) = 0;
+
+  /// This function should return true if the type of the \p AA is
+  /// AAExecutionDomain.
+  static bool classof(const AbstractAttribute *AA) {
+    return (AA->getIdAddr() == &ID);
+  }
+
+  /// Unique ID (due to the unique address)
+  static const char ID;
+};
+
 /// Run options, used by the pass manager.
 enum AttributorRunOption {
   NONE = 0,
   MODULE = 1 << 0,
   CGSCC = 1 << 1,
   ALL = MODULE | CGSCC
 };
 
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_IPO_ATTRIBUTOR_H

llvm/lib/Transforms/IPO/AttributorAttributes.cpp

 //===- AttributorAttributes.cpp - Attributes for Attributor deduction -----===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // See the Attributor.h file comment and the class descriptions in that file for
 // more information.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Transforms/IPO/Attributor.h"
 
+#include "llvm/ADT/PostOrderIterator.h"
 #include "llvm/ADT/SCCIterator.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/AliasAnalysis.h"
 #include "llvm/Analysis/AssumeBundleQueries.h"
 #include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/CaptureTracking.h"
 #include "llvm/Analysis/LazyValueInfo.h"

 PIPE_OPERATOR(AAReachability)
 PIPE_OPERATOR(AAMemoryBehavior)
 PIPE_OPERATOR(AAMemoryLocation)
 PIPE_OPERATOR(AAValueConstantRange)
 PIPE_OPERATOR(AAPrivatizablePtr)
 PIPE_OPERATOR(AAUndefinedBehavior)
 PIPE_OPERATOR(AAPotentialValues)
 PIPE_OPERATOR(AANoUndef)
+PIPE_OPERATOR(AAExecutionDomain)
 
 #undef PIPE_OPERATOR
 } // namespace llvm
 
 namespace {
 
 static Optional<ConstantInt *>
 getAssumedConstantInt(Attributor &A, const Value &V,
private:
 
   /// Set of potential copies of the tracked value.
   SmallVectorImpl<const Value *> &PotentialCopies;
 
   /// Global counter to limit the number of explored uses.
   unsigned &RemainingUsesToExplore;
 };
 
+// interested in AA for specific positions.
 ChangeStatus AANoCaptureImpl::updateImpl(Attributor &A) {
   const IRPosition &IRP = getIRPosition();
   const Value *V = isArgumentPosition() ? IRP.getAssociatedArgument()
                                         : &IRP.getAssociatedValue();
   if (!V)
     return indicatePessimisticFixpoint();
 
   const Function *F =

 struct AANoUndefCallSiteReturned final
     : AACallSiteReturnedFromReturned<AANoUndef, AANoUndefImpl> {
   AANoUndefCallSiteReturned(const IRPosition &IRP, Attributor &A)
       : AACallSiteReturnedFromReturned<AANoUndef, AANoUndefImpl>(IRP, A) {}
 
   /// See AbstractAttribute::trackStatistics()
   void trackStatistics() const override { STATS_DECLTRACK_CSRET_ATTR(noundef) }
 };
+
+/// ----------------------- Execution Domain Analysis --------------------------
+
+struct AAExecutionDomainImpl : public AAExecutionDomain {
+  AAExecutionDomainImpl(const IRPosition &IRP, Attributor &A)
+      : AAExecutionDomain(IRP, A) {}
+
+  const std::string getAsStr() const override { return "[ED]"; }
+
+  ChangeStatus manifest(Attributor &A) override {
+    return ChangeStatus::UNCHANGED;
+  }
+
+  ChangeStatus updateImpl(Attributor &A) override;
+
+  /// Check if an instruction is executed by a single thread.
+  bool isSingleThreadExeuction(const Instruction *I) override {
+    return isSingleThreadExeuction(I->getParent());
+  }
+
+  bool isSingleThreadExeuction(const BasicBlock *BB) override {
+    return SingleThreadedBBs[BB];
+  }
+
+  DenseMap<const BasicBlock *, bool> SingleThreadedBBs;
+};
+
+ChangeStatus AAExecutionDomainImpl::updateImpl(Attributor &A) {
+  Function *F = getAnchorScope();
+  llvm::ReversePostOrderTraversal<Function *> RPOT(F);
+
+  // TODO: Simplify this value to propogate constants.
+  // TODO: Check more than a single use.
+  // TODO: Add a LibraryInfo for functions that return a thread ID.
+  auto IsMasterEdge = [](BranchInst *Edge, BasicBlock *TrueBB) {
+    if (!Edge || !Edge->isConditional())
+      return false;
+    if (Edge->getSuccessor(0) != TrueBB)
+      return false;
+
+    auto *Cmp = dyn_cast<CmpInst>(Edge->getCondition());
+    if (!Cmp || !Cmp->isTrueWhenEqual())
+      return false;
+
+    ConstantInt *C = dyn_cast<ConstantInt>(Cmp->getOperand(1));
+    if (!C || !C->isZero())
+      return false;
+    ;
+    if (auto *CB = dyn_cast<CallBase>(Cmp->getOperand(0))) {
+      if (CB->getCalledFunction()->getName() == "omp_get_thread_num" ||
+          CB->getCalledFunction()->getName() == "__kmpc_global_thread_num" ||
+          CB->getCalledFunction()->getName() == "__kmpc_master")
+        return true;
+    }
+
+    return false;
+  };
+
+  // TODO: Handle more complex cases than only the master thread.
+  auto MergePredecessorStates = [&](BasicBlock *BB) {
+    if (pred_begin(BB) == pred_end(BB))
+      return false;
+
+    bool IsSingleThreaded = true;
+    for (auto I = pred_begin(BB), E = pred_end(BB); I != E; ++I) {
+      if (!IsMasterEdge(dyn_cast<BranchInst>((*I)->getTerminator()), BB))
+        IsSingleThreaded &= SingleThreadedBBs[*I];
+    }
+
+    return IsSingleThreaded;
+  };
+
+  for (auto *BB : RPOT) {
+    SingleThreadedBBs[BB] = MergePredecessorStates(BB);
+  }
+
+  return ChangeStatus::UNCHANGED;
+}
+
+struct AAExecutionDomainFunction final : public AAExecutionDomainImpl {
+  AAExecutionDomainFunction(const IRPosition &IRP, Attributor &A)
+      : AAExecutionDomainImpl(IRP, A) {}
+
+  /// See AbstractAttribute::trackStatistics().
+  void trackStatistics() const override {}
+};
+
 } // namespace
 
 const char AAReturnedValues::ID = 0;
 const char AANoUnwind::ID = 0;
 const char AANoSync::ID = 0;
 const char AANoFree::ID = 0;
 const char AANonNull::ID = 0;
 const char AANoRecurse::ID = 0;

 const char AAValueSimplify::ID = 0;
 const char AAHeapToStack::ID = 0;
 const char AAPrivatizablePtr::ID = 0;
 const char AAMemoryBehavior::ID = 0;
 const char AAMemoryLocation::ID = 0;
 const char AAValueConstantRange::ID = 0;
 const char AAPotentialValues::ID = 0;
 const char AANoUndef::ID = 0;
+const char AAExecutionDomain::ID = 0;
 
 // Macro magic to create the static generator function for attributes that
 // follow the naming scheme.
 
 #define SWITCH_PK_INV(CLASS, PK, POS_NAME)                                     \
   case IRPosition::PK:                                                         \
     llvm_unreachable("Cannot create " #CLASS " for a " POS_NAME " position!");
 

 
 CREATE_ALL_ABSTRACT_ATTRIBUTE_FOR_POSITION(AAValueSimplify)
 CREATE_ALL_ABSTRACT_ATTRIBUTE_FOR_POSITION(AAIsDead)
 CREATE_ALL_ABSTRACT_ATTRIBUTE_FOR_POSITION(AANoFree)
 
 CREATE_FUNCTION_ONLY_ABSTRACT_ATTRIBUTE_FOR_POSITION(AAHeapToStack)
 CREATE_FUNCTION_ONLY_ABSTRACT_ATTRIBUTE_FOR_POSITION(AAReachability)
 CREATE_FUNCTION_ONLY_ABSTRACT_ATTRIBUTE_FOR_POSITION(AAUndefinedBehavior)
+CREATE_FUNCTION_ONLY_ABSTRACT_ATTRIBUTE_FOR_POSITION(AAExecutionDomain)
 
 CREATE_NON_RET_ABSTRACT_ATTRIBUTE_FOR_POSITION(AAMemoryBehavior)
 
 #undef CREATE_FUNCTION_ONLY_ABSTRACT_ATTRIBUTE_FOR_POSITION
 #undef CREATE_FUNCTION_ABSTRACT_ATTRIBUTE_FOR_POSITION
 #undef CREATE_NON_RET_ABSTRACT_ATTRIBUTE_FOR_POSITION
 #undef CREATE_VALUE_ABSTRACT_ATTRIBUTE_FOR_POSITION
 #undef CREATE_ALL_ABSTRACT_ATTRIBUTE_FOR_POSITION
 #undef SWITCH_PK_CREATE
 #undef SWITCH_PK_INV

llvm/lib/Transforms/IPO/OpenMPOpt.cpp

static cl::opt<bool> PrintOpenMPKernels("openmp-print-gpu-kernels",
                                         cl::init(false), cl::Hidden);
 
 static cl::opt<bool> HideMemoryTransferLatency(
     "openmp-hide-memory-transfer-latency",
     cl::desc("[WIP] Tries to hide the latency of host to device memory"
              " transfers"),
     cl::Hidden, cl::init(false));
 
+static cl::opt<bool> IdentifySingleThreads("openmp-identify-single-threads",
+                                           cl::init(false), cl::Hidden);
+
 STATISTIC(NumOpenMPRuntimeCallsDeduplicated,
           "Number of OpenMP runtime calls deduplicated");
 STATISTIC(NumOpenMPParallelRegionsDeleted,
           "Number of OpenMP parallel regions deleted");
 STATISTIC(NumOpenMPRuntimeFunctionsIdentified,
           "Number of OpenMP runtime functions identified");
 STATISTIC(NumOpenMPRuntimeFunctionUsesIdentified,
           "Number of OpenMP runtime function uses identified");
if (SCC.empty())
       return false;
 
     bool Changed = false;
 
     LLVM_DEBUG(dbgs() << TAG << "Run on SCC with " << SCC.size()
                       << " functions in a slice with "
                       << OMPInfoCache.ModuleSlice.size() << " functions\n");
 
+    Changed |= runAttributor();
+
     if (IsModulePass) {
       if (remarksEnabled())
         analysisGlobalization();
+      if (IdentifySingleThreads)
+        identifySingleThreadedBBs();
     } else {
       if (PrintICVValues)
         printICVs();
       if (PrintOpenMPKernels)
         printKernels();
 
       Changed |= rewriteDeviceCodeStateMachine();
 
-      Changed |= runAttributor();
 
       // Recollect uses, in case Attributor deleted any.
       OMPInfoCache.recollectUses();
 
       Changed |= deleteParallelRegions();
       if (HideMemoryTransferLatency)
         Changed |= hideMemTransfersLatency();
       Changed |= deduplicateRuntimeCalls();
auto SplitMemTransfers = [&](Use &U, Function &Decl) {
       Changed |= WasSplit;
       return WasSplit;
     };
     RFI.foreachUse(SCC, SplitMemTransfers);
 
     return Changed;
   }
 
+  void identifySingleThreadedBBs() {
+    for (Function &F : M) {
+      IRPosition FnPos = IRPosition::function(F);
+      for (BasicBlock &BB : F) {
+        for (Instruction &I : BB) {
+          if (auto ED = A.lookupAAFor<AAExecutionDomain>(FnPos))
+            if (!ED->isSingleThreadExeuction(&I))
+              continue;
+
+          auto Remark = [&](OptimizationRemarkAnalysis ORA) {
+            return ORA << "Found Instruction executed by a single thread.";
+          };
+          emitRemark<OptimizationRemarkAnalysis>(&I, "OpenMPSingleThread",
+                                                 Remark);
+        }
+      }
+    }
+  }
+
   void analysisGlobalization() {
     RuntimeFunction GlobalizationRuntimeIDs[] = {
         OMPRTL___kmpc_data_sharing_coalesced_push_stack,
         OMPRTL___kmpc_data_sharing_push_stack};
 
     for (const auto GlobalizationCallID : GlobalizationRuntimeIDs) {
       auto &RFI = OMPInfoCache.RFIs[GlobalizationCallID];
 
for (int Idx = 0; Idx < OMPInfoCache.ICVs.size() - 1; ++Idx) {
 
         IRPosition CBPos = IRPosition::callsite_function(CB);
         A.getOrCreateAAFor<AAICVTracker>(CBPos);
         return false;
       };
 
       GetterRFI.foreachUse(SCC, CreateAA);
     }
+    for (auto &F : M) {
+      if (F.isDeclaration())
+        continue;
+
+      IRPosition FnPos = IRPosition::function(F);
+      A.getOrCreateAAFor<AAExecutionDomain>(FnPos);
+    }
   }
 };
 
 Kernel OpenMPOpt::getUniqueKernelFor(Function &F) {
   if (!OMPInfoCache.ModuleSlice.count(&F))
     return nullptr;
 
   // Use a scope to keep the lifetime of the CachedKernel short.
ChangeStatus updateImpl(Attributor &A) override {
     return Changed;
   }
 };
 } // namespace
 
 const char AAICVTracker::ID = 0;
 
 AAICVTracker &AAICVTracker::createForPosition(const IRPosition &IRP,
                                               Attributor &A) {
   AAICVTracker *AA = nullptr;
   switch (IRP.getPositionKind()) {
   case IRPosition::IRP_INVALID:
   case IRPosition::IRP_FLOAT:
   case IRPosition::IRP_ARGUMENT:
   case IRPosition::IRP_CALL_SITE_ARGUMENT:
     llvm_unreachable("ICVTracker can only be created for function position!");
   case IRPosition::IRP_RETURNED:
     AA = new (A.Allocator) AAICVTrackerFunctionReturned(IRP, A);
     break;
   case IRPosition::IRP_CALL_SITE_RETURNED:
     AA = new (A.Allocator) AAICVTrackerCallSiteReturned(IRP, A);
     break;
   case IRPosition::IRP_CALL_SITE:
     AA = new (A.Allocator) AAICVTrackerCallSite(IRP, A);
     break;
   case IRPosition::IRP_FUNCTION:
     AA = new (A.Allocator) AAICVTrackerFunction(IRP, A);
     break;
   }
 
   return *AA;
 }
 
 PreservedAnalyses OpenMPOptPass::run(Module &M, ModuleAnalysisManager &AM) {
   if (!containsOpenMP(M, OMPInModule))
     return PreservedAnalyses::all();
 
   if (DisableOpenMPOptimizations)
     return PreservedAnalyses::all();
 
   // Look at every function definition in the Module.
   SmallVector<Function *, 16> SCC;
   for (Function &Fn : M)
     if (!Fn.isDeclaration())
       SCC.push_back(&Fn);
 
   if (SCC.empty())
     return PreservedAnalyses::all();
 
   FunctionAnalysisManager &FAM =
       AM.getResult<FunctionAnalysisManagerModuleProxy>(M).getManager();
 
   AnalysisGetter AG(FAM);
 
   auto OREGetter = [&FAM](Function *F) -> OptimizationRemarkEmitter & {
     return FAM.getResult<OptimizationRemarkEmitterAnalysis>(*F);
   };
 
   BumpPtrAllocator Allocator;
   CallGraphUpdater CGUpdater;
 
   SetVector<Function *> Functions(SCC.begin(), SCC.end());
   OMPInformationCache InfoCache(M, AG, Allocator, /*CGSCC*/ Functions,
                                 OMPInModule.getKernels());
 
   Attributor A(Functions, InfoCache, CGUpdater);
 
   OpenMPOpt OMPOpt(SCC, CGUpdater, OREGetter, InfoCache, A);
   bool Changed = OMPOpt.run(true);
   if (Changed)
     return PreservedAnalyses::none();
 
   return PreservedAnalyses::all();
 }
 
 PreservedAnalyses OpenMPOptCGSCCPass::run(LazyCallGraph::SCC &C,
                                           CGSCCAnalysisManager &AM,
PreservedAnalyses OpenMPOptCGSCCPass::run(LazyCallGraph::SCC &C,
   for (LazyCallGraph::Node &N : C) {
     Function *Fn = &N.getFunction();
     SCC.push_back(Fn);
 
     // Do we already know that the SCC contains kernels,
     // or that OpenMP functions are called from this SCC?
     if (SCCIsInteresting)
       continue;
     // If not, let's check that.
     SCCIsInteresting |= OMPInModule.containsOMPRuntimeCalls(Fn);
   }
 
   if (!SCCIsInteresting || SCC.empty())
     return PreservedAnalyses::all();
 
   FunctionAnalysisManager &FAM =
       AM.getResult<FunctionAnalysisManagerCGSCCProxy>(C, CG).getManager();
 
   AnalysisGetter AG(FAM);
 
   auto OREGetter = [&FAM](Function *F) -> OptimizationRemarkEmitter & {
     return FAM.getResult<OptimizationRemarkEmitterAnalysis>(*F);
   };
 
   BumpPtrAllocator Allocator;
   CallGraphUpdater CGUpdater;
   CGUpdater.initialize(CG, C, AM, UR);
 
   SetVector<Function *> Functions(SCC.begin(), SCC.end());
   OMPInformationCache InfoCache(*(Functions.back()->getParent()), AG, Allocator,

llvm/test/Transforms/OpenMP/single_threaded_execution.ll

+; RUN: opt -passes=openmp-opt -pass-remarks-analysis=openmp-opt -openmp-identify-single-threads -disable-output %s 2>&1 | FileCheck %s
+; ModuleID = 'single_threaded_execution.c'
+%struct.ident_t = type { i32, i32, i32, i32, i8* }
+
+@x = dso_local global i32 0, align 4
+@.str = private unnamed_addr constant [4 x i8] c"%d\0A\00", align 1
+@0 = private unnamed_addr constant [40 x i8] c";single_threaded_execution.c;main;7;1;;\00", align 1
+@1 = private unnamed_addr constant %struct.ident_t { i32 0, i32 2, i32 0, i32 0, i8* getelementptr inbounds ([40 x i8], [40 x i8]* @0, i32 0, i32 0) }, align 8
+
+; Function Attrs: noinline nounwind uwtable
+define dso_local i32 @main() #0 !dbg !7 {
+entry:
+  call void (%struct.ident_t*, i32, void (i32*, i32*, ...)*, ...) @__kmpc_fork_call(%struct.ident_t* @1, i32 0, void (i32*, i32*, ...)* bitcast (void (i32*, i32*)* @.omp_outlined. to void (i32*, i32*, ...)*)), !dbg !9
+  ret i32 0, !dbg !10
+}
+
+; Function Attrs: noinline norecurse nounwind uwtable
+define internal void @.omp_outlined.(i32* noalias %.global_tid., i32* noalias %.bound_tid.) #1 !dbg !11 {
+entry:
+  %.global_tid..addr = alloca i32*, align 8
+  %.bound_tid..addr = alloca i32*, align 8
+  store i32* %.global_tid., i32** %.global_tid..addr, align 8
+  store i32* %.bound_tid., i32** %.bound_tid..addr, align 8
+  %call = call i32 @omp_get_thread_num(), !dbg !12
+  %cmp = icmp eq i32 %call, 0, !dbg !13
+  br i1 %cmp, label %if.then, label %if.end5, !dbg !12
+
+; CHECK: remark: single_threaded_execution.c:{{[0-9]+}}
+; CHECK: remark: single_threaded_execution.c:{{[0-9]+}}
+; CHECK: remark: single_threaded_execution.c:{{[0-9]+}}
+if.then:                                          ; preds = %entry
+  %0 = load i32, i32* @x, align 4, !dbg !14
+  %cmp1 = icmp eq i32 %0, 1, !dbg !15
+  br i1 %cmp1, label %if.then2, label %if.end, !dbg !14
+
+; CHECK: remark: single_threaded_execution.c:{{[0-9]+}}
+; CHECK: remark: single_threaded_execution.c:{{[0-9]+}}
+; CHECK: remark: single_threaded_execution.c:{{[0-9]+}}
+if.then2:                                         ; preds = %if.then
+  %call3 = call i32 @omp_get_thread_num(), !dbg !16
+  %call4 = call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([4 x i8], [4 x i8]* @.str, i64 0, i64 0), i32 %call3), !dbg !17
+  br label %if.end, !dbg !17
+
+; CHECK: remark: single_threaded_execution.c:{{[0-9]+}}
+if.end:                                           ; preds = %if.then2, %if.then
+  br label %if.end5, !dbg !18
+
+if.end5:                                          ; preds = %if.end, %entry
+  ret void, !dbg !19
+}
+
+declare dso_local i32 @omp_get_thread_num() #2
+
+declare dso_local i32 @printf(i8*, ...) #2
+
+; Function Attrs: nounwind
+declare !callback !20 void @__kmpc_fork_call(%struct.ident_t*, i32, void (i32*, i32*, ...)*, ...) #3
+
+attributes #0 = { noinline nounwind uwtable "frame-pointer"="all" "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" }
+attributes #1 = { noinline norecurse nounwind uwtable "frame-pointer"="all" "min-legal-vector-width"="0" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" }
+attributes #2 = { "frame-pointer"="all" "no-trapping-math"="true" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "tune-cpu"="generic" }
+attributes #3 = { nounwind }
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3, !4, !5}
+!llvm.ident = !{!6}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 13.0.0", isOptimized: false, runtimeVersion: 0, emissionKind: LineTablesOnly, enums: !2, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "single_threaded_execution.c", directory: "/home/jhuber/Documents/llvm-project/build")
+!2 = !{}
+!3 = !{i32 7, !"Dwarf Version", i32 4}
+!4 = !{i32 2, !"Debug Info Version", i32 3}
+!5 = !{i32 1, !"wchar_size", i32 4}
+!6 = !{!"clang version 13.0.0"}
+!7 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line: 6, type: !8, scopeLine: 6, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!8 = !DISubroutineType(types: !2)
+!9 = !DILocation(line: 7, column: 1, scope: !7)
+!10 = !DILocation(line: 13, column: 1, scope: !7)
+!11 = distinct !DISubprogram(name: ".omp_outlined.", scope: !1, file: !1, line: 7, type: !8, scopeLine: 7, flags: DIFlagPrototyped, spFlags: DISPFlagLocalToUnit | DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!12 = !DILocation(line: 9, column: 9, scope: !11)
+!13 = !DILocation(line: 9, column: 30, scope: !11)
+!14 = !DILocation(line: 10, column: 11, scope: !11)
+!15 = !DILocation(line: 10, column: 13, scope: !11)
+!16 = !DILocation(line: 11, column: 25, scope: !11)
+!17 = !DILocation(line: 11, column: 9, scope: !11)
+!18 = !DILocation(line: 10, column: 16, scope: !11)
+!19 = !DILocation(line: 12, column: 3, scope: !11)
+!20 = !{!21}
+!21 = !{i64 2, i64 -1, i64 -1, i1 true}