[OpenMPIRBuilder] Various changes required for tileLoops.

[Meinersbur]

Summary

Extract some changes not directly related to tileLoops out of D92974:

• Refactor createLoopSkeleton out of createCanonicalLoop.
• Introduce ComputeIP parameter to the createCanonicalLoop overload inserts instructions to compute the trip count. Specifying the location is necessary to make these instructions appear before the outermost loop of a loop nest that is tiled.
• Introduce Name parameter to createCanonicalLoop. This can help better understanding the origin of values of basic blocks with many loops. The default value is "loop" instead of "for" which could be confused with the "for directive" (aka worksharing-loop) and does not apply to Fortran.
• Remove CanonicalLoopInfo::eraseFromParent which is currently unused and untested and was added in anticipation to be used by tileLoops. eraseFromParent has shown to be insufficient when more than a single loop is involved and is replaced by removeUnusedBlocksFromParent in D92974.

[SouraVX]

LGTM Thank You @Meinersbur for taking out time :).

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

public:
   /// This generator operates on the logical iteration space of the loop, i.e.
   /// the caller only has to provide a loop trip count of the loop as defined by
   /// base language semantics. The trip count is interpreted as an unsigned
   /// integer. The induction variable passed to \p BodyGenCB will be of the same
   /// type and run from 0 to \p TripCount - 1. It is up to the callback to
   /// convert the logical iteration variable to the loop counter variable in the
   /// loop body.
   ///
-  /// \param Loc       The insert and source location description.
+  /// \param Loc       The insert and source location description. The insert
+  ///                  location can be between two instructions or the end of a
+  ///                  degenerate block (e.g. a BB under construction).
   /// \param BodyGenCB Callback that will generate the loop body code.
   /// \param TripCount Number of iterations the loop body is executed.
+  /// \param Name      Base name used to derive BB and instruction names.
   ///
   /// \returns An object representing the created control flow structure which
   ///          can be used for loop-associated directives.
   CanonicalLoopInfo *createCanonicalLoop(const LocationDescription &Loc,
                                          LoopBodyGenCallbackTy BodyGenCB,
-                                         Value *TripCount);
+                                         Value *TripCount,
+                                         const Twine &Name = "loop");
 
   /// Generator for the control flow structure of an OpenMP canonical loop.
   ///
   /// Instead of a logical iteration space, this allows specifying user-defined
   /// loop counter values using increment, upper- and lower bounds. To
   /// disambiguate the terminology when counting downwards, instead of lower
   /// bounds we use \p Start for the loop counter value in the first body
   /// iteration.
public:
   /// \param Start     Value of the loop counter for the first iterations.
   /// \param Stop      Loop counter values past this will stop the the
   ///                  iterations.
   /// \param Step      Loop counter increment after each iteration; negative
   ///                  means counting down. \param IsSigned  Whether Start, Stop
   ///                  and Stop are signed integers.
   /// \param InclusiveStop Whether  \p Stop itself is a valid value for the loop
   ///                      counter.
+  /// \param ComputeIP Insertion point for instructions computing the trip
+  ///                  count. Can be used to ensure the trip count is available
+  ///                  at the outermost loop of a loop nest. If not set,
+  ///                  defaults to the preheader of the generated loop.
+  /// \param Name      Base name used to derive BB and instruction names.
   ///
   /// \returns An object representing the created control flow structure which
   ///          can be used for loop-associated directives.
   CanonicalLoopInfo *createCanonicalLoop(const LocationDescription &Loc,
                                          LoopBodyGenCallbackTy BodyGenCB,
                                          Value *Start, Value *Stop, Value *Step,
-                                         bool IsSigned, bool InclusiveStop);
+                                         bool IsSigned, bool InclusiveStop,
+                                         InsertPointTy ComputeIP = {},
+                                         const Twine &Name = "loop");
 
   /// Modifies the canonical loop to be a statically-scheduled workshare loop.
   ///
   /// This takes a \p LoopInfo representing a canonical loop, such as the one
   /// created by \p createCanonicalLoop and emits additional instructions to
   /// turn it into a workshare loop. In particular, it calls to an OpenMP
   /// runtime function in the preheader to obtain the loop bounds to be used in
   /// the current thread, updates the relevant instructions in the canonical
Constant *getOrCreateOMPInternalVariable(Type *Ty, const Twine &Name,
                                            unsigned AddressSpace = 0);
 
   /// Returns corresponding lock object for the specified critical region
   /// name. If the lock object does not exist it is created, otherwise the
   /// reference to the existing copy is returned.
   /// \param CriticalName Name of the critical region.
   ///
   Value *getOMPCriticalRegionLock(StringRef CriticalName);
+
+  /// Create the control flow structure of a canonical OpenMP loop.
+  ///
+  /// The emitted loop will be disconnected, i.e. no edge to the loop's
+  /// preheader and no terminator in the AfterBB. The OpenMPIRBuilder's
+  /// IRBuilder location is not preserved.
+  ///
+  /// \param DL        DebugLoc used for the instructions in the skeleton.
+  /// \param TripCount Value to be used for the trip count.
+  /// \param F         Function in which to insert the BasicBlocks.
+  /// \param PreInsertBefore  Where to insert BBs that execute before the body,
+  ///                         typically the body itself.
+  /// \param PostInsertBefore Where to insert BBs that execute after the body.
+  /// \param Name      Base name used to derive BB
+  ///                  and instruction names.
+  ///
+  /// \returns The CanonicalLoopInfo that represents the emitted loop.
+  CanonicalLoopInfo *createLoopSkeleton(DebugLoc DL, Value *TripCount,
+                                        Function *F,
+                                        BasicBlock *PreInsertBefore,
+                                        BasicBlock *PostInsertBefore,
+                                        const Twine &Name = {});
 };
 
 /// Class to represented the control flow structure of an OpenMP canonical loop.
 ///
 /// The control-flow structure is standardized for easy consumption by
 /// directives associated with loops. For instance, the worksharing-loop
 /// construct may change this control flow such that each loop iteration is
 /// executed on only one thread.
private:
   BasicBlock *Preheader;
   BasicBlock *Header;
   BasicBlock *Cond;
   BasicBlock *Body;
   BasicBlock *Latch;
   BasicBlock *Exit;
   BasicBlock *After;
 
-  /// Delete this loop if unused.
-  void eraseFromParent();
-
 public:
   /// The preheader ensures that there is only a single edge entering the loop.
   /// Code that must be execute before any loop iteration can be emitted here,
   /// such as computing the loop trip count and begin lifetime markers. Code in
   /// the preheader is not considered part of the canonical loop.
   BasicBlock *getPreheader() const { return Preheader; }
 
   /// The header is the entry for each iteration. In the canonical control flow,
public:
   /// Returns the instruction representing the current logical induction
   /// variable. Always unsigned, always starting at 0 with an increment of one.
   Instruction *getIndVar() const {
     Instruction *IndVarPHI = &Header->front();
     assert(isa<PHINode>(IndVarPHI) && "First inst must be the IV PHI");
     return IndVarPHI;
   }
 
+  /// Return the insertion point for user code in the body.
+  OpenMPIRBuilder::InsertPointTy getBodyIP() const {
+    return {Body, Body->begin()};
+  };
+
   /// Return the insertion point for user code after the loop.
   OpenMPIRBuilder::InsertPointTy getAfterIP() const {
     return {After, After->begin()};
   };
 
   /// Consistency self-check.
   void assertOK() const;
 };
 
 } // end namespace llvm
 
 #endif // LLVM_IR_IRBUILDER_H

llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp

OpenMPIRBuilder::createMaster(const LocationDescription &Loc,
 
   Function *ExitRTLFn = getOrCreateRuntimeFunctionPtr(OMPRTL___kmpc_end_master);
   Instruction *ExitCall = Builder.CreateCall(ExitRTLFn, Args);
 
   return EmitOMPInlinedRegion(OMPD, EntryCall, ExitCall, BodyGenCB, FiniCB,
                               /*Conditional*/ true, /*hasFinalize*/ true);
 }
 
-CanonicalLoopInfo *
-OpenMPIRBuilder::createCanonicalLoop(const LocationDescription &Loc,
-                                     LoopBodyGenCallbackTy BodyGenCB,
-                                     Value *TripCount) {
-  BasicBlock *BB = Loc.IP.getBlock();
-  BasicBlock *NextBB = BB->getNextNode();
-  Function *F = BB->getParent();
+CanonicalLoopInfo *OpenMPIRBuilder::createLoopSkeleton(
+    DebugLoc DL, Value *TripCount, Function *F, BasicBlock *PreInsertBefore,
+    BasicBlock *PostInsertBefore, const Twine &Name) {
+  Module *M = F->getParent();
+  LLVMContext &Ctx = M->getContext();
   Type *IndVarTy = TripCount->getType();
 
   // Create the basic block structure.
   BasicBlock *Preheader =
-      BasicBlock::Create(M.getContext(), "omp_for.preheader", F, NextBB);
+      BasicBlock::Create(Ctx, "omp_" + Name + ".preheader", F, PreInsertBefore);
   BasicBlock *Header =
-      BasicBlock::Create(M.getContext(), "omp_for.header", F, NextBB);
+      BasicBlock::Create(Ctx, "omp_" + Name + ".header", F, PreInsertBefore);
   BasicBlock *Cond =
-      BasicBlock::Create(M.getContext(), "omp_for.cond", F, NextBB);
+      BasicBlock::Create(Ctx, "omp_" + Name + ".cond", F, PreInsertBefore);
   BasicBlock *Body =
-      BasicBlock::Create(M.getContext(), "omp_for.body", F, NextBB);
+      BasicBlock::Create(Ctx, "omp_" + Name + ".body", F, PreInsertBefore);
   BasicBlock *Latch =
-      BasicBlock::Create(M.getContext(), "omp_for.inc", F, NextBB);
+      BasicBlock::Create(Ctx, "omp_" + Name + ".inc", F, PostInsertBefore);
   BasicBlock *Exit =
-      BasicBlock::Create(M.getContext(), "omp_for.exit", F, NextBB);
+      BasicBlock::Create(Ctx, "omp_" + Name + ".exit", F, PostInsertBefore);
   BasicBlock *After =
-      BasicBlock::Create(M.getContext(), "omp_for.after", F, NextBB);
+      BasicBlock::Create(Ctx, "omp_" + Name + ".after", F, PostInsertBefore);
 
-  updateToLocation(Loc);
-  Builder.CreateBr(Preheader);
+  // Use specified DebugLoc for new instructions.
+  Builder.SetCurrentDebugLocation(DL);
 
   Builder.SetInsertPoint(Preheader);
   Builder.CreateBr(Header);
 
   Builder.SetInsertPoint(Header);
-  PHINode *IndVarPHI = Builder.CreatePHI(IndVarTy, 2, "omp_for.iv");
+  PHINode *IndVarPHI = Builder.CreatePHI(IndVarTy, 2, "omp_" + Name + ".iv");
   IndVarPHI->addIncoming(ConstantInt::get(IndVarTy, 0), Preheader);
   Builder.CreateBr(Cond);
 
   Builder.SetInsertPoint(Cond);
-  Value *Cmp = Builder.CreateICmpULT(IndVarPHI, TripCount, "omp_for.cmp");
+  Value *Cmp =
+      Builder.CreateICmpULT(IndVarPHI, TripCount, "omp_" + Name + ".cmp");
   Builder.CreateCondBr(Cmp, Body, Exit);
 
   Builder.SetInsertPoint(Body);
   Builder.CreateBr(Latch);
 
   Builder.SetInsertPoint(Latch);
   Value *Next = Builder.CreateAdd(IndVarPHI, ConstantInt::get(IndVarTy, 1),
-                                  "omp_for.next", /*HasNUW=*/true);
+                                  "omp_" + Name + ".next", /*HasNUW=*/true);
   Builder.CreateBr(Header);
   IndVarPHI->addIncoming(Next, Latch);
 
   Builder.SetInsertPoint(Exit);
   Builder.CreateBr(After);
 
-  // After all control flow has been created, insert the body user code.
-  BodyGenCB(InsertPointTy(Body, Body->begin()), IndVarPHI);
-
   // Remember and return the canonical control flow.
   LoopInfos.emplace_front();
   CanonicalLoopInfo *CL = &LoopInfos.front();
 
   CL->Preheader = Preheader;
   CL->Header = Header;
   CL->Cond = Cond;
   CL->Body = Body;
   CL->Latch = Latch;
   CL->Exit = Exit;
   CL->After = After;
 
   CL->IsValid = true;
 
 #ifndef NDEBUG
   CL->assertOK();
 #endif
   return CL;
 }
 
+CanonicalLoopInfo *
+OpenMPIRBuilder::createCanonicalLoop(const LocationDescription &Loc,
+                                     LoopBodyGenCallbackTy BodyGenCB,
+                                     Value *TripCount, const Twine &Name) {
+  BasicBlock *BB = Loc.IP.getBlock();
+  BasicBlock *NextBB = BB->getNextNode();
+
+  CanonicalLoopInfo *CL = createLoopSkeleton(Loc.DL, TripCount, BB->getParent(),
+                                             NextBB, NextBB, Name);
+  BasicBlock *After = CL->getAfter();
+
+  // If location is not set, don't connect the loop.
+  if (updateToLocation(Loc)) {
+    // Split the loop at the insertion point: Branch to the preheader and move
+    // every following instruction to after the loop (the After BB). Also, the
+    // new successor is the loop's after block.
+    Builder.CreateBr(CL->Preheader);
+    After->getInstList().splice(After->begin(), BB->getInstList(),
+                                Builder.GetInsertPoint(), BB->end());
+    After->replaceSuccessorsPhiUsesWith(BB, After);
+  }
+
+  // Emit the body content. We do it after connecting the loop to the CFG to
+  // avoid that the callback encounters degenerate BBs.
+  BodyGenCB(CL->getBodyIP(), CL->getIndVar());
+
+#ifndef NDEBUG
+  CL->assertOK();
+#endif
+  return CL;
+}
+
 CanonicalLoopInfo *OpenMPIRBuilder::createCanonicalLoop(
     const LocationDescription &Loc, LoopBodyGenCallbackTy BodyGenCB,
-    Value *Start, Value *Stop, Value *Step, bool IsSigned, bool InclusiveStop) {
+    Value *Start, Value *Stop, Value *Step, bool IsSigned, bool InclusiveStop,
+    InsertPointTy ComputeIP, const Twine &Name) {
+
   // Consider the following difficulties (assuming 8-bit signed integers):
   //  * Adding \p Step to the loop counter which passes \p Stop may overflow:
   //      DO I = 1, 100, 50
   ///  * A \p Step of INT_MIN cannot not be normalized to a positive direction:
   //      DO I = 100, 0, -128
 
   // Start, Stop and Step must be of the same integer type.
   auto *IndVarTy = cast<IntegerType>(Start->getType());
   assert(IndVarTy == Stop->getType() && "Stop type mismatch");
   assert(IndVarTy == Step->getType() && "Step type mismatch");
 
-  updateToLocation(Loc);
+  LocationDescription ComputeLoc =
+      ComputeIP.isSet() ? LocationDescription(ComputeIP, Loc.DL) : Loc;
+  updateToLocation(ComputeLoc);
 
   ConstantInt *Zero = ConstantInt::get(IndVarTy, 0);
   ConstantInt *One = ConstantInt::get(IndVarTy, 1);
 
   // Like Step, but always positive.
   Value *Incr = Step;
 
   // Distance between Start and Stop; always positive.
CanonicalLoopInfo *OpenMPIRBuilder::createCanonicalLoop(
   } else {
     // Avoid incrementing past stop since it could overflow.
     Value *CountIfTwo = Builder.CreateAdd(
         Builder.CreateUDiv(Builder.CreateSub(Span, One), Incr), One);
     Value *OneCmp = Builder.CreateICmp(
         InclusiveStop ? CmpInst::ICMP_ULT : CmpInst::ICMP_ULE, Span, Incr);
     CountIfLooping = Builder.CreateSelect(OneCmp, One, CountIfTwo);
   }
-  Value *TripCount = Builder.CreateSelect(ZeroCmp, Zero, CountIfLooping);
+  Value *TripCount = Builder.CreateSelect(ZeroCmp, Zero, CountIfLooping,
+                                          "omp_" + Name + ".tripcount");
 
   auto BodyGen = [=](InsertPointTy CodeGenIP, Value *IV) {
     Builder.restoreIP(CodeGenIP);
     Value *Span = Builder.CreateMul(IV, Step);
     Value *IndVar = Builder.CreateAdd(Span, Start);
     BodyGenCB(Builder.saveIP(), IndVar);
   };
-  return createCanonicalLoop(Builder.saveIP(), BodyGen, TripCount);
+  LocationDescription LoopLoc = ComputeIP.isSet() ? Loc.IP : Builder.saveIP();
+  return createCanonicalLoop(LoopLoc, BodyGen, TripCount, Name);
 }
 
 // Returns an LLVM function to call for initializing loop bounds using OpenMP
 // static scheduling depending on `type`. Only i32 and i64 are supported by the
 // runtime. Always interpret integers as unsigned similarly to
 // CanonicalLoopInfo.
 static FunctionCallee getKmpcForStaticInitForType(Type *Ty, Module &M,
                                                   OpenMPIRBuilder &OMPBuilder) {
if (NeedsBarrier)
     createBarrier(LocationDescription(Builder.saveIP(), Loc.DL),
                   omp::Directive::OMPD_for, /* ForceSimpleCall */ false,
                   /* CheckCancelFlag */ false);
 
   CLI->assertOK();
   return CLI;
 }
 
-void CanonicalLoopInfo::eraseFromParent() {
-  assert(IsValid && "can only erase previously valid loop cfg");
-  IsValid = false;
-
-  SmallVector<BasicBlock *, 5> BBsToRemove{Header, Cond, Latch, Exit};
-  SmallVector<Instruction *, 16> InstsToRemove;
-
-  // Only remove preheader if not re-purposed somewhere else.
-  if (Preheader->getNumUses() == 0)
-    BBsToRemove.push_back(Preheader);
-
-  DeleteDeadBlocks(BBsToRemove);
-}
-
 OpenMPIRBuilder::InsertPointTy
 OpenMPIRBuilder::createCopyPrivate(const LocationDescription &Loc,
                                    llvm::Value *BufSize, llvm::Value *CpyBuf,
                                    llvm::Value *CpyFn, llvm::Value *DidIt) {
   if (!updateToLocation(Loc))
     return Loc.IP;
 
   Constant *SrcLocStr = getOrCreateSrcLocStr(Loc);