[Flang][OpenMP] Add support for lastprivate clause for worksharing loop.

[arnamoy10]

Summary

This patch adds an initial support to the lastprivate clause for worksharing loop. The patch creates necessary control flow to guarantee the store of the value from the logical last iteration of the workshare loop.

[kiranchandramohan]

The test along with this patch is failing. Please have a look.

flang/lib/Lower/OpenMP.cpp

108–116

Can we use FIR/SCF/CF operations here instead of llvm dialect operations? Did you face any issue if we put the store in an scf.if/fir.if operation close to the omp.yield?

159–190

Here and else where please use MLIR coding guidelines (camelBack syntax). https://mlir.llvm.org/getting_started/DeveloperGuide/#style-guide

[arnamoy10]

1. Handling the control flow through scf.if
2. Trying to fix the test case.

[arnamoy10]

camelBack-ization!

[kiranchandramohan]

Patch looks good. The main concern remaining is the resetting and resetting of insertion points and passing state around that. If we can reduce this it will be great.

flang/lib/Lower/Bridge.cpp

515–530

Nit: remove auto.

516

Nit: https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements

547

Nit: https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements

flang/lib/Lower/OpenMP.cpp

89

Can we inline this now, that the code is not big?

116

Can you add a hard TODO for the else case?

117–121

I think the loop is not necessary. Will the following work?

firOpBuilder.setInsertionPoint(&op->region().back().back());

Also, I am assuming that you expect this to go before the Yield instruction, if so it would be good to and get the terminator explicitly and assert that it is the Yield terminator.

125

I guess, this will not work with collapse, Can you add a hard TODO for that?

457–470

Not for this patch: I think we need to clean this up a bit. Can you leave a TODO comment to clean up the setting/resetting of the insertion point?

flang/test/Lower/OpenMP/omp-parallel-lastprivate-clause-scalar.f90

2

Can you add a test with
-> a lastprivate and firstprivate?
-> multiple lastprivate clauses?

4

Nit: You can use the default CHECK prefix.
Nit: Add a driver test as well.

[kiranchandramohan]

In D130027#3668360, @kiranchandramohan wrote:

Patch looks good. The main concern remaining is the resetting and resetting of insertion points and passing state around that. If we can reduce this it will be great.

The setting and resetting can be cleaned up in a separate patch. Add a TODO for that.

Can you add a test with
-> a lastprivate and firstprivate?
-> multiple lastprivate clauses?

If these do not work then add appropriate TODOs.

Can you get this ready for submission today?

[peixin]

flang/lib/Lower/Bridge.cpp

497–498

Please add a FIXME: Generalize this function.. Ideally, this function should be generic so not to restricted only for OpenMP lastprivate lause.

flang/test/Lower/OpenMP/omp-parallel-lastprivate-clause-scalar.f90

2

Can you add a test with
-> a lastprivate and firstprivate?

You may use the code from D128595.

[arnamoy10]

Addressing all reviewers comments and adding TODOs and more test cases as per suggestion.

[kiranchandramohan]

This looks mostly good. A few more comments to address before submission.

flang/include/flang/Lower/AbstractConverter.h

64

Nit: The style in this file seems to be to use the namespace along with the type. Can you use mlir along with Block and remove this using namespace

flang/lib/Lower/OpenMP.cpp

149

Nit: spelling: presence

152

Nit: I missed this before, please use the arithmetic dialect comparison. It is probably very similar to the LLVM one.

flang/test/Lower/OpenMP/omp-parallel-lastprivate-clause-scalar.f90

173

Nit: I meant a test where the list item variable is the same. Also add a check for the barrier between the two. If we finally remove the barrier for firstprivate, the barrier when both are present should exist. If there are issues, please add a TODO.

If a list item appears in both firstprivate and lastprivate clauses, the update required for the lastprivate clause occurs after all initializations for the firstprivate clause.

A list item may not appear in more than one clause on the same directive, except that it may be specified in both firstprivate and lastprivate clauses.

[arnamoy10]

flang/test/Lower/OpenMP/omp-parallel-lastprivate-clause-scalar.f90

173

I see, thanks for the suggestion. Currently same var being firstprivate and lastprivate has issues. I will submit a separate patch to fix the issue. Adding a TODO here.

[arnamoy10]

Addressing comments before landing

[arnamoy10]

fixing clang-format

[klausler]

GCC 9.3.0 is complaining:

flang/lib/Lower/OpenMP.cpp:66:41: error: parameter 'lastPrivBlock' set but not used [-Werror=unused-but-set-parameter]
   66 |                                  Block *lastPrivBlock = nullptr) {
      |                                  ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~

[arnamoy10]

Not sure why, in line 516 you will see a use.

[shraiysh]

flang/lib/Lower/OpenMP.cpp

66

The complain is from here. The if conditions on line 77 and 79 gets evaluated at compile time and so when T is not Lastprivate, lastPrivBlock isn't used. You can add maybe_unused to handle this, example here

flang/include/flang/Lower/AbstractConverter.h

 #ifndef FORTRAN_LOWER_ABSTRACTCONVERTER_H
 #define FORTRAN_LOWER_ABSTRACTCONVERTER_H
 
 #include "flang/Common/Fortran.h"
 #include "flang/Lower/PFTDefs.h"
 #include "flang/Optimizer/Builder/BoxValue.h"
 #include "flang/Semantics/symbol.h"
 #include "mlir/IR/BuiltinOps.h"
+#include "mlir/IR/Operation.h"
 #include "llvm/ADT/ArrayRef.h"
 
 namespace fir {
 class KindMapping;
 class FirOpBuilder;
 } // namespace fir
 
 namespace fir {

 namespace lower {
 namespace pft {
 struct Variable;
 }
 
 using SomeExpr = Fortran::evaluate::Expr<Fortran::evaluate::SomeType>;
 using SymbolRef = Fortran::common::Reference<const Fortran::semantics::Symbol>;
 class StatementContext;
 
 //===----------------------------------------------------------------------===//
 // AbstractConverter interface
 //===----------------------------------------------------------------------===//
 
 /// The abstract interface for converter implementations to lower Fortran
 /// front-end fragments such as expressions, types, etc. to the FIR dialect of
 /// MLIR.
 class AbstractConverter {
public:
   /// Get the code defined by a label
   virtual pft::Evaluation *lookupLabel(pft::Label label) = 0;
 
   /// For a given symbol which is host-associated, create a clone using
   /// parameters from the host-associated symbol.
   virtual bool
   createHostAssociateVarClone(const Fortran::semantics::Symbol &sym) = 0;
 
-  virtual void copyHostAssociateVar(const Fortran::semantics::Symbol &sym) = 0;
+  virtual void copyHostAssociateVar(const Fortran::semantics::Symbol &sym,
+                                    mlir::Block *lastPrivBlock = nullptr) = 0;
 
   /// Collect the set of ultimate symbols of symbols with \p flag in \p eval
   /// region if \p isUltimateSymbol is true. Otherwise, collect the set of
   /// symbols with \p flag.
   virtual void collectSymbolSet(
       pft::Evaluation &eval,
       llvm::SetVector<const Fortran::semantics::Symbol *> &symbolSet,
       Fortran::semantics::Symbol::Flag flag, bool isUltimateSymbol = true) = 0;

flang/lib/Lower/Bridge.cpp

 #include "flang/Optimizer/Support/FIRContext.h"
 #include "flang/Optimizer/Support/FatalError.h"
 #include "flang/Optimizer/Support/InternalNames.h"
 #include "flang/Optimizer/Transforms/Passes.h"
 #include "flang/Parser/parse-tree.h"
 #include "flang/Runtime/iostat.h"
 #include "flang/Semantics/tools.h"
 #include "mlir/Dialect/ControlFlow/IR/ControlFlowOps.h"
+#include "mlir/Dialect/OpenMP/OpenMPDialect.h"
 #include "mlir/IR/PatternMatch.h"
 #include "mlir/Parser/Parser.h"
 #include "mlir/Transforms/RegionUtils.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/ErrorHandling.h"
 
 #define DEBUG_TYPE "flang-lower-bridge"
for (auto &oper : curRegion.getOps()) {
         if (oper.getOperand(ii) == oldVal) {
           oper.setOperand(ii, cloneVal);
         }
       }
     }
     return bindIfNewSymbol(sym, exv);
   }
 
+  // FIXME: Generalize this function, so that lastPrivBlock can be removed
   void
-  copyHostAssociateVar(const Fortran::semantics::Symbol &sym) override final {
+  copyHostAssociateVar(const Fortran::semantics::Symbol &sym,
+                       mlir::Block *lastPrivBlock = nullptr) override final {
     // 1) Fetch the original copy of the variable.
     assert(sym.has<Fortran::semantics::HostAssocDetails>() &&
            "No host-association found");
     const Fortran::semantics::Symbol &hsym = sym.GetUltimate();
     Fortran::lower::SymbolBox hsb = lookupOneLevelUpSymbol(hsym);
     assert(hsb && "Host symbol box not found");
     fir::ExtendedValue hexv = getExtendedValue(hsb);
 
     // 2) Fetch the copied one that will mask the original.
     Fortran::lower::SymbolBox sb = shallowLookupSymbol(sym);
     assert(sb && "Host-associated symbol box not found");
     assert(hsb.getAddr() != sb.getAddr() &&
            "Host and associated symbol boxes are the same");
     fir::ExtendedValue exv = getExtendedValue(sb);
 
     // 3) Perform the assignment.
+    mlir::OpBuilder::InsertPoint insPt = builder->saveInsertionPoint();
+    if (lastPrivBlock)
+      builder->setInsertionPointToStart(lastPrivBlock);
+    else
       builder->setInsertionPointAfter(fir::getBase(exv).getDefiningOp());
+
+    fir::ExtendedValue lhs, rhs;
+    if (lastPrivBlock) {
+      // lastprivate case
+      lhs = hexv;
+      rhs = exv;
+    } else {
+      lhs = exv;
+      rhs = hexv;
+    }
+
     mlir::Location loc = genLocation(sym.name());
     mlir::Type symType = genType(sym);
     if (auto seqTy = symType.dyn_cast<fir::SequenceType>()) {
       Fortran::lower::StatementContext stmtCtx;
-      Fortran::lower::createSomeArrayAssignment(*this, exv, hexv, localSymbols,
+      Fortran::lower::createSomeArrayAssignment(*this, lhs, rhs, localSymbols,
                                                 stmtCtx);
       stmtCtx.finalize();
     } else if (hexv.getBoxOf<fir::CharBoxValue>()) {
-      fir::factory::CharacterExprHelper{*builder, loc}.createAssign(exv, hexv);
+      fir::factory::CharacterExprHelper{*builder, loc}.createAssign(lhs, rhs);
     } else if (hexv.getBoxOf<fir::MutableBoxValue>()) {
       TODO(loc, "firstprivatisation of allocatable variables");
     } else {
-      auto loadVal = builder->create<fir::LoadOp>(loc, fir::getBase(hexv));
-      builder->create<fir::StoreOp>(loc, loadVal, fir::getBase(exv));
+      auto loadVal = builder->create<fir::LoadOp>(loc, fir::getBase(rhs));
+      builder->create<fir::StoreOp>(loc, loadVal, fir::getBase(lhs));
     }
+
+    if (lastPrivBlock)
+      builder->restoreInsertionPoint(insPt);
   }
 
   //===--------------------------------------------------------------------===//
   // Utility methods
   //===--------------------------------------------------------------------===//
 
   void collectSymbolSet(
       Fortran::lower::pft::Evaluation &eval,

flang/lib/Lower/OpenMP.cpp

 #include "flang/Lower/PFTBuilder.h"
 #include "flang/Lower/StatementContext.h"
 #include "flang/Optimizer/Builder/BoxValue.h"
 #include "flang/Optimizer/Builder/FIRBuilder.h"
 #include "flang/Optimizer/Builder/Todo.h"
 #include "flang/Parser/parse-tree.h"
 #include "flang/Semantics/tools.h"
 #include "mlir/Dialect/OpenMP/OpenMPDialect.h"
+#include "mlir/Dialect/SCF/IR/SCF.h"
 #include "llvm/Frontend/OpenMP/OMPConstants.h"
 
 using namespace mlir;
 
 int64_t Fortran::lower::getCollapseValue(
     const Fortran::parser::OmpClauseList &clauseList) {
   for (const auto &clause : clauseList.v) {
     if (const auto &collapseClause =
std::visit(Fortran::common::visitors{
                  },
                  [&](const Fortran::parser::Name &name) { sym = name.symbol; }},
              ompObject.u);
   return sym;
 }
 
 template <typename T>
 static void createPrivateVarSyms(Fortran::lower::AbstractConverter &converter,
-                                 const T *clause) {
+                                 const T *clause,
+                                 Block *lastPrivBlock = nullptr) {
   const Fortran::parser::OmpObjectList &ompObjectList = clause->v;
   for (const Fortran::parser::OmpObject &ompObject : ompObjectList.v) {
     Fortran::semantics::Symbol *sym = getOmpObjectSymbol(ompObject);
     // Privatization for symbols which are pre-determined (like loop index
     // variables) happen separately, for everything else privatize here.
     if (sym->test(Fortran::semantics::Symbol::Flag::OmpPreDetermined))
       continue;
     bool success = converter.createHostAssociateVarClone(*sym);
     (void)success;
     assert(success && "Privatization failed due to existing binding");
     if constexpr (std::is_same_v<T, Fortran::parser::OmpClause::Firstprivate>) {
       converter.copyHostAssociateVar(*sym);
+    } else if constexpr (std::is_same_v<
+                             T, Fortran::parser::OmpClause::Lastprivate>) {
+      converter.copyHostAssociateVar(*sym, lastPrivBlock);
     }
   }
 }
 
-static void privatizeVars(Fortran::lower::AbstractConverter &converter,
+template <typename Op>
+static bool privatizeVars(Op &op, Fortran::lower::AbstractConverter &converter,
                           const Fortran::parser::OmpClauseList &opClauseList) {
   fir::FirOpBuilder &firOpBuilder = converter.getFirOpBuilder();
   auto insPt = firOpBuilder.saveInsertionPoint();
   firOpBuilder.setInsertionPointToStart(firOpBuilder.getAllocaBlock());
   bool hasFirstPrivateOp = false;
+  bool hasLastPrivateOp = false;
+  Block *lastPrivBlock = nullptr;
+  // We need just one ICmpOp for multiple LastPrivate clauses.
+  mlir::arith::CmpIOp cmpOp;
+
   for (const Fortran::parser::OmpClause &clause : opClauseList.v) {
     if (const auto &privateClause =
             std::get_if<Fortran::parser::OmpClause::Private>(&clause.u)) {
       createPrivateVarSyms(converter, privateClause);
     } else if (const auto &firstPrivateClause =
                    std::get_if<Fortran::parser::OmpClause::Firstprivate>(
                        &clause.u)) {
       createPrivateVarSyms(converter, firstPrivateClause);
       hasFirstPrivateOp = true;
+    } else if (const auto &lastPrivateClause =
+                   std::get_if<Fortran::parser::OmpClause::Lastprivate>(
+                       &clause.u)) {
+      // TODO: Add lastprivate support for sections construct, simd construct
+      if (std::is_same_v<Op, omp::WsLoopOp>) {
+        omp::WsLoopOp *wsLoopOp = dyn_cast<omp::WsLoopOp>(&op);
+        fir::FirOpBuilder &firOpBuilder = converter.getFirOpBuilder();
+        auto insPt = firOpBuilder.saveInsertionPoint();
+
+        // Our goal here is to introduce the following control flow
+        // just before exiting the worksharing loop.
+        // Say our wsloop is as follows:
+        //
+        // omp.wsloop {
+        //    ...
+        //    store
+        //    omp.yield
+        // }
+        //
+        // We want to convert it to the following:
+        //
+        // omp.wsloop {
+        //    ...
+        //    store
+        //    %cmp = llvm.icmp "eq" %iv %ub
+        //    scf.if %cmp {
+        //      ^%lpv_update_blk:
+        //    }
+        //    omp.yield
+        // }
+
+        Operation *lastOper = wsLoopOp->region().back().getTerminator();
+
+        firOpBuilder.setInsertionPoint(lastOper);
+
+        // TODO: The following will not work when there is collapse present.
+        // Have to modify this in future.
+        for (const Fortran::parser::OmpClause &clause : opClauseList.v)
+          if (const auto &collapseClause =
+                  std::get_if<Fortran::parser::OmpClause::Collapse>(&clause.u))
+            TODO(converter.getCurrentLocation(),
+                 "Collapse clause with lastprivate");
+        // Only generate the compare once in presence of multiple LastPrivate
+        // clauses
+        if (!hasLastPrivateOp) {
+          cmpOp = firOpBuilder.create<mlir::arith::CmpIOp>(
+              wsLoopOp->getLoc(), mlir::arith::CmpIPredicate::eq,
+              wsLoopOp->getRegion().front().getArguments()[0],
+              wsLoopOp->upperBound()[0]);
+        }
+        mlir::scf::IfOp ifOp = firOpBuilder.create<mlir::scf::IfOp>(
+            wsLoopOp->getLoc(), cmpOp, /*else*/ false);
+
+        firOpBuilder.restoreInsertionPoint(insPt);
+        createPrivateVarSyms(converter, lastPrivateClause,
+                             &(ifOp.getThenRegion().front()));
+      } else {
+        TODO(converter.getCurrentLocation(),
+             "lastprivate clause in constructs other than work-share loop");
+      }
+      hasLastPrivateOp = true;
     }
   }
   if (hasFirstPrivateOp)
     firOpBuilder.create<mlir::omp::BarrierOp>(converter.getCurrentLocation());
   firOpBuilder.restoreInsertionPoint(insPt);
+  return hasLastPrivateOp;
 }
 
 /// The COMMON block is a global structure. \p commonValue is the base address
 /// of the the COMMON block. As the offset from the symbol \p sym, generate the
 /// COMMON block member value (commonValue + offset) for the symbol.
 /// FIXME: Share the code with `instantiateCommon` in ConvertVariable.cpp.
 static mlir::Value
 genCommonBlockMember(Fortran::lower::AbstractConverter &converter,
                      const Fortran::semantics::Symbol &sym,
                      mlir::Value commonValue) {
   auto &firOpBuilder = converter.getFirOpBuilder();
   mlir::Location currentLocation = converter.getCurrentLocation();
   mlir::IntegerType i8Ty = firOpBuilder.getIntegerType(8);
   mlir::Type i8Ptr = firOpBuilder.getRefType(i8Ty);
   mlir::Type seqTy = firOpBuilder.getRefType(firOpBuilder.getVarLenSeqTy(i8Ty));
   mlir::Value base =
       firOpBuilder.createConvert(currentLocation, seqTy, commonValue);
   std::size_t byteOffset = sym.GetUltimate().offset();
   mlir::Value offs = firOpBuilder.createIntegerConstant(
       currentLocation, firOpBuilder.getIndexType(), byteOffset);
   mlir::Value varAddr = firOpBuilder.create<fir::CoordinateOp>(
       currentLocation, i8Ptr, base, mlir::ValueRange{offs});
   mlir::Type symType = converter.genType(sym);
   return firOpBuilder.createConvert(currentLocation,
                                     firOpBuilder.getRefType(symType), varAddr);
if (eval.block) {
                "expected terminator op");
       }
     }
     if (!eval.isDirective() && eval.hasNestedEvaluations())
       createEmptyRegionBlocks(firOpBuilder, eval.getNestedEvaluations());
   }
 }
 
+void resetBeforeTerminator(fir::FirOpBuilder &firOpBuilder,
+                           mlir::Operation *storeOp, mlir::Block &block) {
+  if (storeOp)
+    firOpBuilder.setInsertionPointAfter(storeOp);
+  else
+    firOpBuilder.setInsertionPointToStart(&block);
+}
+
 /// Create the body (block) for an OpenMP Operation.
 ///
 /// \param [in]    op - the operation the body belongs to.
 /// \param [inout] converter - converter to use for the clauses.
 /// \param [in]    loc - location in source code.
 /// \param [in]    eval - current PFT node/evaluation.
 /// \oaran [in]    clauses - list of clauses to process.
 /// \param [in]    args - block arguments (induction variable[s]) for the
createBodyOfOp(Op &op, Fortran::lower::AbstractConverter &converter,
   if constexpr (std::is_same_v<Op, omp::WsLoopOp> ||
                 std::is_same_v<Op, omp::SimdLoopOp>) {
     mlir::ValueRange results;
     firOpBuilder.create<mlir::omp::YieldOp>(loc, results);
   } else {
     firOpBuilder.create<mlir::omp::TerminatorOp>(loc);
   }
 
   // Reset the insert point to before the terminator.
-  if (storeOp)
-    firOpBuilder.setInsertionPointAfter(storeOp);
-  else
-    firOpBuilder.setInsertionPointToStart(&block);
+  resetBeforeTerminator(firOpBuilder, storeOp, block);
 
   // Handle privatization. Do not privatize if this is the outer operation.
-  if (clauses && !outerCombined)
-    privatizeVars(converter, *clauses);
+  if (clauses && !outerCombined) {
+    bool lastPrivateOp = privatizeVars(op, converter, *clauses);
+    // LastPrivatization, due to introduction of
+    // new control flow, changes the insertion point,
+    // thus restore it.
+    // TODO: Clean up later a bit to avoid this many sets and resets.
+    if (lastPrivateOp)
+      resetBeforeTerminator(firOpBuilder, storeOp, block);
+  }
 
   if constexpr (std::is_same_v<Op, omp::ParallelOp>) {
     threadPrivatizeVars(converter, eval);
     if (clauses)
       genCopyinClause(converter, *clauses);
   }
 }
 
 static void genOMP(Fortran::lower::AbstractConverter &converter,

flang/test/Lower/OpenMP/omp-parallel-lastprivate-clause-scalar.f90

+! This test checks lowering of `FIRSTPRIVATE` clause for scalar types.
+! TODO: Add a test for same var being first and lastprivate when support is there.
+
+! RUN: bbc -fopenmp -emit-fir %s -o - | FileCheck %s
+! RUN: flang-new -fc1 -fopenmp -emit-fir %s -o - | FileCheck %s
+
+!CHECK: func @_QPlastprivate_character(%[[ARG1:.*]]: !fir.boxchar<1>{{.*}}) {
+!CHECK-DAG: %[[ARG1_UNBOX:.*]]:2 = fir.unboxchar
+!CHECK-DAG: %[[FIVE:.*]] = arith.constant 5 : index
+
+!CHECK: omp.parallel {
+!CHECK-DAG: %[[ARG1_PVT:.*]] = fir.alloca !fir.char<1,5> {bindc_name = "arg1", 
+
+! Check that we are accessing the clone inside the loop
+!CHECK-DAG: omp.wsloop for (%[[INDX_WS:.*]]) : {{.*}} {
+!CHECK-DAG: %[[NEG_ONE:.*]] = arith.constant -1 : i32
+!CHECK-NEXT: %[[ADDR:.*]] = fir.address_of(@_QQcl.
+!CHECK-NEXT: %[[CVT0:.*]] = fir.convert %[[ADDR]] 
+!CHECK-NEXT: %[[CNST:.*]] = arith.constant
+!CHECK-NEXT: %[[CALL_BEGIN_IO:.*]] = fir.call @_FortranAioBeginExternalListOutput(%[[NEG_ONE]], %[[CVT0]], %[[CNST]]) : (i32, !fir.ref<i8>, i32) -> !fir.ref<i8>
+!CHECK-NEXT: %[[CVT_0_1:.*]] = fir.convert %[[ARG1_PVT]] 
+!CHECK-NEXT: %[[CVT_0_2:.*]] = fir.convert %[[FIVE]]
+!CHECK-NEXT: %[[CALL_OP_ASCII:.*]] = fir.call @_FortranAioOutputAscii(%[[CALL_BEGIN_IO]], %[[CVT_0_1]], %[[CVT_0_2]])
+!CHECK-NEXT: %[[CALL_END_IO:.*]] = fir.call @_FortranAioEndIoStatement(%[[CALL_BEGIN_IO]])
+
+! Testing last iteration check
+!CHECK-NEXT: %[[IV_CMP:.*]] = arith.cmpi eq, %[[INDX_WS]]
+!CHECK: scf.if %[[IV_CMP]] {
+
+! Testing lastprivate val update
+!CHECK-DAG: %[[CVT:.*]] = fir.convert %[[ARG1_UNBOX]]#0 : (!fir.ref<!fir.char<1,?>>) -> !fir.ref<i8>
+!CHECK-DAG: %[[CVT1:.*]] = fir.convert %[[ARG1_PVT]] : (!fir.ref<!fir.char<1,5>>) -> !fir.ref<i8>
+!CHECK-DAG: fir.call @llvm.memmove.p0.p0.i64(%[[CVT]], %[[CVT1]]{{.*}})
+!CHECK: %[[THIRTY_TWO:.*]] = arith.constant 32 : i8
+!CHECK-DAG: %[[UNDEF:.*]] = fir.undefined !fir.char<1>
+!CHECK-DAG: %[[INSERT:.*]] = fir.insert_value %[[UNDEF]], %[[THIRTY_TWO]], [0 : index] : (!fir.char<1>, i8) -> !fir.char<1>
+!CHECK-DAG: %[[ONE_3:.*]] = arith.constant 1 : index
+!CHECK: fir.do_loop %[[ARG2:.*]] = {{.*}} {
+!CHECK-DAG: %[[CVT_2:.*]] = fir.convert %[[ARG1_UNBOX]]#0 : (!fir.ref<!fir.char<1,?>>) -> !fir.ref<!fir.array<?x!fir.char<1>>>
+!CHECK-DAG: %[[COORD:.*]] = fir.coordinate_of %[[CVT_2]], %[[ARG2]] : (!fir.ref<!fir.array<?x!fir.char<1>>>, index) -> !fir.ref<!fir.char<1>>
+!CHECK-DAG: fir.store %[[INSERT]] to %[[COORD]] : !fir.ref<!fir.char<1>>
+!CHECK-DAG: }
+!CHECK-DAG: } 
+!CHECK-DAG: omp.yield
+
+subroutine lastprivate_character(arg1)
+        character(5) :: arg1
+!$OMP PARALLEL 
+!$OMP DO LASTPRIVATE(arg1)
+do n = 1, 5
+        arg1(n:n) = 'c'
+        print *, arg1
+end do
+!$OMP END DO
+!$OMP END PARALLEL
+end subroutine
+
+!CHECK: func @_QPlastprivate_int(%[[ARG1:.*]]: !fir.ref<i32> {fir.bindc_name = "arg1"}) {
+!CHECK-DAG: omp.parallel  {
+!CHECK-DAG: %[[CLONE:.*]] = fir.alloca i32 {bindc_name = "arg1"
+!CHECK: omp.wsloop for (%[[INDX_WS:.*]]) : {{.*}} {
+
+! Testing last iteration check
+!CHECK-DAG: %[[IV_CMP:.*]] = arith.cmpi eq, %[[INDX_WS]]
+!CHECK-DAG: scf.if %[[IV_CMP]] {
+
+! Testing lastprivate val update
+!CHECK-NEXT: %[[CLONE_LD:.*]] = fir.load %[[CLONE]] : !fir.ref<i32>
+!CHECK-NEXT: fir.store %[[CLONE_LD]] to %[[ARG1]] : !fir.ref<i32>
+!CHECK-DAG: }
+!CHECK-DAG: omp.yield
+
+subroutine lastprivate_int(arg1)
+        integer :: arg1
+!$OMP PARALLEL 
+!$OMP DO LASTPRIVATE(arg1)
+do n = 1, 5
+        arg1 = 2
+        print *, arg1
+end do
+!$OMP END DO
+!$OMP END PARALLEL
+print *, arg1
+end subroutine
+
+!CHECK: func.func @_QPmult_lastprivate_int(%[[ARG1:.*]]: !fir.ref<i32> {fir.bindc_name = "arg1"}, %[[ARG2:.*]]: !fir.ref<i32> {fir.bindc_name = "arg2"}) {
+!CHECK-DAG: omp.parallel  {
+!CHECK-DAG: %[[CLONE1:.*]] = fir.alloca i32 {bindc_name = "arg1"
+!CHECK-DAG: %[[CLONE2:.*]] = fir.alloca i32 {bindc_name = "arg2"
+!CHECK: omp.wsloop for (%[[INDX_WS:.*]]) : {{.*}} {
+
+! Testing last iteration check
+!CHECK-DAG: %[[IV_CMP1:.*]] = arith.cmpi eq, %[[INDX_WS]]
+!CHECK-DAG: scf.if %[[IV_CMP1]] {
+! Testing lastprivate val update
+!CHECK-NEXT: %[[CLONE_LD1:.*]] = fir.load %[[CLONE1]] : !fir.ref<i32>
+!CHECK-NEXT: fir.store %[[CLONE_LD1]] to %[[ARG1]] : !fir.ref<i32>
+!CHECK-DAG: }
+!CHECK-DAG: scf.if %[[IV_CMP1]] {
+! Testing lastprivate val update
+!CHECK-NEXT: %[[CLONE_LD2:.*]] = fir.load %[[CLONE2]] : !fir.ref<i32>
+!CHECK-NEXT: fir.store %[[CLONE_LD2]] to %[[ARG2]] : !fir.ref<i32>
+!CHECK-DAG: }
+!CHECK-DAG: omp.yield
+
+subroutine mult_lastprivate_int(arg1, arg2)
+        integer :: arg1, arg2
+!$OMP PARALLEL 
+!$OMP DO LASTPRIVATE(arg1) LASTPRIVATE(arg2)
+do n = 1, 5
+        arg1 = 2
+        arg2 = 3
+        print *, arg1, arg2
+end do
+!$OMP END DO
+!$OMP END PARALLEL
+print *, arg1, arg2
+end subroutine
+
+!CHECK: func.func @_QPmult_lastprivate_int2(%[[ARG1:.*]]: !fir.ref<i32> {fir.bindc_name = "arg1"}, %[[ARG2:.*]]: !fir.ref<i32> {fir.bindc_name = "arg2"}) {
+!CHECK-DAG: omp.parallel  {
+!CHECK-DAG: %[[CLONE1:.*]] = fir.alloca i32 {bindc_name = "arg1"
+!CHECK-DAG: %[[CLONE2:.*]] = fir.alloca i32 {bindc_name = "arg2"
+!CHECK: omp.wsloop for (%[[INDX_WS:.*]]) : {{.*}} {
+
+! Testing last iteration check
+!CHECK-DAG: %[[IV_CMP1:.*]] = arith.cmpi eq, %[[INDX_WS]]
+!CHECK-DAG: scf.if %[[IV_CMP1]] {
+! Testing lastprivate val update
+!CHECK-NEXT: %[[CLONE_LD2:.*]] = fir.load %[[CLONE2]] : !fir.ref<i32>
+!CHECK-NEXT: fir.store %[[CLONE_LD2]] to %[[ARG2]] : !fir.ref<i32>
+!CHECK-NEXT: %[[CLONE_LD1:.*]] = fir.load %[[CLONE1]] : !fir.ref<i32>
+!CHECK-NEXT: fir.store %[[CLONE_LD1]] to %[[ARG1]] : !fir.ref<i32>
+!CHECK-NEXT: }
+!CHECK-NEXT: omp.yield
+
+subroutine mult_lastprivate_int2(arg1, arg2)
+        integer :: arg1, arg2
+!$OMP PARALLEL 
+!$OMP DO LASTPRIVATE(arg1, arg2)
+do n = 1, 5
+        arg1 = 2
+        arg2 = 3
+        print *, arg1, arg2
+end do
+!$OMP END DO
+!$OMP END PARALLEL
+print *, arg1, arg2
+end subroutine
+
+!CHECK: func.func @_QPfirstpriv_lastpriv_int(%[[ARG1:.*]]: !fir.ref<i32> {fir.bindc_name = "arg1"}, %[[ARG2:.*]]: !fir.ref<i32> {fir.bindc_name = "arg2"}) {
+!CHECK-DAG: omp.parallel  {
+!CHECK-DAG: %[[CLONE1:.*]] = fir.alloca i32 {bindc_name = "arg1"
+! Firstprivate update
+!CHECK-NEXT: %[[FPV_LD:.*]] = fir.load %[[ARG1]] : !fir.ref<i32>
+!CHECK-NEXT: fir.store %[[FPV_LD]] to %[[CLONE1]] : !fir.ref<i32>
+! Lastprivate Allocation
+!CHECK-NEXT: %[[CLONE2:.*]] = fir.alloca i32 {bindc_name = "arg2"
+!CHECK-NEXT: omp.barrier
+!CHECK: omp.wsloop for (%[[INDX_WS:.*]]) : {{.*}} {
+
+! Testing last iteration check
+!CHECK-DAG: %[[IV_CMP1:.*]] = arith.cmpi eq, %[[INDX_WS]]
+!CHECK-DAG: scf.if %[[IV_CMP1]] {
+! Testing lastprivate val update
+!CHECK-NEXT: %[[CLONE_LD:.*]] = fir.load %[[CLONE2]] : !fir.ref<i32>
+!CHECK-NEXT: fir.store %[[CLONE_LD]] to %[[ARG2]] : !fir.ref<i32>
+!CHECK-NEXT: }
+!CHECK-NEXT: omp.yield
+
+subroutine firstpriv_lastpriv_int(arg1, arg2)
+        integer :: arg1, arg2
+!$OMP PARALLEL 
+!$OMP DO FIRSTPRIVATE(arg1) LASTPRIVATE(arg2)
+do n = 1, 5
+        arg1 = 2
+        arg2 = 3
+        print *, arg1, arg2
+end do
+!$OMP END DO
+!$OMP END PARALLEL
+print *, arg1, arg2
+end subroutine
+
+