[OPENMP] Loop collapsing and initial codegen for 'omp simd' directive

[amusman]

Summary

This patch implements collapsing of the loops (in particular, in presense of clause 'collapse'). It calculates number of iterations N and expressions nesessary to calculate the nested loops counters values based on new iteration variable (that goes from 0 to N-1) in Sema. It also adds Codegen for 'omp simd', which uses (and tests) this feature.

[amusman]

Ping.

[amusman]

Ping.

[hfinkel]

lib/CodeGen/CGStmtOpenMP.cpp

67 ↗

(On Diff #14071)

Is there something missing for you to support this? Looks like everything is in place below.

lib/Sema/SemaOpenMP.cpp

2534 ↗

(On Diff #14071)

The OpenMP specification says:

The integer type (or kind, for Fortran) used to compute the iteration count for the  collapsed loop is implementation defined.

And I'm afraid that, especially if the iteration variables are originally 32-bits, with the current scheme we'll overflow when calculating the total number of iterations. Can we always use a larger integer type?

[amusman]

Hal,

I updated the code to make the number of iterations 64-bit if we cannot prove that 32 bits would be enough and added a test with such example.
I'm not sure if it's worth to do widening to 128 bits in similar cases with several nested loops with 64-bit counters - in particular, in case of worksharing loop directives the library only supports 32-bit and 64-bit counters, so we'll need truncate it to 64 bits for such directives anyway.

Regarding 'lastprivate' codegen at: lib/CodeGen/CGStmtOpenMP.cpp:67 -- this will need helper expressions in OMPLastprivateClause (e.g., Copy Constructor to copy from private var to the non-private), this would be better to do in a separate patch.

Thanks,
Alexander

[ABataev]

There will be a separate patch for lastprivate variables - the codegen for lastprivates is very similar to codegen for privates and firstprivates. I'll prepare it as soon as codegen for privates and firstprivates is approved and committed.

[hfinkel]

LGTM, thanks!

Regarding 'lastprivate' codegen at: lib/CodeGen/CGStmtOpenMP.cpp:67 -- this will need helper expressions in OMPLastprivateClause (e.g., Copy Constructor to copy from private var to the non-private), this would be better to do in a separate patch.

Okay, please make this clear in the comment (and please add an assert, to be removed later, if the code will be functionally-incorrect without this logic).

lib/Sema/SemaOpenMP.cpp

2597 ↗

(On Diff #14210)

Add "the" before 32-bit

[amusman]

Closed by commit rL218743 (authored by @amusman).

cfe/trunk/include/clang/AST/Stmt.h

public:
   /// viewAST - Visualize an AST rooted at this Stmt* using GraphViz.  Only
   ///   works on systems with GraphViz (Mac OS X) or dot+gv installed.
   void viewAST() const;
 
   /// Skip past any implicit AST nodes which might surround this
   /// statement, such as ExprWithCleanups or ImplicitCastExpr nodes.
   Stmt *IgnoreImplicit();
 
+  /// \brief Skip no-op (attributed, compound) container stmts and skip captured
+  /// stmt at the top, if \a IgnoreCaptured is true.
+  Stmt *IgnoreContainers(bool IgnoreCaptured = false);
+
   const Stmt *stripLabelLikeStatements() const;
   Stmt *stripLabelLikeStatements() {
     return const_cast<Stmt*>(
       const_cast<const Stmt*>(this)->stripLabelLikeStatements());
   }
 
   /// Child Iterators: All subclasses must implement 'children'
   /// to permit easy iteration over the substatements/subexpessions of an

cfe/trunk/include/clang/AST/StmtOpenMP.h

 /// \brief This is a common base class for loop directives ('omp simd', 'omp
 /// for', 'omp for simd' etc.). It is responsible for the loop code generation.
 ///
 class OMPLoopDirective : public OMPExecutableDirective {
   friend class ASTStmtReader;
   /// \brief Number of collapsed loops as specified by 'collapse' clause.
   unsigned CollapsedNum;
 
+  /// \brief Offsets to the stored exprs.
+  enum {
+    AssociatedStmtOffset = 0,
+    IterationVariableOffset = 1,
+    LastIterationOffset = 2,
+    CalcLastIterationOffset = 3,
+    PreConditionOffset = 4,
+    CondOffset = 5,
+    SeparatedCondOffset = 6,
+    InitOffset = 7,
+    IncOffset = 8,
+    ArraysOffset = 9
+  };
+
+  /// \brief Get the counters storage.
+  MutableArrayRef<Expr *> getCounters() {
+    Expr **Storage =
+        reinterpret_cast<Expr **>(&(*(std::next(child_begin(), ArraysOffset))));
+    return MutableArrayRef<Expr *>(Storage, CollapsedNum);
+  }
+
+  /// \brief Get the updates storage.
+  MutableArrayRef<Expr *> getUpdates() {
+    Expr **Storage = reinterpret_cast<Expr **>(
+        &*std::next(child_begin(), ArraysOffset + CollapsedNum));
+    return MutableArrayRef<Expr *>(Storage, CollapsedNum);
+  }
+
+  /// \brief Get the final counter updates storage.
+  MutableArrayRef<Expr *> getFinals() {
+    Expr **Storage = reinterpret_cast<Expr **>(
+        &*std::next(child_begin(), ArraysOffset + 2 * CollapsedNum));
+    return MutableArrayRef<Expr *>(Storage, CollapsedNum);
+  }
+
 protected:
   /// \brief Build instance of loop directive of class \a Kind.
   ///
   /// \param SC Statement class.
   /// \param Kind Kind of OpenMP directive.
   /// \param StartLoc Starting location of the directive (directive keyword).
   /// \param EndLoc Ending location of the directive.
   /// \param CollapsedNum Number of collapsed loops from 'collapse' clause.
   /// \param NumClauses Number of clauses.
   /// \param NumSpecialChildren Number of additional directive-specific stmts.
   ///
   template <typename T>
   OMPLoopDirective(const T *That, StmtClass SC, OpenMPDirectiveKind Kind,
                    SourceLocation StartLoc, SourceLocation EndLoc,
                    unsigned CollapsedNum, unsigned NumClauses,
                    unsigned NumSpecialChildren = 0)
       : OMPExecutableDirective(That, SC, Kind, StartLoc, EndLoc, NumClauses,
-                               1 + NumSpecialChildren),
+                               numLoopChildren(CollapsedNum) +
+                                   NumSpecialChildren),
         CollapsedNum(CollapsedNum) {}
 
+  /// \brief Children number.
+  static unsigned numLoopChildren(unsigned CollapsedNum) {
+    return ArraysOffset + 3 * CollapsedNum; // Counters, Updates and Finals
+  }
+
+  void setIterationVariable(Expr *IV) {
+    *std::next(child_begin(), IterationVariableOffset) = IV;
+  }
+  void setLastIteration(Expr *LI) {
+    *std::next(child_begin(), LastIterationOffset) = LI;
+  }
+  void setCalcLastIteration(Expr *CLI) {
+    *std::next(child_begin(), CalcLastIterationOffset) = CLI;
+  }
+  void setPreCond(Expr *PC) {
+    *std::next(child_begin(), PreConditionOffset) = PC;
+  }
+  void setCond(Expr *Cond, Expr *SeparatedCond) {
+    *std::next(child_begin(), CondOffset) = Cond;
+    *std::next(child_begin(), SeparatedCondOffset) = SeparatedCond;
+  }
+  void setInit(Expr *Init) { *std::next(child_begin(), InitOffset) = Init; }
+  void setInc(Expr *Inc) { *std::next(child_begin(), IncOffset) = Inc; }
+  void setCounters(ArrayRef<Expr *> A);
+  void setUpdates(ArrayRef<Expr *> A);
+  void setFinals(ArrayRef<Expr *> A);
+
 public:
   /// \brief Get number of collapsed loops.
   unsigned getCollapsedNumber() const { return CollapsedNum; }
 
+  Expr *getIterationVariable() const {
+    return const_cast<Expr *>(reinterpret_cast<const Expr *>(
+        *std::next(child_begin(), IterationVariableOffset)));
+  }
+  Expr *getLastIteration() const {
+    return const_cast<Expr *>(reinterpret_cast<const Expr *>(
+        *std::next(child_begin(), LastIterationOffset)));
+  }
+  Expr *getCalcLastIteration() const {
+    return const_cast<Expr *>(reinterpret_cast<const Expr *>(
+        *std::next(child_begin(), CalcLastIterationOffset)));
+  }
+  Expr *getPreCond() const {
+    return const_cast<Expr *>(reinterpret_cast<const Expr *>(
+        *std::next(child_begin(), PreConditionOffset)));
+  }
+  Expr *getCond(bool SeparateIter) const {
+    return const_cast<Expr *>(reinterpret_cast<const Expr *>(
+        *std::next(child_begin(),
+                   (SeparateIter ? SeparatedCondOffset : CondOffset))));
+  }
+  Expr *getInit() const {
+    return const_cast<Expr *>(
+        reinterpret_cast<const Expr *>(*std::next(child_begin(), InitOffset)));
+  }
+  Expr *getInc() const {
+    return const_cast<Expr *>(
+        reinterpret_cast<const Expr *>(*std::next(child_begin(), IncOffset)));
+  }
+  const Stmt *getBody() const {
+    // This relies on the loop form is already checked by Sema.
+    Stmt *Body = getAssociatedStmt()->IgnoreContainers(true);
+    Body = cast<ForStmt>(Body)->getBody();
+    for (unsigned Cnt = 1; Cnt < CollapsedNum; ++Cnt) {
+      Body = Body->IgnoreContainers();
+      Body = cast<ForStmt>(Body)->getBody();
+    }
+    return Body;
+  }
+
+  ArrayRef<Expr *> counters() { return getCounters(); }
+
+  ArrayRef<Expr *> counters() const {
+    return const_cast<OMPLoopDirective *>(this)->getCounters();
+  }
+
+  ArrayRef<Expr *> updates() { return getUpdates(); }
+
+  ArrayRef<Expr *> updates() const {
+    return const_cast<OMPLoopDirective *>(this)->getUpdates();
+  }
+
+  ArrayRef<Expr *> finals() { return getFinals(); }
+
+  ArrayRef<Expr *> finals() const {
+    return const_cast<OMPLoopDirective *>(this)->getFinals();
+  }
+
   static bool classof(const Stmt *T) {
     return T->getStmtClass() == OMPSimdDirectiveClass ||
            T->getStmtClass() == OMPForDirectiveClass ||
            T->getStmtClass() == OMPForSimdDirectiveClass ||
            T->getStmtClass() == OMPParallelForDirectiveClass ||
            T->getStmtClass() == OMPParallelForSimdDirectiveClass;
   }
 };
public:
   /// \brief Creates directive with a list of \a Clauses.
   ///
   /// \param C AST context.
   /// \param StartLoc Starting location of the directive kind.
   /// \param EndLoc Ending Location of the directive.
   /// \param CollapsedNum Number of collapsed loops.
   /// \param Clauses List of clauses.
   /// \param AssociatedStmt Statement, associated with the directive.
+  /// \param IV Loop iteration variable for CodeGen.
+  /// \param LastIteration Loop last iteration number for CodeGen.
+  /// \param CalcLastIteration Calculation of last iteration.
+  /// \param PreCond Pre-condition.
+  /// \param Cond Condition.
+  /// \param SeparatedCond Condition with 1 iteration separated.
+  /// \param Inc Loop increment.
+  /// \param Counters Loop counters.
+  /// \param Updates Expressions for loop counters update for CodeGen.
+  /// \param Finals Final loop counter values for GodeGen.
   ///
-  static OMPSimdDirective *Create(const ASTContext &C, SourceLocation StartLoc,
-                                  SourceLocation EndLoc, unsigned CollapsedNum,
-                                  ArrayRef<OMPClause *> Clauses,
-                                  Stmt *AssociatedStmt);
+  static OMPSimdDirective *
+  Create(const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
+         unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses,
+         Stmt *AssociatedStmt, Expr *IV, Expr *LastIteration,
+         Expr *CalcLastIteration, Expr *PreCond, Expr *Cond,
+         Expr *SeparatedCond, Expr *Init, Expr *Inc, ArrayRef<Expr *> Counters,
+         ArrayRef<Expr *> Updates, ArrayRef<Expr *> Finals);
 
   /// \brief Creates an empty directive with the place
   /// for \a NumClauses clauses.
   ///
   /// \param C AST context.
   /// \param CollapsedNum Number of collapsed nested loops.
   /// \param NumClauses Number of clauses.
   ///
public:
   /// \brief Creates directive with a list of \a Clauses.
   ///
   /// \param C AST context.
   /// \param StartLoc Starting location of the directive kind.
   /// \param EndLoc Ending Location of the directive.
   /// \param CollapsedNum Number of collapsed loops.
   /// \param Clauses List of clauses.
   /// \param AssociatedStmt Statement, associated with the directive.
+  /// \param IV Loop iteration variable for CodeGen.
+  /// \param LastIteration Loop last iteration number for CodeGen.
+  /// \param CalcLastIteration Calculation of last iteration.
+  /// \param PreCond Pre-condition.
+  /// \param Cond Condition.
+  /// \param SeparatedCond Condition with 1 iteration separated.
+  /// \param Inc Loop increment.
+  /// \param Counters Loop counters.
+  /// \param Updates Expressions for loop counters update for CodeGen.
+  /// \param Finals Final loop counter values for GodeGen.
   ///
-  static OMPForDirective *Create(const ASTContext &C, SourceLocation StartLoc,
-                                 SourceLocation EndLoc, unsigned CollapsedNum,
-                                 ArrayRef<OMPClause *> Clauses,
-                                 Stmt *AssociatedStmt);
+  static OMPForDirective *
+  Create(const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
+         unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses,
+         Stmt *AssociatedStmt, Expr *IV, Expr *LastIteration,
+         Expr *CalcLastIteration, Expr *PreCond, Expr *Cond,
+         Expr *SeparatedCond, Expr *Init, Expr *Inc, ArrayRef<Expr *> Counters,
+         ArrayRef<Expr *> Updates, ArrayRef<Expr *> Finals);
 
   /// \brief Creates an empty directive with the place
   /// for \a NumClauses clauses.
   ///
   /// \param C AST context.
   /// \param CollapsedNum Number of collapsed nested loops.
   /// \param NumClauses Number of clauses.
   ///
public:
   /// \brief Creates directive with a list of \a Clauses.
   ///
   /// \param C AST context.
   /// \param StartLoc Starting location of the directive kind.
   /// \param EndLoc Ending Location of the directive.
   /// \param CollapsedNum Number of collapsed loops.
   /// \param Clauses List of clauses.
   /// \param AssociatedStmt Statement, associated with the directive.
+  /// \param IV Loop iteration variable for CodeGen.
+  /// \param LastIteration Loop last iteration number for CodeGen.
+  /// \param CalcLastIteration Calculation of last iteration.
+  /// \param PreCond Pre-condition.
+  /// \param Cond Condition.
+  /// \param SeparatedCond Condition with 1 iteration separated.
+  /// \param Inc Loop increment.
+  /// \param Counters Loop counters.
+  /// \param Updates Expressions for loop counters update for CodeGen.
+  /// \param Finals Final loop counter values for GodeGen.
   ///
   static OMPForSimdDirective *
   Create(const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
          unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses,
-         Stmt *AssociatedStmt);
+         Stmt *AssociatedStmt, Expr *IV, Expr *LastIteration,
+         Expr *CalcLastIteration, Expr *PreCond, Expr *Cond,
+         Expr *SeparatedCond, Expr *Init, Expr *Inc, ArrayRef<Expr *> Counters,
+         ArrayRef<Expr *> Updates, ArrayRef<Expr *> Finals);
 
   /// \brief Creates an empty directive with the place
   /// for \a NumClauses clauses.
   ///
   /// \param C AST context.
   /// \param CollapsedNum Number of collapsed nested loops.
   /// \param NumClauses Number of clauses.
   ///
public:
   /// \brief Creates directive with a list of \a Clauses.
   ///
   /// \param C AST context.
   /// \param StartLoc Starting location of the directive kind.
   /// \param EndLoc Ending Location of the directive.
   /// \param CollapsedNum Number of collapsed loops.
   /// \param Clauses List of clauses.
   /// \param AssociatedStmt Statement, associated with the directive.
+  /// \param IV Loop iteration variable for CodeGen.
+  /// \param LastIteration Loop last iteration number for CodeGen.
+  /// \param CalcLastIteration Calculation of last iteration.
+  /// \param PreCond Pre-condition.
+  /// \param Cond Condition.
+  /// \param SeparatedCond Condition with 1 iteration separated.
+  /// \param Inc Loop increment.
+  /// \param Counters Loop counters.
+  /// \param Updates Expressions for loop counters update for CodeGen.
+  /// \param Finals Final loop counter values for GodeGen.
   ///
   static OMPParallelForDirective *
   Create(const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
          unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses,
-         Stmt *AssociatedStmt);
+         Stmt *AssociatedStmt, Expr *IV, Expr *LastIteration,
+         Expr *CalcLastIteration, Expr *PreCond, Expr *Cond,
+         Expr *SeparatedCond, Expr *Init, Expr *Inc, ArrayRef<Expr *> Counters,
+         ArrayRef<Expr *> Updates, ArrayRef<Expr *> Finals);
 
   /// \brief Creates an empty directive with the place
   /// for \a NumClauses clauses.
   ///
   /// \param C AST context.
   /// \param CollapsedNum Number of collapsed nested loops.
   /// \param NumClauses Number of clauses.
   ///
public:
   /// \brief Creates directive with a list of \a Clauses.
   ///
   /// \param C AST context.
   /// \param StartLoc Starting location of the directive kind.
   /// \param EndLoc Ending Location of the directive.
   /// \param CollapsedNum Number of collapsed loops.
   /// \param Clauses List of clauses.
   /// \param AssociatedStmt Statement, associated with the directive.
+  /// \param IV Loop iteration variable for CodeGen.
+  /// \param LastIteration Loop last iteration number for CodeGen.
+  /// \param CalcLastIteration Calculation of last iteration.
+  /// \param PreCond Pre-condition.
+  /// \param Cond Condition.
+  /// \param SeparatedCond Condition with 1 iteration separated.
+  /// \param Inc Loop increment.
+  /// \param Counters Loop counters.
+  /// \param Updates Expressions for loop counters update for CodeGen.
+  /// \param Finals Final loop counter values for GodeGen.
   ///
   static OMPParallelForSimdDirective *
   Create(const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
          unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses,
-         Stmt *AssociatedStmt);
+         Stmt *AssociatedStmt, Expr *IV, Expr *LastIteration,
+         Expr *CalcLastIteration, Expr *PreCond, Expr *Cond,
+         Expr *SeparatedCond, Expr *Init, Expr *Inc, ArrayRef<Expr *> Counters,
+         ArrayRef<Expr *> Updates, ArrayRef<Expr *> Finals);
 
   /// \brief Creates an empty directive with the place
   /// for \a NumClauses clauses.
   ///
   /// \param C AST context.
   /// \param CollapsedNum Number of collapsed nested loops.
   /// \param NumClauses Number of clauses.
   ///

cfe/trunk/include/clang/Basic/DiagnosticSemaKinds.td

def err_omp_loop_not_canonical_incr : Error<
   "or subtraction on loop variable %0">;
 def err_omp_loop_variable_type : Error<
   "variable must be of integer or %select{pointer|random access iterator}0 type">;
 def err_omp_loop_incr_not_compatible : Error<
   "increment expression must cause %0 to %select{decrease|increase}1 "
   "on each iteration of OpenMP for loop">;
 def note_omp_loop_cond_requres_compatible_incr : Note<
   "loop step is expected to be %select{negative|positive}0 due to this condition">;
+def err_omp_loop_diff_cxx : Error<
+  "could not calculate number of iterations calling 'operator-' with "
+  "upper and lower loop bounds">;
 def err_omp_loop_cannot_use_stmt : Error<
   "'%0' statement cannot be used in OpenMP for loop">;
 def err_omp_simd_region_cannot_use_stmt : Error<
   "'%0' statement cannot be used in OpenMP simd region">;
 def err_omp_unknown_reduction_identifier : Error<
   "incorrect reduction identifier, expected one of '+', '-', '*', '&', '|', '^', '&&', '||', 'min' or 'max'">;
 def err_omp_reduction_type_array : Error<
   "a reduction variable with array type %0">;

cfe/trunk/lib/AST/Stmt.cpp

if (ExprWithCleanups *ewc = dyn_cast<ExprWithCleanups>(s))
     s = ewc->getSubExpr();
 
   while (ImplicitCastExpr *ice = dyn_cast<ImplicitCastExpr>(s))
     s = ice->getSubExpr();
 
   return s;
 }
 
+/// \brief Skip no-op (attributed, compound) container stmts and skip captured
+/// stmt at the top, if \a IgnoreCaptured is true.
+Stmt *Stmt::IgnoreContainers(bool IgnoreCaptured) {
+  Stmt *S = this;
+  if (IgnoreCaptured)
+    if (auto CapS = dyn_cast_or_null<CapturedStmt>(S))
+      S = CapS->getCapturedStmt();
+  while (true) {
+    if (auto AS = dyn_cast_or_null<AttributedStmt>(S))
+      S = AS->getSubStmt();
+    else if (auto CS = dyn_cast_or_null<CompoundStmt>(S)) {
+      if (CS->size() != 1)
+        break;
+      S = CS->body_back();
+    } else
+      break;
+  }
+  return S;
+}
+
 /// \brief Strip off all label-like statements.
 ///
 /// This will strip off label statements, case statements, attributed
 /// statements and default statements recursively.
 const Stmt *Stmt::stripLabelLikeStatements() const {
   const Stmt *S = this;
   while (true) {
     if (const LabelStmt *LS = dyn_cast<LabelStmt>(S))

 }
 
 void OMPExecutableDirective::setClauses(ArrayRef<OMPClause *> Clauses) {
   assert(Clauses.size() == getNumClauses() &&
          "Number of clauses is not the same as the preallocated buffer");
   std::copy(Clauses.begin(), Clauses.end(), getClauses().begin());
 }
 
+void OMPLoopDirective::setCounters(ArrayRef<Expr *> A) {
+  assert(A.size() == getCollapsedNumber() &&
+         "Number of loop counters is not the same as the collapsed number");
+  std::copy(A.begin(), A.end(), getCounters().begin());
+}
+
+void OMPLoopDirective::setUpdates(ArrayRef<Expr *> A) {
+  assert(A.size() == getCollapsedNumber() &&
+         "Number of counter updates is not the same as the collapsed number");
+  std::copy(A.begin(), A.end(), getUpdates().begin());
+}
+
+void OMPLoopDirective::setFinals(ArrayRef<Expr *> A) {
+  assert(A.size() == getCollapsedNumber() &&
+         "Number of counter finals is not the same as the collapsed number");
+  std::copy(A.begin(), A.end(), getFinals().begin());
+}
+
 OMPReductionClause *OMPReductionClause::Create(
     const ASTContext &C, SourceLocation StartLoc, SourceLocation LParenLoc,
     SourceLocation EndLoc, SourceLocation ColonLoc, ArrayRef<Expr *> VL,
     NestedNameSpecifierLoc QualifierLoc, const DeclarationNameInfo &NameInfo) {
   void *Mem = C.Allocate(llvm::RoundUpToAlignment(sizeof(OMPReductionClause),
                                                   llvm::alignOf<Expr *>()) +
                          sizeof(Expr *) * VL.size());
   OMPReductionClause *Clause = new (Mem) OMPReductionClause(
OMPParallelDirective *OMPParallelDirective::CreateEmpty(const ASTContext &C,
   void *Mem = C.Allocate(Size + sizeof(OMPClause *) * NumClauses +
                          sizeof(Stmt *));
   return new (Mem) OMPParallelDirective(NumClauses);
 }
 
 OMPSimdDirective *
 OMPSimdDirective::Create(const ASTContext &C, SourceLocation StartLoc,
                          SourceLocation EndLoc, unsigned CollapsedNum,
-                         ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt) {
+                         ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt,
+                         Expr *IV, Expr *LastIteration, Expr *CalcLastIteration,
+                         Expr *PreCond, Expr *Cond, Expr *SeparatedCond,
+                         Expr *Init, Expr *Inc, ArrayRef<Expr *> Counters,
+                         ArrayRef<Expr *> Updates, ArrayRef<Expr *> Finals) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPSimdDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   OMPSimdDirective *Dir = new (Mem)
       OMPSimdDirective(StartLoc, EndLoc, CollapsedNum, Clauses.size());
   Dir->setClauses(Clauses);
   Dir->setAssociatedStmt(AssociatedStmt);
+  Dir->setIterationVariable(IV);
+  Dir->setLastIteration(LastIteration);
+  Dir->setCalcLastIteration(CalcLastIteration);
+  Dir->setPreCond(PreCond);
+  Dir->setCond(Cond, SeparatedCond);
+  Dir->setInit(Init);
+  Dir->setInc(Inc);
+  Dir->setCounters(Counters);
+  Dir->setUpdates(Updates);
+  Dir->setFinals(Finals);
   return Dir;
 }
 
 OMPSimdDirective *OMPSimdDirective::CreateEmpty(const ASTContext &C,
                                                 unsigned NumClauses,
                                                 unsigned CollapsedNum,
                                                 EmptyShell) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPSimdDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * NumClauses + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * NumClauses +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   return new (Mem) OMPSimdDirective(CollapsedNum, NumClauses);
 }
 
 OMPForDirective *
 OMPForDirective::Create(const ASTContext &C, SourceLocation StartLoc,
                         SourceLocation EndLoc, unsigned CollapsedNum,
-                        ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt) {
+                        ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt,
+                        Expr *IV, Expr *LastIteration, Expr *CalcLastIteration,
+                        Expr *PreCond, Expr *Cond, Expr *SeparatedCond,
+                        Expr *Init, Expr *Inc, ArrayRef<Expr *> Counters,
+                        ArrayRef<Expr *> Updates, ArrayRef<Expr *> Finals) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPForDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   OMPForDirective *Dir =
       new (Mem) OMPForDirective(StartLoc, EndLoc, CollapsedNum, Clauses.size());
   Dir->setClauses(Clauses);
   Dir->setAssociatedStmt(AssociatedStmt);
+  Dir->setIterationVariable(IV);
+  Dir->setLastIteration(LastIteration);
+  Dir->setCalcLastIteration(CalcLastIteration);
+  Dir->setPreCond(PreCond);
+  Dir->setCond(Cond, SeparatedCond);
+  Dir->setInit(Init);
+  Dir->setInc(Inc);
+  Dir->setCounters(Counters);
+  Dir->setUpdates(Updates);
+  Dir->setFinals(Finals);
   return Dir;
 }
 
 OMPForDirective *OMPForDirective::CreateEmpty(const ASTContext &C,
                                               unsigned NumClauses,
                                               unsigned CollapsedNum,
                                               EmptyShell) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPForDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * NumClauses + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * NumClauses +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   return new (Mem) OMPForDirective(CollapsedNum, NumClauses);
 }
 
-OMPForSimdDirective *OMPForSimdDirective::Create(const ASTContext &C,
-                                                 SourceLocation StartLoc,
-                                                 SourceLocation EndLoc,
-                                                 unsigned CollapsedNum,
-                                                 ArrayRef<OMPClause *> Clauses,
-                                                 Stmt *AssociatedStmt) {
+OMPForSimdDirective *OMPForSimdDirective::Create(
+    const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
+    unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt,
+    Expr *IV, Expr *LastIteration, Expr *CalcLastIteration, Expr *PreCond,
+    Expr *Cond, Expr *SeparatedCond, Expr *Init, Expr *Inc,
+    ArrayRef<Expr *> Counters, ArrayRef<Expr *> Updates,
+    ArrayRef<Expr *> Finals) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPForSimdDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   OMPForSimdDirective *Dir = new (Mem)
       OMPForSimdDirective(StartLoc, EndLoc, CollapsedNum, Clauses.size());
   Dir->setClauses(Clauses);
   Dir->setAssociatedStmt(AssociatedStmt);
+  Dir->setIterationVariable(IV);
+  Dir->setLastIteration(LastIteration);
+  Dir->setCalcLastIteration(CalcLastIteration);
+  Dir->setPreCond(PreCond);
+  Dir->setCond(Cond, SeparatedCond);
+  Dir->setInit(Init);
+  Dir->setInc(Inc);
+  Dir->setCounters(Counters);
+  Dir->setUpdates(Updates);
+  Dir->setFinals(Finals);
   return Dir;
 }
 
 OMPForSimdDirective *OMPForSimdDirective::CreateEmpty(const ASTContext &C,
                                                       unsigned NumClauses,
                                                       unsigned CollapsedNum,
                                                       EmptyShell) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPForSimdDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * NumClauses + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * NumClauses +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   return new (Mem) OMPForSimdDirective(CollapsedNum, NumClauses);
 }
 
 OMPSectionsDirective *OMPSectionsDirective::Create(
     const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
     ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPSectionsDirective),
                                            llvm::alignOf<OMPClause *>());

 OMPCriticalDirective *OMPCriticalDirective::CreateEmpty(const ASTContext &C,
                                                         EmptyShell) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPCriticalDirective),
                                            llvm::alignOf<Stmt *>());
   void *Mem = C.Allocate(Size + sizeof(Stmt *));
   return new (Mem) OMPCriticalDirective();
 }
 
-OMPParallelForDirective *
-OMPParallelForDirective::Create(const ASTContext &C, SourceLocation StartLoc,
-                                SourceLocation EndLoc, unsigned CollapsedNum,
-                                ArrayRef<OMPClause *> Clauses,
-                                Stmt *AssociatedStmt) {
+OMPParallelForDirective *OMPParallelForDirective::Create(
+    const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
+    unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt,
+    Expr *IV, Expr *LastIteration, Expr *CalcLastIteration, Expr *PreCond,
+    Expr *Cond, Expr *SeparatedCond, Expr *Init, Expr *Inc,
+    ArrayRef<Expr *> Counters, ArrayRef<Expr *> Updates,
+    ArrayRef<Expr *> Finals) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPParallelForDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   OMPParallelForDirective *Dir = new (Mem)
       OMPParallelForDirective(StartLoc, EndLoc, CollapsedNum, Clauses.size());
   Dir->setClauses(Clauses);
   Dir->setAssociatedStmt(AssociatedStmt);
+  Dir->setIterationVariable(IV);
+  Dir->setLastIteration(LastIteration);
+  Dir->setCalcLastIteration(CalcLastIteration);
+  Dir->setPreCond(PreCond);
+  Dir->setCond(Cond, SeparatedCond);
+  Dir->setInit(Init);
+  Dir->setInc(Inc);
+  Dir->setCounters(Counters);
+  Dir->setUpdates(Updates);
+  Dir->setFinals(Finals);
   return Dir;
 }
 
 OMPParallelForDirective *
 OMPParallelForDirective::CreateEmpty(const ASTContext &C, unsigned NumClauses,
                                      unsigned CollapsedNum, EmptyShell) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPParallelForDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * NumClauses + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * NumClauses +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   return new (Mem) OMPParallelForDirective(CollapsedNum, NumClauses);
 }
 
 OMPParallelForSimdDirective *OMPParallelForSimdDirective::Create(
     const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
-    unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses,
-    Stmt *AssociatedStmt) {
+    unsigned CollapsedNum, ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt,
+    Expr *IV, Expr *LastIteration, Expr *CalcLastIteration, Expr *PreCond,
+    Expr *Cond, Expr *SeparatedCond, Expr *Init, Expr *Inc,
+    ArrayRef<Expr *> Counters, ArrayRef<Expr *> Updates,
+    ArrayRef<Expr *> Finals) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPParallelForSimdDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * Clauses.size() +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   OMPParallelForSimdDirective *Dir = new (Mem) OMPParallelForSimdDirective(
       StartLoc, EndLoc, CollapsedNum, Clauses.size());
   Dir->setClauses(Clauses);
   Dir->setAssociatedStmt(AssociatedStmt);
+  Dir->setIterationVariable(IV);
+  Dir->setLastIteration(LastIteration);
+  Dir->setCalcLastIteration(CalcLastIteration);
+  Dir->setPreCond(PreCond);
+  Dir->setCond(Cond, SeparatedCond);
+  Dir->setInit(Init);
+  Dir->setInc(Inc);
+  Dir->setCounters(Counters);
+  Dir->setUpdates(Updates);
+  Dir->setFinals(Finals);
   return Dir;
 }
 
 OMPParallelForSimdDirective *
 OMPParallelForSimdDirective::CreateEmpty(const ASTContext &C,
                                          unsigned NumClauses,
                                          unsigned CollapsedNum, EmptyShell) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPParallelForSimdDirective),
                                            llvm::alignOf<OMPClause *>());
-  void *Mem =
-      C.Allocate(Size + sizeof(OMPClause *) * NumClauses + sizeof(Stmt *));
+  void *Mem = C.Allocate(Size + sizeof(OMPClause *) * NumClauses +
+                         sizeof(Stmt *) * numLoopChildren(CollapsedNum));
   return new (Mem) OMPParallelForSimdDirective(CollapsedNum, NumClauses);
 }
 
 OMPParallelSectionsDirective *OMPParallelSectionsDirective::Create(
     const ASTContext &C, SourceLocation StartLoc, SourceLocation EndLoc,
     ArrayRef<OMPClause *> Clauses, Stmt *AssociatedStmt) {
   unsigned Size = llvm::RoundUpToAlignment(sizeof(OMPParallelSectionsDirective),
                                            llvm::alignOf<OMPClause *>());

cfe/trunk/lib/CodeGen/CGStmtOpenMP.cpp

llvm::Value *Args[] = {
       Builder.CreateBitCast(OutlinedFn,
                             CGM.getOpenMPRuntime().getKmpc_MicroPointerTy()),
       EmitCastToVoidPtr(CapturedStruct)};
   llvm::Constant *RTLFn = CGM.getOpenMPRuntime().CreateRuntimeFunction(
       CGOpenMPRuntime::OMPRTL__kmpc_fork_call);
   EmitRuntimeCall(RTLFn, Args);
 }
 
+void CodeGenFunction::EmitOMPSimdBody(const OMPLoopDirective &S,
+                                      bool SeparateIter) {
+  RunCleanupsScope BodyScope(*this);
+  // Update counters values on current iteration.
+  for (auto I : S.updates()) {
+    EmitIgnoredExpr(I);
+  }
+  // On a continue in the body, jump to the end.
+  auto Continue = getJumpDestInCurrentScope("simd.continue");
+  BreakContinueStack.push_back(BreakContinue(JumpDest(), Continue));
+  // Emit loop body.
+  EmitStmt(S.getBody());
+  // The end (updates/cleanups).
+  EmitBlock(Continue.getBlock());
+  BreakContinueStack.pop_back();
+  if (SeparateIter) {
+    // TODO: Update lastprivates if the SeparateIter flag is true.
+    // This will be implemented in a follow-up OMPLastprivateClause patch, but
+    // result should be still correct without it, as we do not make these
+    // variables private yet.
+  }
+}
+
+void CodeGenFunction::EmitOMPSimdLoop(const OMPLoopDirective &S,
+                                      OMPPrivateScope &LoopScope,
+                                      bool SeparateIter) {
+  auto LoopExit = getJumpDestInCurrentScope("simd.for.end");
+  auto Cnt = getPGORegionCounter(&S);
+
+  // Start the loop with a block that tests the condition.
+  auto CondBlock = createBasicBlock("simd.for.cond");
+  EmitBlock(CondBlock);
+  LoopStack.push(CondBlock);
+
+  // If there are any cleanups between here and the loop-exit scope,
+  // create a block to stage a loop exit along.
+  auto ExitBlock = LoopExit.getBlock();
+  if (LoopScope.requiresCleanups())
+    ExitBlock = createBasicBlock("simd.for.cond.cleanup");
+
+  auto LoopBody = createBasicBlock("simd.for.body");
+
+  // Emit condition: "IV < LastIteration + 1 [ - 1]"
+  // ("- 1" when lastprivate clause is present - separate one iteration).
+  llvm::Value *BoolCondVal = EvaluateExprAsBool(S.getCond(SeparateIter));
+  Builder.CreateCondBr(BoolCondVal, LoopBody, ExitBlock,
+                       PGO.createLoopWeights(S.getCond(SeparateIter), Cnt));
+
+  if (ExitBlock != LoopExit.getBlock()) {
+    EmitBlock(ExitBlock);
+    EmitBranchThroughCleanup(LoopExit);
+  }
+
+  EmitBlock(LoopBody);
+  Cnt.beginRegion(Builder);
+
+  // Create a block for the increment.
+  auto Continue = getJumpDestInCurrentScope("simd.for.inc");
+  BreakContinueStack.push_back(BreakContinue(LoopExit, Continue));
+
+  EmitOMPSimdBody(S, /* SeparateIter */ false);
+  EmitStopPoint(&S);
+
+  // Emit "IV = IV + 1" and a back-edge to the condition block.
+  EmitBlock(Continue.getBlock());
+  EmitIgnoredExpr(S.getInc());
+  BreakContinueStack.pop_back();
+  EmitBranch(CondBlock);
+  LoopStack.pop();
+  // Emit the fall-through block.
+  EmitBlock(LoopExit.getBlock());
+}
+
+void CodeGenFunction::EmitOMPSimdFinal(const OMPLoopDirective &S) {
+  auto IC = S.counters().begin();
+  for (auto F : S.finals()) {
+    if (LocalDeclMap.lookup(cast<DeclRefExpr>((*IC))->getDecl())) {
+      EmitIgnoredExpr(F);
+    }
+    ++IC;
+  }
+}
+
 static void EmitOMPAlignedClause(CodeGenFunction &CGF, CodeGenModule &CGM,
                                  const OMPAlignedClause &Clause) {
   unsigned ClauseAlignment = 0;
   if (auto AlignmentExpr = Clause.getAlignment()) {
     auto AlignmentCI =
         cast<llvm::ConstantInt>(CGF.EmitScalarExpr(AlignmentExpr));
     ClauseAlignment = static_cast<unsigned>(AlignmentCI->getZExtValue());
   }
for (auto E : Clause.varlists()) {
     if (Alignment != 0) {
       llvm::Value *PtrValue = CGF.EmitScalarExpr(E);
       CGF.EmitAlignmentAssumption(PtrValue, Alignment);
     }
   }
 }
 
 void CodeGenFunction::EmitOMPSimdDirective(const OMPSimdDirective &S) {
-  const CapturedStmt *CS = cast<CapturedStmt>(S.getAssociatedStmt());
-  const Stmt *Body = CS->getCapturedStmt();
+  // Pragma 'simd' code depends on presence of 'lastprivate'.
+  // If present, we have to separate last iteration of the loop:
+  //
+  // if (LastIteration != 0) {
+  //   for (IV in 0..LastIteration-1) BODY;
+  //   BODY with updates of lastprivate vars;
+  //   <Final counter/linear vars updates>;
+  // }
+  //
+  // otherwise (when there's no lastprivate):
+  //
+  //   for (IV in 0..LastIteration) BODY;
+  //   <Final counter/linear vars updates>;
+  //
+
+  // Walk clauses and process safelen/lastprivate.
+  bool SeparateIter = false;
   LoopStack.setParallel();
   LoopStack.setVectorizerEnable(true);
   for (auto C : S.clauses()) {
     switch (C->getClauseKind()) {
     case OMPC_safelen: {
       RValue Len = EmitAnyExpr(cast<OMPSafelenClause>(C)->getSafelen(),
                                AggValueSlot::ignored(), true);
       llvm::ConstantInt *Val = cast<llvm::ConstantInt>(Len.getScalarVal());
       LoopStack.setVectorizerWidth(Val->getZExtValue());
       // In presence of finite 'safelen', it may be unsafe to mark all
       // the memory instructions parallel, because loop-carried
       // dependences of 'safelen' iterations are possible.
       LoopStack.setParallel(false);
       break;
     }
     case OMPC_aligned:
       EmitOMPAlignedClause(*this, CGM, cast<OMPAlignedClause>(*C));
       break;
+    case OMPC_lastprivate:
+      SeparateIter = true;
+      break;
     default:
       // Not handled yet
       ;
     }
   }
-  EmitStmt(Body);
+
+  RunCleanupsScope DirectiveScope(*this);
+
+  CGDebugInfo *DI = getDebugInfo();
+  if (DI)
+    DI->EmitLexicalBlockStart(Builder, S.getSourceRange().getBegin());
+
+  // Emit the loop iteration variable.
+  const Expr *IVExpr = S.getIterationVariable();
+  const VarDecl *IVDecl = cast<VarDecl>(cast<DeclRefExpr>(IVExpr)->getDecl());
+  EmitVarDecl(*IVDecl);
+  EmitIgnoredExpr(S.getInit());
+
+  // Emit the iterations count variable.
+  // If it is not a variable, Sema decided to calculate iterations count on each
+  // iteration (e.g., it is foldable into a constant).
+  if (auto LIExpr = dyn_cast<DeclRefExpr>(S.getLastIteration())) {
+    EmitVarDecl(*cast<VarDecl>(LIExpr->getDecl()));
+    // Emit calculation of the iterations count.
+    EmitIgnoredExpr(S.getCalcLastIteration());
+  }
+
+  if (SeparateIter) {
+    // Emit: if (LastIteration > 0) - begin.
+    RegionCounter Cnt = getPGORegionCounter(&S);
+    auto ThenBlock = createBasicBlock("simd.if.then");
+    auto ContBlock = createBasicBlock("simd.if.end");
+    EmitBranchOnBoolExpr(S.getPreCond(), ThenBlock, ContBlock, Cnt.getCount());
+    EmitBlock(ThenBlock);
+    Cnt.beginRegion(Builder);
+    // Emit 'then' code.
+    {
+      OMPPrivateScope LoopScope(*this);
+      LoopScope.addPrivates(S.counters());
+      EmitOMPSimdLoop(S, LoopScope, /* SeparateIter */ true);
+      EmitOMPSimdBody(S, /* SeparateIter */ true);
+    }
+    EmitOMPSimdFinal(S);
+    // Emit: if (LastIteration != 0) - end.
+    EmitBranch(ContBlock);
+    EmitBlock(ContBlock, true);
+  } else {
+    {
+      OMPPrivateScope LoopScope(*this);
+      LoopScope.addPrivates(S.counters());
+      EmitOMPSimdLoop(S, LoopScope, /* SeparateIter */ false);
+    }
+    EmitOMPSimdFinal(S);
+  }
+
+  if (DI)
+    DI->EmitLexicalBlockEnd(Builder, S.getSourceRange().getEnd());
 }
 
 void CodeGenFunction::EmitOMPForDirective(const OMPForDirective &) {
   llvm_unreachable("CodeGen for 'omp for' is not supported yet.");
 }
 
 void CodeGenFunction::EmitOMPForSimdDirective(const OMPForSimdDirective &) {
   llvm_unreachable("CodeGen for 'omp for simd' is not supported yet.");

cfe/trunk/lib/CodeGen/CodeGenFunction.h

struct BreakContinue {
     BreakContinue(JumpDest Break, JumpDest Continue)
       : BreakBlock(Break), ContinueBlock(Continue) {}
 
     JumpDest BreakBlock;
     JumpDest ContinueBlock;
   };
   SmallVector<BreakContinue, 8> BreakContinueStack;
 
+  /// \brief The scope used to remap some variables as private in the OpenMP
+  /// loop body (or other captured region emitted without outlining), and to
+  /// restore old vars back on exit.
+  class OMPPrivateScope : public RunCleanupsScope {
+    DeclMapTy SavedLocals;
+
+  private:
+    OMPPrivateScope(const OMPPrivateScope &) LLVM_DELETED_FUNCTION;
+    void operator=(const OMPPrivateScope &) LLVM_DELETED_FUNCTION;
+
+  public:
+    /// \brief Enter a new OpenMP private scope.
+    explicit OMPPrivateScope(CodeGenFunction &CGF) : RunCleanupsScope(CGF) {}
+
+    /// \brief Add and remap private variables (without initialization).
+    /// \param Vars - a range of DeclRefExprs for the private variables.
+    template <class IT> void addPrivates(IT Vars) {
+      assert(PerformCleanup && "adding private to dead scope");
+      for (auto E : Vars) {
+        auto D = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
+        assert(!SavedLocals.lookup(D) && "remapping a var twice");
+        SavedLocals[D] = CGF.LocalDeclMap.lookup(D);
+        CGF.LocalDeclMap.erase(D);
+        // Emit var without initialization.
+        auto VarEmission = CGF.EmitAutoVarAlloca(*D);
+        CGF.EmitAutoVarCleanups(VarEmission);
+      }
+    }
+
+    void ForceCleanup() {
+      RunCleanupsScope::ForceCleanup();
+      // Remap vars back to the original values.
+      for (auto I : SavedLocals) {
+        CGF.LocalDeclMap[I.first] = I.second;
+      }
+      SavedLocals.clear();
+    }
+
+    /// \brief Exit scope - all the mapped variables are restored.
+    ~OMPPrivateScope() { ForceCleanup(); }
+  };
+
   CodeGenPGO PGO;
 
 public:
   /// Get a counter for instrumentation of the region associated with the given
   /// statement.
   RegionCounter getPGORegionCounter(const Stmt *S) {
     return RegionCounter(PGO, S);
   }
#endif
   void EmitOMPTaskyieldDirective(const OMPTaskyieldDirective &S);
   void EmitOMPBarrierDirective(const OMPBarrierDirective &S);
   void EmitOMPTaskwaitDirective(const OMPTaskwaitDirective &S);
   void EmitOMPFlushDirective(const OMPFlushDirective &S);
   void EmitOMPOrderedDirective(const OMPOrderedDirective &S);
   void EmitOMPAtomicDirective(const OMPAtomicDirective &S);
   void EmitOMPTargetDirective(const OMPTargetDirective &S);
 
+  /// Helpers for 'omp simd' directive.
+  void EmitOMPSimdBody(const OMPLoopDirective &Directive, bool SeparateIter);
+  void EmitOMPSimdLoop(const OMPLoopDirective &S, OMPPrivateScope &LoopScope,
+                       bool SeparateIter);
+  void EmitOMPSimdFinal(const OMPLoopDirective &S);
+
   //===--------------------------------------------------------------------===//
   //                         LValue Expression Emission
   //===--------------------------------------------------------------------===//
 
   /// GetUndefRValue - Get an appropriate 'undef' rvalue for the given type.
   RValue GetUndefRValue(QualType Ty);
 
   /// EmitUnsupportedRValue - Emit a dummy r-value using the type of E

cfe/trunk/lib/Sema/SemaOpenMP.cpp

 /// for IR generation.
 class OpenMPIterationSpaceChecker {
   /// \brief Reference to Sema.
   Sema &SemaRef;
   /// \brief A location for diagnostics (when there is no some better location).
   SourceLocation DefaultLoc;
   /// \brief A location for diagnostics (when increment is not compatible).
   SourceLocation ConditionLoc;
+  /// \brief A source location for referring to loop init later.
+  SourceRange InitSrcRange;
   /// \brief A source location for referring to condition later.
   SourceRange ConditionSrcRange;
+  /// \brief A source location for referring to increment later.
+  SourceRange IncrementSrcRange;
   /// \brief Loop variable.
   VarDecl *Var;
   /// \brief Reference to loop variable.
   DeclRefExpr *VarRef;
   /// \brief Lower bound (initializer for the var).
   Expr *LB;
   /// \brief Upper bound.
   Expr *UB;
   /// \brief Loop step (increment).
   Expr *Step;
   /// \brief This flag is true when condition is one of:
   ///   Var <  UB
   ///   Var <= UB
   ///   UB  >  Var
   ///   UB  >= Var
   bool TestIsLessOp;
   /// \brief This flag is true when condition is strict ( < or > ).
   bool TestIsStrictOp;
   /// \brief This flag is true when step is subtracted on each iteration.
   bool SubtractStep;
 
 public:
   OpenMPIterationSpaceChecker(Sema &SemaRef, SourceLocation DefaultLoc)
       : SemaRef(SemaRef), DefaultLoc(DefaultLoc), ConditionLoc(DefaultLoc),
-        ConditionSrcRange(SourceRange()), Var(nullptr), VarRef(nullptr),
+        InitSrcRange(SourceRange()), ConditionSrcRange(SourceRange()),
+        IncrementSrcRange(SourceRange()), Var(nullptr), VarRef(nullptr),
         LB(nullptr), UB(nullptr), Step(nullptr), TestIsLessOp(false),
         TestIsStrictOp(false), SubtractStep(false) {}
   /// \brief Check init-expr for canonical loop form and save loop counter
   /// variable - #Var and its initialization value - #LB.
   bool CheckInit(Stmt *S);
   /// \brief Check test-expr for canonical form, save upper-bound (#UB), flags
   /// for less/greater and for strict/non-strict comparison.
   bool CheckCond(Expr *S);
   /// \brief Check incr-expr for canonical loop form and return true if it
   /// does not conform, otherwise save loop step (#Step).
   bool CheckInc(Expr *S);
   /// \brief Return the loop counter variable.
   VarDecl *GetLoopVar() const { return Var; }
   /// \brief Return the reference expression to loop counter variable.
   DeclRefExpr *GetLoopVarRefExpr() const { return VarRef; }
+  /// \brief Source range of the loop init.
+  SourceRange GetInitSrcRange() const { return InitSrcRange; }
+  /// \brief Source range of the loop condition.
+  SourceRange GetConditionSrcRange() const { return ConditionSrcRange; }
+  /// \brief Source range of the loop increment.
+  SourceRange GetIncrementSrcRange() const { return IncrementSrcRange; }
+  /// \brief True if the step should be subtracted.
+  bool ShouldSubtractStep() const { return SubtractStep; }
+  /// \brief Build the expression to calculate the number of iterations.
+  Expr *BuildNumIterations(Scope *S) const;
+  /// \brief Build reference expression to the counter be used for codegen.
+  Expr *BuildCounterVar() const;
+  /// \brief Build initization of the counter be used for codegen.
+  Expr *BuildCounterInit() const;
+  /// \brief Build step of the counter be used for codegen.
+  Expr *BuildCounterStep() const;
   /// \brief Return true if any expression is dependent.
   bool Dependent() const;
 
 private:
   /// \brief Check the right-hand side of an assignment in the increment
   /// expression.
   bool CheckIncRHS(Expr *RHS);
   /// \brief Helper to set loop counter variable and its initializer.
if (!NewStep->isValueDependent()) {
     //  loop. If test-expr is of form b relational-op var and relational-op is
     //  > or >= then incr-expr must cause var to increase on each iteration of
     //  the loop.
     llvm::APSInt Result;
     bool IsConstant = NewStep->isIntegerConstantExpr(Result, SemaRef.Context);
     bool IsUnsigned = !NewStep->getType()->hasSignedIntegerRepresentation();
     bool IsConstNeg =
         IsConstant && Result.isSigned() && (Subtract != Result.isNegative());
+    bool IsConstPos =
+        IsConstant && Result.isSigned() && (Subtract == Result.isNegative());
     bool IsConstZero = IsConstant && !Result.getBoolValue();
     if (UB && (IsConstZero ||
                (TestIsLessOp ? (IsConstNeg || (IsUnsigned && Subtract))
-                             : (!IsConstNeg || (IsUnsigned && !Subtract))))) {
+                             : (IsConstPos || (IsUnsigned && !Subtract))))) {
       SemaRef.Diag(NewStep->getExprLoc(),
                    diag::err_omp_loop_incr_not_compatible)
           << Var << TestIsLessOp << NewStep->getSourceRange();
       SemaRef.Diag(ConditionLoc,
                    diag::note_omp_loop_cond_requres_compatible_incr)
           << TestIsLessOp << ConditionSrcRange;
       return true;
     }
+    if (TestIsLessOp == Subtract) {
+      NewStep = SemaRef.CreateBuiltinUnaryOp(NewStep->getExprLoc(), UO_Minus,
+                                             NewStep).get();
+      Subtract = !Subtract;
+    }
   }
 
   Step = NewStep;
   SubtractStep = Subtract;
   return false;
 }
 
 bool OpenMPIterationSpaceChecker::CheckInit(Stmt *S) {
   // Check init-expr for canonical loop form and save loop counter
   // variable - #Var and its initialization value - #LB.
   // OpenMP [2.6] Canonical loop form. init-expr may be one of the following:
   //   var = lb
   //   integer-type var = lb
   //   random-access-iterator-type var = lb
   //   pointer-type var = lb
   //
   if (!S) {
     SemaRef.Diag(DefaultLoc, diag::err_omp_loop_not_canonical_init);
     return true;
   }
+  InitSrcRange = S->getSourceRange();
   if (Expr *E = dyn_cast<Expr>(S))
     S = E->IgnoreParens();
   if (auto BO = dyn_cast<BinaryOperator>(S)) {
     if (BO->getOpcode() == BO_Assign)
       if (auto DRE = dyn_cast<DeclRefExpr>(BO->getLHS()->IgnoreParens()))
         return SetVarAndLB(dyn_cast<VarDecl>(DRE->getDecl()), DRE,
-                           BO->getLHS());
+                           BO->getRHS());
   } else if (auto DS = dyn_cast<DeclStmt>(S)) {
     if (DS->isSingleDecl()) {
       if (auto Var = dyn_cast_or_null<VarDecl>(DS->getSingleDecl())) {
         if (Var->hasInit()) {
           // Accept non-canonical init form here but emit ext. warning.
           if (Var->getInitStyle() != VarDecl::CInit)
             SemaRef.Diag(S->getLocStart(),
                          diag::ext_omp_loop_not_canonical_init)
bool OpenMPIterationSpaceChecker::CheckInc(Expr *S) {
   //   var = var + incr
   //   var = incr + var
   //   var = var - incr
   //
   if (!S) {
     SemaRef.Diag(DefaultLoc, diag::err_omp_loop_not_canonical_incr) << Var;
     return true;
   }
+  IncrementSrcRange = S->getSourceRange();
   S = S->IgnoreParens();
   if (auto UO = dyn_cast<UnaryOperator>(S)) {
     if (UO->isIncrementDecrementOp() && GetInitVarDecl(UO->getSubExpr()) == Var)
       return SetStep(
           SemaRef.ActOnIntegerConstant(UO->getLocStart(),
                                        (UO->isDecrementOp() ? -1 : 1)).get(),
           false);
   } else if (auto BO = dyn_cast<BinaryOperator>(S)) {
if (auto UO = dyn_cast<UnaryOperator>(S)) {
     default:
       break;
     }
   }
   SemaRef.Diag(S->getLocStart(), diag::err_omp_loop_not_canonical_incr)
       << S->getSourceRange() << Var;
   return true;
 }
+
+/// \brief Build the expression to calculate the number of iterations.
+Expr *OpenMPIterationSpaceChecker::BuildNumIterations(Scope *S) const {
+  ExprResult Diff;
+  if (Var->getType()->isIntegerType() || Var->getType()->isPointerType() ||
+      SemaRef.getLangOpts().CPlusPlus) {
+    // Upper - Lower
+    Expr *Upper = TestIsLessOp ? UB : LB;
+    Expr *Lower = TestIsLessOp ? LB : UB;
+
+    Diff = SemaRef.BuildBinOp(S, DefaultLoc, BO_Sub, Upper, Lower);
+
+    if (!Diff.isUsable() && Var->getType()->getAsCXXRecordDecl()) {
+      // BuildBinOp already emitted error, this one is to point user to upper
+      // and lower bound, and to tell what is passed to 'operator-'.
+      SemaRef.Diag(Upper->getLocStart(), diag::err_omp_loop_diff_cxx)
+          << Upper->getSourceRange() << Lower->getSourceRange();
+      return nullptr;
+    }
+  }
+
+  if (!Diff.isUsable())
+    return nullptr;
+
+  // Upper - Lower [- 1]
+  if (TestIsStrictOp)
+    Diff = SemaRef.BuildBinOp(
+        S, DefaultLoc, BO_Sub, Diff.get(),
+        SemaRef.ActOnIntegerConstant(SourceLocation(), 1).get());
+  if (!Diff.isUsable())
+    return nullptr;
+
+  // Upper - Lower [- 1] + Step
+  Diff = SemaRef.BuildBinOp(S, DefaultLoc, BO_Add, Diff.get(),
+                            Step->IgnoreImplicit());
+  if (!Diff.isUsable())
+    return nullptr;
+
+  // Parentheses (for dumping/debugging purposes only).
+  Diff = SemaRef.ActOnParenExpr(DefaultLoc, DefaultLoc, Diff.get());
+  if (!Diff.isUsable())
+    return nullptr;
+
+  // (Upper - Lower [- 1] + Step) / Step
+  Diff = SemaRef.BuildBinOp(S, DefaultLoc, BO_Div, Diff.get(),
+                            Step->IgnoreImplicit());
+  if (!Diff.isUsable())
+    return nullptr;
+
+  return Diff.get();
+}
+
+/// \brief Build reference expression to the counter be used for codegen.
+Expr *OpenMPIterationSpaceChecker::BuildCounterVar() const {
+  return DeclRefExpr::Create(SemaRef.Context, NestedNameSpecifierLoc(),
+                             GetIncrementSrcRange().getBegin(), Var, false,
+                             DefaultLoc, Var->getType(), VK_LValue);
+}
+
+/// \brief Build initization of the counter be used for codegen.
+Expr *OpenMPIterationSpaceChecker::BuildCounterInit() const { return LB; }
+
+/// \brief Build step of the counter be used for codegen.
+Expr *OpenMPIterationSpaceChecker::BuildCounterStep() const { return Step; }
+
+/// \brief Iteration space of a single for loop.
+struct LoopIterationSpace {
+  /// \brief This expression calculates the number of iterations in the loop.
+  /// It is always possible to calculate it before starting the loop.
+  Expr *NumIterations;
+  /// \brief The loop counter variable.
+  Expr *CounterVar;
+  /// \brief This is initializer for the initial value of #CounterVar.
+  Expr *CounterInit;
+  /// \brief This is step for the #CounterVar used to generate its update:
+  /// #CounterVar = #CounterInit + #CounterStep * CurrentIteration.
+  Expr *CounterStep;
+  /// \brief Should step be subtracted?
+  bool Subtract;
+  /// \brief Source range of the loop init.
+  SourceRange InitSrcRange;
+  /// \brief Source range of the loop condition.
+  SourceRange CondSrcRange;
+  /// \brief Source range of the loop increment.
+  SourceRange IncSrcRange;
+};
+
+/// \brief The resulting expressions built for the OpenMP loop CodeGen for the
+/// whole collapsed loop nest. See class OMPLoopDirective for their description.
+struct BuiltLoopExprs {
+  Expr *IterationVarRef;
+  Expr *LastIteration;
+  Expr *CalcLastIteration;
+  Expr *PreCond;
+  Expr *Cond;
+  Expr *SeparatedCond;
+  Expr *Init;
+  Expr *Inc;
+  SmallVector<Expr *, 4> Counters;
+  SmallVector<Expr *, 4> Updates;
+  SmallVector<Expr *, 4> Finals;
+
+  bool builtAll() {
+    return IterationVarRef != nullptr && LastIteration != nullptr &&
+           PreCond != nullptr && Cond != nullptr && SeparatedCond != nullptr &&
+           Init != nullptr && Inc != nullptr;
+  }
+  void clear(unsigned size) {
+    IterationVarRef = nullptr;
+    LastIteration = nullptr;
+    CalcLastIteration = nullptr;
+    PreCond = nullptr;
+    Cond = nullptr;
+    SeparatedCond = nullptr;
+    Init = nullptr;
+    Inc = nullptr;
+    Counters.resize(size);
+    Updates.resize(size);
+    Finals.resize(size);
+    for (unsigned i = 0; i < size; ++i) {
+      Counters[i] = nullptr;
+      Updates[i] = nullptr;
+      Finals[i] = nullptr;
+    }
+  }
+};
+
 } // namespace
 
 /// \brief Called on a for stmt to check and extract its iteration space
 /// for further processing (such as collapsing).
 static bool CheckOpenMPIterationSpace(
     OpenMPDirectiveKind DKind, Stmt *S, Sema &SemaRef, DSAStackTy &DSA,
     unsigned CurrentNestedLoopCount, unsigned NestedLoopCount,
     Expr *NestedLoopCountExpr,
-    llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA) {
+    llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA,
+    LoopIterationSpace &ResultIterSpace) {
   // OpenMP [2.6, Canonical Loop Form]
   //   for (init-expr; test-expr; incr-expr) structured-block
   auto For = dyn_cast_or_null<ForStmt>(S);
   if (!For) {
     SemaRef.Diag(S->getLocStart(), diag::err_omp_not_for)
         << (NestedLoopCountExpr != nullptr) << getOpenMPDirectiveName(DKind)
         << NestedLoopCount << (CurrentNestedLoopCount > 0)
         << CurrentNestedLoopCount;
static bool CheckOpenMPIterationSpace(
   assert(isOpenMPLoopDirective(DKind) && "DSA for non-loop vars");
 
   // Check test-expr.
   HasErrors |= ISC.CheckCond(For->getCond());
 
   // Check incr-expr.
   HasErrors |= ISC.CheckInc(For->getInc());
 
-  if (ISC.Dependent())
+  if (ISC.Dependent() || SemaRef.CurContext->isDependentContext() || HasErrors)
     return HasErrors;
 
-  // FIXME: Build loop's iteration space representation.
+  // Build the loop's iteration space representation.
+  ResultIterSpace.NumIterations = ISC.BuildNumIterations(DSA.getCurScope());
+  ResultIterSpace.CounterVar = ISC.BuildCounterVar();
+  ResultIterSpace.CounterInit = ISC.BuildCounterInit();
+  ResultIterSpace.CounterStep = ISC.BuildCounterStep();
+  ResultIterSpace.InitSrcRange = ISC.GetInitSrcRange();
+  ResultIterSpace.CondSrcRange = ISC.GetConditionSrcRange();
+  ResultIterSpace.IncSrcRange = ISC.GetIncrementSrcRange();
+  ResultIterSpace.Subtract = ISC.ShouldSubtractStep();
+
+  HasErrors |= (ResultIterSpace.NumIterations == nullptr ||
+                ResultIterSpace.CounterVar == nullptr ||
+                ResultIterSpace.CounterInit == nullptr ||
+                ResultIterSpace.CounterStep == nullptr);
+
   return HasErrors;
 }
 
-/// \brief A helper routine to skip no-op (attributed, compound) stmts get the
-/// next nested for loop. If \a IgnoreCaptured is true, it skips captured stmt
-/// to get the first for loop.
-static Stmt *IgnoreContainerStmts(Stmt *S, bool IgnoreCaptured) {
-  if (IgnoreCaptured)
-    if (auto CapS = dyn_cast_or_null<CapturedStmt>(S))
-      S = CapS->getCapturedStmt();
-  // OpenMP [2.8.1, simd construct, Restrictions]
-  // All loops associated with the construct must be perfectly nested; that is,
-  // there must be no intervening code nor any OpenMP directive between any two
-  // loops.
-  while (true) {
-    if (auto AS = dyn_cast_or_null<AttributedStmt>(S))
-      S = AS->getSubStmt();
-    else if (auto CS = dyn_cast_or_null<CompoundStmt>(S)) {
-      if (CS->size() != 1)
-        break;
-      S = CS->body_back();
-    } else
-      break;
-  }
-  return S;
+/// \brief Build a variable declaration for OpenMP loop iteration variable.
+static VarDecl *BuildVarDecl(Sema &SemaRef, SourceLocation Loc, QualType Type,
+                             StringRef Name) {
+  DeclContext *DC = SemaRef.CurContext;
+  IdentifierInfo *II = &SemaRef.PP.getIdentifierTable().get(Name);
+  TypeSourceInfo *TInfo = SemaRef.Context.getTrivialTypeSourceInfo(Type, Loc);
+  VarDecl *Decl =
+      VarDecl::Create(SemaRef.Context, DC, Loc, Loc, II, Type, TInfo, SC_None);
+  Decl->setImplicit();
+  return Decl;
+}
+
+/// \brief Build 'VarRef = Start + Iter * Step'.
+static ExprResult BuildCounterUpdate(Sema &SemaRef, Scope *S,
+                                     SourceLocation Loc, ExprResult VarRef,
+                                     ExprResult Start, ExprResult Iter,
+                                     ExprResult Step, bool Subtract) {
+  // Add parentheses (for debugging purposes only).
+  Iter = SemaRef.ActOnParenExpr(Loc, Loc, Iter.get());
+  if (!VarRef.isUsable() || !Start.isUsable() || !Iter.isUsable() ||
+      !Step.isUsable())
+    return ExprError();
+
+  ExprResult Update = SemaRef.BuildBinOp(S, Loc, BO_Mul, Iter.get(),
+                                         Step.get()->IgnoreImplicit());
+  if (!Update.isUsable())
+    return ExprError();
+
+  // Build 'VarRef = Start + Iter * Step'.
+  Update = SemaRef.BuildBinOp(S, Loc, (Subtract ? BO_Sub : BO_Add),
+                              Start.get()->IgnoreImplicit(), Update.get());
+  if (!Update.isUsable())
+    return ExprError();
+
+  Update = SemaRef.PerformImplicitConversion(
+      Update.get(), VarRef.get()->getType(), Sema::AA_Converting, true);
+  if (!Update.isUsable())
+    return ExprError();
+
+  Update = SemaRef.BuildBinOp(S, Loc, BO_Assign, VarRef.get(), Update.get());
+  return Update;
+
+/// \brief Convert integer expression \a E to make it have at least \a Bits
+/// bits.
+static ExprResult WidenIterationCount(unsigned Bits, Expr *E,
+                                      Sema &SemaRef) {
+  if (E == nullptr)
+    return ExprError();
+  auto &C = SemaRef.Context;
+  QualType OldType = E->getType();
+  unsigned HasBits = C.getTypeSize(OldType);
+  if (HasBits >= Bits)
+    return ExprResult(E);
+  // OK to convert to signed, because new type has more bits than old.
+  QualType NewType = C.getIntTypeForBitwidth(Bits, /* Signed */ true);
+  return SemaRef.PerformImplicitConversion(E, NewType, Sema::AA_Converting,
+                                           true);
+}
+
+/// \brief Check if the given expression \a E is a constant integer that fits
+/// into \a Bits bits.
+static bool FitsInto(unsigned Bits, bool Signed, Expr *E, Sema &SemaRef) {
+  if (E == nullptr)
+    return false;
+  llvm::APSInt Result;
+  if (E->isIntegerConstantExpr(Result, SemaRef.Context))
+    return Signed ? Result.isSignedIntN(Bits) : Result.isIntN(Bits);
+  return false;
 }
 
 /// \brief Called on a for stmt to check itself and nested loops (if any).
 /// \return Returns 0 if one of the collapsed stmts is not canonical for loop,
 /// number of collapsed loops otherwise.
 static unsigned
 CheckOpenMPLoop(OpenMPDirectiveKind DKind, Expr *NestedLoopCountExpr,
                 Stmt *AStmt, Sema &SemaRef, DSAStackTy &DSA,
-                llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA) {
+                llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA,
+                BuiltLoopExprs &Built) {
   unsigned NestedLoopCount = 1;
   if (NestedLoopCountExpr) {
     // Found 'collapse' clause - calculate collapse number.
     llvm::APSInt Result;
     if (NestedLoopCountExpr->EvaluateAsInt(Result, SemaRef.getASTContext()))
       NestedLoopCount = Result.getLimitedValue();
   }
   // This is helper routine for loop directives (e.g., 'for', 'simd',
   // 'for simd', etc.).
-  Stmt *CurStmt = IgnoreContainerStmts(AStmt, true);
+  SmallVector<LoopIterationSpace, 4> IterSpaces;
+  IterSpaces.resize(NestedLoopCount);
+  Stmt *CurStmt = AStmt->IgnoreContainers(/* IgnoreCaptured */ true);
   for (unsigned Cnt = 0; Cnt < NestedLoopCount; ++Cnt) {
     if (CheckOpenMPIterationSpace(DKind, CurStmt, SemaRef, DSA, Cnt,
                                   NestedLoopCount, NestedLoopCountExpr,
-                                  VarsWithImplicitDSA))
+                                  VarsWithImplicitDSA, IterSpaces[Cnt]))
       return 0;
     // Move on to the next nested for loop, or to the loop body.
-    CurStmt = IgnoreContainerStmts(cast<ForStmt>(CurStmt)->getBody(), false);
+    // OpenMP [2.8.1, simd construct, Restrictions]
+    // All loops associated with the construct must be perfectly nested; that
+    // is, there must be no intervening code nor any OpenMP directive between
+    // any two loops.
+    CurStmt = cast<ForStmt>(CurStmt)->getBody()->IgnoreContainers();
   }
 
-  // FIXME: Build resulting iteration space for IR generation (collapsing
-  // iteration spaces when loop count > 1 ('collapse' clause)).
+  Built.clear(/* size */ NestedLoopCount);
+
+  if (SemaRef.CurContext->isDependentContext())
+    return NestedLoopCount;
+
+  // An example of what is generated for the following code:
+  //
+  //   #pragma omp simd collapse(2)
+  //   for (i = 0; i < NI; ++i)
+  //     for (j = J0; j < NJ; j+=2) {
+  //     <loop body>
+  //   }
+  //
+  // We generate the code below.
+  // Note: the loop body may be outlined in CodeGen.
+  // Note: some counters may be C++ classes, operator- is used to find number of
+  // iterations and operator+= to calculate counter value.
+  // Note: decltype(NumIterations) must be integer type (in 'omp for', only i32
+  // or i64 is currently supported).
+  //
+  //   #define NumIterations (NI * ((NJ - J0 - 1 + 2) / 2))
+  //   for (int[32|64]_t IV = 0; IV < NumIterations; ++IV ) {
+  //     .local.i = IV / ((NJ - J0 - 1 + 2) / 2);
+  //     .local.j = J0 + (IV % ((NJ - J0 - 1 + 2) / 2)) * 2;
+  //     // similar updates for vars in clauses (e.g. 'linear')
+  //     <loop body (using local i and j)>
+  //   }
+  //   i = NI; // assign final values of counters
+  //   j = NJ;
+  //
+
+  // Last iteration number is (I1 * I2 * ... In) - 1, where I1, I2 ... In are
+  // the iteration counts of the collapsed for loops.
+  auto N0 = IterSpaces[0].NumIterations;
+  ExprResult LastIteration32 = WidenIterationCount(32 /* Bits */, N0, SemaRef);
+  ExprResult LastIteration64 = WidenIterationCount(64 /* Bits */, N0, SemaRef);
+
+  if (!LastIteration32.isUsable() || !LastIteration64.isUsable())
+    return NestedLoopCount;
+
+  auto &C = SemaRef.Context;
+  bool AllCountsNeedLessThan32Bits = C.getTypeSize(N0->getType()) < 32;
+
+  Scope *CurScope = DSA.getCurScope();
+  for (unsigned Cnt = 1; Cnt < NestedLoopCount; ++Cnt) {
+    auto N = IterSpaces[Cnt].NumIterations;
+    AllCountsNeedLessThan32Bits &= C.getTypeSize(N->getType()) < 32;
+    if (LastIteration32.isUsable())
+      LastIteration32 = SemaRef.BuildBinOp(CurScope, SourceLocation(), BO_Mul,
+                                           LastIteration32.get(), N);
+    if (LastIteration64.isUsable())
+      LastIteration64 = SemaRef.BuildBinOp(CurScope, SourceLocation(), BO_Mul,
+                                           LastIteration64.get(), N);
+  }
+
+  // Choose either the 32-bit or 64-bit version.
+  ExprResult LastIteration = LastIteration64;
+  if (LastIteration32.isUsable() &&
+      C.getTypeSize(LastIteration32.get()->getType()) == 32 &&
+      (AllCountsNeedLessThan32Bits || NestedLoopCount == 1 ||
+       FitsInto(
+           32 /* Bits */,
+           LastIteration32.get()->getType()->hasSignedIntegerRepresentation(),
+           LastIteration64.get(), SemaRef)))
+    LastIteration = LastIteration32;
+
+  if (!LastIteration.isUsable())
+    return 0;
+
+  // Save the number of iterations.
+  ExprResult NumIterations = LastIteration;
+  {
+    LastIteration = SemaRef.BuildBinOp(
+        CurScope, SourceLocation(), BO_Sub, LastIteration.get(),
+        SemaRef.ActOnIntegerConstant(SourceLocation(), 1).get());
+    if (!LastIteration.isUsable())
+      return 0;
+  }
+
+  // Calculate the last iteration number beforehand instead of doing this on
+  // each iteration. Do not do this if the number of iterations may be kfold-ed.
+  llvm::APSInt Result;
+  bool IsConstant =
+      LastIteration.get()->isIntegerConstantExpr(Result, SemaRef.Context);
+  ExprResult CalcLastIteration;
+  if (!IsConstant) {
+    SourceLocation SaveLoc;
+    VarDecl *SaveVar =
+        BuildVarDecl(SemaRef, SaveLoc, LastIteration.get()->getType(),
+                     ".omp.last.iteration");
+    ExprResult SaveRef = SemaRef.BuildDeclRefExpr(
+        SaveVar, LastIteration.get()->getType(), VK_LValue, SaveLoc);
+    CalcLastIteration = SemaRef.BuildBinOp(CurScope, SaveLoc, BO_Assign,
+                                           SaveRef.get(), LastIteration.get());
+    LastIteration = SaveRef;
+
+    // Prepare SaveRef + 1.
+    NumIterations = SemaRef.BuildBinOp(
+        CurScope, SaveLoc, BO_Add, SaveRef.get(),
+        SemaRef.ActOnIntegerConstant(SourceLocation(), 1).get());
+    if (!NumIterations.isUsable())
+      return 0;
+  }
+
+  SourceLocation InitLoc = IterSpaces[0].InitSrcRange.getBegin();
+
+  // Precondition tests if there is at least one iteration (LastIteration > 0).
+  ExprResult PreCond = SemaRef.BuildBinOp(
+      CurScope, InitLoc, BO_GT, LastIteration.get(),
+      SemaRef.ActOnIntegerConstant(SourceLocation(), 0).get());
+
+  // Build the iteration variable and its initialization to zero before loop.
+  ExprResult IV;
+  ExprResult Init;
+  {
+    VarDecl *IVDecl = BuildVarDecl(SemaRef, InitLoc,
+                                   LastIteration.get()->getType(), ".omp.iv");
+    IV = SemaRef.BuildDeclRefExpr(IVDecl, LastIteration.get()->getType(),
+                                  VK_LValue, InitLoc);
+    Init = SemaRef.BuildBinOp(
+        CurScope, InitLoc, BO_Assign, IV.get(),
+        SemaRef.ActOnIntegerConstant(SourceLocation(), 0).get());
+  }
+
+  // Loop condition (IV < NumIterations)
+  SourceLocation CondLoc;
+  ExprResult Cond = SemaRef.BuildBinOp(CurScope, CondLoc, BO_LT, IV.get(),
+                                       NumIterations.get());
+  // Loop condition with 1 iteration separated (IV < LastIteration)
+  ExprResult SeparatedCond = SemaRef.BuildBinOp(CurScope, CondLoc, BO_LT,
+                                                IV.get(), LastIteration.get());
+
+  // Loop increment (IV = IV + 1)
+  SourceLocation IncLoc;
+  ExprResult Inc =
+      SemaRef.BuildBinOp(CurScope, IncLoc, BO_Add, IV.get(),
+                         SemaRef.ActOnIntegerConstant(IncLoc, 1).get());
+  if (!Inc.isUsable())
+    return 0;
+  Inc = SemaRef.BuildBinOp(CurScope, IncLoc, BO_Assign, IV.get(), Inc.get());
+
+  // Build updates and final values of the loop counters.
+  bool HasErrors = false;
+  Built.Counters.resize(NestedLoopCount);
+  Built.Updates.resize(NestedLoopCount);
+  Built.Finals.resize(NestedLoopCount);
+  {
+    ExprResult Div;
+    // Go from inner nested loop to outer.
+    for (int Cnt = NestedLoopCount - 1; Cnt >= 0; --Cnt) {
+      LoopIterationSpace &IS = IterSpaces[Cnt];
+      SourceLocation UpdLoc = IS.IncSrcRange.getBegin();
+      // Build: Iter = (IV / Div) % IS.NumIters
+      // where Div is product of previous iterations' IS.NumIters.
+      ExprResult Iter;
+      if (Div.isUsable()) {
+        Iter =
+            SemaRef.BuildBinOp(CurScope, UpdLoc, BO_Div, IV.get(), Div.get());
+      } else {
+        Iter = IV;
+        assert((Cnt == (int)NestedLoopCount - 1) &&
+               "unusable div expected on first iteration only");
+      }
+
+      if (Cnt != 0 && Iter.isUsable())
+        Iter = SemaRef.BuildBinOp(CurScope, UpdLoc, BO_Rem, Iter.get(),
+                                  IS.NumIterations);
+      if (!Iter.isUsable()) {
+        HasErrors = true;
+        break;
+      }
+
+      // Build update: IS.CounterVar = IS.Start + Iter * IS.Step
+      ExprResult Update =
+          BuildCounterUpdate(SemaRef, CurScope, UpdLoc, IS.CounterVar,
+                             IS.CounterInit, Iter, IS.CounterStep, IS.Subtract);
+      if (!Update.isUsable()) {
+        HasErrors = true;
+        break;
+      }
+
+      // Build final: IS.CounterVar = IS.Start + IS.NumIters * IS.Step
+      ExprResult Final = BuildCounterUpdate(
+          SemaRef, CurScope, UpdLoc, IS.CounterVar, IS.CounterInit,
+          IS.NumIterations, IS.CounterStep, IS.Subtract);
+      if (!Final.isUsable()) {
+        HasErrors = true;
+        break;
+      }
+
+      // Build Div for the next iteration: Div <- Div * IS.NumIters
+      if (Cnt != 0) {
+        if (Div.isUnset())
+          Div = IS.NumIterations;
+        else
+          Div = SemaRef.BuildBinOp(CurScope, UpdLoc, BO_Mul, Div.get(),
+                                   IS.NumIterations);
+
+        // Add parentheses (for debugging purposes only).
+        if (Div.isUsable())
+          Div = SemaRef.ActOnParenExpr(UpdLoc, UpdLoc, Div.get());
+        if (!Div.isUsable()) {
+          HasErrors = true;
+          break;
+        }
+      }
+      if (!Update.isUsable() || !Final.isUsable()) {
+        HasErrors = true;
+        break;
+      }
+      // Save results
+      Built.Counters[Cnt] = IS.CounterVar;
+      Built.Updates[Cnt] = Update.get();
+      Built.Finals[Cnt] = Final.get();
+    }
+  }
+
+  if (HasErrors)
+    return 0;
+
+  // Save results
+  Built.IterationVarRef = IV.get();
+  Built.LastIteration = LastIteration.get();
+  Built.CalcLastIteration = CalcLastIteration.get();
+  Built.PreCond = PreCond.get();
+  Built.Cond = Cond.get();
+  Built.SeparatedCond = SeparatedCond.get();
+  Built.Init = Init.get();
+  Built.Inc = Inc.get();
+
   return NestedLoopCount;
 }
 
 static Expr *GetCollapseNumberExpr(ArrayRef<OMPClause *> Clauses) {
   auto CollapseFilter = [](const OMPClause *C) -> bool {
     return C->getClauseKind() == OMPC_collapse;
   };
   OMPExecutableDirective::filtered_clause_iterator<decltype(CollapseFilter)> I(
       Clauses, CollapseFilter);
   if (I)
     return cast<OMPCollapseClause>(*I)->getNumForLoops();
   return nullptr;
 }
 
 StmtResult Sema::ActOnOpenMPSimdDirective(
     ArrayRef<OMPClause *> Clauses, Stmt *AStmt, SourceLocation StartLoc,
     SourceLocation EndLoc,
     llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA) {
+  BuiltLoopExprs B;
   // In presence of clause 'collapse', it will define the nested loops number.
   unsigned NestedLoopCount =
       CheckOpenMPLoop(OMPD_simd, GetCollapseNumberExpr(Clauses), AStmt, *this,
-                      *DSAStack, VarsWithImplicitDSA);
+                      *DSAStack, VarsWithImplicitDSA, B);
   if (NestedLoopCount == 0)
     return StmtError();
 
+  assert((CurContext->isDependentContext() || B.builtAll()) &&
+         "omp simd loop exprs were not built");
+
   getCurFunction()->setHasBranchProtectedScope();
-  return OMPSimdDirective::Create(Context, StartLoc, EndLoc, NestedLoopCount,
-                                  Clauses, AStmt);
+  return OMPSimdDirective::Create(
+      Context, StartLoc, EndLoc, NestedLoopCount, Clauses, AStmt,
+      B.IterationVarRef, B.LastIteration, B.CalcLastIteration, B.PreCond,
+      B.Cond, B.SeparatedCond, B.Init, B.Inc, B.Counters, B.Updates, B.Finals);
 }
 
 StmtResult Sema::ActOnOpenMPForDirective(
     ArrayRef<OMPClause *> Clauses, Stmt *AStmt, SourceLocation StartLoc,
     SourceLocation EndLoc,
     llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA) {
+  BuiltLoopExprs B;
   // In presence of clause 'collapse', it will define the nested loops number.
   unsigned NestedLoopCount =
       CheckOpenMPLoop(OMPD_for, GetCollapseNumberExpr(Clauses), AStmt, *this,
-                      *DSAStack, VarsWithImplicitDSA);
+                      *DSAStack, VarsWithImplicitDSA, B);
   if (NestedLoopCount == 0)
     return StmtError();
 
+  assert((CurContext->isDependentContext() || B.builtAll()) &&
+         "omp for loop exprs were not built");
+
   getCurFunction()->setHasBranchProtectedScope();
-  return OMPForDirective::Create(Context, StartLoc, EndLoc, NestedLoopCount,
-                                 Clauses, AStmt);
+  return OMPForDirective::Create(
+      Context, StartLoc, EndLoc, NestedLoopCount, Clauses, AStmt,
+      B.IterationVarRef, B.LastIteration, B.CalcLastIteration, B.PreCond,
+      B.Cond, B.SeparatedCond, B.Init, B.Inc, B.Counters, B.Updates, B.Finals);
 }
 
 StmtResult Sema::ActOnOpenMPForSimdDirective(
     ArrayRef<OMPClause *> Clauses, Stmt *AStmt, SourceLocation StartLoc,
     SourceLocation EndLoc,
     llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA) {
+  BuiltLoopExprs B;
   // In presence of clause 'collapse', it will define the nested loops number.
   unsigned NestedLoopCount =
       CheckOpenMPLoop(OMPD_for_simd, GetCollapseNumberExpr(Clauses), AStmt,
-                      *this, *DSAStack, VarsWithImplicitDSA);
+                      *this, *DSAStack, VarsWithImplicitDSA, B);
   if (NestedLoopCount == 0)
     return StmtError();
 
   getCurFunction()->setHasBranchProtectedScope();
-  return OMPForSimdDirective::Create(Context, StartLoc, EndLoc, NestedLoopCount,
-                                     Clauses, AStmt);
+  return OMPForSimdDirective::Create(
+      Context, StartLoc, EndLoc, NestedLoopCount, Clauses, AStmt,
+      B.IterationVarRef, B.LastIteration, B.CalcLastIteration, B.PreCond,
+      B.Cond, B.SeparatedCond, B.Init, B.Inc, B.Counters, B.Updates, B.Finals);
 }
 
 StmtResult Sema::ActOnOpenMPSectionsDirective(ArrayRef<OMPClause *> Clauses,
                                               Stmt *AStmt,
                                               SourceLocation StartLoc,
                                               SourceLocation EndLoc) {
   assert(AStmt && isa<CapturedStmt>(AStmt) && "Captured statement expected");
   auto BaseStmt = AStmt;
StmtResult Sema::ActOnOpenMPParallelForDirective(
   CapturedStmt *CS = cast<CapturedStmt>(AStmt);
   // 1.2.2 OpenMP Language Terminology
   // Structured block - An executable statement with a single entry at the
   // top and a single exit at the bottom.
   // The point of exit cannot be a branch out of the structured block.
   // longjmp() and throw() must not violate the entry/exit criteria.
   CS->getCapturedDecl()->setNothrow();
 
+  BuiltLoopExprs B;
   // In presence of clause 'collapse', it will define the nested loops number.
   unsigned NestedLoopCount =
       CheckOpenMPLoop(OMPD_parallel_for, GetCollapseNumberExpr(Clauses), AStmt,
-                      *this, *DSAStack, VarsWithImplicitDSA);
+                      *this, *DSAStack, VarsWithImplicitDSA, B);
   if (NestedLoopCount == 0)
     return StmtError();
 
+  assert((CurContext->isDependentContext() || B.builtAll()) &&
+         "omp parallel for loop exprs were not built");
+
   getCurFunction()->setHasBranchProtectedScope();
-  return OMPParallelForDirective::Create(Context, StartLoc, EndLoc,
-                                         NestedLoopCount, Clauses, AStmt);
+  return OMPParallelForDirective::Create(
+      Context, StartLoc, EndLoc, NestedLoopCount, Clauses, AStmt,
+      B.IterationVarRef, B.LastIteration, B.CalcLastIteration, B.PreCond,
+      B.Cond, B.SeparatedCond, B.Init, B.Inc, B.Counters, B.Updates, B.Finals);
 }
 
 StmtResult Sema::ActOnOpenMPParallelForSimdDirective(
     ArrayRef<OMPClause *> Clauses, Stmt *AStmt, SourceLocation StartLoc,
     SourceLocation EndLoc,
     llvm::DenseMap<VarDecl *, Expr *> &VarsWithImplicitDSA) {
   assert(AStmt && isa<CapturedStmt>(AStmt) && "Captured statement expected");
   CapturedStmt *CS = cast<CapturedStmt>(AStmt);
   // 1.2.2 OpenMP Language Terminology
   // Structured block - An executable statement with a single entry at the
   // top and a single exit at the bottom.
   // The point of exit cannot be a branch out of the structured block.
   // longjmp() and throw() must not violate the entry/exit criteria.
   CS->getCapturedDecl()->setNothrow();
 
+  BuiltLoopExprs B;
   // In presence of clause 'collapse', it will define the nested loops number.
   unsigned NestedLoopCount =
       CheckOpenMPLoop(OMPD_parallel_for_simd, GetCollapseNumberExpr(Clauses),
-                      AStmt, *this, *DSAStack, VarsWithImplicitDSA);
+                      AStmt, *this, *DSAStack, VarsWithImplicitDSA, B);
   if (NestedLoopCount == 0)
     return StmtError();
 
   getCurFunction()->setHasBranchProtectedScope();
-  return OMPParallelForSimdDirective::Create(Context, StartLoc, EndLoc,
-                                             NestedLoopCount, Clauses, AStmt);
+  return OMPParallelForSimdDirective::Create(
+      Context, StartLoc, EndLoc, NestedLoopCount, Clauses, AStmt,
+      B.IterationVarRef, B.LastIteration, B.CalcLastIteration, B.PreCond,
+      B.Cond, B.SeparatedCond, B.Init, B.Inc, B.Counters, B.Updates, B.Finals);
 }
 
 StmtResult
 Sema::ActOnOpenMPParallelSectionsDirective(ArrayRef<OMPClause *> Clauses,
                                            Stmt *AStmt, SourceLocation StartLoc,
                                            SourceLocation EndLoc) {
   assert(AStmt && isa<CapturedStmt>(AStmt) && "Captured statement expected");
   auto BaseStmt = AStmt;

cfe/trunk/lib/Serialization/ASTReaderStmt.cpp

if (E->hasAssociatedStmt())
     E->setAssociatedStmt(Reader.ReadSubStmt());
 }
 
 void ASTStmtReader::VisitOMPLoopDirective(OMPLoopDirective *D) {
   VisitStmt(D);
   // Two fields (NumClauses and CollapsedNum) were read in ReadStmtFromStream.
   Idx += 2;
   VisitOMPExecutableDirective(D);
+  D->setIterationVariable(Reader.ReadSubExpr());
+  D->setLastIteration(Reader.ReadSubExpr());
+  D->setCalcLastIteration(Reader.ReadSubExpr());
+  D->setPreCond(Reader.ReadSubExpr());
+  auto Fst = Reader.ReadSubExpr();
+  auto Snd = Reader.ReadSubExpr();
+  D->setCond(Fst, Snd);
+  D->setInit(Reader.ReadSubExpr());
+  D->setInc(Reader.ReadSubExpr());
+  SmallVector<Expr *, 4> Sub;
+  unsigned CollapsedNum = D->getCollapsedNumber();
+  Sub.reserve(CollapsedNum);
+  for (unsigned i = 0; i < CollapsedNum; ++i)
+    Sub.push_back(Reader.ReadSubExpr());
+  D->setCounters(Sub);
+  Sub.clear();
+  for (unsigned i = 0; i < CollapsedNum; ++i)
+    Sub.push_back(Reader.ReadSubExpr());
+  D->setUpdates(Sub);
+  Sub.clear();
+  for (unsigned i = 0; i < CollapsedNum; ++i)
+    Sub.push_back(Reader.ReadSubExpr());
+  D->setFinals(Sub);
 }
 
 void ASTStmtReader::VisitOMPParallelDirective(OMPParallelDirective *D) {
   VisitStmt(D);
   // The NumClauses field was read in ReadStmtFromStream.
   ++Idx;
   VisitOMPExecutableDirective(D);
 }

cfe/trunk/lib/Serialization/ASTWriterStmt.cpp

if (E->hasAssociatedStmt())
     Writer.AddStmt(E->getAssociatedStmt());
 }
 
 void ASTStmtWriter::VisitOMPLoopDirective(OMPLoopDirective *D) {
   VisitStmt(D);
   Record.push_back(D->getNumClauses());
   Record.push_back(D->getCollapsedNumber());
   VisitOMPExecutableDirective(D);
+  Writer.AddStmt(D->getIterationVariable());
+  Writer.AddStmt(D->getLastIteration());
+  Writer.AddStmt(D->getCalcLastIteration());
+  Writer.AddStmt(D->getPreCond());
+  Writer.AddStmt(D->getCond(/* SeparateIter */ false));
+  Writer.AddStmt(D->getCond(/* SeparateIter */ true));
+  Writer.AddStmt(D->getInit());
+  Writer.AddStmt(D->getInc());
+  for (auto I : D->counters()) {
+    Writer.AddStmt(I);
+  }
+  for (auto I : D->updates()) {
+    Writer.AddStmt(I);
+  }
+  for (auto I : D->finals()) {
+    Writer.AddStmt(I);
+  }
 }
 
 void ASTStmtWriter::VisitOMPParallelDirective(OMPParallelDirective *D) {
   VisitStmt(D);
   Record.push_back(D->getNumClauses());
   VisitOMPExecutableDirective(D);
   Code = serialization::STMT_OMP_PARALLEL_DIRECTIVE;
 }

cfe/trunk/test/OpenMP/for_loop_messages.cpp

 class Iter0 {
 public:
   Iter0() {}
   Iter0(const Iter0 &) {}
   Iter0 operator++() { return *this; }
   Iter0 operator--() { return *this; }
   bool operator<(Iter0 a) { return true; }
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'Iter0' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'Iter0' for 1st argument}}
 int operator-(Iter0 a, Iter0 b) { return 0; }
 class Iter1 {
 public:
   Iter1(float f = 0.0f, double d = 0.0) {}
   Iter1(const Iter1 &) {}
   Iter1 operator++() { return *this; }
   Iter1 operator--() { return *this; }
   bool operator<(Iter1 a) { return true; }
   bool operator>=(Iter1 a) { return false; }
 };
 class GoodIter {
 public:
   GoodIter() {}
   GoodIter(const GoodIter &) {}
   GoodIter(int fst, int snd) {}
   GoodIter &operator=(const GoodIter &that) { return *this; }
   GoodIter &operator=(const Iter0 &that) { return *this; }
   GoodIter &operator+=(int x) { return *this; }
+  GoodIter &operator-=(int x) { return *this; }
   explicit GoodIter(void *) {}
   GoodIter operator++() { return *this; }
   GoodIter operator--() { return *this; }
   bool operator!() { return true; }
   bool operator<(GoodIter a) { return true; }
   bool operator<=(GoodIter a) { return true; }
   bool operator>=(GoodIter a) { return false; }
   typedef int difference_type;
   typedef std::random_access_iterator_tag iterator_category;
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'GoodIter' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 int operator-(GoodIter a, GoodIter b) { return 0; }
+// expected-note@+1 3 {{candidate function not viable: requires single argument 'a', but 2 arguments were provided}}
 GoodIter operator-(GoodIter a) { return a; }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'int' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 GoodIter operator-(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'GoodIter' for 1st argument}}
 GoodIter operator+(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'int' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'int' for 1st argument}}
 GoodIter operator-(int v, GoodIter a) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'int' for 1st argument}}
 GoodIter operator+(int v, GoodIter a) { return GoodIter(); }
 
 int test_with_random_access_iterator() {
   GoodIter begin, end;
   Iter0 begin0, end0;
 #pragma omp parallel
 #pragma omp for
   for (GoodIter I = begin; I < end; ++I)

 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp for
   for (GoodIter I(1, 2); I < end; ++I)
     ++I;
 #pragma omp parallel
 #pragma omp for
   for (begin = GoodIter(0); begin < end; ++begin)
     ++begin;
+// expected-error@+4 {{invalid operands to binary expression ('GoodIter' and 'const Iter0')}}
+// expected-error@+3 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp parallel
 #pragma omp for
   for (begin = begin0; begin < end; ++begin)
     ++begin;
 #pragma omp parallel
 // expected-error@+2 {{initialization clause of OpenMP for loop must be of the form 'var = init' or 'T var = init'}}
 #pragma omp for
   for (++begin; begin < end; ++begin)

 #pragma omp for
   for (GoodIter I = begin; I >= end; I = 2 + I)
     ++I;
 #pragma omp parallel
 // expected-error@+2 {{increment clause of OpenMP for loop must perform simple addition or subtraction on loop variable 'I'}}
 #pragma omp for
   for (GoodIter I = begin; I >= end; I = 2 - I)
     ++I;
+// In the following example, we cannot update the loop variable using '+='
+// expected-error@+3 {{invalid operands to binary expression ('Iter0' and 'int')}}
 #pragma omp parallel
 #pragma omp for
   for (Iter0 I = begin0; I < end0; ++I)
     ++I;
 #pragma omp parallel
 // Initializer is constructor without params.
+// expected-error@+3 {{invalid operands to binary expression ('Iter0' and 'int')}}
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp for
   for (Iter0 I; I < end0; ++I)
     ++I;
   Iter1 begin1, end1;
+// expected-error@+4 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+3 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp parallel
 #pragma omp for
   for (Iter1 I = begin1; I < end1; ++I)
     ++I;
 #pragma omp parallel
 // expected-note@+3 {{loop step is expected to be negative due to this condition}}
 // expected-error@+2 {{increment expression must cause 'I' to decrease on each iteration of OpenMP for loop}}
 #pragma omp for
   for (Iter1 I = begin1; I >= end1; ++I)
     ++I;
 #pragma omp parallel
+// expected-error@+5 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+4 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 // Initializer is constructor with all default params.
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp for
   for (Iter1 I; I < end1; ++I) {
   }
   return 0;
 }
 

cfe/trunk/test/OpenMP/for_simd_loop_messages.cpp

 class Iter0 {
 public:
   Iter0() {}
   Iter0(const Iter0 &) {}
   Iter0 operator++() { return *this; }
   Iter0 operator--() { return *this; }
   bool operator<(Iter0 a) { return true; }
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'Iter0' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'Iter0' for 1st argument}}
 int operator-(Iter0 a, Iter0 b) { return 0; }
 class Iter1 {
 public:
   Iter1(float f = 0.0f, double d = 0.0) {}
   Iter1(const Iter1 &) {}
   Iter1 operator++() { return *this; }
   Iter1 operator--() { return *this; }
   bool operator<(Iter1 a) { return true; }
public:
   GoodIter operator--() { return *this; }
   bool operator!() { return true; }
   bool operator<(GoodIter a) { return true; }
   bool operator<=(GoodIter a) { return true; }
   bool operator>=(GoodIter a) { return false; }
   typedef int difference_type;
   typedef std::random_access_iterator_tag iterator_category;
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'GoodIter' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 int operator-(GoodIter a, GoodIter b) { return 0; }
+// expected-note@+1 3 {{candidate function not viable: requires single argument 'a', but 2 arguments were provided}}
 GoodIter operator-(GoodIter a) { return a; }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'int' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 GoodIter operator-(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'GoodIter' for 1st argument}}
 GoodIter operator+(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'int' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'int' for 1st argument}}
 GoodIter operator-(int v, GoodIter a) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'int' for 1st argument}}
 GoodIter operator+(int v, GoodIter a) { return GoodIter(); }
 
 int test_with_random_access_iterator() {
   GoodIter begin, end;
   Iter0 begin0, end0;
 #pragma omp parallel
 #pragma omp for simd
   for (GoodIter I = begin; I < end; ++I)

 #pragma omp for simd
   for (GoodIter I(1, 2); I < end; ++I)
     ++I;
 #pragma omp parallel
 #pragma omp for simd
   for (begin = GoodIter(0); begin < end; ++begin)
     ++begin;
 #pragma omp parallel
+// expected-error@+3 {{invalid operands to binary expression ('GoodIter' and 'const Iter0')}}
+// expected-error@+2 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp for simd
   for (begin = begin0; begin < end; ++begin)
     ++begin;
 #pragma omp parallel
 // expected-error@+2 {{initialization clause of OpenMP for loop must be of the form 'var = init' or 'T var = init'}}
 #pragma omp for simd
   for (++begin; begin < end; ++begin)
     ++begin;
#pragma omp for simd
   for (GoodIter I = begin; I >= end; I = 2 + I)
     ++I;
 #pragma omp parallel
 // expected-error@+2 {{increment clause of OpenMP for loop must perform simple addition or subtraction on loop variable 'I'}}
 #pragma omp for simd
   for (GoodIter I = begin; I >= end; I = 2 - I)
     ++I;
 #pragma omp parallel
+// expected-error@+2 {{invalid operands to binary expression ('Iter0' and 'int')}}
 #pragma omp for simd
   for (Iter0 I = begin0; I < end0; ++I)
     ++I;
 #pragma omp parallel
 // Initializer is constructor without params.
+// expected-error@+3 {{invalid operands to binary expression ('Iter0' and 'int')}}
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp for simd
   for (Iter0 I; I < end0; ++I)
     ++I;
   Iter1 begin1, end1;
 #pragma omp parallel
+// expected-error@+3 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+2 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp for simd
   for (Iter1 I = begin1; I < end1; ++I)
     ++I;
 #pragma omp parallel
 // expected-note@+3 {{loop step is expected to be negative due to this condition}}
 // expected-error@+2 {{increment expression must cause 'I' to decrease on each iteration of OpenMP for loop}}
 #pragma omp for simd
   for (Iter1 I = begin1; I >= end1; ++I)
     ++I;
 #pragma omp parallel
+// expected-error@+5 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+4 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 // Initializer is constructor with all default params.
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp for simd
   for (Iter1 I; I < end1; ++I) {
   }
   return 0;
 }
 

cfe/trunk/test/OpenMP/parallel_for_loop_messages.cpp

 class Iter0 {
 public:
   Iter0() {}
   Iter0(const Iter0 &) {}
   Iter0 operator++() { return *this; }
   Iter0 operator--() { return *this; }
   bool operator<(Iter0 a) { return true; }
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'Iter0' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'Iter0' for 1st argument}}
 int operator-(Iter0 a, Iter0 b) { return 0; }
 class Iter1 {
 public:
   Iter1(float f = 0.0f, double d = 0.0) {}
   Iter1(const Iter1 &) {}
   Iter1 operator++() { return *this; }
   Iter1 operator--() { return *this; }
   bool operator<(Iter1 a) { return true; }
   bool operator>=(Iter1 a) { return false; }
 };
 class GoodIter {
 public:
   GoodIter() {}
   GoodIter(const GoodIter &) {}
   GoodIter(int fst, int snd) {}
   GoodIter &operator=(const GoodIter &that) { return *this; }
   GoodIter &operator=(const Iter0 &that) { return *this; }
   GoodIter &operator+=(int x) { return *this; }
+  GoodIter &operator-=(int x) { return *this; }
   explicit GoodIter(void *) {}
   GoodIter operator++() { return *this; }
   GoodIter operator--() { return *this; }
   bool operator!() { return true; }
   bool operator<(GoodIter a) { return true; }
   bool operator<=(GoodIter a) { return true; }
   bool operator>=(GoodIter a) { return false; }
   typedef int difference_type;
   typedef std::random_access_iterator_tag iterator_category;
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'GoodIter' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 int operator-(GoodIter a, GoodIter b) { return 0; }
+// expected-note@+1 3 {{candidate function not viable: requires single argument 'a', but 2 arguments were provided}}
 GoodIter operator-(GoodIter a) { return a; }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'int' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 GoodIter operator-(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'GoodIter' for 1st argument}}
 GoodIter operator+(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'int' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'int' for 1st argument}}
 GoodIter operator-(int v, GoodIter a) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'int' for 1st argument}}
 GoodIter operator+(int v, GoodIter a) { return GoodIter(); }
 
 int test_with_random_access_iterator() {
   GoodIter begin, end;
   Iter0 begin0, end0;
 #pragma omp parallel for
   for (GoodIter I = begin; I < end; ++I)
     ++I;
for (GoodIter I(0); I < end; ++I)
     ++I;
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp parallel for
   for (GoodIter I(1, 2); I < end; ++I)
     ++I;
 #pragma omp parallel for
   for (begin = GoodIter(0); begin < end; ++begin)
     ++begin;
+// expected-error@+3 {{invalid operands to binary expression ('GoodIter' and 'const Iter0')}}
+// expected-error@+2 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp parallel for
   for (begin = begin0; begin < end; ++begin)
     ++begin;
 // expected-error@+2 {{initialization clause of OpenMP for loop must be of the form 'var = init' or 'T var = init'}}
 #pragma omp parallel for
   for (++begin; begin < end; ++begin)
     ++begin;
 #pragma omp parallel for

 // expected-error@+2 {{increment expression must cause 'I' to decrease on each iteration of OpenMP for loop}}
 #pragma omp parallel for
   for (GoodIter I = begin; I >= end; I = 2 + I)
     ++I;
 // expected-error@+2 {{increment clause of OpenMP for loop must perform simple addition or subtraction on loop variable 'I'}}
 #pragma omp parallel for
   for (GoodIter I = begin; I >= end; I = 2 - I)
     ++I;
+// expected-error@+2 {{invalid operands to binary expression ('Iter0' and 'int')}}
 #pragma omp parallel for
   for (Iter0 I = begin0; I < end0; ++I)
     ++I;
 // Initializer is constructor without params.
+// expected-error@+3 {{invalid operands to binary expression ('Iter0' and 'int')}}
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp parallel for
   for (Iter0 I; I < end0; ++I)
     ++I;
   Iter1 begin1, end1;
+// expected-error@+3 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+2 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp parallel for
   for (Iter1 I = begin1; I < end1; ++I)
     ++I;
 // expected-note@+3 {{loop step is expected to be negative due to this condition}}
 // expected-error@+2 {{increment expression must cause 'I' to decrease on each iteration of OpenMP for loop}}
 #pragma omp parallel for
   for (Iter1 I = begin1; I >= end1; ++I)
     ++I;
+// expected-error@+5 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+4 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 // Initializer is constructor with all default params.
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp parallel for
   for (Iter1 I; I < end1; ++I) {
   }
   return 0;
 }
 

cfe/trunk/test/OpenMP/parallel_for_simd_loop_messages.cpp

 class Iter0 {
 public:
   Iter0() {}
   Iter0(const Iter0 &) {}
   Iter0 operator++() { return *this; }
   Iter0 operator--() { return *this; }
   bool operator<(Iter0 a) { return true; }
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'Iter0' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'Iter0' for 1st argument}}
 int operator-(Iter0 a, Iter0 b) { return 0; }
 class Iter1 {
 public:
   Iter1(float f = 0.0f, double d = 0.0) {}
   Iter1(const Iter1 &) {}
   Iter1 operator++() { return *this; }
   Iter1 operator--() { return *this; }
   bool operator<(Iter1 a) { return true; }
public:
   GoodIter operator--() { return *this; }
   bool operator!() { return true; }
   bool operator<(GoodIter a) { return true; }
   bool operator<=(GoodIter a) { return true; }
   bool operator>=(GoodIter a) { return false; }
   typedef int difference_type;
   typedef std::random_access_iterator_tag iterator_category;
 };
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'GoodIter' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 int operator-(GoodIter a, GoodIter b) { return 0; }
+// expected-note@+1 3 {{candidate function not viable: requires single argument 'a', but 2 arguments were provided}}
 GoodIter operator-(GoodIter a) { return a; }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'const Iter0' to 'int' for 2nd argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 GoodIter operator-(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'GoodIter' for 1st argument}}
 GoodIter operator+(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+2 {{candidate function not viable: no known conversion from 'GoodIter' to 'int' for 1st argument}}
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'int' for 1st argument}}
 GoodIter operator-(int v, GoodIter a) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter0' to 'int' for 1st argument}}
 GoodIter operator+(int v, GoodIter a) { return GoodIter(); }
 
 int test_with_random_access_iterator() {
   GoodIter begin, end;
   Iter0 begin0, end0;
 #pragma omp parallel for simd
   for (GoodIter I = begin; I < end; ++I)
     ++I;
for (GoodIter I(0); I < end; ++I)
     ++I;
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp parallel for simd
   for (GoodIter I(1, 2); I < end; ++I)
     ++I;
 #pragma omp parallel for simd
   for (begin = GoodIter(0); begin < end; ++begin)
     ++begin;
+// expected-error@+3 {{invalid operands to binary expression ('GoodIter' and 'const Iter0')}}
+// expected-error@+2 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp parallel for simd
   for (begin = begin0; begin < end; ++begin)
     ++begin;
 // expected-error@+2 {{initialization clause of OpenMP for loop must be of the form 'var = init' or 'T var = init'}}
 #pragma omp parallel for simd
   for (++begin; begin < end; ++begin)
     ++begin;
 #pragma omp parallel for simd

 // expected-error@+2 {{increment expression must cause 'I' to decrease on each iteration of OpenMP for loop}}
 #pragma omp parallel for simd
   for (GoodIter I = begin; I >= end; I = 2 + I)
     ++I;
 // expected-error@+2 {{increment clause of OpenMP for loop must perform simple addition or subtraction on loop variable 'I'}}
 #pragma omp parallel for simd
   for (GoodIter I = begin; I >= end; I = 2 - I)
     ++I;
+// expected-error@+2 {{invalid operands to binary expression ('Iter0' and 'int')}}
 #pragma omp parallel for simd
   for (Iter0 I = begin0; I < end0; ++I)
     ++I;
 // Initializer is constructor without params.
+// expected-error@+3 {{invalid operands to binary expression ('Iter0' and 'int')}}
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp parallel for simd
   for (Iter0 I; I < end0; ++I)
     ++I;
   Iter1 begin1, end1;
+// expected-error@+3 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+2 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 #pragma omp parallel for simd
   for (Iter1 I = begin1; I < end1; ++I)
     ++I;
 // expected-note@+3 {{loop step is expected to be negative due to this condition}}
 // expected-error@+2 {{increment expression must cause 'I' to decrease on each iteration of OpenMP for loop}}
 #pragma omp parallel for simd
   for (Iter1 I = begin1; I >= end1; ++I)
     ++I;
+// expected-error@+5 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+// expected-error@+4 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
 // Initializer is constructor with all default params.
 // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
 #pragma omp parallel for simd
   for (Iter1 I; I < end1; ++I) {
   }
   return 0;
 }
 

cfe/trunk/test/OpenMP/simd_codegen.cpp

+// RUN: %clang_cc1 -verify -fopenmp=libiomp5 -x c++ -emit-llvm %s -fexceptions -fcxx-exceptions -o - | FileCheck %s
+// RUN: %clang_cc1 -fopenmp=libiomp5 -x c++ -std=c++11 -triple x86_64-unknown-unknown -fexceptions -fcxx-exceptions -emit-pch -o %t %s
+// RUN: %clang_cc1 -fopenmp=libiomp5 -x c++ -triple x86_64-unknown-unknown -fexceptions -fcxx-exceptions -g -std=c++11 -include-pch %t -verify %s -emit-llvm -o - | FileCheck %s
+//
+// expected-no-diagnostics
+#ifndef HEADER
+#define HEADER
+
+// CHECK-LABEL: define {{.*void}} @{{.*}}simple{{.*}}(float* {{.+}}, float* {{.+}}, float* {{.+}}, float* {{.+}})
+void simple(float *a, float *b, float *c, float *d) {
+  #pragma omp simd
+// CHECK: store i32 0, i32* [[OMP_IV:%[^,]+]]
+
+// CHECK: [[IV:%.+]] = load i32* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP1_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP:%.+]] = icmp slt i32 [[IV]], 6
+// CHECK-NEXT: br i1 [[CMP]], label %[[SIMPLE_LOOP1_BODY:.+]], label %[[SIMPLE_LOOP1_END:[^,]+]]
+  for (int i = 3; i < 32; i += 5) {
+// CHECK: [[SIMPLE_LOOP1_BODY]]
+// Start of body: calculate i from IV:
+// CHECK: [[IV1_1:%.+]] = load i32* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP1_ID]]
+// CHECK: [[CALC_I_1:%.+]] = mul nsw i32 [[IV1_1]], 5
+// CHECK-NEXT: [[CALC_I_2:%.+]] = add nsw i32 3, [[CALC_I_1]]
+// CHECK-NEXT: store i32 [[CALC_I_2]], i32* [[LC_I:.+]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP1_ID]]
+// ... loop body ...
+// End of body: store into a[i]:
+// CHECK: store float [[RESULT:%.+]], float* {{%.+}}{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP1_ID]]
+    a[i] = b[i] * c[i] * d[i];
+// CHECK: [[IV1_2:%.+]] = load i32* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP1_ID]]
+// CHECK-NEXT: [[ADD1_2:%.+]] = add nsw i32 [[IV1_2]], 1
+// CHECK-NEXT: store i32 [[ADD1_2]], i32* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP1_ID]]
+// br label %{{.+}}, !llvm.loop !{{.+}}
+  }
+// CHECK: [[SIMPLE_LOOP1_END]]
+
+  #pragma omp simd
+// CHECK: store i32 0, i32* [[OMP_IV2:%[^,]+]]
+
+// CHECK: [[IV2:%.+]] = load i32* [[OMP_IV2]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP2_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP2:%.+]] = icmp slt i32 [[IV2]], 9
+// CHECK-NEXT: br i1 [[CMP2]], label %[[SIMPLE_LOOP2_BODY:.+]], label %[[SIMPLE_LOOP2_END:[^,]+]]
+  for (int i = 10; i > 1; i--) {
+// CHECK: [[SIMPLE_LOOP2_BODY]]
+// Start of body: calculate i from IV:
+// CHECK: [[IV2_0:%.+]] = load i32* [[OMP_IV2]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP2_ID]]
+// FIXME: It is interesting, why the following "mul 1" was not constant folded?
+// CHECK-NEXT: [[IV2_1:%.+]] = mul nsw i32 [[IV2_0]], 1
+// CHECK-NEXT: [[LC_I_1:%.+]] = sub nsw i32 10, [[IV2_1]]
+// CHECK-NEXT: store i32 [[LC_I_1]], i32* {{.+}}, !llvm.mem.parallel_loop_access ![[SIMPLE_LOOP2_ID]]
+    a[i]++;
+// CHECK: [[IV2_2:%.+]] = load i32* [[OMP_IV2]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP2_ID]]
+// CHECK-NEXT: [[ADD2_2:%.+]] = add nsw i32 [[IV2_2]], 1
+// CHECK-NEXT: store i32 [[ADD2_2]], i32* [[OMP_IV2]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP2_ID]]
+// br label {{.+}}, !llvm.loop ![[SIMPLE_LOOP2_ID]]
+  }
+// CHECK: [[SIMPLE_LOOP2_END]]
+
+  #pragma omp simd
+// CHECK: store i64 0, i64* [[OMP_IV3:%[^,]+]]
+
+// CHECK: [[IV3:%.+]] = load i64* [[OMP_IV3]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP3_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP3:%.+]] = icmp ult i64 [[IV3]], 4
+// CHECK-NEXT: br i1 [[CMP3]], label %[[SIMPLE_LOOP3_BODY:.+]], label %[[SIMPLE_LOOP3_END:[^,]+]]
+  for (unsigned long long it = 2000; it >= 600; it-=400) {
+// CHECK: [[SIMPLE_LOOP3_BODY]]
+// Start of body: calculate it from IV:
+// CHECK: [[IV3_0:%.+]] = load i64* [[OMP_IV3]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP3_ID]]
+// CHECK-NEXT: [[LC_IT_1:%.+]] = mul i64 [[IV3_0]], 400
+// CHECK-NEXT: [[LC_IT_2:%.+]] = sub i64 2000, [[LC_IT_1]]
+// CHECK-NEXT: store i64 [[LC_IT_2]], i64* {{.+}}, !llvm.mem.parallel_loop_access ![[SIMPLE_LOOP3_ID]]
+    a[it]++;
+// CHECK: [[IV3_2:%.+]] = load i64* [[OMP_IV3]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP3_ID]]
+// CHECK-NEXT: [[ADD3_2:%.+]] = add i64 [[IV3_2]], 1
+// CHECK-NEXT: store i64 [[ADD3_2]], i64* [[OMP_IV3]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP3_ID]]
+  }
+// CHECK: [[SIMPLE_LOOP3_END]]
+
+  #pragma omp simd
+// CHECK: store i32 0, i32* [[OMP_IV4:%[^,]+]]
+
+// CHECK: [[IV4:%.+]] = load i32* [[OMP_IV4]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP4_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP4:%.+]] = icmp slt i32 [[IV4]], 4
+// CHECK-NEXT: br i1 [[CMP4]], label %[[SIMPLE_LOOP4_BODY:.+]], label %[[SIMPLE_LOOP4_END:[^,]+]]
+  for (short it = 6; it <= 20; it-=-4) {
+// CHECK: [[SIMPLE_LOOP4_BODY]]
+// Start of body: calculate it from IV:
+// CHECK: [[IV4_0:%.+]] = load i32* [[OMP_IV4]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP4_ID]]
+// CHECK-NEXT: [[LC_IT_1:%.+]] = mul nsw i32 [[IV4_0]], 4
+// CHECK-NEXT: [[LC_IT_2:%.+]] = add nsw i32 6, [[LC_IT_1]]
+// CHECK-NEXT: [[LC_IT_3:%.+]] = trunc i32 [[LC_IT_2]] to i16
+// CHECK-NEXT: store i16 [[LC_IT_3]], i16* {{.+}}, !llvm.mem.parallel_loop_access ![[SIMPLE_LOOP4_ID]]
+
+// CHECK: [[IV4_2:%.+]] = load i32* [[OMP_IV4]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP4_ID]]
+// CHECK-NEXT: [[ADD4_2:%.+]] = add nsw i32 [[IV4_2]], 1
+// CHECK-NEXT: store i32 [[ADD4_2]], i32* [[OMP_IV4]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP4_ID]]
+  }
+// CHECK: [[SIMPLE_LOOP4_END]]
+
+  #pragma omp simd
+// CHECK: store i32 0, i32* [[OMP_IV5:%[^,]+]]
+
+// CHECK: [[IV5:%.+]] = load i32* [[OMP_IV5]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP5_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP5:%.+]] = icmp slt i32 [[IV5]], 26
+// CHECK-NEXT: br i1 [[CMP5]], label %[[SIMPLE_LOOP5_BODY:.+]], label %[[SIMPLE_LOOP5_END:[^,]+]]
+  for (unsigned char it = 'z'; it >= 'a'; it+=-1) {
+// CHECK: [[SIMPLE_LOOP5_BODY]]
+// Start of body: calculate it from IV:
+// CHECK: [[IV5_0:%.+]] = load i32* [[OMP_IV5]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP5_ID]]
+// CHECK-NEXT: [[IV5_1:%.+]] = mul nsw i32 [[IV5_0]], 1
+// CHECK-NEXT: [[LC_IT_1:%.+]] = sub nsw i32 122, [[IV5_1]]
+// CHECK-NEXT: [[LC_IT_2:%.+]] = trunc i32 [[LC_IT_1]] to i8
+// CHECK-NEXT: store i8 [[LC_IT_2]], i8* {{.+}}, !llvm.mem.parallel_loop_access ![[SIMPLE_LOOP5_ID]]
+
+// CHECK: [[IV5_2:%.+]] = load i32* [[OMP_IV5]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP5_ID]]
+// CHECK-NEXT: [[ADD5_2:%.+]] = add nsw i32 [[IV5_2]], 1
+// CHECK-NEXT: store i32 [[ADD5_2]], i32* [[OMP_IV5]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP5_ID]]
+  }
+// CHECK: [[SIMPLE_LOOP5_END]]
+
+  #pragma omp simd
+// FIXME: I think we would get wrong result using 'unsigned' in the loop below.
+// So we'll need to add zero trip test for 'unsigned' counters.
+//
+// CHECK: store i32 0, i32* [[OMP_IV6:%[^,]+]]
+
+// CHECK: [[IV6:%.+]] = load i32* [[OMP_IV6]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP6_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP6:%.+]] = icmp slt i32 [[IV6]], -8
+// CHECK-NEXT: br i1 [[CMP6]], label %[[SIMPLE_LOOP6_BODY:.+]], label %[[SIMPLE_LOOP6_END:[^,]+]]
+  for (int i=100; i<10; i+=10) {
+// CHECK: [[SIMPLE_LOOP6_BODY]]
+// Start of body: calculate i from IV:
+// CHECK: [[IV6_0:%.+]] = load i32* [[OMP_IV6]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP6_ID]]
+// CHECK-NEXT: [[LC_IT_1:%.+]] = mul nsw i32 [[IV6_0]], 10
+// CHECK-NEXT: [[LC_IT_2:%.+]] = add nsw i32 100, [[LC_IT_1]]
+// CHECK-NEXT: store i32 [[LC_IT_2]], i32* {{.+}}, !llvm.mem.parallel_loop_access ![[SIMPLE_LOOP6_ID]]
+
+// CHECK: [[IV6_2:%.+]] = load i32* [[OMP_IV6]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP6_ID]]
+// CHECK-NEXT: [[ADD6_2:%.+]] = add nsw i32 [[IV6_2]], 1
+// CHECK-NEXT: store i32 [[ADD6_2]], i32* [[OMP_IV6]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP6_ID]]
+  }
+// CHECK: [[SIMPLE_LOOP6_END]]
+
+  int A;
+  #pragma omp simd lastprivate(A)
+// Clause 'lastprivate' implementation is not completed yet.
+// Test checks that one iteration is separated in presence of lastprivate.
+//
+// CHECK: store i64 0, i64* [[OMP_IV7:%[^,]+]]
+// CHECK: br i1 true, label %[[SIMPLE_IF7_THEN:.+]], label %[[SIMPLE_IF7_END:[^,]+]]
+// CHECK: [[SIMPLE_IF7_THEN]]
+// CHECK: br label %[[SIMD_LOOP7_COND:[^,]+]]
+// CHECK: [[SIMD_LOOP7_COND]]
+// CHECK-NEXT: [[IV7:%.+]] = load i64* [[OMP_IV7]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP7_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP7:%.+]] = icmp slt i64 [[IV7]], 6
+// CHECK-NEXT: br i1 [[CMP7]], label %[[SIMPLE_LOOP7_BODY:.+]], label %[[SIMPLE_LOOP7_END:[^,]+]]
+  for (long long i = -10; i < 10; i += 3) {
+// CHECK: [[SIMPLE_LOOP7_BODY]]
+// Start of body: calculate i from IV:
+// CHECK: [[IV7_0:%.+]] = load i64* [[OMP_IV7]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP7_ID]]
+// CHECK-NEXT: [[LC_IT_1:%.+]] = mul nsw i64 [[IV7_0]], 3
+// CHECK-NEXT: [[LC_IT_2:%.+]] = add nsw i64 -10, [[LC_IT_1]]
+// CHECK-NEXT: store i64 [[LC_IT_2]], i64* {{.+}}, !llvm.mem.parallel_loop_access ![[SIMPLE_LOOP7_ID]]
+    A = i;
+// CHECK: [[IV7_2:%.+]] = load i64* [[OMP_IV7]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP7_ID]]
+// CHECK-NEXT: [[ADD7_2:%.+]] = add nsw i64 [[IV7_2]], 1
+// CHECK-NEXT: store i64 [[ADD7_2]], i64* [[OMP_IV7]]{{.*}}!llvm.mem.parallel_loop_access ![[SIMPLE_LOOP7_ID]]
+  }
+// CHECK: [[SIMPLE_LOOP7_END]]
+// Separated last iteration.
+// CHECK: [[IV7_4:%.+]] = load i64* [[OMP_IV7]]
+// CHECK-NEXT: [[LC_FIN_1:%.+]] = mul nsw i64 [[IV7_4]], 3
+// CHECK-NEXT: [[LC_FIN_2:%.+]] = add nsw i64 -10, [[LC_FIN_1]]
+// CHECK-NEXT: store i64 [[LC_FIN_2]], i64* [[ADDR_I:%[^,]+]]
+// CHECK: [[LOAD_I:%.+]] = load i64* [[ADDR_I]]
+// CHECK-NEXT: [[CONV_I:%.+]] = trunc i64 [[LOAD_I]] to i32
+//
+// CHECK: br label %[[SIMPLE_IF7_END]]
+// CHECK: [[SIMPLE_IF7_END]]
+//
+
+// CHECK: ret void
+}
+
+template <class T, unsigned K> T tfoo(T a) { return a + K; }
+
+template <typename T, unsigned N>
+int templ1(T a, T *z) {
+  #pragma omp simd collapse(N)
+  for (int i = 0; i < N * 2; i++) {
+    for (long long j = 0; j < (N + N + N + N); j += 2) {
+      z[i + j] = a + tfoo<T, N>(i + j);
+    }
+  }
+  return 0;
+}
+
+// Instatiation templ1<float,2>
+// CHECK-LABEL: define {{.*i32}} @{{.*}}templ1{{.*}}(float {{.+}}, float* {{.+}})
+// CHECK: store i64 0, i64* [[T1_OMP_IV:[^,]+]]
+// ...
+// CHECK: [[IV:%.+]] = load i64* [[T1_OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[T1_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP1:%.+]] = icmp slt i64 [[IV]], 16
+// CHECK-NEXT: br i1 [[CMP1]], label %[[T1_BODY:.+]], label %[[T1_END:[^,]+]]
+// CHECK: [[T1_BODY]]
+// Loop counters i and j updates:
+// CHECK: [[IV1:%.+]] = load i64* [[T1_OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[T1_ID]]
+// CHECK-NEXT: [[I_1:%.+]] = sdiv i64 [[IV1]], 4
+// CHECK-NEXT: [[I_1_MUL1:%.+]] = mul nsw i64 [[I_1]], 1
+// CHECK-NEXT: [[I_1_ADD0:%.+]] = add nsw i64 0, [[I_1_MUL1]]
+// CHECK-NEXT: [[I_2:%.+]] = trunc i64 [[I_1_ADD0]] to i32
+// CHECK-NEXT: store i32 [[I_2]], i32* {{%.+}}{{.*}}!llvm.mem.parallel_loop_access ![[T1_ID]]
+// CHECK: [[IV2:%.+]] = load i64* [[T1_OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[T1_ID]]
+// CHECK-NEXT: [[J_1:%.+]] = srem i64 [[IV2]], 4
+// CHECK-NEXT: [[J_2:%.+]] = mul nsw i64 [[J_1]], 2
+// CHECK-NEXT: [[J_2_ADD0:%.+]] = add nsw i64 0, [[J_2]]
+// CHECK-NEXT: store i64 [[J_2_ADD0]], i64* {{%.+}}{{.*}}!llvm.mem.parallel_loop_access ![[T1_ID]]
+// simd.for.inc:
+// CHECK: [[IV3:%.+]] = load i64* [[T1_OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[T1_ID]]
+// CHECK-NEXT: [[INC:%.+]] = add nsw i64 [[IV3]], 1
+// CHECK-NEXT: store i64 [[INC]], i64* [[T1_OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[T1_ID]]
+// CHECK-NEXT: br label {{%.+}}
+// CHECK: [[T1_END]]
+// CHECK: ret i32 0
+//
+void inst_templ1() {
+  float a;
+  float z[100];
+  templ1<float,2> (a, z);
+}
+
+
+typedef int MyIdx;
+
+class IterDouble {
+  double *Ptr;
+public:
+  IterDouble operator++ () const {
+    IterDouble n;
+    n.Ptr = Ptr + 1;
+    return n;
+  }
+  bool operator < (const IterDouble &that) const {
+    return Ptr < that.Ptr;
+  }
+  double & operator *() const {
+    return *Ptr;
+  }
+  MyIdx operator - (const IterDouble &that) const {
+    return (MyIdx) (Ptr - that.Ptr);
+  }
+  IterDouble operator + (int Delta) {
+    IterDouble re;
+    re.Ptr = Ptr + Delta;
+    return re;
+  }
+
+  ///~IterDouble() {}
+};
+
+// CHECK-LABEL: define {{.*void}} @{{.*}}iter_simple{{.*}}
+void iter_simple(IterDouble ia, IterDouble ib, IterDouble ic) {
+//
+// CHECK: store i32 0, i32* [[IT_OMP_IV:%[^,]+]]
+// Calculate number of iterations before the loop body.
+// CHECK: [[DIFF1:%.+]] = call {{.*}}i32 @{{.*}}IterDouble{{.*}}
+// CHECK-NEXT: [[DIFF2:%.+]] = sub nsw i32 [[DIFF1]], 1
+// CHECK-NEXT: [[DIFF3:%.+]] = add nsw i32 [[DIFF2]], 1
+// CHECK-NEXT: [[DIFF4:%.+]] = sdiv i32 [[DIFF3]], 1
+// CHECK-NEXT: [[DIFF5:%.+]] = sub nsw i32 [[DIFF4]], 1
+// CHECK-NEXT: store i32 [[DIFF5]], i32* [[OMP_LAST_IT:%[^,]+]]{{.+}}
+  #pragma omp simd
+
+// CHECK: [[IV:%.+]] = load i32* [[IT_OMP_IV]]{{.+}} !llvm.mem.parallel_loop_access ![[ITER_LOOP_ID:[0-9]+]]
+// CHECK-NEXT: [[LAST_IT:%.+]] = load i32* [[OMP_LAST_IT]]{{.+}}!llvm.mem.parallel_loop_access ![[ITER_LOOP_ID]]
+// CHECK-NEXT: [[NUM_IT:%.+]] = add nsw i32 [[LAST_IT]], 1
+// CHECK-NEXT: [[CMP:%.+]] = icmp slt i32 [[IV]], [[NUM_IT]]
+// CHECK-NEXT: br i1 [[CMP]], label %[[IT_BODY:[^,]+]], label %[[IT_END:[^,]+]]
+  for (IterDouble i = ia; i < ib; ++i) {
+// CHECK: [[IT_BODY]]
+// Start of body: calculate i from index:
+// CHECK: [[IV1:%.+]] = load i32* [[IT_OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[ITER_LOOP_ID]]
+// Call of operator+ (i, IV).
+// CHECK: {{%.+}} = call {{.+}} @{{.*}}IterDouble{{.*}}!llvm.mem.parallel_loop_access ![[ITER_LOOP_ID]]
+// ... loop body ...
+   *i = *ic * 0.5;
+// Float multiply and save result.
+// CHECK: [[MULR:%.+]] = fmul double {{%.+}}, 5.000000e-01
+// CHECK-NEXT: call {{.+}} @{{.*}}IterDouble{{.*}}
+// CHECK: store double [[MULR:%.+]], double* [[RESULT_ADDR:%.+]], !llvm.mem.parallel_loop_access ![[ITER_LOOP_ID]]
+   ++ic;
+//
+// CHECK: [[IV2:%.+]] = load i32* [[IT_OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[ITER_LOOP_ID]]
+// CHECK-NEXT: [[ADD2:%.+]] = add nsw i32 [[IV2]], 1
+// CHECK-NEXT: store i32 [[ADD2]], i32* [[IT_OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[ITER_LOOP_ID]]
+// br label %{{.*}}, !llvm.loop ![[ITER_LOOP_ID]]
+  }
+// CHECK: [[IT_END]]
+// CHECK: ret void
+}
+
+
+// CHECK-LABEL: define {{.*void}} @{{.*}}collapsed{{.*}}
+void collapsed(float *a, float *b, float *c, float *d) {
+  int i; // outer loop counter
+  unsigned j; // middle loop couter, leads to unsigned icmp in loop header.
+  // k declared in the loop init below
+  short l; // inner loop counter
+// CHECK: store i32 0, i32* [[OMP_IV:[^,]+]]
+//
+  #pragma omp simd collapse(4)
+
+// CHECK: [[IV:%.+]] = load i32* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID:[0-9]+]]
+// CHECK-NEXT: [[CMP:%.+]] = icmp ult i32 [[IV]], 120
+// CHECK-NEXT: br i1 [[CMP]], label %[[COLL1_BODY:[^,]+]], label %[[COLL1_END:[^,]+]]
+  for (i = 1; i < 3; i++) // 2 iterations
+    for (j = 2u; j < 5u; j++) //3 iterations
+      for (int k = 3; k <= 6; k++) // 4 iterations
+        for (l = 4; l < 9; ++l) // 5 iterations
+        {
+// CHECK: [[COLL1_BODY]]
+// Start of body: calculate i from index:
+// CHECK: [[IV1:%.+]] = load i32* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID]]
+// Calculation of the loop counters values.
+// CHECK: [[CALC_I_1:%.+]] = udiv i32 [[IV1]], 60
+// CHECK-NEXT: [[CALC_I_1_MUL1:%.+]] = mul i32 [[CALC_I_1]], 1
+// CHECK-NEXT: [[CALC_I_2:%.+]] = add i32 1, [[CALC_I_1_MUL1]]
+// CHECK-NEXT: store i32 [[CALC_I_2]], i32* [[LC_I:.+]]
+// CHECK: [[IV1_2:%.+]] = load i32* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID]]
+// CHECK-NEXT: [[CALC_J_1:%.+]] = udiv i32 [[IV1_2]], 20
+// CHECK-NEXT: [[CALC_J_2:%.+]] = urem i32 [[CALC_J_1]], 3
+// CHECK-NEXT: [[CALC_J_2_MUL1:%.+]] = mul i32 [[CALC_J_2]], 1
+// CHECK-NEXT: [[CALC_J_3:%.+]] = add i32 2, [[CALC_J_2_MUL1]]
+// CHECK-NEXT: store i32 [[CALC_J_3]], i32* [[LC_J:.+]]
+// CHECK: [[IV1_3:%.+]] = load i32* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID]]
+// CHECK-NEXT: [[CALC_K_1:%.+]] = udiv i32 [[IV1_3]], 5
+// CHECK-NEXT: [[CALC_K_2:%.+]] = urem i32 [[CALC_K_1]], 4
+// CHECK-NEXT: [[CALC_K_2_MUL1:%.+]] = mul i32 [[CALC_K_2]], 1
+// CHECK-NEXT: [[CALC_K_3:%.+]] = add i32 3, [[CALC_K_2_MUL1]]
+// CHECK-NEXT: store i32 [[CALC_K_3]], i32* [[LC_K:.+]]
+// CHECK: [[IV1_4:%.+]] = load i32* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID]]
+// CHECK-NEXT: [[CALC_L_1:%.+]] = urem i32 [[IV1_4]], 5
+// CHECK-NEXT: [[CALC_L_1_MUL1:%.+]] = mul i32 [[CALC_L_1]], 1
+// CHECK-NEXT: [[CALC_L_2:%.+]] = add i32 4, [[CALC_L_1_MUL1]]
+// CHECK-NEXT: [[CALC_L_3:%.+]] = trunc i32 [[CALC_L_2]] to i16
+// CHECK-NEXT: store i16 [[CALC_L_3]], i16* [[LC_L:.+]]
+// ... loop body ...
+// End of body: store into a[i]:
+// CHECK: store float [[RESULT:%.+]], float* [[RESULT_ADDR:%.+]]{{.+}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID]]
+    float res = b[j] * c[k];
+    a[i] = res * d[l];
+// CHECK: [[IV2:%.+]] = load i32* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID]]
+// CHECK-NEXT: [[ADD2:%.+]] = add i32 [[IV2]], 1
+// CHECK-NEXT: store i32 [[ADD2]], i32* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[COLL1_LOOP_ID]]
+// br label %{{[^,]+}}, !llvm.loop ![[COLL1_LOOP_ID]]
+// CHECK: [[COLL1_END]]
+  }
+// i,j,l are updated; k is not updated.
+// CHECK: store i32 3, i32* [[I:%[^,]+]]
+// CHECK-NEXT: store i32 5, i32* [[I:%[^,]+]]
+// CHECK-NEXT: store i16 9, i16* [[I:%[^,]+]]
+// CHECK: ret void
+}
+
+extern char foo();
+
+// CHECK-LABEL: define {{.*void}} @{{.*}}widened{{.*}}
+void widened(float *a, float *b, float *c, float *d) {
+  int i; // outer loop counter
+  short j; // inner loop counter
+// Counter is widened to 64 bits.
+// CHECK: store i64 0, i64* [[OMP_IV:[^,]+]]
+//
+  #pragma omp simd collapse(2)
+
+// CHECK: [[IV:%.+]] = load i64* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[WIDE1_LOOP_ID:[0-9]+]]
+// CHECK-NEXT: [[LI:%.+]] = load i64* [[OMP_LI:%[^,]+]]{{.+}}!llvm.mem.parallel_loop_access ![[WIDE1_LOOP_ID]]
+// CHECK-NEXT: [[NUMIT:%.+]] = add nsw i64 [[LI]], 1
+// CHECK-NEXT: [[CMP:%.+]] = icmp slt i64 [[IV]], [[NUMIT]]
+// CHECK-NEXT: br i1 [[CMP]], label %[[WIDE1_BODY:[^,]+]], label %[[WIDE1_END:[^,]+]]
+  for (i = 1; i < 3; i++) // 2 iterations
+    for (j = 0; j < foo(); j++) // foo() iterations
+  {
+// CHECK: [[WIDE1_BODY]]
+// Start of body: calculate i from index:
+// CHECK: [[IV1:%.+]] = load i64* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[WIDE1_LOOP_ID]]
+// Calculation of the loop counters values...
+// CHECK: store i32 {{[^,]+}}, i32* [[LC_I:.+]]
+// CHECK: [[IV1_2:%.+]] = load i64* [[OMP_IV]]{{.+}}!llvm.mem.parallel_loop_access ![[WIDE1_LOOP_ID]]
+// CHECK: store i16 {{[^,]+}}, i16* [[LC_J:.+]]
+// ... loop body ...
+// End of body: store into a[i]:
+// CHECK: store float [[RESULT:%.+]], float* [[RESULT_ADDR:%.+]]{{.+}}!llvm.mem.parallel_loop_access ![[WIDE1_LOOP_ID]]
+    float res = b[j] * c[j];
+    a[i] = res * d[i];
+// CHECK: [[IV2:%.+]] = load i64* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[WIDE1_LOOP_ID]]
+// CHECK-NEXT: [[ADD2:%.+]] = add nsw i64 [[IV2]], 1
+// CHECK-NEXT: store i64 [[ADD2]], i64* [[OMP_IV]]{{.*}}!llvm.mem.parallel_loop_access ![[WIDE1_LOOP_ID]]
+// br label %{{[^,]+}}, !llvm.loop ![[WIDE1_LOOP_ID]]
+// CHECK: [[WIDE1_END]]
+  }
+// i,j are updated.
+// CHECK: store i32 3, i32* [[I:%[^,]+]]
+// CHECK: store i16
+// CHECK: ret void
+}
+
+#endif // HEADER
+

cfe/trunk/test/OpenMP/simd_loop_messages.cpp

 distance(Iter first, Iter last) { return first - last; }
 }
 class Iter0 {
   public:
     Iter0() { }
     Iter0(const Iter0 &) { }
     Iter0 operator ++() { return *this; }
     Iter0 operator --() { return *this; }
+    Iter0 operator + (int delta) { return *this; }
     bool operator <(Iter0 a) { return true; }
 };
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'Iter0' for 1st argument}}
 int operator -(Iter0 a, Iter0 b) { return 0; }
 class Iter1 {
   public:
     Iter1(float f=0.0f, double d=0.0) { }
     Iter1(const Iter1 &) { }
     Iter1 operator ++() { return *this; }
     Iter1 operator --() { return *this; }
     bool operator <(Iter1 a) { return true; }
public:
     GoodIter operator --() { return *this; }
     bool operator !() { return true; }
     bool operator <(GoodIter a) { return true; }
     bool operator <=(GoodIter a) { return true; }
     bool operator >=(GoodIter a) { return false; }
     typedef int difference_type;
     typedef std::random_access_iterator_tag iterator_category;
 };
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 int operator -(GoodIter a, GoodIter b) { return 0; }
+// expected-note@+1 2 {{candidate function not viable: requires single argument 'a', but 2 arguments were provided}}
 GoodIter operator -(GoodIter a) { return a; }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'GoodIter' for 1st argument}}
 GoodIter operator -(GoodIter a, int v) { return GoodIter(); }
 GoodIter operator +(GoodIter a, int v) { return GoodIter(); }
+// expected-note@+1 2 {{candidate function not viable: no known conversion from 'Iter1' to 'int' for 1st argument}}
 GoodIter operator -(int v, GoodIter a) { return GoodIter(); }
 GoodIter operator +(int v, GoodIter a) { return GoodIter(); }
 
 int test_with_random_access_iterator() {
   GoodIter begin, end;
   Iter0 begin0, end0;
   #pragma omp simd
   for (GoodIter I = begin; I < end; ++I)
int test_with_random_access_iterator() {
   // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
   #pragma omp simd
   for (GoodIter I(1,2); I < end; ++I)
     ++I;
   #pragma omp simd
   for (begin = GoodIter(0); begin < end; ++begin)
     ++begin;
   #pragma omp simd
-  for (begin = begin0; begin < end; ++begin)
+  for (begin = GoodIter(1,2); begin < end; ++begin)
     ++begin;
   // expected-error@+2 {{initialization clause of OpenMP for loop must be of the form 'var = init' or 'T var = init'}}
   #pragma omp simd
   for (++begin; begin < end; ++begin)
     ++begin;
   #pragma omp simd
   for (begin = end; begin < end; ++begin)
     ++begin;
for (GoodIter I = begin; I >= end; I = 2 + I)
     ++I;
   // expected-error@+2 {{increment clause of OpenMP for loop must perform simple addition or subtraction on loop variable 'I'}}
   #pragma omp simd
   for (GoodIter I = begin; I >= end; I = 2 - I)
     ++I;
   #pragma omp simd
   for (Iter0 I = begin0; I < end0; ++I)
     ++I;
+
   // Initializer is constructor without params.
   // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
   #pragma omp simd
   for (Iter0 I; I < end0; ++I)
     ++I;
+
   Iter1 begin1, end1;
+  // expected-error@+3 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+  // expected-error@+2 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
   #pragma omp simd
   for (Iter1 I = begin1; I < end1; ++I)
     ++I;
   // expected-note@+3 {{loop step is expected to be negative due to this condition}}
   // expected-error@+2 {{increment expression must cause 'I' to decrease on each iteration of OpenMP for loop}}
   #pragma omp simd
   for (Iter1 I = begin1; I >= end1; ++I)
     ++I;
+
   // Initializer is constructor with all default params.
+  // expected-error@+4 {{invalid operands to binary expression ('Iter1' and 'Iter1')}}
+  // expected-error@+3 {{could not calculate number of iterations calling 'operator-' with upper and lower loop bounds}}
   // expected-warning@+2 {{initialization clause of OpenMP for loop is not in canonical form ('var = init' or 'T var = init')}}
   #pragma omp simd
   for (Iter1 I; I < end1; ++I) {
   }
+
   return 0;
 }
 
 template <typename IT, int ST> class TC {
   public:
     int dotest_lt(IT begin, IT end) {
       // expected-note@+3 {{loop step is expected to be positive due to this condition}}
       // expected-error@+2 {{increment expression must cause 'I' to increase on each iteration of OpenMP for loop}}