Add #pragma clang loop vectorize_assume_alignment(n)

[m-happy]

Summary

This patch adds functionality '#pragma clang loop vectorize_assume_alignment'. The load/store access in for loop following the '#pragma clang loop vectorize_assume_alignment(n)' uses n bit for align access.

#pragma clang loop vectorize_assume_alignment(n)
for(){
  ....
}

This pragma depends on D69900

[lebedev.ri]

clang/docs/LanguageExtensions.rst

3135

What does this actually mean, in the end?
Will it assume that whatever alignment is required
for whatever vectorization width chosen,
is the actual alignment?

[hfinkel]

clang/docs/LanguageExtensions.rst

3135

Also, just arrays, or also pointers?

[hfinkel]

clang/docs/LanguageExtensions.rst

3135

Currently, it is using 32 bit.

You'll need to use the preferred alignment of the element type.

By pointers, if you mean pointers to arrays? Then Yes.

No, I mean pointers.

What do you intend for these cases:

double A[Size], B[Size];

void foo() {
#pragma clang loop aligned
  for (int i = 0; i < n; ++i) {
    A[i] = B[i] + 1;
  }
}

or this:

void foo(double *A, double *B) {
#pragma clang loop aligned
  for (int i = 0; i < n; ++i) {
    A[i] = B[i] + 1;
  }
}

or this:

void foo(double *A, double *B, double *C) {
#pragma clang loop aligned
  for (int i = 0; i < n; ++i) {
    A[i] = B[i] + *C + 1;
  }
}

or this:

void foo(double *A, double *B) {
#pragma clang loop aligned
  for (int i = 0; i < n; ++i) {
    *A++ = *B++ + 1;
  }
}

what about arrays of structs?

[Meinersbur]

Please see my comments in D69900 as well.

For the font-end side, I suggest the syntax vectorize_assume_alignment(32) instead. We also need to define what this assumes to be aligned. The first element of the array? Every array element? In an array-of-structs, the struct of the element that is accessed? Scalars?

[lebedev.ri]

In D69897#1735961, @Meinersbur wrote:

Please see my comments in D69900 as well.

For the font-end side, I suggest the syntax vectorize_assume_alignment(32) instead. We also need to define what this assumes to be aligned. The first element of the array? Every array element? In an array-of-structs, the struct of the element that is accessed? Scalars?

It would be really best if this could just make use of CodeGenFunction::EmitAlignmentAssumption(),
else this is a heavy hammer with loose handle.

[Meinersbur]

In D69897#1735983, @lebedev.ri wrote:

It would be really best if this could just make use of CodeGenFunction::EmitAlignmentAssumption(),
else this is a heavy hammer with loose handle.

Since it is a property of the data structure, not of the vectorization, __builtin_assume_aligned might be preferable anyway.

[m-happy]

Updated the syntax as suggested by Michael Kruse, and added functionality to specify the number used for alignment instead of just using 32bits.

[lebedev.ri]

To reword, if vectorize_assume_alignment(32) is *NOT* lowered
via CodeGenFunction::EmitAlignmentAssumption() in clang frontend,
but lowered into LoopAttributes::VectorizeAssumeAlignment,
then once the alignment that was specified does not match reality,
there will be a miscompile instead of an UBSan error.

[Meinersbur]

Could you elaborate why this is better than __builtin_assume_aligned?

clang/docs/LanguageExtensions.rst

 these two instantiations, ``twice`` will be optimized (because its definition
 was outside the region) and ``thrice`` will not be optimized.
 
 Extensions for loop hint optimizations
 ======================================
 
 The ``#pragma clang loop`` directive is used to specify hints for optimizing the
 subsequent for, while, do-while, or c++11 range-based for loop. The directive
-provides options for vectorization, interleaving, predication, unrolling and
+provides options for vectorization, interleaving, predication, aligning, unrolling and
 distribution. Loop hints can be specified before any loop and will be ignored if
 the optimization is not safe to apply.
 
 Vectorization, Interleaving, and Predication
 --------------------------------------------
 
 A vectorized loop performs multiple iterations of the original loop
 in parallel using vector instructions. The instruction set of the target
for(...) {
     ...
   }
 
 This predicates (masks) all instructions in the loop, which allows the scalar
 remainder loop (the tail) to be folded into the main vectorized loop. This
 might be more efficient when vector predication is efficiently supported by the
 target platform.
 
+Alignment of references inside loop is enabled by ``vectorize_assume_aligned(_value_)`` where _value_ is the alignment specified and it should be a multiple of 2, for example:
+
+.. code-block::c++
+  #pragma clang loop vectorize_assume_alignment(32)
+  for(...) {
+    ...
+  }
+
+This predicates all the references inside the loop to be aligned to the specified alignment value. The aligned access to them can increase fetch time and increase the performance. 
+
+
 Loop Unrolling
 --------------
 
 Unrolling a loop reduces the loop control overhead and exposes more
 opportunities for ILP. Loops can be fully or partially unrolled. Full unrolling
 eliminates the loop and replaces it with an enumerated sequence of loop
 iterations. Full unrolling is only possible if the loop trip count is known at
 compile time. Partial unrolling replicates the loop body within the loop and

clang/include/clang/Basic/Attr.td

let Spellings = [Pragma<"clang", "loop">, Pragma<"", "unroll">,
                    Pragma<"", "nounroll">, Pragma<"", "unroll_and_jam">,
                    Pragma<"", "nounroll_and_jam">];
 
   /// State of the loop optimization specified by the spelling.
   let Args = [EnumArgument<"Option", "OptionType",
                           ["vectorize", "vectorize_width", "interleave", "interleave_count",
                            "unroll", "unroll_count", "unroll_and_jam", "unroll_and_jam_count",
                            "pipeline", "pipeline_initiation_interval", "distribute",
-                           "vectorize_predicate"],
+                           "vectorize_predicate", 
+                           "vectorize_assume_alignment"],
                           ["Vectorize", "VectorizeWidth", "Interleave", "InterleaveCount",
                            "Unroll", "UnrollCount", "UnrollAndJam", "UnrollAndJamCount",
                            "PipelineDisabled", "PipelineInitiationInterval", "Distribute",
-                           "VectorizePredicate"]>,
+                           "VectorizePredicate", 
+                           "VectorizeAssumeAlignment"]>,
               EnumArgument<"State", "LoopHintState",
                            ["enable", "disable", "numeric", "assume_safety", "full"],
                            ["Enable", "Disable", "Numeric", "AssumeSafety", "Full"]>,
               ExprArgument<"Value">];
 
   let AdditionalMembers = [{
   static const char *getOptionName(int Option) {
     switch(Option) {
     case Vectorize: return "vectorize";
     case VectorizeWidth: return "vectorize_width";
     case Interleave: return "interleave";
     case InterleaveCount: return "interleave_count";
     case Unroll: return "unroll";
     case UnrollCount: return "unroll_count";
     case UnrollAndJam: return "unroll_and_jam";
     case UnrollAndJamCount: return "unroll_and_jam_count";
     case PipelineDisabled: return "pipeline";
     case PipelineInitiationInterval: return "pipeline_initiation_interval";
     case Distribute: return "distribute";
     case VectorizePredicate: return "vectorize_predicate";
+    case VectorizeAssumeAlignment: return "vectorize_assume_alignment";
     }
     llvm_unreachable("Unhandled LoopHint option.");
   }
 
   void printPrettyPragma(raw_ostream &OS, const PrintingPolicy &Policy) const {
     unsigned SpellingIndex = getSpellingListIndex();
     // For "#pragma unroll" and "#pragma nounroll" the string "unroll" or
     // "nounroll" is already emitted as the pragma name.

clang/include/clang/Basic/AttrDocs.td

 }
 
 def LoopHintDocs : Documentation {
   let Category = DocCatStmt;
   let Heading = "#pragma clang loop";
   let Content = [{
 The ``#pragma clang loop`` directive allows loop optimization hints to be
 specified for the subsequent loop. The directive allows pipelining to be
-disabled, or vectorization, vector predication, interleaving, and unrolling to
+disabled, or vectorization, vector predication, vectorize_assume_alignment, interleaving, and unrolling to
 be enabled or disabled. Vector width, vector predication, interleave count,
 unrolling count, and the initiation interval for pipelining can be explicitly
 specified. See `language extensions
 <http://clang.llvm.org/docs/LanguageExtensions.html#extensions-for-loop-hint-optimizations>`_
 for details.
   }];
 }
 

clang/include/clang/Basic/DiagnosticParseKinds.td

 
 // Pragma loop support.
 def err_pragma_loop_missing_argument : Error<
   "missing argument; expected %select{an integer value|"
   "'enable'%select{|, 'full'}1%select{|, 'assume_safety'}2 or 'disable'}0">;
 def err_pragma_loop_invalid_option : Error<
   "%select{invalid|missing}0 option%select{ %1|}0; expected vectorize, "
   "vectorize_width, interleave, interleave_count, unroll, unroll_count, "
-  "pipeline, pipeline_initiation_interval, vectorize_predicate, or distribute">;
+  "pipeline, pipeline_initiation_interval, vectorize_predicate, vectorize_assume_alignment, or distribute">;
 
 def err_pragma_fp_invalid_option : Error<
   "%select{invalid|missing}0 option%select{ %1|}0; expected contract">;
 def err_pragma_fp_invalid_argument : Error<
   "unexpected argument '%0' to '#pragma clang fp %1'; "
   "expected 'on', 'fast' or 'off'">;
 def err_pragma_fp_scope : Error<
   "'#pragma clang fp' can only appear at file scope or at the start of a "

clang/lib/CodeGen/CGLoopInfo.h

struct LoopAttributes {
   LVEnableState UnrollEnable;
 
   /// Value for llvm.loop.unroll_and_jam.* metadata (enable, disable, or full).
   LVEnableState UnrollAndJamEnable;
 
   /// Value for llvm.loop.vectorize.predicate metadata
   LVEnableState VectorizePredicateEnable;
 
+  ///  Value of llvm.loop.vectorize_assume_aligned metadata
+  unsigned VectorizeAssumeAlignment;
+
   /// Value for llvm.loop.vectorize.width metadata.
   unsigned VectorizeWidth;
 
   /// Value for llvm.loop.interleave.count metadata.
   unsigned InterleaveCount;
 
   /// llvm.unroll.
   unsigned UnrollCount;
void setVectorizePredicateState(const LoopAttributes::LVEnableState &State) {
     StagedAttrs.VectorizePredicateEnable = State;
   }
 
   /// Set the next pushed loop unroll_and_jam state.
   void setUnrollAndJamState(const LoopAttributes::LVEnableState &State) {
     StagedAttrs.UnrollAndJamEnable = State;
   }
 
+  /// Set the next pushed loop 'vectorize_assume_aligned'
+  void setVectorizeAssumeAlignment(unsigned A) {
+    StagedAttrs.VectorizeAssumeAlignment = A;
+  }
+
   /// Set the vectorize width for the next loop pushed.
   void setVectorizeWidth(unsigned W) { StagedAttrs.VectorizeWidth = W; }
 
   /// Set the interleave count for the next loop pushed.
   void setInterleaveCount(unsigned C) { StagedAttrs.InterleaveCount = C; }
 
   /// Set the unroll count for the next loop pushed.
   void setUnrollCount(unsigned C) { StagedAttrs.UnrollCount = C; }

clang/lib/CodeGen/CGLoopInfo.cpp

LoopInfo::createLoopVectorizeMetadata(const LoopAttributes &Attrs,
                                       bool &HasUserTransforms) {
   LLVMContext &Ctx = Header->getContext();
 
   Optional<bool> Enabled;
   if (Attrs.VectorizeEnable == LoopAttributes::Disable)
     Enabled = false;
   else if (Attrs.VectorizeEnable != LoopAttributes::Unspecified ||
            Attrs.VectorizePredicateEnable != LoopAttributes::Unspecified ||
+           Attrs.VectorizeAssumeAlignment != 0 ||
            Attrs.InterleaveCount != 0 || Attrs.VectorizeWidth != 0)
     Enabled = true;
 
   if (Enabled != true) {
     SmallVector<Metadata *, 4> NewLoopProperties;
     if (Enabled == false) {
       NewLoopProperties.append(LoopProperties.begin(), LoopProperties.end());
       NewLoopProperties.push_back(
if (Attrs.VectorizePredicateEnable != LoopAttributes::Unspecified &&
 
     Metadata *Vals[] = {
         MDString::get(Ctx, "llvm.loop.vectorize.predicate.enable"),
         ConstantAsMetadata::get(ConstantInt::get(llvm::Type::getInt1Ty(Ctx),
                                                  IsVectorPredicateEnabled))};
     Args.push_back(MDNode::get(Ctx, Vals));
   }
 
+  // Setting vectorize_assume_alignment
+  if (Attrs.VectorizeAssumeAlignment > 0) {
+
+    Metadata *Vals[] = {MDString::get(Ctx, "llvm.loop.vectorize_assume_alignment"),
+                        ConstantAsMetadata::get(ConstantInt::get(
+                            llvm::Type::getInt32Ty(Ctx), 
+                              Attrs.VectorizeAssumeAlignment))};
+    Args.push_back(MDNode::get(Ctx, Vals));
+  }
   // Setting vectorize.width
   if (Attrs.VectorizeWidth > 0) {
     Metadata *Vals[] = {
         MDString::get(Ctx, "llvm.loop.vectorize.width"),
         ConstantAsMetadata::get(ConstantInt::get(llvm::Type::getInt32Ty(Ctx),
                                                  Attrs.VectorizeWidth))};
     Args.push_back(MDNode::get(Ctx, Vals));
   }
LoopProperties.insert(LoopProperties.end(), AdditionalLoopProperties.begin(),
                         AdditionalLoopProperties.end());
   return createFullUnrollMetadata(Attrs, LoopProperties, HasUserTransforms);
 }
 
 LoopAttributes::LoopAttributes(bool IsParallel)
     : IsParallel(IsParallel), VectorizeEnable(LoopAttributes::Unspecified),
       UnrollEnable(LoopAttributes::Unspecified),
       UnrollAndJamEnable(LoopAttributes::Unspecified),
-      VectorizePredicateEnable(LoopAttributes::Unspecified), VectorizeWidth(0),
+      VectorizePredicateEnable(LoopAttributes::Unspecified),
+      VectorizeAssumeAlignment(0), VectorizeWidth(0),
       InterleaveCount(0), UnrollCount(0), UnrollAndJamCount(0),
       DistributeEnable(LoopAttributes::Unspecified), PipelineDisabled(false),
       PipelineInitiationInterval(0) {}
 
 void LoopAttributes::clear() {
   IsParallel = false;
   VectorizeWidth = 0;
   InterleaveCount = 0;
   UnrollCount = 0;
   UnrollAndJamCount = 0;
+  VectorizeAssumeAlignment = 0;
   VectorizeEnable = LoopAttributes::Unspecified;
   UnrollEnable = LoopAttributes::Unspecified;
   UnrollAndJamEnable = LoopAttributes::Unspecified;
   VectorizePredicateEnable = LoopAttributes::Unspecified;
   DistributeEnable = LoopAttributes::Unspecified;
   PipelineDisabled = false;
   PipelineInitiationInterval = 0;
 }
if (Attrs.IsParallel) {
     LLVMContext &Ctx = Header->getContext();
     AccGroup = MDNode::getDistinct(Ctx, {});
   }
 
   if (!Attrs.IsParallel && Attrs.VectorizeWidth == 0 &&
       Attrs.InterleaveCount == 0 && Attrs.UnrollCount == 0 &&
       Attrs.UnrollAndJamCount == 0 && !Attrs.PipelineDisabled &&
       Attrs.PipelineInitiationInterval == 0 &&
+      Attrs.VectorizeAssumeAlignment == 0 &&
       Attrs.VectorizePredicateEnable == LoopAttributes::Unspecified &&
       Attrs.VectorizeEnable == LoopAttributes::Unspecified &&
       Attrs.UnrollEnable == LoopAttributes::Unspecified &&
       Attrs.UnrollAndJamEnable == LoopAttributes::Unspecified &&
       Attrs.DistributeEnable == LoopAttributes::Unspecified && !StartLoc &&
       !EndLoc)
     return;
 
if (Parent && (Parent->Attrs.UnrollAndJamEnable ||
 
     BeforeJam.IsParallel = AfterJam.IsParallel = Attrs.IsParallel;
 
     BeforeJam.VectorizeWidth = Attrs.VectorizeWidth;
     BeforeJam.InterleaveCount = Attrs.InterleaveCount;
     BeforeJam.VectorizeEnable = Attrs.VectorizeEnable;
     BeforeJam.DistributeEnable = Attrs.DistributeEnable;
     BeforeJam.VectorizePredicateEnable = Attrs.VectorizePredicateEnable;
+    BeforeJam.VectorizeAssumeAlignment = Attrs.VectorizeAssumeAlignment;
 
     switch (Attrs.UnrollEnable) {
     case LoopAttributes::Unspecified:
     case LoopAttributes::Disable:
       BeforeJam.UnrollEnable = Attrs.UnrollEnable;
       AfterJam.UnrollEnable = Attrs.UnrollEnable;
       break;
     case LoopAttributes::Full:
       BeforeJam.UnrollEnable = LoopAttributes::Full;
       break;
     case LoopAttributes::Enable:
       AfterJam.UnrollEnable = LoopAttributes::Enable;
       break;
     }
 
+    AfterJam.VectorizeAssumeAlignment = Attrs.VectorizeAssumeAlignment;
     AfterJam.VectorizePredicateEnable = Attrs.VectorizePredicateEnable;
     AfterJam.UnrollCount = Attrs.UnrollCount;
     AfterJam.PipelineDisabled = Attrs.PipelineDisabled;
     AfterJam.PipelineInitiationInterval = Attrs.PipelineInitiationInterval;
 
     // If this loop is subject of an unroll-and-jam by the parent loop, and has
     // an unroll-and-jam annotation itself, we have to decide whether to first
     // apply the parent's unroll-and-jam or this loop's unroll-and-jam. The
     // UnrollAndJam pass processes loops from inner to outer, so we apply the
     // inner first.
     BeforeJam.UnrollAndJamCount = Attrs.UnrollAndJamCount;
     BeforeJam.UnrollAndJamEnable = Attrs.UnrollAndJamEnable;
 
     // Set the inner followup metadata to process by the outer loop. Only
     // consider the first inner loop.
     if (!Parent->UnrollAndJamInnerFollowup) {
       // Splitting the attributes into a BeforeJam and an AfterJam part will
       // stop 'llvm.loop.isvectorized' (generated by vectorization in BeforeJam)
       // to be forwarded to the AfterJam part. We detect the situation here and
       // add it manually.
       SmallVector<Metadata *, 1> BeforeLoopProperties;
       if (BeforeJam.VectorizeEnable != LoopAttributes::Unspecified ||
           BeforeJam.VectorizePredicateEnable != LoopAttributes::Unspecified ||
+          BeforeJam.VectorizeAssumeAlignment != 0 ||
           BeforeJam.InterleaveCount != 0 || BeforeJam.VectorizeWidth != 0)
         BeforeLoopProperties.push_back(
             MDNode::get(Ctx, MDString::get(Ctx, "llvm.loop.isvectorized")));
 
       bool InnerFollowupHasTransform = false;
       MDNode *InnerFollowup = createMetadata(AfterJam, BeforeLoopProperties,
                                              InnerFollowupHasTransform);
       if (InnerFollowupHasTransform)
case LoopHintAttr::Disable:
         break;
       case LoopHintAttr::PipelineDisabled:
         setPipelineDisabled(true);
         break;
       case LoopHintAttr::UnrollCount:
       case LoopHintAttr::UnrollAndJamCount:
       case LoopHintAttr::VectorizeWidth:
       case LoopHintAttr::InterleaveCount:
+      case LoopHintAttr::VectorizeAssumeAlignment:
       case LoopHintAttr::PipelineInitiationInterval:
         llvm_unreachable("Options cannot be disabled.");
         break;
       }
       break;
     case LoopHintAttr::Enable:
       switch (Option) {
       case LoopHintAttr::Vectorize:
case LoopHintAttr::Enable:
         setVectorizePredicateState(LoopAttributes::Enable);
         break;
       case LoopHintAttr::Distribute:
         setDistributeState(true);
         break;
       case LoopHintAttr::UnrollCount:
       case LoopHintAttr::UnrollAndJamCount:
       case LoopHintAttr::VectorizeWidth:
+      case LoopHintAttr::VectorizeAssumeAlignment:
       case LoopHintAttr::InterleaveCount:
       case LoopHintAttr::PipelineDisabled:
       case LoopHintAttr::PipelineInitiationInterval:
         llvm_unreachable("Options cannot enabled.");
         break;
       }
       break;
     case LoopHintAttr::AssumeSafety:
       switch (Option) {
       case LoopHintAttr::Vectorize:
       case LoopHintAttr::Interleave:
         // Apply "llvm.mem.parallel_loop_access" metadata to load/stores.
         setParallel(true);
         setVectorizeEnable(true);
         break;
       case LoopHintAttr::Unroll:
       case LoopHintAttr::UnrollAndJam:
       case LoopHintAttr::VectorizePredicate:
+      case LoopHintAttr::VectorizeAssumeAlignment:
       case LoopHintAttr::UnrollCount:
       case LoopHintAttr::UnrollAndJamCount:
       case LoopHintAttr::VectorizeWidth:
       case LoopHintAttr::InterleaveCount:
       case LoopHintAttr::Distribute:
       case LoopHintAttr::PipelineDisabled:
       case LoopHintAttr::PipelineInitiationInterval:
         llvm_unreachable("Options cannot be used to assume mem safety.");
case LoopHintAttr::Full:
       case LoopHintAttr::UnrollCount:
       case LoopHintAttr::UnrollAndJamCount:
       case LoopHintAttr::VectorizeWidth:
       case LoopHintAttr::InterleaveCount:
       case LoopHintAttr::Distribute:
       case LoopHintAttr::PipelineDisabled:
       case LoopHintAttr::PipelineInitiationInterval:
       case LoopHintAttr::VectorizePredicate:
+      case LoopHintAttr::VectorizeAssumeAlignment:
         llvm_unreachable("Options cannot be used with 'full' hint.");
         break;
       }
       break;
     case LoopHintAttr::Numeric:
       switch (Option) {
       case LoopHintAttr::VectorizeWidth:
         setVectorizeWidth(ValueInt);
         break;
       case LoopHintAttr::InterleaveCount:
         setInterleaveCount(ValueInt);
         break;
+      case LoopHintAttr::VectorizeAssumeAlignment:
+        setVectorizeAssumeAlignment(ValueInt);
+        break;
       case LoopHintAttr::UnrollCount:
         setUnrollCount(ValueInt);
         break;
       case LoopHintAttr::UnrollAndJamCount:
         setUnrollAndJamCount(ValueInt);
         break;
       case LoopHintAttr::PipelineInitiationInterval:
         setPipelineInitiationInterval(ValueInt);

clang/lib/Parse/ParsePragma.cpp

 ///  loop-hints:
 ///    loop-hint loop-hints[opt]
 ///
 ///  loop-hint:
 ///    'vectorize' '(' loop-hint-keyword ')'
 ///    'interleave' '(' loop-hint-keyword ')'
 ///    'unroll' '(' unroll-hint-keyword ')'
 ///    'vectorize_predicate' '(' loop-hint-keyword ')'
+///    'vectorize_assume_alignment' '(' loop-hint-value')'
 ///    'vectorize_width' '(' loop-hint-value ')'
 ///    'interleave_count' '(' loop-hint-value ')'
 ///    'unroll_count' '(' loop-hint-value ')'
 ///    'pipeline' '(' disable ')'
 ///    'pipeline_initiation_interval' '(' loop-hint-value ')'
 ///
 ///  loop-hint-keyword:
 ///    'enable'
while (Tok.is(tok::identifier)) {
     IdentifierInfo *OptionInfo = Tok.getIdentifierInfo();
 
     bool OptionValid = llvm::StringSwitch<bool>(OptionInfo->getName())
                            .Case("vectorize", true)
                            .Case("interleave", true)
                            .Case("unroll", true)
                            .Case("distribute", true)
                            .Case("vectorize_predicate", true)
+                           .Case("vectorize_assume_alignment", true)
                            .Case("vectorize_width", true)
                            .Case("interleave_count", true)
                            .Case("unroll_count", true)
                            .Case("pipeline", true)
                            .Case("pipeline_initiation_interval", true)
                            .Default(false);
     if (!OptionValid) {
       PP.Diag(Tok.getLocation(), diag::err_pragma_loop_invalid_option)

clang/lib/Sema/SemaStmtAttr.cpp

+
 //===--- SemaStmtAttr.cpp - Statement Attribute Handling ------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
if (PragmaName == "nounroll") {
     assert(OptionLoc && OptionLoc->Ident &&
            "Attribute must have valid option info.");
     Option = llvm::StringSwitch<LoopHintAttr::OptionType>(
                  OptionLoc->Ident->getName())
                  .Case("vectorize", LoopHintAttr::Vectorize)
                  .Case("vectorize_width", LoopHintAttr::VectorizeWidth)
                  .Case("interleave", LoopHintAttr::Interleave)
                  .Case("vectorize_predicate", LoopHintAttr::VectorizePredicate)
+                 .Case("vectorize_assume_alignment", LoopHintAttr::VectorizeAssumeAlignment)
                  .Case("interleave_count", LoopHintAttr::InterleaveCount)
                  .Case("unroll", LoopHintAttr::Unroll)
                  .Case("unroll_count", LoopHintAttr::UnrollCount)
                  .Case("pipeline", LoopHintAttr::PipelineDisabled)
                  .Case("pipeline_initiation_interval",
                        LoopHintAttr::PipelineInitiationInterval)
                  .Case("distribute", LoopHintAttr::Distribute)
                  .Default(LoopHintAttr::Vectorize);
     if (Option == LoopHintAttr::VectorizeWidth ||
         Option == LoopHintAttr::InterleaveCount ||
         Option == LoopHintAttr::UnrollCount ||
+        Option == LoopHintAttr::VectorizeAssumeAlignment ||
         Option == LoopHintAttr::PipelineInitiationInterval) {
       assert(ValueExpr && "Attribute must have a valid value expression.");
       if (S.CheckLoopHintExpr(ValueExpr, St->getBeginLoc()))
         return nullptr;
       State = LoopHintAttr::Numeric;
     } else if (Option == LoopHintAttr::Vectorize ||
                Option == LoopHintAttr::Interleave ||
                Option == LoopHintAttr::VectorizePredicate ||
CheckForIncompatibleAttributes(Sema &S,
   // example, unroll count) to the transformer. The following array accumulates
   // the hints encountered while iterating through the attributes to check for
   // compatibility.
   struct {
     const LoopHintAttr *StateAttr;
     const LoopHintAttr *NumericAttr;
   } HintAttrs[] = {{nullptr, nullptr}, {nullptr, nullptr}, {nullptr, nullptr},
                    {nullptr, nullptr}, {nullptr, nullptr}, {nullptr, nullptr},
-                   {nullptr, nullptr}};
+                   {nullptr, nullptr}, {nullptr, nullptr}};
 
   for (const auto *I : Attrs) {
     const LoopHintAttr *LH = dyn_cast<LoopHintAttr>(I);
 
     // Skip non loop hint attributes
     if (!LH)
       continue;
 
     LoopHintAttr::OptionType Option = LH->getOption();
     enum {
       Vectorize,
       Interleave,
       Unroll,
       UnrollAndJam,
       Distribute,
       Pipeline,
-      VectorizePredicate
+      VectorizePredicate,
+      Alignment
     } Category;
     switch (Option) {
     case LoopHintAttr::Vectorize:
     case LoopHintAttr::VectorizeWidth:
       Category = Vectorize;
       break;
     case LoopHintAttr::Interleave:
     case LoopHintAttr::InterleaveCount:
       Category = Interleave;
       break;
+    case LoopHintAttr::VectorizeAssumeAlignment:
+      Category = Alignment;
+      break;
     case LoopHintAttr::Unroll:
     case LoopHintAttr::UnrollCount:
       Category = Unroll;
       break;
     case LoopHintAttr::UnrollAndJam:
     case LoopHintAttr::UnrollAndJamCount:
       Category = UnrollAndJam;
       break;

clang/test/AST/ast-print-pragmas.cpp

 // RUN: %clang_cc1 -ast-print %s -o - | FileCheck %s
 // RUN: %clang_cc1 -DMS_EXT -fsyntax-only -fms-extensions %s -triple x86_64-pc-win32 -ast-print | FileCheck %s --check-prefix=MS-EXT
 
 // CHECK: #pragma clang loop vectorize_width(4)
 // CHECK-NEXT: #pragma clang loop interleave_count(8){{$}}
 
 void test(int *List, int Length) {
   int i = 0;
 #pragma clang loop vectorize_width(4)
 #pragma clang loop interleave_count(8)
+#pragma clang loop vectorize_assume_alignment(32)
   // CHECK-NEXT: while (i < Length)
   while (i < Length) {
     List[i] = i * 2;
     i++;
   }
 
   // CHECK: #pragma clang loop distribute(disable)
   // CHECK-NEXT: #pragma clang loop vectorize(enable)
   // CHECK-NEXT: #pragma clang loop interleave(disable)
   // CHECK-NEXT: #pragma clang loop vectorize_predicate(disable)
 
 #pragma clang loop distribute(disable)
 #pragma clang loop vectorize(enable)
 #pragma clang loop interleave(disable)
 #pragma clang loop vectorize_predicate(disable)
   // CHECK-NEXT: while (i - 1 < Length)
   while (i - 1 < Length) {
     List[i] = i * 2;
     i++;
   }
 
   // CHECK: #pragma clang loop distribute(enable)
   // CHECK-NEXT: #pragma clang loop vectorize(disable)
   // CHECK-NEXT: #pragma clang loop interleave(enable)
   // CHECK-NEXT: #pragma clang loop vectorize_predicate(enable)
 
 #pragma clang loop distribute(enable)
 #pragma clang loop vectorize(disable)
 #pragma clang loop interleave(enable)
 #pragma clang loop vectorize_predicate(enable)
   // CHECK-NEXT: while (i - 2 < Length)
   while (i - 2 < Length) {
     List[i] = i * 2;
     i++;
   }
 }
 
 template <int V, int I>
 void test_nontype_template_param(int *List, int Length) {

 }
 
 #ifdef MS_EXT
 #pragma init_seg(compiler)
 // MS-EXT: #pragma init_seg (.CRT$XCC){{$}}
 // MS-EXT-NEXT: int x = 3 __declspec(thread);
 int __declspec(thread) x = 3;
 #endif //MS_EXT
-

clang/test/CodeGenCXX/pragma-loop-aligned.cpp

+// RUN: %clang_cc1 -triple x86_64-apple-darwin -std=c++11 -emit-llvm -o - %s | FileCheck %s
+
+// RUN: %clang_cc1 -triple x86_64-apple-darwin -std=c++11 -emit-llvm -o - %s | FileCheck %s
+
+void test0(int *List, int Length) {
+// CHECK-LABEL: @{{.*}}test0{{.*}}(
+// CHECK: br label {{.*}}, !llvm.loop ![[LOOP0:.*]]
+
+  #pragma clang loop vectorize(enable)
+  for (int i = 0; i < Length; i++)
+    List[i] = i * 2;
+}
+
+void test1(int *List, int Length) {
+// CHECK-LABEL: @{{.*}}test1{{.*}}(
+// CHECK: br label {{.*}}, !llvm.loop ![[LOOP1:.*]]
+
+  #pragma clang loop vectorize(enable) vectorize_assume_alignment(32)
+  for (int i = 0; i < Length; i++)
+    List[i] = i * 2;
+}
+
+void test2(int *List, int Length) {
+// CHECK-LABEL: @{{.*}}test2{{.*}}(
+// CHECK:       br label {{.*}}, !llvm.loop ![[LOOP2:.*]]
+
+  #pragma clang loop vectorize_assume_alignment(32)
+  for (int i = 0; i < Length; i++)
+    List[i] = i * 2;
+}
+
+// vectorize_predicate(enable) implies vectorize(enable)
+void test3(int *List, int Length) {
+// CHECK-LABEL: @{{.*}}test3{{.*}}(
+// CHECK:       br label {{.*}}, !llvm.loop ![[LOOP3:.*]]
+
+  #pragma clang loop vectorize_assume_alignment(32)
+  for (int i = 0; i < Length; i++)
+    List[i] = i * 2;
+}
+
+// Check that disabling vectorization means a vectorization width of 1, and
+// also that vectorization_predicate isn't enabled.
+void test4(int *List, int Length) {
+// CHECK-LABEL: @{{.*}}test4{{.*}}(
+// CHECK: br label {{.*}}, !llvm.loop ![[LOOP4:.*]]
+
+  #pragma clang loop vectorize(disable)
+  for (int i = 0; i < Length; i++)
+    List[i] = i * 2;
+}
+
+// Check that vectorize and aligned are disabled.
+void test5(int *List, int Length) {
+// CHECK-LABEL: @{{.*}}test5{{.*}}(
+// CHECK: br label {{.*}}, !llvm.loop ![[LOOP5:.*]]
+
+  #pragma clang loop vectorize(disable) vectorize_assume_alignment(32)
+  for (int i = 0; i < Length; i++)
+    List[i] = i * 2;
+}
+
+
+// CHECK:      ![[LOOP0]] = distinct !{![[LOOP0]], !3}
+// CHECK-NEXT: !3 = !{!"llvm.loop.vectorize.enable", i1 true}
+
+// CHECK-NEXT: ![[LOOP1]] = distinct !{![[LOOP1]], !5, !3}
+// CHECK-NEXT: !5 = !{!"llvm.loop.vectorize_assume_alignment", i32 32}
+
+// CHECK-NEXT: ![[LOOP2]] = distinct !{![[LOOP2]], !7, !3}
+// CHECK-NEXT: !7 = !{!"llvm.loop.vectorize_assume_alignment", i32 32}
+
+// CHECK-NEXT: ![[LOOP3]] = distinct !{![[LOOP3]], !5, !3}
+
+// CHECK-NEXT: ![[LOOP4]] = distinct !{![[LOOP4]], !10}
+// CHECK-NEXT: !10 = !{!"llvm.loop.vectorize.width", i32 1}
+
+// CHECK-NEXT: ![[LOOP5]] = distinct !{![[LOOP5]], !10}

clang/test/Parser/pragma-loop.cpp

 #pragma clang loop vectorize_width(V + 4) interleave_count(I + 4)
   for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 }
 
 template <int V>
 void test_nontype_template_vectorize(int *List, int Length) {
-  /* expected-error {{invalid value '-1'; must be positive}} */ #pragma clang loop vectorize_width(V)
-  for (int i = 0; i < Length; i++) {
+  /* expected-error {{invalid value '-1'; must be positive}} */ #pragma clang loop vectorize_width(V) for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 
-  /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop vectorize_width(V / 2)
-  for (int i = 0; i < Length; i++) {
+  /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop vectorize_width(V / 2) for (int i = 0; i < Length; i++) {
     List[i] += i;
   }
 }
 
 template <int I>
 void test_nontype_template_interleave(int *List, int Length) {
-  /* expected-error {{invalid value '-1'; must be positive}} */ #pragma clang loop interleave_count(I)
-  for (int i = 0; i < Length; i++) {
+  /* expected-error {{invalid value '-1'; must be positive}} */ #pragma clang loop interleave_count(I) for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 
-  /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop interleave_count(2 % I)
-  for (int i = 0; i < Length; i++) {
+  /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop interleave_count(2 % I) for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 }
 
 template <char V>
 void test_nontype_template_char(int *List, int Length) {
-  /* expected-error {{invalid argument of type 'char'; expected an integer type}} */ #pragma clang loop vectorize_width(V)
-  for (int i = 0; i < Length; i++) {
+  /* expected-error {{invalid argument of type 'char'; expected an integer type}} */ #pragma clang loop vectorize_width(V) for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 }
 
 template <bool V>
 void test_nontype_template_bool(int *List, int Length) {
-  /* expected-error {{invalid argument of type 'bool'; expected an integer type}} */ #pragma clang loop vectorize_width(V)
-  for (int i = 0; i < Length; i++) {
+  /* expected-error {{invalid argument of type 'bool'; expected an integer type}} */ #pragma clang loop vectorize_width(V) for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 }
 
 template <int V, int I>
 void test_nontype_template_badarg(int *List, int Length) {
   /* expected-error {{use of undeclared identifier 'Vec'}} */ #pragma clang loop vectorize_width(Vec) interleave_count(I)
-  /* expected-error {{use of undeclared identifier 'Int'}} */ #pragma clang loop vectorize_width(V) interleave_count(Int)
-  for (int i = 0; i < Length; i++) {
+      /* expected-error {{use of undeclared identifier 'Int'}} */ #pragma clang loop
+      vectorize_width(V) interleave_count(Int) for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 }
 
 template <typename T>
 void test_type_template_vectorize(int *List, int Length) {
   const T Value = -1;
-  /* expected-error {{invalid value '-1'; must be positive}} */ #pragma clang loop vectorize_width(Value)
-  for (int i = 0; i < Length; i++) {
+  /* expected-error {{invalid value '-1'; must be positive}} */ #pragma clang loop vectorize_width(Value) for (int i = 0; i < Length; i++) {
     List[i] = i;
   }
 }
 
 void test(int *List, int Length) {
   int i = 0;
 
 #pragma clang loop vectorize(enable)
 #pragma clang loop interleave(enable)
 #pragma clang loop vectorize_predicate(enable)
 #pragma clang loop unroll(full)
   while (i + 1 < Length) {
     List[i] = i;
   }
 
 #pragma clang loop vectorize_width(4)
 #pragma clang loop interleave_count(8)
 #pragma clang loop unroll_count(16)
+#pragma clang loop vectorize_assume_alignment(32)
   while (i < Length) {
     List[i] = i;
   }
 
 #pragma clang loop vectorize(disable)
 #pragma clang loop interleave(disable)
 #pragma clang loop vectorize_predicate(disable)
 #pragma clang loop unroll(disable)
   while (i - 1 < Length) {
     List[i] = i;
   }
 
-#pragma clang loop vectorize_width(4) interleave_count(8) unroll_count(16)
+#pragma clang loop vectorize_width(4) interleave_count(8) unroll_count(16) vectorize_assume_alignment(32)
   while (i - 2 < Length) {
     List[i] = i;
   }
 
 #pragma clang loop interleave_count(16)
   while (i - 3 < Length) {
     List[i] = i;
   }
 
   int VList[Length];
-#pragma clang loop vectorize(disable) interleave(disable) unroll(disable) vectorize_predicate(disable)
+#pragma clang loop vectorize(disable) interleave(disable) unroll(disable) vectorize_predicate(disable) aligned(disable)
   for (int j : VList) {
     VList[j] = List[j];
   }
 
 #pragma clang loop distribute(enable)
   for (int j : VList) {
     VList[j] = List[j];
   }

 /* expected-error {{expected ')'}} */ #pragma clang loop interleave(enable
 /* expected-error {{expected ')'}} */ #pragma clang loop vectorize_predicate(enable
 /* expected-error {{expected ')'}} */ #pragma clang loop unroll(full
 /* expected-error {{expected ')'}} */ #pragma clang loop distribute(enable
 
 /* expected-error {{expected ')'}} */ #pragma clang loop vectorize_width(4
 /* expected-error {{expected ')'}} */ #pragma clang loop interleave_count(4
 /* expected-error {{expected ')'}} */ #pragma clang loop unroll_count(4
+/* expected-error {{expected ')'}} */ #pragma clang loop vectorize_assume_alignment(32
 
 /* expected-error {{missing argument; expected 'enable', 'assume_safety' or 'disable'}} */ #pragma clang loop vectorize()
 /* expected-error {{missing argument; expected an integer value}} */ #pragma clang loop interleave_count()
+/* expected-error {{missing argument; expected an integer value}} */ #pragma clang loop vectorize_assume_alignment()  
 /* expected-error {{missing argument; expected 'enable', 'full' or 'disable'}} */ #pragma clang loop unroll()
 /* expected-error {{missing argument; expected 'enable' or 'disable'}} */ #pragma clang loop distribute()
 
-/* expected-error {{missing option; expected vectorize, vectorize_width, interleave, interleave_count, unroll, unroll_count, pipeline, pipeline_initiation_interval, vectorize_predicate, or distribute}} */ #pragma clang loop
+/* expected-error {{missing option; expected vectorize, vectorize_width, interleave, interleave_count, unroll, unroll_count, pipeline, pipeline_initiation_interval, vectorize_predicate, aligned, or distribute}} */ #pragma clang loop
 /* expected-error {{invalid option 'badkeyword'}} */ #pragma clang loop badkeyword
 /* expected-error {{invalid option 'badkeyword'}} */ #pragma clang loop badkeyword(enable)
 /* expected-error {{invalid option 'badkeyword'}} */ #pragma clang loop vectorize(enable) badkeyword(4)
 /* expected-warning {{extra tokens at end of '#pragma clang loop'}} */ #pragma clang loop vectorize(enable) ,
   while (i-4 < Length) {
     List[i] = i;
   }
 
 /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop vectorize_width(0)
+/* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop vectorize_assume_alignment(0)
 /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop interleave_count(0)
 /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop unroll_count(0)
 
 /* expected-error {{expression is not an integral constant expression}} expected-note {{division by zero}} */ #pragma clang loop vectorize_width(10 / 0)
 /* expected-error {{invalid value '0'; must be positive}} */ #pragma clang loop interleave_count(10 / 5 - 2)
   while (i-5 < Length) {
     List[i] = i;
   }
 
 test_nontype_template_vectorize<4>(List, Length);
 /* expected-note {{in instantiation of function template specialization}} */ test_nontype_template_vectorize<-1>(List, Length);
 test_nontype_template_interleave<8>(List, Length);
 /* expected-note {{in instantiation of function template specialization}} */ test_nontype_template_interleave<-1>(List, Length);
 
 /* expected-note {{in instantiation of function template specialization}} */ test_nontype_template_char<'A'>(List, Length); // Loop hint arg cannot be a char.
 /* expected-note {{in instantiation of function template specialization}} */ test_nontype_template_bool<true>(List, Length);  // Or a bool.
 /* expected-note {{in instantiation of function template specialization}} */ test_type_template_vectorize<int>(List, Length); // Or a template type.
 
 /* expected-error {{value '3000000000' is too large}} */ #pragma clang loop vectorize_width(3000000000)
 /* expected-error {{value '3000000000' is too large}} */ #pragma clang loop interleave_count(3000000000)
+/* expected-error {{value '3000000000' is too large}} */ #pragma clang loop vectorize_assume_alignment(3000000000)
 /* expected-error {{value '3000000000' is too large}} */ #pragma clang loop unroll_count(3000000000)
   while (i-6 < Length) {
     List[i] = i;
   }
 
 /* expected-warning {{extra tokens at end of '#pragma clang loop'}} */ #pragma clang loop vectorize_width(1 +) 1
 /* expected-warning {{extra tokens at end of '#pragma clang loop'}} */ #pragma clang loop vectorize_width(1) +1
 const int VV = 4;

 
 // PR20069 - Loop pragma arguments that are not identifiers or numeric
 // constants crash FE.
 /* expected-error {{expected ')'}} */ #pragma clang loop vectorize(()
 /* expected-error {{invalid argument; expected 'enable', 'assume_safety' or 'disable'}} */ #pragma clang loop interleave(*)
 /* expected-error {{invalid argument; expected 'enable', 'full' or 'disable'}} */ #pragma clang loop unroll(=)
 /* expected-error {{invalid argument; expected 'enable' or 'disable'}} */ #pragma clang loop distribute(+)
 /* expected-error {{type name requires a specifier or qualifier}} expected-error {{expected expression}} */ #pragma clang loop vectorize_width(^)
+/* expected-error {{expected expression}} expected-error {{expected expression}} */ #pragma clang loop vectorize_assume_alignment(/)
 /* expected-error {{expected expression}} expected-error {{expected expression}} */ #pragma clang loop interleave_count(/)
 /* expected-error {{expected expression}} expected-error {{expected expression}} */ #pragma clang loop unroll_count(==)
   while (i-8 < Length) {
     List[i] = i;
   }
 
 #pragma clang loop vectorize(enable)
 /* expected-error {{expected a for, while, or do-while loop to follow '#pragma clang loop'}} */ int j = Length;

 #pragma clang loop unroll(disable)
 /* expected-error {{incompatible directives 'unroll(disable)' and 'unroll_count(4)'}} */ #pragma clang loop unroll_count(4)
   while (i-10 < Length) {
     List[i] = i;
   }
 
 #pragma clang loop vectorize_width(8)
 /* expected-error {{duplicate directives 'vectorize_width(8)' and 'vectorize_width(4)'}} */ #pragma clang loop vectorize_width(4)
+#pragma clang loop vectorize_assume_alignment(32)
+/* expected-error {{duplicate directives 'vectorize_width(8)' and 'vectorize_width(4)'}} */ #pragma clang loop vectorize_assume_alignment(4)
 #pragma clang loop interleave_count(8)
 /* expected-error {{duplicate directives 'interleave_count(8)' and 'interleave_count(4)'}} */ #pragma clang loop interleave_count(4)
 #pragma clang loop unroll_count(8)
 /* expected-error {{duplicate directives 'unroll_count(8)' and 'unroll_count(4)'}} */ #pragma clang loop unroll_count(4)
   while (i-11 < Length) {
     List[i] = i;
   }
 

clang/test/Parser/pragma-unroll-and-jam.cpp

 #pragma unroll(4)
   for (int i = 0; i < Length; i++) {
     for (int j = 0; j < Length; j++) {
       List[i * Length + j] = Value;
     }
   }
 
 // pragma clang unroll_and_jam is disabled for the moment
-/* expected-error {{invalid option 'unroll_and_jam'; expected vectorize, vectorize_width, interleave, interleave_count, unroll, unroll_count, pipeline, pipeline_initiation_interval, vectorize_predicate, or distribute}} */ #pragma clang loop unroll_and_jam(4)
+/* expected-error {{invalid option 'unroll_and_jam'; expected vectorize, vectorize_width, interleave, interleave_count, unroll, unroll_count, pipeline, pipeline_initiation_interval, vectorize_predicate, vectorize_assume_alignment, or distribute}} */ #pragma clang loop unroll_and_jam(4)
   for (int i = 0; i < Length; i++) {
     for (int j = 0; j < Length; j++) {
       List[i * Length + j] = Value;
     }
   }
 
 #pragma unroll_and_jam
 /* expected-error {{expected statement}} */ }