llvm.noalias - Clang CodeGen for local restrict-qualified pointers

[hfinkel]

Summary

This is part of the series started by D9375, but the first Clang patch. It teaches Clang to emit llvm.noalias calls and noalias metadata to represent local restrict-qualified pointers.

There are a few design considerations:

First, a local restrict-qualified pointer can have multiple values assigned to it, and they all need to fall into the same "noalias scope".

Second, the set of local scopes relevant to a given block cannot be determined without first visiting all restrict-qualified declarations and generating their corresponding scope MDNodes. Because the aliasing assumptions apply "retroactively" (to all accesses within the block, not just those sequenced after the declaration statement), we need to generate all MDNodes before applying the metadata. When we visit the declaration as the variable is being emitted we have an opportunity to record the address so that we can wrap all assigned values with an llvm.noalias call.

In this implementation, I'm generating the metadata nodes when we visit the decl when the local address is created (but before the initializer is evaluated), and adding the address to a map of address that, when stored to, are wrapped in llvm.noalias calls. To apply the metadata, we need to record all memory instructions in each block (and parent blocks, etc.). To avoid incurring this expense when no restrict-qualified decls are present, I have a pre-check that enables this recording of memory accesses (a small recursive visitor that looks for restrict-qualified decls).

After we're done emitting a block, we iterate over the memory-access instructions generated for it, and add metadata for all relevant noalias scopes of variables within that block.

[majnemer]

lib/CodeGen/CGDecl.cpp

1325–1330

It's a shame that createAnonymousAARoot doesn't take a twine, it would save a memory allocation through here.

[rjmccall]

lib/CodeGen/CGExpr.cpp

1577

This is very strange. Why do the aliasing properties of an address cause metadata to be attached to the value stored into it? Shouldn't the metadata be on the address (in which case, perhaps it should have been attached to the load from the restrict pointer?) or the store?

Also, please find some way to avoid doing a DenseMap lookup on every single store. I would guess that the address only has an entry in that table if it was restrict-qualified; you can probably find a way to pass that information down.

lib/CodeGen/CGStmt.cpp

363

RecursiveASTVisitors are actually really big; the last time I checked, a RecursiveASTVisitor compiled to several hundred kilobytes of code, no matter how few traversals you actually have in it.

Also, doing a recursive walk every time you enter a scope is inherently quadratic with deeply-nested scopes.

You don't really need a fully recursive walk, though. The language specification for 'restrict' talks about the scope B that declares the restrict-qualified pointer; you just need to scan the current compound statement for variable declarations.

[hfinkel]

lib/CodeGen/CGDecl.cpp

1325–1330

I'm not sure what you're proposing, exactly. MDBuilder::createString takes a StringRef (and, correspondingly MDString::get does). If createAnonymousAARoot took a Twine, it would need to allocate some SmallVector and then call toStringRef. Is that what you have in mind?

[hfinkel]

lib/CodeGen/CGExpr.cpp

1577

This is very strange. Why do the aliasing properties of an address cause metadata to be attached to the value stored into it? Shouldn't the metadata be on the address (in which case, perhaps it should have been attached to the load from the restrict pointer?) or the store?

To be clear, this is not the metadata, this is the intrinsic; the metadata needs to be attached to all access in the scope. The intrinsic wraps the 'noalias' address. And this is "wrapping" the restrict-qualified address with the llvm.noalias intrinsic before it is stored into the stack space allocated for the local variable.

Also, please find some way to avoid doing a DenseMap lookup on every single store. I would guess that the address only has an entry in that table if it was restrict-qualified; you can probably find a way to pass that information down.

You are right: some (indirect) caller of this function should see a restrict-qualified variable. Maybe this information should end up in the LValue structure?

Maybe it is worth now thinking about that should be the follow-on patch, because that also might influence the design. This patch handles direct restrict-qualified local variables, but does not handle the case where the variables are inside a structure. Specifically, I mean this:

struct V {
  double * restrict x;
  double * restrict y;
  double * restrict z;
};

void foo() {
  V v = { getX(), getY(), getZ() };
  // The stores that compose this initializer need to be wrapped in @llvm.noalias also.
}

So I'd like to end up with a design for this patch that is easy to extend to handle the restrict-qualified-local-structure-member case as well.

lib/CodeGen/CGStmt.cpp

363

RecursiveASTVisitors are actually really big; the last time I checked, a RecursiveASTVisitor compiled to several hundred kilobytes of code, no matter how few traversals you actually have in it.

Hrmm.

Also, doing a recursive walk every time you enter a scope is inherently quadratic with deeply-nested scopes.

Good point.

You don't really need a fully recursive walk, though. The language specification for 'restrict' talks about the scope B that declares the restrict-qualified pointer; you just need to scan the current compound statement for variable declarations.

You're right, I'll write a more-targeted function.

[rjmccall]

lib/CodeGen/CGExpr.cpp

1577

"Value" is the value being stored to the address.

If I've got code like this:

double * restrict addr = get_addr();
double value = get_value();
*addr = value;

you are producing IR like

%addr = call double* @get_addr()
%value = call double get_value()
%value2 = call @llvm.noalias(double %value)
store %value2, %addr

This is what I'm saying does not make any sense, because there's nothing special about %value; the restrictions are on %addr. I think you probably just have a bug in your patch and meant to use Addr.

You are right: some (indirect) caller of this function should see a restrict-qualified variable. Maybe this information should end up in the LValue structure?

Yes, we already store when l-values are volatile, so storing that they're restrict should be easy.

[hfinkel]

lib/CodeGen/CGExpr.cpp

1577

No, this is not correct. If I have code like this:

double * restrict addr = get_addr();
double value = get_value();
*addr = value;

then we don't get code like this:

%addr = call double* @get_addr()
%value = call double get_value()
%value2 = call @llvm.noalias(double %value)
store %value2, %addr

The code that Clang generates does not look like that: it does not generate an SSA variable for %addr, it generates a local stack location for it, and then loads/stores from that location when accessed. The only address that appear in NoAliasAddrMap are the address of the local allocas for the restrict-qualified variables themselves. It is only the pointer values being stored into the local restrict-qualified variables themselves that get wrapped.

If you look at the regression test, I think it is clear that the code does the right thing. (If that's not clear, I should improve the test.)

1577

Yes, we already store when l-values are volatile, so storing that they're restrict should be easy.

FWIW, LValue already keeps track of whether or not the value is restrict qualified (along with whether or not it is volatile). I'll see if I can predicate the map lookup on that, which may be the best solution (does not increase the size of LValue for an uncommon case, but also does not perform a map lookup for every store).

[hfinkel]

lib/CodeGen/CGExpr.cpp

1577

Before I forget ;) -- Here's the actual IR for this case you've provided above...

$ cat /tmp/ta.c
double *get_addr(void);
double get_value(void);

void foo() {
  double * restrict addr = get_addr();
  double value = get_value();
  *addr = value;
}

The optimized IR is:

$ clang -O3 -S -emit-llvm -o - /tmp/ta.c 
; ModuleID = '/tmp/ta.c'
target datalayout = "E-m:e-i64:64-n32:64"
target triple = "powerpc64-unknown-linux-gnu"

; Function Attrs: nounwind
define void @foo() #0 {
entry:
  %call = tail call double* @get_addr() #2, !noalias !1
  %0 = tail call double* @llvm.noalias.p0f64(double* %call, metadata !1) #2
  %call1 = tail call double @get_value() #2, !noalias !1
  store double %call1, double* %0, align 8, !tbaa !4, !noalias !1
  ret void
}

and the original IR is:

$ clang -O3 -S -emit-llvm -o - /tmp/ta.c -mllvm -disable-llvm-optzns
; ModuleID = '/tmp/ta.c'
target datalayout = "E-m:e-i64:64-n32:64"
target triple = "powerpc64-unknown-linux-gnu"

; Function Attrs: nounwind
define void @foo() #0 {
entry:
  %addr = alloca double*, align 8
  %value = alloca double, align 8
  %0 = bitcast double** %addr to i8*
  call void @llvm.lifetime.start(i64 8, i8* %0) #1, !noalias !1
  %call = call double* @get_addr(), !noalias !1
  %1 = call double* @llvm.noalias.p0f64(double* %call, metadata !1) #1
  store double* %1, double** %addr, align 8, !tbaa !4, !noalias !1
  %2 = bitcast double* %value to i8*
  call void @llvm.lifetime.start(i64 8, i8* %2) #1, !noalias !1
  %call1 = call double @get_value(), !noalias !1
  store double %call1, double* %value, align 8, !tbaa !8, !noalias !1
  %3 = load double, double* %value, align 8, !tbaa !8, !noalias !1
  %4 = load double*, double** %addr, align 8, !tbaa !4, !noalias !1
  store double %3, double* %4, align 8, !tbaa !8, !noalias !1
  %5 = bitcast double* %value to i8*
  call void @llvm.lifetime.end(i64 8, i8* %5) #1
  %6 = bitcast double** %addr to i8*
  call void @llvm.lifetime.end(i64 8, i8* %6) #1
  ret void
}

Thanks again!

[rjmccall]

Okay, thank you, I understand what you're saying now and you're completely right. It makes sense for this to be tied to the value stored into the l-value, as that actions is basically what's establishing a stronger rule about that pointer.

Side question: is the use of an overloaded intrinsic here going to be a problem, since intrinsics can't be overloaded on arbitrary user types (e.g. structs)? Is there anything about the analysis that breaks if we bitcast the operand to i8* and then bitcast the result back to the actual type? Is this question eventually being mooted by the Great Pointer-Typing Change? :)

[hfinkel]

In D9403#207395, @rjmccall wrote:

Okay, thank you, I understand what you're saying now and you're completely right. It makes sense for this to be tied to the value stored into the l-value, as that actions is basically what's establishing a stronger rule about that pointer.

Side question: is the use of an overloaded intrinsic here going to be a problem, since intrinsics can't be overloaded on arbitrary user types (e.g. structs)?

Philip Reames fixed that (for almost all types) a few months ago. The IRBuilder code works around the remaining cases by casting to i8*.

Is there anything about the analysis that breaks if we bitcast the operand to i8* and then bitcast the result back to the actual type?

No, it is just more IR.

Is this question eventually being mooted by the Great Pointer-Typing Change? :)

Yes :)

[hfinkel]

Rebased. Recursive AST visitor (which John pointed out was bad because it caused quadratic behavior and was wasteful in terms of code size) replaced with a function with some loops over the statements in each compound statement.

[hfinkel]

lib/CodeGen/CGExpr.cpp

1577

Yes, we already store when l-values are volatile, so storing that they're restrict should be easy.

FWIW, LValue already keeps track of whether or not the value is restrict qualified (along with whether or not it is volatile). I'll see if I can predicate the map lookup on that, which may be the best solution (does not increase the size of LValue for an uncommon case, but also does not perform a map lookup for every store).

I've posted this enhancement as D22189.

[rjmccall]

lib/CodeGen/CGStmt.cpp

580

This is a very strange representation. Every memory operation in the lexical block is annotated with a list of all of the scopes that were entered within the block, even if they were entered after the operation. But for some reason, not with nested scopes?

What's the right patch for me to read about this representation?

lib/CodeGen/CodeGenFunction.h

638

These should be TinyPtrVectors. Restrict-qualified local variables are very, very uncommon.

658

Our normal convention in Clang is to declare the type separately from a variable of it.

688

This is a completely useless parameter name.

1528

Star placement.

[hfinkel]

lib/CodeGen/CGStmt.cpp

580

Perhaps unfortunately, this is an artifact of the way that restrict is defined in C. It applies to all accesses in the block in which the variable is declared, even those before the declaration of the restrict-qualified local itself.

It should work with nested scopes, in the sense that we add these things as we complete each scope. So we add things to the inner scope, and then when we complete the outer scope, we go back over the instructions (including those in the inner scope because the scope recording recurses up the scope hierarchy), and adds the outer scopes - it concatenates them to any added by the inner (nested) scopes.

The noalias intrinsic's LangRef updates are in D9375.

[rjmccall]

lib/CodeGen/CGStmt.cpp

568

It's much more likely that NoAliasScopes will be empty than that MemoryInsts will be empty. You should probably fast-path using that, or better yet, with the RecordMemoryInsts bit.

580

Ah, I see the recursion now. Please add a comment explaining that expectation here.

lib/CodeGen/CodeGenFunction.cpp

2125

Is it intentional that this includes calls and invokes? If so, please leave a comment describing which instructions we want to apply this to and why.

In general, this entire patch is really light on comments.

lib/CodeGen/CodeGenFunction.h

640

Why this is defaultable?

[hfinkel]

lib/CodeGen/CGStmt.cpp

568

I'm not sure that's true; we only record memory-accessing instructions in the first place if there are relevant restrict-qualified pointers around.

lib/CodeGen/CodeGenFunction.cpp

2125

In general, this entire patch is really light on comments.

Agreed; adding more comments...

lib/CodeGen/CodeGenFunction.h

640

It seems like a reasonable default (i.e. we default to not recording memory instructions); in the current patch, this is used by the LexicalNoAliasInfo FnNoAliasInfo; member of CodeGenFunction.

[hfinkel]

Rebased; added more comments and addressed other review feedback.

[hfinkel]

Ping.

[rjmccall]

lib/CodeGen/CGDecl.cpp

1312

This should really be a subroutine of EmitAutoVarAlloca; that will implicitly pick this logic up for a bunch of less standard places in IRGen that create local variables. Please make it a static function (if possible) called EmitNoAliasScope and call it immediately after EmitVarAnnotations.

lib/CodeGen/CGStmt.cpp

568

Ok, that makes sense.

580

I would still like this comment. It should be enough to explain how MemoryInsts actually gets filled: there's a callback from inserting an instruction which adds all memory instructions to every active lexical scope that's recording them.

lib/CodeGen/CodeGenFunction.cpp

2131

This condition will include calls, which I assume is unnecessary (except maybe memory intrinsics?). More seriously, it will also include loads and stores into allocas, which will sweep in all sorts of bookkeeping temporaries that IRGen uses in more complex code-generation situations. You might want to consider ways to avoid recording this kind of instruction that are very likely to just get immediately optimized away by e.g. mem2reg.

Please also modify LexicalScope so that it records whether there's any active recording scope in the CodeGenFunction. Lexical scopes are nested, so that should be as easy as saving the current state when you enter a scope and restoring it when you leave. That will allow you to optimize this code by completely bypassing the recursion and even the check for whether the instruction is a memory instruction in the extremely common case of a function with no restrict variables.

In general, a lot of things about this approach seem to have worryingly poor asymptotic performance in the face of large functions or (especially) many restrict variables. (You could, for example, have chosen to have each memory instruction link to its enclosing noalias scope, which would contain a link to its enclosing scope and a list of the variables it directly declares. This would be a more complex representation to consume, but it would not require inherently quadratic frontend work building all these lists.) The only saving grace is that we expect very little code to using restrict, so it becomes vital to ensure that your code is immediately short-circuited when nothing is using restrict.

lib/CodeGen/CodeGenFunction.h

715

I would suggest this wording:

/// Record the given memory access instruction in this and all of its enclosing scopes.
/// When we close this scope, we'll add the list of all the restrict-qualified local variables
/// it declared to all the memory accesses which occurred within it.  Recording is only
/// enabled under optimization, and it's disabled in a scope unless it actually declares
/// any local restrict variables.

[hfinkel]

lib/CodeGen/CGDecl.cpp

1312

Done (I didn't make it static, which is certainly possible, but didn't look much cleaner to me - let me know if you want it static anyway).

lib/CodeGen/CGStmt.cpp

580

Added.

lib/CodeGen/CodeGenFunction.cpp

2131

This condition will include calls, which I assume is unnecessary (except maybe memory intrinsics?).

No, getting calls here was intentional. The AA implementation can specifically deal with calls (especially those that are known only to access memory given by their arguments).

More seriously, it will also include loads and stores into allocas, which will sweep in all sorts of bookkeeping temporaries that IRGen uses in more complex code-generation situations. You might want to consider ways to avoid recording this kind of instruction that are very likely to just get immediately optimized away by e.g. mem2reg.

I agree, but this might be difficult to do in general. We should be able to avoid annotating accesses to allocas that don't escape. Do we have a sound way to determine at this stage whether a local variable has had its address taken?

Please also modify LexicalScope so that it records whether there's any active recording scope in the CodeGenFunction. Lexical scopes are nested, so that should be as easy as saving the current state when you enter a scope and restoring it when you leave. That will allow you to optimize this code by completely bypassing the recursion and even the check for whether the instruction is a memory instruction in the extremely common case of a function with no restrict variables.

I added a variable to the scope to cache whether or not anything is being recorded. This avoids the quadratic recording-check behavior (unless some scope is actually recording something).

In general, a lot of things about this approach seem to have worryingly poor asymptotic performance in the face of large functions or (especially) many restrict variables. (You could, for example, have chosen to have each memory instruction link to its enclosing noalias scope, which would contain a link to its enclosing scope and a list of the variables it directly declares. This would be a more complex representation to consume, but it would not require inherently quadratic frontend work building all these lists.) The only saving grace is that we expect very little code to using restrict, so it becomes vital to ensure that your code is immediately short-circuited when nothing is using restrict.

We're definitely on the same page here. The reasons I didn't worry about this too much is that, as you note, we expect this to be rare, and moreover, the underlying metadata representation being created is itself quadratic (#restricts x #accesses). The good news is that this new representation is actually less verbose than the existing noalias metadata. The bad news is that, if this turns out to be a problem, and it could, then we'll need to either add cutoffs or design some different representation (or both).

lib/CodeGen/CodeGenFunction.h

715

Sounds good (updated).

[hfinkel]

Rebased and addressing review comments.

[troyj]

Hi, I got here via llvm-dev => D9375 => D9403 (this) and have read through everything. I see that this patch has been stalled for about a year and I would like to know its status. Is it waiting on a resolution in LLVM for this problem that Jeroen mentioned on llvm-dev?

"One of the bigger problems is, that some optimization passes optimize away some (but not all) of the llvm.noalias intrinsics in a function. Because of that, the alias analysis can get confused and can identify two pointers as not-aliasing where they should be aliasing."

That's an improper optimization though, right? If the semantics of the intrinsic are that all of them or none of them must be present for correctness, then any optimization that removes some but not all is defective. Is the issue that these optimizations cannot tell that they are doing anything wrong?

I am interested in seeing this patch merged, so I would like to know what the obstacles are. Thanks.

lib/CodeGen/CGDecl.cpp

 #include "clang/AST/DeclOpenMP.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/CodeGen/CGFunctionInfo.h"
 #include "clang/Frontend/CodeGenOptions.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/MDBuilder.h"
 #include "llvm/IR/Type.h"
 
 using namespace clang;
 using namespace CodeGen;
 
 void CodeGenFunction::EmitDecl(const Decl &D) {
   switch (D.getKind()) {
   case Decl::BuiltinTemplate:
if (CGDebugInfo *DI = getDebugInfo()) {
         DI->setLocation(D.getLocation());
         DI->EmitDeclareOfAutoVariable(&D, address.getPointer(), Builder);
       }
     }
 
   if (D.hasAttr<AnnotateAttr>())
     EmitVarAnnotations(&D, address.getPointer());
 
+  EmitNoAliasScope(emission);
+
   // Make sure we call @llvm.lifetime.end.
   if (emission.useLifetimeMarkers())
     EHStack.pushCleanup<CallLifetimeEnd>(NormalEHLifetimeMarker,
                                          emission.getAllocatedAddress(),
                                          emission.getSizeForLifetimeMarkers());
 
   return emission;
 }
if (shouldUseMemSetPlusStoresToInitialize(constant,
     Address SrcPtr = Address(GV, Loc.getAlignment());
     if (SrcPtr.getType() != BP)
       SrcPtr = Builder.CreateBitCast(SrcPtr, BP);
 
     Builder.CreateMemCpy(Loc, SrcPtr, SizeVal, isVolatile);
   }
 }
 
+// For all local restrict-qualified local variables, we create a noalias
+// metadata scope. This scope is used to identify each restrict-qualified
+// variable and the other memory accesses within the scope where its aliasing
+// assumptions apply. The scope metadata is stored in the NoAliasAddrMap map
+// where the pointer to the local variable is the key in the map.
+void CodeGenFunction::EmitNoAliasScope(const AutoVarEmission &emission) {
+  assert(emission.Variable && "emission was not valid!");
+
+  // Don't emit noalias intrinsics unless we're optimizing.
+  if (CGM.getCodeGenOpts().OptimizationLevel == 0)
+    return;
+
+  const VarDecl &D = *emission.Variable;
+  QualType type = D.getType();
+
+  // Emit a noalias intrinsic for restrict-qualified variables.
+  if (!type.isRestrictQualified())
+    return;
+
+  llvm::MDBuilder MDB(CurFn->getContext());
+  if (!NoAliasDomain)
+    NoAliasDomain = MDB.createAnonymousAliasScopeDomain(CurFn->getName());
+
+  std::string Name = CurFn->getName();
+  Name += ": ";
+  Name += D.getName();
+
+  llvm::MDNode *Scope =
+    MDB.createAnonymousAliasScope(NoAliasDomain, Name);
+  addNoAliasScope(Scope);
+
+  SmallVector<llvm::Metadata *, 8> ScopeListEntries(1, Scope);
+  llvm::MDNode *ScopeList =
+    llvm::MDNode::get(CurFn->getContext(), ScopeListEntries);
+
+  // Check whether this is a byref variable that's potentially
+  // captured and moved by its own initializer.  If so, we'll need to
+  // emit the initializer first, then copy into the variable.
+  const Expr *Init = D.getInit();
+  bool capturedByInit = emission.IsByRef && isCapturedBy(D, Init);
+
+  Address Loc =
+    capturedByInit ? emission.Addr : emission.getObjectAddress(*this);
+  NoAliasAddrMap[Loc.getPointer()] = ScopeList;
+}
+
 /// Emit an expression as an initializer for a variable at the given
 /// location.  The expression is not necessarily the normal
 /// initializer for the variable, and the address is not necessarily
 /// its normal location.
 ///
 /// \param init the initializing expression
 /// \param var the variable to act as if we're initializing
 /// \param loc the address to initialize; its type is a pointer

lib/CodeGen/CGExpr.cpp

if (Ty->isVectorType()) {
       if (Addr.getElementType() != SrcTy) {
         Addr = Builder.CreateElementBitCast(Addr, SrcTy, "storetmp");
       }
     }
   }
 
   Value = EmitToMemory(Value, Ty);
 
+  // If this is an assignment to a restrict-qualified local variable, then we
+  // have pointer aliasing assumptions that can be applied to the pointer value
+  // being stored.
+  auto NAI = NoAliasAddrMap.find(Addr.getPointer());
+  if (NAI != NoAliasAddrMap.end())
+    Value = Builder.CreateNoAliasPointer(Value, NAI->second);
+
   LValue AtomicLValue =
       LValue::MakeAddr(Addr, Ty, getContext(), BaseInfo, TBAAInfo);
   if (Ty->isAtomicType() ||
       (!isInit && LValueIsSuitableForInlineAtomic(AtomicLValue))) {
     EmitAtomicStore(RValue::get(Value), AtomicLValue, isInit);
     return;
   }
 

lib/CodeGen/CGStmt.cpp

bool CodeGenFunction::EmitSimpleStmt(const Stmt *S) {
   case Stmt::DefaultStmtClass:  EmitDefaultStmt(cast<DefaultStmt>(*S));   break;
   case Stmt::CaseStmtClass:     EmitCaseStmt(cast<CaseStmt>(*S));         break;
   case Stmt::SEHLeaveStmtClass: EmitSEHLeaveStmt(cast<SEHLeaveStmt>(*S)); break;
   }
 
   return true;
 }
 
+bool CodeGenFunction::hasLocalRestrictVars(const CompoundStmt &S) {
+  // We may have restrict-qualified variables, but if we're not optimizing, we
+  // don't do anything special with them.
+  if (CGM.getCodeGenOpts().OptimizationLevel == 0)
+    return false;
+
+  for (const auto *C : S.body())
+    if (const auto *DS = dyn_cast<DeclStmt>(C))
+      for (const auto *I : DS->decls())
+        if (const auto *VD = dyn_cast<VarDecl>(I))
+          if (VD->getType().isRestrictQualified())
+            return true;
+
+  return false;
+}
+
 /// EmitCompoundStmt - Emit a compound statement {..} node.  If GetLast is true,
 /// this captures the expression result of the last sub-statement and returns it
 /// (for use by the statement expression extension).
 Address CodeGenFunction::EmitCompoundStmt(const CompoundStmt &S, bool GetLast,
                                           AggValueSlot AggSlot) {
   PrettyStackTraceLoc CrashInfo(getContext().getSourceManager(),S.getLBracLoc(),
                              "LLVM IR generation of compound statement ('{}')");
 
   // Keep track of the current cleanup stack depth, including debug scopes.
-  LexicalScope Scope(*this, S.getSourceRange());
+  LexicalScope Scope(*this, S.getSourceRange(), hasLocalRestrictVars(S));
 
   return EmitCompoundStmtWithoutScope(S, GetLast, AggSlot);
 }
 
 Address
 CodeGenFunction::EmitCompoundStmtWithoutScope(const CompoundStmt &S,
                                               bool GetLast,
                                               AggValueSlot AggSlot) {
void CodeGenFunction::LexicalScope::rescopeLabels() {
   }
 
   // Reparent the labels if the new scope also has cleanups.
   if (innermostScope != EHScopeStack::stable_end() && ParentScope) {
     ParentScope->Labels.append(Labels.begin(), Labels.end());
   }
 }
 
+// For all of the instructions generated for this lexical scope that access
+// memory, add the noalias metadata associated with any block-local
+// restrict-qualified pointers from this scope.
+void CodeGenFunction::LexicalNoAliasInfo::addNoAliasMD() {
+  if (MemoryInsts.empty() || NoAliasScopes.empty())
+    return;
+
+  llvm::MDNode *NewScopeList =
+    llvm::MDNode::get(MemoryInsts[0]->getParent()->getContext(),
+                      NoAliasScopes);
+
+  // Note that we're adding here metadata on all memory-accessing instructions
+  // in this lexical scope. Perhaps surprisingly, as provided by the C
+  // specification, this is even true for instructions corresponding to
+  // accesses before the declaration of the associated restrict-qualified
+  // variables. This list will include all memory instructions from this scope
+  // as well as from all inner scopes. There's a callback from inserting an
+  // instruction which adds all memory instructions to every active lexical
+  // scope that's recording them.
+
+  for (auto &I : MemoryInsts)
+    I->setMetadata(
+          llvm::LLVMContext::MD_noalias,
+          llvm::MDNode::concatenate(I->getMetadata(
+                                      llvm::LLVMContext::MD_noalias),
+                                      NewScopeList));
+
+  MemoryInsts.clear();
+}
 
 void CodeGenFunction::EmitLabelStmt(const LabelStmt &S) {
   EmitLabel(S.getDecl());
   EmitStmt(S.getSubStmt());
 }
 
 void CodeGenFunction::EmitAttributedStmt(const AttributedStmt &S) {
   const Stmt *SubStmt = S.getSubStmt();

lib/CodeGen/CodeGenFunction.h

 #include "clang/Basic/ABI.h"
 #include "clang/Basic/CapturedStmt.h"
 #include "clang/Basic/OpenMPKinds.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/Frontend/CodeGenOptions.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/TinyPtrVector.h"
 #include "llvm/IR/ValueHandle.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Transforms/Utils/SanitizerStats.h"
 
 namespace llvm {
 class BasicBlock;
 class LLVMContext;
 class MDNode;
typedef llvm::function_ref<std::pair<llvm::Value *, llvm::Value *>(
       CodeGenFunction &, const OMPExecutableDirective &S, Address LB,
       Address UB)>
       CodeGenDispatchBoundsTy;
 
   /// \brief CGBuilder insert helper. This function is called after an
   /// instruction is created using Builder.
   void InsertHelper(llvm::Instruction *I, const llvm::Twine &Name,
                     llvm::BasicBlock *BB,
-                    llvm::BasicBlock::iterator InsertPt) const;
+                    llvm::BasicBlock::iterator InsertPt);
 
   /// CurFuncDecl - Holds the Decl for the current outermost
   /// non-closure context.
   const Decl *CurFuncDecl;
   /// CurCodeDecl - This is the inner-most code context, which includes blocks.
   const Decl *CurCodeDecl;
   const CGFunctionInfo *CurFnInfo;
   QualType FnRetTy;
void ForceCleanup(std::initializer_list<llvm::Value**> ValuesToReload = {}) {
       assert(PerformCleanup && "Already forced cleanup");
       CGF.DidCallStackSave = OldDidCallStackSave;
       CGF.PopCleanupBlocks(CleanupStackDepth, LifetimeExtendedCleanupStackSize,
                            ValuesToReload);
       PerformCleanup = false;
     }
   };
 
-  class LexicalScope : public RunCleanupsScope {
+  bool hasLocalRestrictVars(const CompoundStmt &S);
+
+  // The noalias scopes used to tag pointer values assigned to block-local
+  // restrict-qualified variables, and the memory-accessing instructions within
+  // this lexical scope to which the associated pointer-aliasing assumptions
+  // might apply. One of these will exist for each lexical scope.
+  struct LexicalNoAliasInfo {
+    bool RecordMemoryInsts;
+    llvm::TinyPtrVector<llvm::Instruction *> MemoryInsts;
+    llvm::TinyPtrVector<llvm::Metadata *> NoAliasScopes;
+
+    LexicalNoAliasInfo(bool RecordMemoryInsts = false)
+      : RecordMemoryInsts(RecordMemoryInsts) {}
+
+    void recordMemoryInsts() {
+      RecordMemoryInsts = true;
+    }
+
+    bool isRecordingMemoryInsts() const {
+      return RecordMemoryInsts;
+    }
+
+    void recordMemoryInstruction(llvm::Instruction *I) {
+      if (RecordMemoryInsts)
+        MemoryInsts.push_back(I);
+    }
+
+    void addNoAliasScope(llvm::MDNode *Scope) {
+      assert(RecordMemoryInsts &&
+             "Adding noalias scope but not recording memory accesses!");
+      NoAliasScopes.push_back(Scope);
+    }
+
+    void addNoAliasMD();
+  };
+
+  LexicalNoAliasInfo FnNoAliasInfo;
+
+  class LexicalScope : public RunCleanupsScope, public LexicalNoAliasInfo {
     SourceRange Range;
     SmallVector<const LabelDecl*, 4> Labels;
     LexicalScope *ParentScope;
 
+    // Cache whether this scope, or any parent scope, is recording memory
+    // instructions (used for later annotating accesses in blocks with local
+    // restrict-qualified variables). If this is false, the code can bypass a
+    // walk up the scope chain for each memory access.
+    bool IsRecordingMemoryInsts;
+
     LexicalScope(const LexicalScope &) = delete;
     void operator=(const LexicalScope &) = delete;
 
   public:
     /// \brief Enter a new cleanup scope.
-    explicit LexicalScope(CodeGenFunction &CGF, SourceRange Range)
-      : RunCleanupsScope(CGF), Range(Range), ParentScope(CGF.CurLexicalScope) {
+    explicit LexicalScope(CodeGenFunction &CGF, SourceRange Range,
+                          bool RecordMemoryInsts = false)
+      : RunCleanupsScope(CGF), LexicalNoAliasInfo(RecordMemoryInsts),
+        Range(Range), ParentScope(CGF.CurLexicalScope) {
       CGF.CurLexicalScope = this;
       if (CGDebugInfo *DI = CGF.getDebugInfo())
         DI->EmitLexicalBlockStart(CGF.Builder, Range.getBegin());
+
+      IsRecordingMemoryInsts = RecordMemoryInsts;
+      if (ParentScope)
+        IsRecordingMemoryInsts |= ParentScope->isRecordingMemoryInsts();
+      else
+        IsRecordingMemoryInsts |= CGF.FnNoAliasInfo.isRecordingMemoryInsts();
+    }
+
+    bool isRecordingMemoryInsts() const {
+      return IsRecordingMemoryInsts;
     }
 
     void addLabel(const LabelDecl *label) {
       assert(PerformCleanup && "adding label to dead scope?");
       Labels.push_back(label);
     }
 
+    // Record the given memory access instruction in this and all of its
+    // enclosing scopes. When we close this scope, we'll add the list of
+    // (metadata corresponding to) all the restrict-qualified local variables
+    // it declared to all the memory accesses which occurred within it.
+    // Recording is only enabled under optimization, and it's disabled in a
+    // scope unless it actually declares any local restrict variables.
+    void recordMemoryInstruction(llvm::Instruction *I) {
+      if (!IsRecordingMemoryInsts)
+        return;
+
+      LexicalNoAliasInfo::recordMemoryInstruction(I);
+      if (ParentScope)
+        ParentScope->recordMemoryInstruction(I);
+      else
+        CGF.FnNoAliasInfo.recordMemoryInstruction(I);
+    }
+
     /// \brief Exit this cleanup scope, emitting any accumulated
     /// cleanups.
     ~LexicalScope() {
       if (CGDebugInfo *DI = CGF.getDebugInfo())
         DI->EmitLexicalBlockEnd(CGF.Builder, Range.getEnd());
 
       // If we should perform a cleanup, force them now.  Note that
       // this ends the cleanup scope before rescoping any labels.
       if (PerformCleanup) {
         ApplyDebugLocation DL(CGF, Range.getEnd());
         ForceCleanup();
       }
     }
 
     /// \brief Force the emission of cleanups now, instead of waiting
     /// until this object is destroyed.
     void ForceCleanup() {
+      addNoAliasMD();
+
       CGF.CurLexicalScope = ParentScope;
       RunCleanupsScope::ForceCleanup();
 
       if (!Labels.empty())
         rescopeLabels();
     }
 
     bool hasLabels() const {
       return !Labels.empty();
     }
 
     void rescopeLabels();
   };
 
   typedef llvm::DenseMap<const Decl *, Address> DeclMapTy;
 
+  // Record this instruction for the purpose of later adding noalias metadata,
+  // is applicible, in order to support block-local restrict-qualified
+  // pointers.
+  void recordMemoryInstruction(llvm::Instruction *I) {
+    if (CurLexicalScope)
+      CurLexicalScope->recordMemoryInstruction(I);
+    else
+      FnNoAliasInfo.recordMemoryInstruction(I);
+  }
+
+  void addNoAliasScope(llvm::MDNode *Scope) {
+    if (CurLexicalScope)
+      CurLexicalScope->addNoAliasScope(Scope);
+    else
+      FnNoAliasInfo.addNoAliasScope(Scope);
+  }
+
   /// \brief The scope used to remap some variables as private in the OpenMP
   /// loop body (or other captured region emitted without outlining), and to
   /// restore old vars back on exit.
   class OMPPrivateScope : public RunCleanupsScope {
     DeclMapTy SavedLocals;
     DeclMapTy SavedPrivates;
 
   private:
private:
   /// True if we need emit the life-time markers.
   const bool ShouldEmitLifetimeMarkers;
 
   /// Add OpenCL kernel arg metadata and the kernel attribute meatadata to
   /// the function metadata.
   void EmitOpenCLKernelMetadata(const FunctionDecl *FD, 
                                 llvm::Function *Fn);
 
+  /// The noalias domain metadata for this function.
+  llvm::MDNode *NoAliasDomain;
+  /// A map between the addresses of local restrict-qualified variables and
+  /// their noalias scope.
+  llvm::DenseMap<llvm::Value *, llvm::MDNode*> NoAliasAddrMap;
+
 public:
   CodeGenFunction(CodeGenModule &cgm, bool suppressNewContext=false);
   ~CodeGenFunction();
 
   CodeGenTypes &getTypes() const { return CGM.getTypes(); }
   ASTContext &getContext() const { return CGM.getContext(); }
   CGDebugInfo *getDebugInfo() { 
     if (DisableDebugInfo) 
public:
     }
   };
   AutoVarEmission EmitAutoVarAlloca(const VarDecl &var);
   void EmitAutoVarInit(const AutoVarEmission &emission);
   void EmitAutoVarCleanups(const AutoVarEmission &emission);  
   void emitAutoVarTypeCleanup(const AutoVarEmission &emission,
                               QualType::DestructionKind dtorKind);
 
+  void EmitNoAliasScope(const AutoVarEmission &emission);
+
   void EmitStaticVarDecl(const VarDecl &D,
                          llvm::GlobalValue::LinkageTypes Linkage);
 
   class ParamValue {
     llvm::Value *Value;
     unsigned Alignment;
     ParamValue(llvm::Value *V, unsigned A) : Value(V), Alignment(A) {}
   public:

lib/CodeGen/CodeGenFunction.cpp

: CodeGenTypeCache(cgm), CGM(cgm), Target(cgm.getTarget()),
       CaseRangeBlock(nullptr), UnreachableBlock(nullptr), NumReturnExprs(0),
       NumSimpleReturnExprs(0), CXXABIThisDecl(nullptr),
       CXXABIThisValue(nullptr), CXXThisValue(nullptr),
       CXXStructorImplicitParamDecl(nullptr),
       CXXStructorImplicitParamValue(nullptr), OutermostConditional(nullptr),
       CurLexicalScope(nullptr), TerminateLandingPad(nullptr),
       TerminateHandler(nullptr), TrapBB(nullptr),
       ShouldEmitLifetimeMarkers(
-          shouldEmitLifetimeMarkers(CGM.getCodeGenOpts(), CGM.getLangOpts())) {
+          shouldEmitLifetimeMarkers(CGM.getCodeGenOpts(), CGM.getLangOpts())),
+      NoAliasDomain(nullptr) {
   if (!suppressNewContext)
     CGM.getCXXABI().getMangleContext().startNewFunction();
 
   llvm::FastMathFlags FMF;
   if (CGM.getLangOpts().FastMath)
     FMF.setUnsafeAlgebra();
   if (CGM.getLangOpts().FiniteMathOnly) {
     FMF.setNoNaNs();
void CodeGenFunction::StartFunction(GlobalDecl GD,
   // Emit a location at the end of the prologue.
   if (CGDebugInfo *DI = getDebugInfo())
     DI->EmitLocation(Builder, StartLoc);
 }
 
 void CodeGenFunction::EmitFunctionBody(FunctionArgList &Args,
                                        const Stmt *Body) {
   incrementProfileCounter(Body);
-  if (const CompoundStmt *S = dyn_cast<CompoundStmt>(Body))
+  if (const CompoundStmt *S = dyn_cast<CompoundStmt>(Body)) {
+    // Block-local restrict-qualified pointers have the property of implying
+    // exclusivity of access within their block, even with respect to accesses
+    // in their block prior to the point of declaraction of the
+    // restrict-qualified pointer. We don't want to incur the expense
+    // associated with recording memory-accessing instructions when we won't
+    // have any block-local restrict-qualified pointers, so we check for any
+    // relevant declaractions first.
+    if (hasLocalRestrictVars(*S))
+      FnNoAliasInfo.recordMemoryInsts();
+
     EmitCompoundStmtWithoutScope(*S);
-  else
+
+    // Now that we're done with the block, add noalias metadata if we had any
+    // block-local restrict-qualified pointers.
+    FnNoAliasInfo.addNoAliasMD();
+  } else {
     EmitStmt(Body);
   }
+}
 
 /// When instrumenting to collect profile data, the counts for some blocks
 /// such as switch cases need to not include the fall-through counts, so
 /// emit a branch around the instrumentation code. When not instrumenting,
 /// this just calls EmitBlock().
 void CodeGenFunction::EmitBlockWithFallThrough(llvm::BasicBlock *BB,
                                                const Stmt *S) {
   llvm::BasicBlock *SkipCountBB = nullptr;

 
 CodeGenFunction::SanitizerScope::~SanitizerScope() {
   CGF->IsSanitizerScope = false;
 }
 
 void CodeGenFunction::InsertHelper(llvm::Instruction *I,
                                    const llvm::Twine &Name,
                                    llvm::BasicBlock *BB,
-                                   llvm::BasicBlock::iterator InsertPt) const {
+                                   llvm::BasicBlock::iterator InsertPt) {
   LoopStack.InsertHelper(I);
   if (IsSanitizerScope)
     CGM.getSanitizerMetadata()->disableSanitizerForInstruction(I);
+
+  // When we have block-local restrict-qualified pointers, we need to record
+  // all memory-accessing instructions (i.e. any kind of instruction for which
+  // AA::getModRefInfo might return something other than NoModRef) so that they
+  // can be tagged with noalias metadata with noalias scopes corresponding to
+  // the applicable restrict-qualified pointers.
+  if (I->mayReadOrWriteMemory())
+    recordMemoryInstruction(I);
 }
 
 void CGBuilderInserter::InsertHelper(
     llvm::Instruction *I, const llvm::Twine &Name, llvm::BasicBlock *BB,
     llvm::BasicBlock::iterator InsertPt) const {
   llvm::IRBuilderDefaultInserter::InsertHelper(I, Name, BB, InsertPt);
   if (CGF)
     CGF->InsertHelper(I, Name, BB, InsertPt);

test/CodeGen/arm_neon_intrinsics.c

 // RUN: %clang_cc1 -triple thumbv7s-apple-darwin -target-abi apcs-gnu\
 // RUN:  -target-cpu swift -fallow-half-arguments-and-returns -ffreestanding \
-// RUN:  -disable-O0-optnone -emit-llvm -o - %s \
+// RUN:  -disable-O0-optnone -mllvm -use-noalias-intrinsic-during-inlining=0 -emit-llvm -o - %s \
 // RUN:  | opt -S -mem2reg | FileCheck %s
+// FIXME: We should update the checks here to accept the llvm.noalias intrinsic
+// that results from inlining the called always_inline functions (and remove
+// the -use-noalias-intrinsic-during-inlining=0 above).
 
 #include <arm_neon.h>
 
 // CHECK-LABEL: @test_vaba_s8(
 // CHECK:   [[VABD_V_I_I:%.*]] = call <8 x i8> @llvm.arm.neon.vabds.v8i8(<8 x i8> %b, <8 x i8> %c)
 // CHECK:   [[ADD_I:%.*]] = add <8 x i8> %a, [[VABD_V_I_I]]
 // CHECK:   ret <8 x i8> [[ADD_I]]
 int8x8_t test_vaba_s8(int8x8_t a, int8x8_t b, int8x8_t c) {

test/CodeGen/noalias.c

+// RUN: %clang_cc1 -triple x86_64-apple-darwin -O1 -disable-llvm-optzns %s -emit-llvm -o - | FileCheck %s
+
+int r;
+void ex1(int *);
+
+int *a;
+int *foo() {
+  int * restrict x = a;
+  return x;
+
+// CHECK-LABEL: define i32* @foo()
+// CHECK: [[x_addr_foo1:%[a-z0-9_.]+]] = alloca i32*
+// CHECK: [[x_foo1:%[a-z0-9_.]+]] = load i32*, i32** @a{{.*}}, !noalias [[TAG_foo1:!.*]]
+// CHECK: [[x_a_foo1:%[a-z0-9_.]+]] = call i32* @llvm.noalias.p0i32(i32* [[x_foo1]], metadata [[TAG_foo1]])
+// CHECK: store i32* [[x_a_foo1]], i32** [[x_addr_foo1]]{{.*}}, !noalias [[TAG_foo1]]
+}
+
+int *a2;
+int *foo1(int b) {
+  int * restrict x;
+
+// CHECK-LABEL: define i32* @foo1(i32 %b)
+// CHECK: [[x_addr_foo2:%[a-z0-9_.]+]] = alloca i32*
+// CHECK: [[x2_addr_foo2:%[a-z0-9_.]+]] = alloca i32*
+
+  if (b) {
+    x = a;
+    r += *x;
+    ex1(x);
+
+// CHECK: [[x_foo2:%[a-z0-9_.]+]] = load i32*, i32** @a{{.*}}, !noalias [[x_x2_tag_foo2:!.*]]
+// CHECK: [[x_a_foo2:%[a-z0-9_.]+]] = call i32* @llvm.noalias.p0i32(i32* [[x_foo2]], metadata [[x_tag_foo2:!.*]])
+// CHECK: store i32* [[x_a_foo2]], i32** [[x_addr_foo2]]{{.*}}, !noalias [[x_x2_tag_foo2]]
+// CHECK: call void @ex1
+    ++x;
+    *x = r;
+    ex1(x);
+
+// CHECK:  [[old_x_foo2:%[a-z0-9_.]+]] = load i32*, i32** [[x_addr_foo2]]{{.*}}, !noalias [[x_x2_tag_foo2]]
+// CHECK:  [[x_foo2:%[a-z0-9_.]+]] = getelementptr inbounds i32, i32* [[old_x_foo2]], i32 1
+// CHECK:  [[x_a_foo2:%[a-z0-9_.]+]] = call i32* @llvm.noalias.p0i32(i32* [[x_foo2]], metadata [[x_tag_foo2]])
+// CHECK:  store i32* [[x_a_foo2]], i32** [[x_addr_foo2]]{{.*}}, !noalias [[x_x2_tag_foo2]]
+// CHECK: call void @ex1
+
+    x += b;
+    *x = r;
+    ex1(x);
+
+// CHECK:  [[old_x_foo2:%[a-z0-9_.]+]] = load i32*, i32** [[x_addr_foo2]]{{.*}}, !noalias [[x_x2_tag_foo2]]
+// CHECK:  [[x_foo2:%[a-z0-9_.]+]] = getelementptr inbounds i32, i32* [[old_x_foo2]], i64
+// CHECK:  [[x_a_foo2:%[a-z0-9_.]+]] = call i32* @llvm.noalias.p0i32(i32* [[x_foo2]], metadata [[x_tag_foo2]])
+// CHECK:  store i32* [[x_a_foo2]], i32** [[x_addr_foo2]]{{.*}}, !noalias [[x_x2_tag_foo2]]
+// CHECK: call void @ex1
+
+    int * restrict x2 = a2;
+    *x2 = r;
+    ex1(x2);
+
+// CHECK: [[x2_foo2:%[a-z0-9_.]+]] = load i32*, i32** @a2{{.*}}, !noalias [[x_x2_tag_foo2]]
+// CHECK: [[x2_a_foo2:%[a-z0-9_.]+]] = call i32* @llvm.noalias.p0i32(i32* [[x2_foo2]], metadata [[x2_tag_foo2:!.*]])
+// CHECK: store i32* [[x2_a_foo2]], i32** [[x2_addr_foo2]]{{.*}}, !noalias [[x_x2_tag_foo2]]
+// CHECK: call void @ex1
+  } else {
+    x = a2;
+    r += *x;
+
+// CHECK: [[x_foo2:%[a-z0-9_.]+]] = load i32*, i32** @a2{{.*}}, !noalias [[x_tag_foo2]]
+// CHECK: [[x_a_foo2:%[a-z0-9_.]+]] = call i32* @llvm.noalias.p0i32(i32* [[x_foo2]], metadata [[x_tag_foo2]])
+// CHECK: store i32* [[x_a_foo2]], i32** [[x_addr_foo2]]{{.*}}, !noalias [[x_tag_foo2]]
+  }
+
+  return x;
+}
+
+int *bar() {
+  int * x = a;
+  return x;
+
+// CHECK-LABEL: define i32* @bar()
+// CHECK-NOT: noalias
+// CHECK: ret i32*
+}
+

test/OpenMP/taskloop_firstprivate_codegen.cpp

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_MAIN_TY]]* noalias)
 
 // CHECK: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_DOUBLE_TY]]*,
 // CHECK: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_DOUBLE_TY]]]*,
 // CHECK: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK: [[PRIV_SIVAR_ADDR:%.+]] = alloca i32*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]], i32** [[PRIV_T_VAR_ADDR]], [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], i32** [[PRIV_SIVAR_ADDR]])
 
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_DOUBLE_TY]]*, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]],
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_DOUBLE_TY]]]*, [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],

 // CHECK: ret void
 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_TMAIN_TY]]* noalias)
 // CHECK: alloca i32*,
 // CHECK-DAG: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK-DAG: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK-DAG: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_INT_TY]]]*,
 // CHECK-DAG: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_INT_TY]]*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, i32** [[PRIV_T_VAR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]], [[S_INT_TY]]** [[PRIV_VAR_ADDR]])
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_INT_TY]]]*, [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** [[PRIV_VAR_ADDR]],
 
 // Privates actually are used.

test/OpenMP/taskloop_lastprivate_codegen.cpp

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_MAIN_TY]]* noalias)
 
 // CHECK: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_DOUBLE_TY]]*,
 // CHECK: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_DOUBLE_TY]]]*,
 // CHECK: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK: [[PRIV_SIVAR_ADDR:%.+]] = alloca i32*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]], i32** [[PRIV_T_VAR_ADDR]], [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], i32** [[PRIV_SIVAR_ADDR]])
 
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_DOUBLE_TY]]*, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]],
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_DOUBLE_TY]]]*, [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],

 // CHECK: ret void
 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_TMAIN_TY]]* noalias)
 // CHECK: alloca i32*,
 // CHECK-DAG: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK-DAG: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK-DAG: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_INT_TY]]]*,
 // CHECK-DAG: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_INT_TY]]*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, i32** [[PRIV_T_VAR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]], [[S_INT_TY]]** [[PRIV_VAR_ADDR]])
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_INT_TY]]]*, [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** [[PRIV_VAR_ADDR]],
 
 // Privates actually are used.

test/OpenMP/taskloop_private_codegen.cpp

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_MAIN_TY]]* noalias)
 
 // CHECK: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_DOUBLE_TY]]*,
 // CHECK: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_DOUBLE_TY]]]*,
 // CHECK: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK: [[PRIV_SIVAR_ADDR:%.+]] = alloca i32*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]], i32** [[PRIV_T_VAR_ADDR]], [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], i32** [[PRIV_SIVAR_ADDR]])
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_DOUBLE_TY]]*, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]],
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_DOUBLE_TY]]]*, [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_SIVAR:%.+]] = load i32*, i32** [[PRIV_SIVAR_ADDR]],
 

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_TMAIN_TY]]* noalias)
 
 // CHECK: alloca i32*,
 // CHECK-DAG: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK-DAG: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK-DAG: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_INT_TY]]]*,
 // CHECK-DAG: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_INT_TY]]*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, i32** [[PRIV_T_VAR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]], [[S_INT_TY]]** [[PRIV_VAR_ADDR]])
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_INT_TY]]]*, [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** [[PRIV_VAR_ADDR]],
 
 // Privates actually are used.

test/OpenMP/taskloop_simd_firstprivate_codegen.cpp

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_MAIN_TY]]* noalias)
 
 // CHECK: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_DOUBLE_TY]]*,
 // CHECK: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_DOUBLE_TY]]]*,
 // CHECK: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK: [[PRIV_SIVAR_ADDR:%.+]] = alloca i32*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]], i32** [[PRIV_T_VAR_ADDR]], [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], i32** [[PRIV_SIVAR_ADDR]])
 
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_DOUBLE_TY]]*, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]],
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_DOUBLE_TY]]]*, [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],

 // CHECK: ret void
 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_TMAIN_TY]]* noalias)
 // CHECK: alloca i32*,
 // CHECK-DAG: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK-DAG: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK-DAG: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_INT_TY]]]*,
 // CHECK-DAG: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_INT_TY]]*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, i32** [[PRIV_T_VAR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]], [[S_INT_TY]]** [[PRIV_VAR_ADDR]])
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_INT_TY]]]*, [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** [[PRIV_VAR_ADDR]],
 
 // Privates actually are used.

test/OpenMP/taskloop_simd_lastprivate_codegen.cpp

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_MAIN_TY]]* noalias)
 
 // CHECK: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_DOUBLE_TY]]*,
 // CHECK: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_DOUBLE_TY]]]*,
 // CHECK: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK: [[PRIV_SIVAR_ADDR:%.+]] = alloca i32*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]], i32** [[PRIV_T_VAR_ADDR]], [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], i32** [[PRIV_SIVAR_ADDR]])
 
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_DOUBLE_TY]]*, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]],
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_DOUBLE_TY]]]*, [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],

 // CHECK: ret void
 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_TMAIN_TY]]* noalias)
 // CHECK: alloca i32*,
 // CHECK-DAG: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK-DAG: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK-DAG: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_INT_TY]]]*,
 // CHECK-DAG: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_INT_TY]]*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, i32** [[PRIV_T_VAR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]], [[S_INT_TY]]** [[PRIV_VAR_ADDR]])
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_INT_TY]]]*, [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** [[PRIV_VAR_ADDR]],
 
 // Privates actually are used.

test/OpenMP/taskloop_simd_private_codegen.cpp

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_MAIN_TY]]* noalias)
 
 // CHECK: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_DOUBLE_TY]]*,
 // CHECK: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_DOUBLE_TY]]]*,
 // CHECK: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK: [[PRIV_SIVAR_ADDR:%.+]] = alloca i32*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_MAIN_TY]]*, [[S_DOUBLE_TY]]**, i32**, [2 x [[S_DOUBLE_TY]]]**, [2 x i32]**, i32**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]], i32** [[PRIV_T_VAR_ADDR]], [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], i32** [[PRIV_SIVAR_ADDR]])
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_DOUBLE_TY]]*, [[S_DOUBLE_TY]]** [[PRIV_VAR_ADDR]],
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_DOUBLE_TY]]]*, [2 x [[S_DOUBLE_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_SIVAR:%.+]] = load i32*, i32** [[PRIV_SIVAR_ADDR]],
 

 
 // CHECK: define internal i32 [[TASK_ENTRY]](i32, [[KMP_TASK_TMAIN_TY]]* noalias)
 
 // CHECK: alloca i32*,
 // CHECK-DAG: [[PRIV_T_VAR_ADDR:%.+]] = alloca i32*,
 // CHECK-DAG: [[PRIV_VEC_ADDR:%.+]] = alloca [2 x i32]*,
 // CHECK-DAG: [[PRIV_S_ARR_ADDR:%.+]] = alloca [2 x [[S_INT_TY]]]*,
 // CHECK-DAG: [[PRIV_VAR_ADDR:%.+]] = alloca [[S_INT_TY]]*,
-// CHECK: store void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*), void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
+// CHECK: [[PRIV_NAFP:%.+]] = call void (i8*, ...)* @llvm.noalias.p0f_isVoidp0i8varargf(void (i8*, ...)* bitcast (void ([[PRIVATES_TMAIN_TY]]*, i32**, [2 x i32]**, [2 x [[S_INT_TY]]]**, [[S_INT_TY]]**)* [[PRIVATES_MAP_FN]] to void (i8*, ...)*),
+// CHECK: store void (i8*, ...)* [[PRIV_NAFP]], void (i8*, ...)** [[MAP_FN_ADDR:%.+]],
 // CHECK: [[MAP_FN:%.+]] = load void (i8*, ...)*, void (i8*, ...)** [[MAP_FN_ADDR]],
 // CHECK: call void (i8*, ...) [[MAP_FN]](i8* %{{.+}}, i32** [[PRIV_T_VAR_ADDR]], [2 x i32]** [[PRIV_VEC_ADDR]], [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]], [[S_INT_TY]]** [[PRIV_VAR_ADDR]])
 // CHECK: [[PRIV_T_VAR:%.+]] = load i32*, i32** [[PRIV_T_VAR_ADDR]],
 // CHECK: [[PRIV_VEC:%.+]] = load [2 x i32]*, [2 x i32]** [[PRIV_VEC_ADDR]],
 // CHECK: [[PRIV_S_ARR:%.+]] = load [2 x [[S_INT_TY]]]*, [2 x [[S_INT_TY]]]** [[PRIV_S_ARR_ADDR]],
 // CHECK: [[PRIV_VAR:%.+]] = load [[S_INT_TY]]*, [[S_INT_TY]]** [[PRIV_VAR_ADDR]],
 
 // Privates actually are used.