CodeGen: Cast alloca to expected address space

[yaxunl]

Summary

Alloca always returns a pointer in alloca address space, which may
be different from the type defined by the language. For example,
in C++ the auto variables are in the default address space. Therefore
cast alloca to the expected address space when necessary.

Depends on https://reviews.llvm.org/D32977

[t-tye]

lib/CodeGen/CGDecl.cpp

1105–1119

Is any assert done to ensure that it is legal to address space cast from variable address space to expected address space? Presumably the language, by definition, will only be causing legal casts. For example from alloca address space to generic (which includes the alloca address space).

For OpenCL, can you explain how the local variable can have the constant address space and use an alloca for allocation? Wouldn't a constant address space mean it was static and so should not be using alloca? And if it is using an alloca, how can it then be accessed as if it was in constant address space?

lib/CodeGen/CodeGenTypes.h

200 ↗

(On Diff #95828)

Should the name reflect that the type returned is not the variable type, but a pointer to the variable type? For example, getVariablePointerType or getPointerToVariableType.

[yaxunl]

lib/CodeGen/CGDecl.cpp

1105–1119

If the auto var has address space qualifier specified through __attribute__((address_space(n))), there is not much we can check in clang since it is target dependent. We will just emit address space cast when necessary and let the backend check the validity of the address space cast.

Otherwise, for OpenCL, we can assert the expected address space is default (for OpenCL default address space in AST represents private address space in source language) or constant. For other languages we can assert the expected address space qualifier is default (no address space qualifier). It is not convenient to further check whether the emitted LLVM address space cast instruction is valid since it requires target specific information, therefore such check is better deferred to the backend.

For OpenCL, currently automatic variable in constant address space is emitted in private address space. For example, currently Clang does not diagnose the following code

void f(global int* a) {
  global int* constant p = a;
}

Instead, it emits alloca for p, essentially treats it as global int* const p. This seems to be a bug to me (or maybe we can call it a feature? since there seems no better way to translate this to LLVM IR, or simply diagnose this as an error). However, this is better addressed by another patch.

lib/CodeGen/CodeGenTypes.h

200 ↗

(On Diff #95828)

I think getPointerToVariableType is better. I will change it.

[rjmccall]

lib/CodeGen/CGDecl.cpp

1118

This is a lot of work to be doing in a pretty common routine for the benefit of one unusual target. Maybe there's some way to fast-path it, like you can cache the AST address space of the stack on the CGM (assuming it's always got its own AST address space, I guess) and just check whether Ty.getAddressSpace() is different from that.

Also, I feel like general routines in IRGen should never be calling CreateAddrSpaceCast; they should always be calling getTargetCodeGenInfo().performAddrSpaceCast instead, just in case the target has some more specific idea about how the conversion should be done. I know we're not consistent about that right now, but it's something to aim for.

Also, I agree with Tony Tye: if it's an intended feature that you can declare locals in the constant address space, you need to figure out what the allocation semantics are for that and emit the appropriate code rather than just hand-waving it here after the fact.

lib/CodeGen/CodeGenTypes.h

200 ↗

(On Diff #95828)

The only time you use this function is to immediately call getAddressSpace on it. Seems to me you should just call getTargetAddressSpace(D.getType()).

[yaxunl]

lib/CodeGen/CGDecl.cpp

1105–1119

Hi Anastasia,

Any comments about the automatic variable in constant address space? Thanks.

[Anastasia]

lib/CodeGen/CGDecl.cpp

1105–1119

From the spec s6.5.3 it feels like we should follow the same implementation path in Clang for constant AS inside kernel function as local AS. Because constant AS objects are essentially global objects.

Although, we didn't have any issues up to now because const just does the trick of optimising the objects out eventually. I am not clear if this creates any issue now with your allocation change. It feels though that it should probably work fine just as is?

[Anastasia]

lib/CodeGen/CGDecl.cpp

1115

Do you need to check this? I thought your change doesn't affect OpenCL because alloca AS == private AS in OpenCL and constant AS local objects are treated as private const objects anyways.

test/CodeGen/address-space.c

3

CHeCK -> CHECK

test/CodeGenCXX/amdgcn-automatic-variable.cpp

26

I am wondering if all these different test cases are really needed. Are we testing any different program paths from the patch?

Also would it make sense to add a test case with an object in an AS different to 0 (i.e. with __attribute__((address_space(n))))

[rjmccall]

lib/CodeGen/CGDecl.cpp

1105–1119

If these __constant locals are required to be const (or are implicitly const?) and have constant initializers, it seems to me the implementation obviously intended by the spec is that you allocate them statically in the constant address space. It's likely that just implicitly making the variable static in Sema, the same way you make it implicitly const, will make IRGen and various other parts of the compiler just do the right thing.

[yaxunl]

lib/CodeGen/CGDecl.cpp

1105–1119

My patch does not change the current behaviour of Clang regarding function-scope variable in constant address space. Basically there is no issue if there is no address taken. However, if there is address taken, there will be assertion due to casting private pointer to constant address space. I think probably it is better to emit function-scope variable in constant address space as global variable in constant address space instead of alloca, as John suggested.

1105–1119

I agree function-scope variable in constant address space should be emitted as global variable in constant address space instead of alloca. However, in OpenCL 1.2 section 6.8, "The static storage-class specifier can only be used for non-kernel functions and global variables declared in program scope." Currently Clang diagnoses function-scope variables with static storage class for OpenCL 1.2. Therefore we cannot make function-scope variable in constant address space have static storage class. However, I think we can still emit function-scope variable in constant address space as global variable in CodeGen.

[rjmccall]

lib/CodeGen/CGDecl.cpp

1105–1119

There's precedent for treating something as implicitly static in the AST. thread_local variables in C++ are considered implicitly static.

You'll need to change VarDecl::hasLocalStorage() and isStaticLocal(), and you should look around for other uses of getTSCSpec() that look like they're implementing the same logic.

[Anastasia]

lib/CodeGen/CGDecl.cpp

1105–1119

I belive we already do similar logic for stack variables with __local AS. Because they are emmited as static globals. There should be no contradictions with spec to do the same for __constant AS objects. They are essentially intended to be located in a "special" memory sections and not on a stack. Also constant AS local variable can only be declared in the kernel functions.

[yaxunl]

lib/CodeGen/CGDecl.cpp

1105–1119

I created another patch for this issue: https://reviews.llvm.org/D32977

[t-tye]

lib/CodeGen/CGDecl.cpp

1115

Given patch D32977 that makes function local variables marked with the constant address space treated as static, will they ever be processed by this function now? If so then the extra test for opencl_constant would not be needed.

[yaxunl]

lib/CodeGen/CGDecl.cpp

1115

Will remove this.

test/CodeGenCXX/amdgcn-automatic-variable.cpp

26

I think at least I should cover the typical use cases of auto var.

I will add a test for attribute((address_space(n))))

[yaxunl]

test/CodeGenCXX/amdgcn-automatic-variable.cpp

26

Sorry. I just checked that C++ does not allow attribute((address_space(n))) on automatic var.

[yaxunl]

Revised by reviewers' comments.

[rjmccall]

lib/CodeGen/TargetInfo.cpp

7291

How about, instead of introducing a second method, we just change performAddrSpaceCast to take two AST address spaces and a flag indicating whether the address is known to be non-null? Does your target have an AST-level address space for the stack?

[yaxunl]

lib/CodeGen/TargetInfo.cpp

7291

In both AST and LLVM, the destination pointee type of address space cast may be different from the source pointee type (e.g. addrspacecast i32 addrspace(1)* %a to i8 addrspace(4)* is valid LLVM instruction), so performAddrSpaceCast needs to know the destination QualType, not just the address space.

[rjmccall]

lib/CodeGen/TargetInfo.cpp

7291

Is there any harm to generating separate cast instructions when this happens?

[yaxunl]

lib/CodeGen/TargetInfo.cpp

7291

They should be equivalent but the LLVM IR will be more verbose and take more space, and LLVM passes will take more time since previously they only need to process one instruction but now they need to process two instructions.

The original representation is more concise, why should we change that to emit two instructions instead of one instruction?

[rjmccall]

lib/CodeGen/TargetInfo.cpp

7291

It's a cleaner API.

But fine, whatever, feel free to pass down an optional llvm::PointerType* for the destination type.

[yaxunl]

Removed adjustAddrSpaceForAutoVar and use performAddrSpaceCast instead.

[t-tye]

lib/CodeGen/CGDecl.cpp

1115

Should allowing specifying an address space on a function local automatic variable in OpenCL be allowed? It seems generating LLVM IR that allocates the variable in the alloca address space then address space casting the pointer to some other address space is not what such a language feature is requesting.

It seems it is really requesting that the variable is allocated in a different address space. That could be indicated by putting a different address space on the alloca itself, but the builder only allows an alloca to use the default alloca address space. No target supports this ability.

So maybe the best solution is to make OpenCL be the same as C and not allow an address space qualifier on function scope automatic variables.

If that is done then this assert will only allow the language default address space.

[yaxunl]

lib/CodeGen/CGDecl.cpp

1115

Agree. I think that is why C++ forbids __attribute__((address_space(n)) on automatic variables. I will make it apply to all languages.

[rjmccall]

lib/CodeGen/CGDecl.cpp

1115

Yeah, the embedded C specification (ISO/IEC TR 18037) says "an address space name cannot be used to qualify an object that has automatic storage duration", so this restriction should be applied regardless of language mode. We can deal with the possibility of auto variables in other address spaces when someone wants to add that capability.

(OpenCL's __constant is an acceptable exception because it implicitly makes the variable static.)

1119

Hmm. I would prefer if we could avoid doing this in the fast path. Suggestion:

• Please add a variable (ASTAllocaAddressSpace) to CodeGenTypeCache that stores the AST address space of the stack. You can add a function to CodeGenTargetInfo to initialize this; in most targets, it will just return 0, but your target can make it ~0U or something, anything works as long as it doesn't incorrectly match an actual target address space. (But you should really consider adding an actual target-specific address space for the stack! It'll probably be useful in builtin code or something eventually.)
• Please add an assert that T.getAddressSpace() == 0, and then make this call conditional on whether ASTAllocaAddressSpace is non-zero.

You can just write address.getElementType() instead of address.getPointer()->getType()->getPointerElementType().

Are you intentionally leaving emission.Addr as the unconverted pointer? That will probably mess everything else downstream that uses 'emission', and the only reason you're not seeing that is that OpenCL probably doesn't actually have anything downstream that uses 'emission' — destructors or whatever else.

Please add an inline comment like

, /*non-null*/ true);

lib/CodeGen/TargetInfo.cpp

7294

Spurious newline?

[yaxunl]

diagnose automatic var with invalid addr space for OpenCL.

[rjmccall]

lib/Sema/SemaDecl.cpp

7210

err_opencl_function_variable seems like a better diagnostic, at least for opencl_global. You can fall back on this more general diagnostic for other address spaces.

I think you can just assert here that T.getAddressSpace() != LangAS::opencl_constant instead of checking it.

Thinking about it, it feels like opencl_local should be handled the same way: they shouldn't get here because they don't really have automatic storage duration. They're much more like a thread-local than an actual local. However, we can tackle that some other time.

[yaxunl]

Fix diagnostic for global addr

[rjmccall]

Looking good. We had some overlapping patches and comments there, so I want to make sure you didn't miss my comment on EmitAutoVarAlloca.

lib/Sema/SemaDecl.cpp

7205

LLVM code style doesn't put a space after assert.

[yaxunl]

Fix format.

[yaxunl]

lib/CodeGen/CGDecl.cpp

1119

Sorry I missed this comment. I will do the changes.

About emission.Addr, I leave the unconverted pointer intentionally. I need this to work for both C++ and OpenCL. I checked the usage of emission.Addr and they seem to be only load or store, so I think it is safe to use the unconverted pointer. Now you mention destructor. I think I need to change that and I am going to add a lit test also.

[yaxunl]

Revised by John's comments.

[rjmccall]

lib/CodeGen/CGDecl.cpp

1120

A lot of this line can be address.getElementType().

lib/CodeGen/CodeGenTypes.cpp

95

Did you intend to change this file?

lib/CodeGen/TargetInfo.cpp

7296

Can we rename LangAS::Count to something more meaningful like LangAS::FirstTargetAddressSpace? I think this would clarify a lot of code.

Is the OpenCL special case in ASTContext::getTargetAddressSpace still correct with this patch? A pointer in LangAS::Default should be lowered as a generic pointer instead of a pointer into the alloca address space, right?

[yaxunl]

lib/CodeGen/TargetInfo.cpp

7296

Will do.

The OpenCL special case in ASTContext::getTargetAddressSpace is still correct with this patch. In OpenCL private address space is still represented in AST by LangAS::Default, so a pointer in LangAS::Default should be lowered as a pointer to alloca address space.

[yaxunl]

Revised by John's comments.

[rjmccall]

lib/CodeGen/TargetInfo.cpp

7296

Then I don't understand. If __private is always the alloca address space, why does IRGen have to add addrspace casts after allocating local variables in it? IRGen's invariant is that the value stored in LocalDeclMap for a variable of AST type T is a value of type [[T]] addrspace([[AS]]) *, where [[T]] is the lowered IR type for T and [[AS]] is the target address space of the variable. For auto variables, AS is always LangAS::Default, which according to you means that [[AS]] is always LLVM's notion of the alloca address space, which is exactly the type produced by alloca. So why is there ever a cast?

My understanding was that, on your target, private is (at least sometimes) a 64-bit extension of the alloca address space, which is 32-bit. But that makes private a different address space from the alloca address space, so the special case in getTargetAddressSpace is still wrong, because that special case means that a pointer-to-__private type will be lowered to be a 32-bit pointer type.

Also, I'm not sure why the normal AddrSpaceMap mechanism isn't sufficient for mapping LangAS::Default.

[yaxunl]

lib/CodeGen/TargetInfo.cpp

7296

This work is mostly for C++.

For OpenCL, the default address space is mapped to alloca address space (5), there is no address space cast inserted.

For C++, the default address space is mapped to generic address space (0), therefore the address space cast is needed.

[rjmccall]

lib/CodeGen/TargetInfo.cpp

7296

Wait, I think I might understand. You're saying that, when you're compiling OpenCL, you want __private to just be the normal 32-bit address space, but when you're compiling other languages, you want LangAS::Default to be this 64-bit extension, probably because those other languages don't break things down in address spaces as consistently as OpenCL and so it's more important for LangAS::Default to be able to reach across address spaces.

I do not feel that it's correct for this to be hard-coded in ASTContext; it really seems like a target-specific policy that should just be reflected in AddrSpaceMap. That does mean that you'll have to find some way of informing your TargetInfo that it's building OpenCL. The normal design for something like that would be a target option that the driver would automatically set when it recognized that it was building OpenCL; or you could change getAddressSpaceMap to take a LangOptions.

[Anastasia]

lib/CodeGen/CodeGenTypes.cpp

95

Why this change?

lib/CodeGen/TargetInfo.cpp

411

Seems like some parameters are unused.

lib/Sema/SemaDecl.cpp

7206

I think it was nicer that all OpenCL related changes were grouped in one spot under the same if statement. Even though it isn't quite done everywhere. But this change makes them even more scattered.

test/CodeGenCXX/amdgcn-automatic-variable.cpp

26

I thought address space was undocumented extension. So essentially you can't have a cast to any address space but a default here? Would it work for the C instead to add attribute((address_space(n)))?

test/SemaOpenCL/storageclass-cl20.cl

16

I am guessing generic should be rejected here as well... although it should only apply to pointers. But it seems we don't handle this properly.

[rjmccall]

lib/CodeGen/TargetInfo.cpp

411

This is just the default implementation. The idea is that targets that need to do something more complex on a particular conversion — e.g. to make sure that null pointers are translated correctly when they have different bit-patterns — can easily do so.

[yaxunl]

lib/CodeGen/CodeGenTypes.cpp

95

Will undo it.

lib/CodeGen/TargetInfo.cpp

7294

will remove it.

lib/Sema/SemaDecl.cpp

7206

Will move them down.

test/CodeGenCXX/amdgcn-automatic-variable.cpp

26

As John pointed out, this due to the embedded C specification (ISO/IEC TR 18037). Clang applies it to all languages except OpenCL.

test/SemaOpenCL/storageclass-cl20.cl

16

will add a line for generic.

[yaxunl]

Removed special handling of OpenCL from getTargetAddressSpace.

[yaxunl]

lib/CodeGen/TargetInfo.cpp

7294

Sorry this line is needed to separate it from the next function.

[rjmccall]

Looks great, thank you for being patient.

[yaxunl]

Any further comments? Thanks.

[tstellar]

cfe/trunk/lib/Basic/Targets.cpp

2037–2038 ↗

(On Diff #99479)

Shouldn't this be 0 ?

include/clang/AST/Type.h

public:
   /// True if the lifetime is either strong or weak.
   bool hasStrongOrWeakObjCLifetime() const {
     ObjCLifetime lifetime = getObjCLifetime();
     return (lifetime == OCL_Strong || lifetime == OCL_Weak);
   }
 
   bool hasAddressSpace() const { return Mask & AddressSpaceMask; }
   unsigned getAddressSpace() const { return Mask >> AddressSpaceShift; }
+  bool hasTargetSpecificAddressSpace() const {
+    return getAddressSpace() >= LangAS::Count;
+  }
   /// Get the address space attribute value to be printed by diagnostics.
   unsigned getAddressSpaceAttributePrintValue() const {
     auto Addr = getAddressSpace();
     // This function is not supposed to be used with language specific
     // address spaces. If that happens, the diagnostic message should consider
     // printing the QualType instead of the address space value.
-    assert(Addr == 0 || Addr >= LangAS::Count);
+    assert(Addr == 0 || hasTargetSpecificAddressSpace());
     if (Addr)
       return Addr - LangAS::Count;
     // TODO: The diagnostic messages where Addr may be 0 should be fixed
     // since it cannot differentiate the situation where 0 denotes the default
     // address space or user specified __attribute__((address_space(0))).
     return 0;
   }
   void setAddressSpace(unsigned space) {

include/clang/Basic/DiagnosticSemaKinds.td

   "address space is negative">;
 def err_attribute_address_space_too_high : Error<
   "address space is larger than the maximum supported (%0)">;
 def err_attribute_address_multiple_qualifiers : Error<
   "multiple address spaces specified for type">;
 def err_attribute_address_function_type : Error<
   "function type may not be qualified with an address space">;
 def err_as_qualified_auto_decl : Error<
-  "automatic variable qualified with an address space">;
+  "automatic variable qualified with an%select{| invalid}0 address space">;
 def err_arg_with_address_space : Error<
   "parameter may not be qualified with an address space">;
 def err_field_with_address_space : Error<
   "field may not be qualified with an address space">;
 def err_attr_objc_ownership_redundant : Error<
   "the type %0 is already explicitly ownership-qualified">;
 def err_invalid_nsnumber_type : Error<
   "%0 is not a valid literal type for NSNumber">;

lib/CodeGen/CGDecl.cpp

 //===--- CGDecl.cpp - Emit LLVM Code for declarations ---------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This contains code to emit Decl nodes as LLVM code.
 //
 //===----------------------------------------------------------------------===//
 
-#include "CodeGenFunction.h"
 #include "CGBlocks.h"
 #include "CGCXXABI.h"
 #include "CGCleanup.h"
 #include "CGDebugInfo.h"
 #include "CGOpenCLRuntime.h"
 #include "CGOpenMPRuntime.h"
+#include "CodeGenFunction.h"
 #include "CodeGenModule.h"
+#include "TargetInfo.h"
 #include "clang/AST/ASTContext.h"
 #include "clang/AST/CharUnits.h"
 #include "clang/AST/Decl.h"
 #include "clang/AST/DeclObjC.h"
 #include "clang/AST/DeclOpenMP.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Basic/TargetInfo.h"
 #include "clang/CodeGen/CGFunctionInfo.h"
if (Ty->isConstantSizeType()) {
     QualType elementType;
     std::tie(elementCount, elementType) = getVLASize(Ty);
 
     llvm::Type *llvmTy = ConvertTypeForMem(elementType);
 
     // Allocate memory for the array.
     llvm::AllocaInst *vla = Builder.CreateAlloca(llvmTy, elementCount, "vla");
     vla->setAlignment(alignment.getQuantity());
 
     address = Address(vla, alignment);
   }
 
-  setAddrOfLocalVar(&D, address);
+  // Alloca always returns a pointer in alloca address space, which may
+  // be different from the type defined by the language. For example,
+  // in C++ the auto variables are in the default address space. Therefore
+  // cast alloca to the expected address space when necessary.
+  auto T = D.getType();
+  assert(T.getAddressSpace() == LangAS::Default);
+  auto *Addr = getTargetHooks().performAddrSpaceCast(
+      *this, address.getPointer(), ~0U /* Unknown language address space */,
+      T.getAddressSpace(),
+      address.getPointer()->getType()->getPointerElementType()->getPointerTo(
+          getContext().getTargetAddressSpace(T.getAddressSpace())),
+      true);
+  setAddrOfLocalVar(&D, Address(Addr, address.getAlignment()));
   emission.Addr = address;
 
   // Emit debug info for local var declaration.
   if (HaveInsertPoint())
     if (CGDebugInfo *DI = getDebugInfo()) {
       if (CGM.getCodeGenOpts().getDebugInfo() >=
           codegenoptions::LimitedDebugInfo) {
         DI->setLocation(D.getLocation());

lib/CodeGen/CGExprScalar.cpp

       // eliminate the useless instructions emitted during translating E.
       if (Result.HasSideEffects)
         Visit(E);
       return CGF.CGM.getNullPointer(cast<llvm::PointerType>(
           ConvertType(DestTy)), DestTy);
     }
     // Since target may map different address spaces in AST to the same address
     // space, an address space conversion may end up as a bitcast.
-    auto *Src = Visit(E);
-    return CGF.CGM.getTargetCodeGenInfo().performAddrSpaceCast(CGF, Src,
-                                                               E->getType(),
-                                                               DestTy);
+    return CGF.CGM.getTargetCodeGenInfo().performAddrSpaceCast(
+        CGF, Visit(E), E->getType()->getPointeeType().getAddressSpace(),
+        DestTy->getPointeeType().getAddressSpace(), ConvertType(DestTy));
   }
   case CK_AtomicToNonAtomic:
   case CK_NonAtomicToAtomic:
   case CK_NoOp:
   case CK_UserDefinedConversion:
     return Visit(const_cast<Expr*>(E));
 
   case CK_BaseToDerived: {

lib/CodeGen/CodeGenTypes.cpp

llvm::Type *CodeGenTypes::ConvertTypeForMem(QualType T) {
   if (!R->isIntegerTy(1))
     return R;
 
   // Otherwise, return an integer of the target-specified size.
   return llvm::IntegerType::get(getLLVMContext(),
                                 (unsigned)Context.getTypeSize(T));
 }
 
-
 /// isRecordLayoutComplete - Return true if the specified type is already
 /// completely laid out.
 bool CodeGenTypes::isRecordLayoutComplete(const Type *Ty) const {
   llvm::DenseMap<const Type*, llvm::StructType *>::const_iterator I = 
   RecordDeclTypes.find(Ty);
   return I != RecordDeclTypes.end() && !I->second->isOpaque();
 }
 

lib/CodeGen/TargetInfo.h

public:
   /// \param QT is the clang QualType of the null pointer.
   /// \return ConstantPointerNull with the given type \p T.
   /// Each target can override it to return its own desired constant value.
   virtual llvm::Constant *getNullPointer(const CodeGen::CodeGenModule &CGM,
       llvm::PointerType *T, QualType QT) const;
 
   /// Perform address space cast of an expression of pointer type.
   /// \param V is the LLVM value to be casted to another address space.
-  /// \param SrcTy is the QualType of \p V.
-  /// \param DestTy is the destination QualType.
+  /// \param SrcAddr is the language address space of \p V.
+  /// \param DestAddr is the targeted language address space.
+  /// \param DestTy is the destination LLVM pointer type.
+  /// \param IsNonNull is the flag indicating \p V is known to be non null.
   virtual llvm::Value *performAddrSpaceCast(CodeGen::CodeGenFunction &CGF,
-      llvm::Value *V, QualType SrcTy, QualType DestTy) const;
-
+                                            llvm::Value *V, unsigned SrcAddr,
+                                            unsigned DestAddr,
+                                            llvm::Type *DestTy,
+                                            bool IsNonNull = false) const;
 };
 
 } // namespace CodeGen
 } // namespace clang
 
 #endif // LLVM_CLANG_LIB_CODEGEN_TARGETINFO_H

lib/CodeGen/TargetInfo.cpp

 }
 
 llvm::Constant *TargetCodeGenInfo::getNullPointer(const CodeGen::CodeGenModule &CGM,
     llvm::PointerType *T, QualType QT) const {
   return llvm::ConstantPointerNull::get(T);
 }
 
 llvm::Value *TargetCodeGenInfo::performAddrSpaceCast(
-    CodeGen::CodeGenFunction &CGF, llvm::Value *Src, QualType SrcTy,
-    QualType DestTy) const {
+    CodeGen::CodeGenFunction &CGF, llvm::Value *Src, unsigned SrcAddr,
+    unsigned DestAddr, llvm::Type *DestTy, bool isNonNull) const {
   // Since target may map different address spaces in AST to the same address
   // space, an address space conversion may end up as a bitcast.
-  return CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(Src,
-             CGF.ConvertType(DestTy));
+  return CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(Src, DestTy);
 }
 
 static bool isEmptyRecord(ASTContext &Context, QualType T, bool AllowArrays);
 
 /// isEmptyField - Return true iff a the field is "empty", that is it
 /// is an unnamed bit-field or an (array of) empty record(s).
 static bool isEmptyField(ASTContext &Context, const FieldDecl *FD,
                          bool AllowArrays) {
if (Align >= 16 && (isSSEVectorType(getContext(), Ty) ||
     return 16;
 
   return MinABIStackAlignInBytes;
 }
 
 ABIArgInfo X86_32ABIInfo::getIndirectResult(QualType Ty, bool ByVal,
                                             CCState &State) const {
   if (!ByVal) {
     if (State.FreeRegs) {

lib/Sema/SemaDecl.cpp

     T = Context.getObjCObjectPointerType(T);
     NewVD->setType(T);
   }
 
   // Emit an error if an address space was applied to decl with local storage.
   // This includes arrays of objects with address space qualifiers, but not
   // automatic variables that point to other address spaces.
   // ISO/IEC TR 18037 S5.1.2
-  if (!getLangOpts().OpenCL
-      && NewVD->hasLocalStorage() && T.getAddressSpace() != 0) {
-    Diag(NewVD->getLocation(), diag::err_as_qualified_auto_decl);
+  if (NewVD->hasLocalStorage() && T.getAddressSpace() != 0) {
+    if (!getLangOpts().OpenCL) {
+      Diag(NewVD->getLocation(), diag::err_as_qualified_auto_decl) << 0;
       NewVD->setInvalidDecl();
       return;
+    }
+    assert(T.getAddressSpace() != LangAS::opencl_constant);
+    if (T.getAddressSpace() == LangAS::opencl_global) {
+      Diag(NewVD->getLocation(), diag::err_opencl_function_variable)
+          << 1 /*is any function*/ << "global";
+      NewVD->setInvalidDecl();
+      return;
+    }
+    if (T.getAddressSpace() != LangAS::opencl_local) {
+      Diag(NewVD->getLocation(), diag::err_as_qualified_auto_decl) << 1;
+      NewVD->setInvalidDecl();
+      return;
+    }
   }
 
   // OpenCL v1.2 s6.8 - The static qualifier is valid only in program
   // scope.
   if (getLangOpts().OpenCLVersion == 120 &&
       !getOpenCLOptions().isEnabled("cl_clang_storage_class_specifiers") &&
       NewVD->isStaticLocal()) {
     Diag(NewVD->getLocation(), diag::err_static_function_scope);

               << Scope << "global or constant";
         else
           Diag(NewVD->getLocation(), diag::err_opencl_global_invalid_addr_space)
               << Scope << "constant";
         NewVD->setInvalidDecl();
         return;
       }
     } else {
-      if (T.getAddressSpace() == LangAS::opencl_global) {
-        Diag(NewVD->getLocation(), diag::err_opencl_function_variable)
-            << 1 /*is any function*/ << "global";
-        NewVD->setInvalidDecl();
-        return;
-      }
       // OpenCL v1.1 s6.5.2 and s6.5.3 no local or constant variables
       // in functions.
       if (T.getAddressSpace() == LangAS::opencl_constant ||
           T.getAddressSpace() == LangAS::opencl_local) {
         FunctionDecl *FD = getCurFunctionDecl();
         if (FD && !FD->hasAttr<OpenCLKernelAttr>()) {
           if (T.getAddressSpace() == LangAS::opencl_constant)
             Diag(NewVD->getLocation(), diag::err_opencl_function_variable)

test/CodeGen/address-space.c

-// RUN: %clang_cc1 -triple x86_64-apple-darwin -emit-llvm < %s | FileCheck -check-prefixes=CHECK,GIZ %s
+// RUN: %clang_cc1 -triple x86_64-apple-darwin -emit-llvm < %s | FileCheck -check-prefixes=CHECK,GIZ,X86 %s
 // RUN: %clang_cc1 -triple amdgcn -emit-llvm < %s | FileCheck -check-prefixes=CHECK,PIZ %s
-// RUN: %clang_cc1 -triple amdgcn---amdgiz -emit-llvm < %s | FileCheck -check-prefixes=CHeCK,GIZ %s
+// RUN: %clang_cc1 -triple amdgcn---amdgiz -emit-llvm < %s | FileCheck -check-prefixes=CHECK,GIZ,AMD %s
 
 // CHECK: @foo = common addrspace(1) global
 int foo __attribute__((address_space(1)));
 
 // CHECK: @ban = common addrspace(1) global
 int ban[10] __attribute__((address_space(1)));
 
 // CHECK: @a = common global

 }
 
 // PR7437
 typedef struct {
   float aData[1];
 } MyStruct;
 
 // CHECK-LABEL: define void @test4(
-// CHECK: call void @llvm.memcpy.p0i8.p2i8
-// CHECK: call void @llvm.memcpy.p2i8.p0i8
+// X86: call void @llvm.memcpy.p0i8.p2i8
+// X86: call void @llvm.memcpy.p2i8.p0i8
+// AMD: call void @llvm.memcpy.p5i8.p2i8
+// AMD: call void @llvm.memcpy.p2i8.p0i8
+// PIZ: call void @llvm.memcpy.p0i8.p2i8
+// PIZ: call void @llvm.memcpy.p2i8.p4i8
 void test4(MyStruct __attribute__((address_space(2))) *pPtr) {
   MyStruct s = pPtr[0];
   pPtr[0] = s;
 }

test/CodeGenCXX/amdgcn-automatic-variable.cpp

+// RUN: %clang_cc1 -O0 -triple amdgcn---amdgiz -emit-llvm %s -o - | FileCheck %s
+
+// CHECK-LABEL: define void @_Z5func1Pi(i32* %x)
+void func1(int *x) {
+  // CHECK: %[[x_addr:.*]] = alloca i32*{{.*}}addrspace(5)
+  // CHECK: store i32* %x, i32* addrspace(5)* %[[x_addr]]
+  // CHECK: %[[r0:.*]] = load i32*, i32* addrspace(5)* %[[x_addr]]
+  // CHECK: store i32 1, i32* %[[r0]]
+  *x = 1;
+}
+
+// CHECK-LABEL: define void @_Z5func2v()
+void func2(void) {
+  // CHECK: %lv1 = alloca i32, align 4, addrspace(5)
+  // CHECK: %lv2 = alloca i32, align 4, addrspace(5)
+  // CHECK: %la = alloca [100 x i32], align 4, addrspace(5)
+  // CHECK: %lp1 = alloca i32*, align 4, addrspace(5)
+  // CHECK: %lp2 = alloca i32*, align 4, addrspace(5)
+  // CHECK: %lvc = alloca i32, align 4, addrspace(5)
+
+  // CHECK: %[[r0:.*]] = addrspacecast i32 addrspace(5)* %lv1 to i32*
+  // CHECK: store i32 1, i32* %[[r0]]
+  int lv1;
+  lv1 = 1;
+  // CHECK: store i32 2, i32 addrspace(5)* %lv2
+  int lv2 = 2;
+
+  // CHECK: %[[r2:.*]] = addrspacecast [100 x i32] addrspace(5)* %la to [100 x i32]*
+  // CHECK: %[[arrayidx:.*]] = getelementptr inbounds [100 x i32], [100 x i32]* %[[r2]], i64 0, i64 0
+  // CHECK: store i32 3, i32* %[[arrayidx]], align 4
+  int la[100];
+  la[0] = 3;
+
+  // CHECK: store i32* %[[r0]], i32* addrspace(5)* %lp1, align 4
+  int *lp1 = &lv1;
+
+  // CHECK: %[[arraydecay:.*]] = getelementptr inbounds [100 x i32], [100 x i32]* %[[r2]], i32 0, i32 0
+  // CHECK: store i32* %[[arraydecay]], i32* addrspace(5)* %lp2, align 4
+  int *lp2 = la;
+
+  // CHECK: call void @_Z5func1Pi(i32* %[[r0]])
+  func1(&lv1);
+
+  // CHECK: store i32 4, i32 addrspace(5)* %lvc
+  // CHECK: store i32 4, i32* %[[r0]]
+  const int lvc = 4;
+  lv1 = lvc;
+}

test/CodeGenOpenCL/amdgcn-automatic-variable.cl

+// RUN: %clang_cc1 -O0 -cl-std=CL1.2 -triple amdgcn---amdgiz -emit-llvm %s -o - | FileCheck -check-prefixes=CHECK,CL12 %s
+// RUN: %clang_cc1 -O0 -cl-std=CL2.0 -triple amdgcn---amdgiz -emit-llvm %s -o - | FileCheck -check-prefixes=CHECK,CL20 %s
+
+// CL12-LABEL: define void @func1(i32 addrspace(5)* %x)
+// CL20-LABEL: define void @func1(i32* %x)
+void func1(int *x) {
+  // CL12: %[[x_addr:.*]] = alloca i32 addrspace(5)*{{.*}}addrspace(5)
+  // CL12: store i32 addrspace(5)* %x, i32 addrspace(5)* addrspace(5)* %[[x_addr]]
+  // CL12: %[[r0:.*]] = load i32 addrspace(5)*, i32 addrspace(5)* addrspace(5)* %[[x_addr]]
+  // CL12: store i32 1, i32 addrspace(5)* %[[r0]]
+  // CL20: %[[x_addr:.*]] = alloca i32*{{.*}}addrspace(5)
+  // CL20: store i32* %x, i32* addrspace(5)* %[[x_addr]]
+  // CL20: %[[r0:.*]] = load i32*, i32* addrspace(5)* %[[x_addr]]
+  // CL20: store i32 1, i32* %[[r0]]
+  *x = 1;
+}
+
+// CHECK-LABEL: define void @func2()
+void func2(void) {
+  // CHECK: %lv1 = alloca i32, align 4, addrspace(5)
+  // CHECK: %lv2 = alloca i32, align 4, addrspace(5)
+  // CHECK: %la = alloca [100 x i32], align 4, addrspace(5)
+  // CL12: %lp1 = alloca i32 addrspace(5)*, align 4, addrspace(5)
+  // CL12: %lp2 = alloca i32 addrspace(5)*, align 4, addrspace(5)
+  // CL20: %lp1 = alloca i32*, align 4, addrspace(5)
+  // CL20: %lp2 = alloca i32*, align 4, addrspace(5)
+  // CHECK: %lvc = alloca i32, align 4, addrspace(5)
+
+  // CHECK: store i32 1, i32 addrspace(5)* %lv1
+  int lv1;
+  lv1 = 1;
+  // CHECK: store i32 2, i32 addrspace(5)* %lv2
+  int lv2 = 2;
+
+  // CHECK: %[[arrayidx:.*]] = getelementptr inbounds [100 x i32], [100 x i32] addrspace(5)* %la, i64 0, i64 0
+  // CHECK: store i32 3, i32 addrspace(5)* %[[arrayidx]], align 4
+  int la[100];
+  la[0] = 3;
+
+  // CL12: store i32 addrspace(5)* %lv1, i32 addrspace(5)* addrspace(5)* %lp1, align 4
+  // CL20: %[[r0:.*]] = addrspacecast i32 addrspace(5)* %lv1 to i32*
+  // CL20: store i32* %[[r0]], i32* addrspace(5)* %lp1, align 4
+  int *lp1 = &lv1;
+
+  // CHECK: %[[arraydecay:.*]] = getelementptr inbounds [100 x i32], [100 x i32] addrspace(5)* %la, i32 0, i32 0
+  // CL12: store i32 addrspace(5)* %[[arraydecay]], i32 addrspace(5)* addrspace(5)* %lp2, align 4
+  // CL20: %[[r1:.*]] = addrspacecast i32 addrspace(5)* %[[arraydecay]] to i32*
+  // CL20: store i32* %[[r1]], i32* addrspace(5)* %lp2, align 4
+  int *lp2 = la;
+
+  // CL12: call void @func1(i32 addrspace(5)* %lv1)
+  // CL20: %[[r2:.*]] = addrspacecast i32 addrspace(5)* %lv1 to i32*
+  // CL20: call void @func1(i32* %[[r2]])
+  func1(&lv1);
+
+  // CHECK: store i32 4, i32 addrspace(5)* %lvc
+  // CHECK: store i32 4, i32 addrspace(5)* %lv1
+  const int lvc = 4;
+  lv1 = lvc;
+}

test/SemaOpenCL/storageclass-cl20.cl

 // RUN: %clang_cc1 %s -verify -pedantic -fsyntax-only -cl-std=CL2.0
 
 static constant int G1 = 0;
 int G2 = 0;
 global int G3 = 0;
 local int G4 = 0;              // expected-error{{program scope variable must reside in global or constant address space}}
 
 void kernel foo() {
   static int S1 = 5;
   static global int S2 = 5;
   static private int S3 = 5;   // expected-error{{static local variable must reside in global or constant address space}}
 
   constant int L1 = 0;
   local int L2;
-  global int L3; // expected-error{{function scope variable cannot be declared in global address space}}
+  global int L3;                              // expected-error{{function scope variable cannot be declared in global address space}}
+  __attribute__((address_space(100))) int L4; // expected-error{{automatic variable qualified with an invalid address space}}
 
   extern global int G5;
   extern int G6; // expected-error{{extern variable must reside in global or constant address space}}
 }

test/SemaOpenCL/storageclass.cl

 // RUN: %clang_cc1 %s -verify -pedantic -fsyntax-only -cl-std=CL1.2
 
 static constant int G1 = 0;
 constant int G2 = 0;
 int G3 = 0;        // expected-error{{program scope variable must reside in constant address space}}
 global int G4 = 0; // expected-error{{program scope variable must reside in constant address space}}
 
 void kernel foo(int x) {
   // static is not allowed at local scope before CL2.0
   static int S1 = 5;          // expected-error{{variables in function scope cannot be declared static}}
   static constant int S2 = 5; // expected-error{{variables in function scope cannot be declared static}}
 
   constant int L1 = 0;
   local int L2;
 
-  auto int L3 = 7; // expected-error{{OpenCL version 1.2 does not support the 'auto' storage class specifier}}
-  global int L4;   // expected-error{{function scope variable cannot be declared in global address space}}
+  auto int L3 = 7;                            // expected-error{{OpenCL version 1.2 does not support the 'auto' storage class specifier}}
+  global int L4;                              // expected-error{{function scope variable cannot be declared in global address space}}
+  __attribute__((address_space(100))) int L5; // expected-error{{automatic variable qualified with an invalid address space}}
 
-  constant int L5 = x; // expected-error {{initializer element is not a compile-time constant}}
-  global int *constant L6 = &G4;
-  private int *constant L7 = &x; // expected-error {{initializer element is not a compile-time constant}}
-  constant int *constant L8 = &L1;
-  local int *constant L9 = &L2; // expected-error {{initializer element is not a compile-time constant}}
+  constant int L6 = x;                        // expected-error {{initializer element is not a compile-time constant}}
+  global int *constant L7 = &G4;
+  private int *constant L8 = &x;              // expected-error {{initializer element is not a compile-time constant}}
+  constant int *constant L9 = &L1;
+  local int *constant L10 = &L2;              // expected-error {{initializer element is not a compile-time constant}}
 }
 
 static void kernel bar() { // expected-error{{kernel functions cannot be declared static}}
 }
 
 void f() {
-  constant int L1 = 0; // expected-error{{non-kernel function variable cannot be declared in constant address space}}
-  local int L2;        // expected-error{{non-kernel function variable cannot be declared in local address space}}
+  constant int L1 = 0;                        // expected-error{{non-kernel function variable cannot be declared in constant address space}}
+  local int L2;                               // expected-error{{non-kernel function variable cannot be declared in local address space}}
+  global int L3;                              // expected-error{{function scope variable cannot be declared in global address space}}
+  __attribute__((address_space(100))) int L4; // expected-error{{automatic variable qualified with an invalid address space}}
+
   {
-    constant int L1 = 0; // expected-error{{non-kernel function variable cannot be declared in constant address space}}
-    local int L2;        // expected-error{{non-kernel function variable cannot be declared in local address space}}
-  }
-  global int L3; // expected-error{{function scope variable cannot be declared in global address space}}
-  extern constant float L4;
-  extern local float L5; // expected-error{{extern variable must reside in constant address space}}
-  static int L6 = 0;     // expected-error{{variables in function scope cannot be declared static}}
-  static int L7;         // expected-error{{variables in function scope cannot be declared static}}
+    constant int L1 = 0;                        // expected-error{{non-kernel function variable cannot be declared in constant address space}}
+    local int L2;                               // expected-error{{non-kernel function variable cannot be declared in local address space}}
+    global int L3;                              // expected-error{{function scope variable cannot be declared in global address space}}
+    __attribute__((address_space(100))) int L4; // expected-error{{automatic variable qualified with an invalid address space}}
+  }
+
+
+  extern constant float L5;
+  extern local float L6; // expected-error{{extern variable must reside in constant address space}}
+
+  static int L7 = 0;     // expected-error{{variables in function scope cannot be declared static}}
+  static int L8;         // expected-error{{variables in function scope cannot be declared static}}
 }