[clang][CodeGen] Refactor CreateTempAlloca function nest. NFC.

[wingo]

Summary

It used to be that there were three layers to create temp alloca
instructions in CodeGenFunction. The lowest level was named
CreateTempAlloca and returned an LLVM instruction (1):

llvm::AllocaInst* CreateTempAlloca(llvm::Type *Ty);

(Leaving off the name argument and array size from the prototype, for
brevity.)

The next level applied frontend-specified alignment to the alloca and
returned an address, but left the value in the alloca address space (2):

Address
CreateTempAllocaWithoutCast(llvm::Type *Ty, CharUnits Align);

Finally the normal function returns an Address, but also makes sure that
the result is in LangAS::Default (3):

Address
CreateTempAlloca(QualType Ty, CharUnits Align);

This is a bit confusing since functions (1) and (3) share a name but
have different behavior, and function (2) has a funny name.
Furthermore, the implementation of function (2) actually calls
function (1), making it seem to the reader like there is a loop in the
call graph.

This patch refactors to remove function (1) and replace code that uses
it with calls to function (2), returning an Address instead of an IR
instruction. This also removes some places in which the frontend wasn't
specifying the alignment of its allocas.

This patch also changes function (2) to explicitly take an address space
argument, which should generally be the alloca address space. There is
usually one target-specified alloca address space, but in the future,
the WebAssembly target may alloca variables in multiple address spaces.
The function name is changed from CreateTempAllocaWithoutCast to
CreateTempAllocaInAS, indicating that the result is left in the given
AS.

Finally, we also replace uses of the somewhat-deprecated
CreateDefaultAlignTempAlloca with CreateTempAllocaInAS, passing the
result of calling the new CodeGenFunction::PreferredAlignmentForIRType
method as the alignnment.

As a result of this patch, a number of llvm::Value* variables are
changed to be Address instead. This allows a more simplified codegen,
as the IR builder doesn't need to take an additional alignment argument.

[wingo]

Sooooo... besides the refactor, this is getting closer to where I'm going in https://lists.llvm.org/pipermail/cfe-dev/2021-July/068559.html, though still NFC. I think you can see where I would replace getASTAllocaAddressSpace with getAllocaAddressSpace(QualType Ty), and possibly (depending on the source language) avoid casting the resulting alloca to LangAS::Default. WDYT, is this sort of thing OK?

[wingo]

clang/lib/CodeGen/CGBuilder.h

115

it's the change to always return an Address from CreateTempAlloca that makes these methods unnecessary.

clang/lib/CodeGen/CGGPUBuiltin.cpp

116–122

this is an open question -- there could be a bug here in the existing code.

[wingo]

In D108464#2957276, @wingo wrote:

Sooooo... besides the refactor, this is getting closer to where I'm going in https://lists.llvm.org/pipermail/cfe-dev/2021-July/068559.html, though still NFC. I think you can see where I would replace getASTAllocaAddressSpace with getAllocaAddressSpace(QualType Ty), and possibly (depending on the source language) avoid casting the resulting alloca to LangAS::Default. WDYT, is this sort of thing OK?

Taking this patch as perhaps a better generic discussion point, @rjmccall graciously gave some general feedback on this approach (thank you!!!):

In D108360#2957844, @rjmccall wrote:

I'm not sure that I agree with your overall plan, though:

• The WebAssembly operand stack is not a good match for an address space at the language level because it's not addressable at all. If you can't meaningfully have a pointer into the address space, then you don't really need this in the type system; it's more like a declaration modifier at best.
• Allocating local variables on the operand stack ought to be a very straightforward analysis in the backend. There's not much optimization value in trying to do it in the frontend, and it's going to be problematic for things like coroutine lowering.
• The security argument seems pretty weak, not because security isn't important but because this is not really an adequate basis for getting the tamper-proof guarantee you want. For example, LLVM passes can and do introduce its own allocas and store scalars into them sometimes. Really you need some sort of "tamper-proof" *type* which the compiler can make an end-to-end guarantee of non-tamper-ability for the values of, and while optimally this would be implemented by just keeping values on the operand stack, in the general case you will need to have some sort of strategy for keeping things in memory.

Thanks for thinking about this! Indeed I started out with the goal of not going deep into clang and if it's possible to avoid going too deeply, that would be better for everyone involved. I am starting to think however that it may be unavoidable for me at least.

So, I am focusing on WebAssembly global and local variables; the WebAssembly operand stack is an artifact of the IR-to-MC lowering and AFAICS doesn't have any bearing on what clang does -- though perhaps I am misunderstanding what you are getting at here. The issue is not to allocate locals on the operand stack, but rather to allocate them as part of the "locals" of a WebAssembly function. Cc @tlively on the WebAssembly side.

I agree that the security argument is weak: it's something but it's not the real motivation.

The main motivator is the ability to have "reference type" (externref/funcref) locals and globals at all. Reference-typed values can't be stored to linear memory. They have no size and no byte representation -- they are opaque values from the host. However, WebAssembly locals and globals can define storage locations of type externref or funcref. The storage locations for WebAssembly locals and globals are not in linear memory, and are not addressable by pointer at run-time -- accesses to them are always by immediate.

Currently, clang always produces LLVM IR that allocates C++ globals and locals in linear memory. LLVM may transform some of these to WebAssembly globals or locals at its discretion. This strategy works because all values for the initial set of types supported by WebAssembly can be stored to linear memory; what you can do with a WebAssembly global or local was a subset of what you could do with linear-memory globals and alloca locals.

However, with reference types (merged into the spec earlier this year), this is no longer the case -- there are now types representable in WebAssembly globals/locals which can't be represented in linear memory.

Because of the limitations in how WebAssembly globals and locals can be used, reference-typed values have associated semantic restrictions in the front-end. If I declare a C++ local of type externref (which must be allocated to a WebAssembly local), I can't take its address:

void f() {
  externref_t x = g();
  h(&x); // error
}

Similarly I can't put an externref in an aggregate type that itself is allocated in linear memory:

// global
struct { int x; externref_t y; } z; // error

But, if we add a generic OpenCL-like address space attribute, that would allow the user to declare some variables to be in alternate address spaces. Then we can apply the ACLE SVE semantic restrictions to these values also, and add on an additional restriction preventing address-of. That way users get to make off-heap definitions, and if they misuse them, they get comprehensible errors. LLVM IR and WebAssembly lowering is ready for these alternate-address-space allocations.

// global
struct { int x; externref_t y; } z __attribute__((wasm_var)); // ok

The builtin externref_t and funcref_t types would probably already have this attribute. (I don't have a complete clang patchset yet, so if you prefer to wait to see what things look like, this is perfectly ok.) But because in the future there will be more kinds of reference types, and that we might want to have have aggregate types which include both number and reference types, it seems that there are two separable concerns here: one about applying the semantic restrictions for WebAssembly global and local storage locations, and another concern about handling "opaque" values (externref) which doesn't impose additional Sema/ restrictions.

The restrictions needed for WebAssembly globals and locals are essentially the same, and they lower to the same LLVM IR address space for the WebAssembly target, hence I would propose a single wasm_var attribute instead of wasm_global and wasm_local. This can change though if it's confusing.

Finally, I would note that it would be useful from an ABI point of view to be able to define named WebAssembly globals (but not locals) in C, if e.g. an external interface expects that a module export an i32 global with name foo. So this patch-set has that use-case also.

Regarding coroutine lowering, I can see how that can be challenging; would it be reasonable to restrict continuations to not include saved off-heap locals, for now? If there were such a local, it would be a compilation error.

OK, lots of words. Thanks for reading. What do you think about this (ab)use of LangAS? If there is a better way to cross reference types with C++, pointers are very much welcome. I will have a better idea what the end size of the patch-set is within a couple weeks; I guess I would propose to continue posting the series and hope that the end set of core changes is stomache-able, and add you to Cc as I go.

[wingo]

Rebase to no longer require Address default constructor.

[rjmccall]

+ JF, who knows something about Web Assembly, or can at least drag in the right people

In D108464#2959591, @wingo wrote:

In D108464#2957276, @wingo wrote:

Sooooo... besides the refactor, this is getting closer to where I'm going in https://lists.llvm.org/pipermail/cfe-dev/2021-July/068559.html, though still NFC. I think you can see where I would replace getASTAllocaAddressSpace with getAllocaAddressSpace(QualType Ty), and possibly (depending on the source language) avoid casting the resulting alloca to LangAS::Default. WDYT, is this sort of thing OK?

Taking this patch as perhaps a better generic discussion point, @rjmccall graciously gave some general feedback on this approach (thank you!!!):

In D108360#2957844, @rjmccall wrote:

I'm not sure that I agree with your overall plan, though:

• The WebAssembly operand stack is not a good match for an address space at the language level because it's not addressable at all. If you can't meaningfully have a pointer into the address space, then you don't really need this in the type system; it's more like a declaration modifier at best.
• Allocating local variables on the operand stack ought to be a very straightforward analysis in the backend. There's not much optimization value in trying to do it in the frontend, and it's going to be problematic for things like coroutine lowering.
• The security argument seems pretty weak, not because security isn't important but because this is not really an adequate basis for getting the tamper-proof guarantee you want. For example, LLVM passes can and do introduce its own allocas and store scalars into them sometimes. Really you need some sort of "tamper-proof" *type* which the compiler can make an end-to-end guarantee of non-tamper-ability for the values of, and while optimally this would be implemented by just keeping values on the operand stack, in the general case you will need to have some sort of strategy for keeping things in memory.

Thanks for thinking about this! Indeed I started out with the goal of not going deep into clang and if it's possible to avoid going too deeply, that would be better for everyone involved. I am starting to think however that it may be unavoidable for me at least.

So, I am focusing on WebAssembly global and local variables; the WebAssembly operand stack is an artifact of the IR-to-MC lowering and AFAICS doesn't have any bearing on what clang does -- though perhaps I am misunderstanding what you are getting at here. The issue is not to allocate locals on the operand stack, but rather to allocate them as part of the "locals" of a WebAssembly function. Cc @tlively on the WebAssembly side.

By "operand stack" I mean the innate, unaddressable stack that the WebAssembly VM maintains in order to make functions reentrant. I don't know what term the VM spec uses for it, but I believe "operand stack" is widely accepted terminology for the unaddressable stack when you've got this kind of dual-stack setup. And yes, VM "locals" would go there.

The main motivator is the ability to have "reference type" (externref/funcref) locals and globals at all. Reference-typed values can't be stored to linear memory. They have no size and no byte representation -- they are opaque values from the host. However, WebAssembly locals and globals can define storage locations of type externref or funcref.

I see. I think you need to think carefully about the best way to represent values of these types in LLVM IR, because it probably cannot just be "treat them as a normal value, emit code a certain way that we know how to lower, and hope nothing goes wrong". It seems to me that you probably need a new IR type for it, since normal types aren't restricted from memory and tokens can't be used as parameters or return values.

Hopefully, someone had a plan for this when they introduced that WebAssembly extension.

But, if we add a generic OpenCL-like address space attribute, that would allow the user to declare some variables to be in alternate address spaces. Then we can apply the ACLE SVE semantic restrictions to these values also, and add on an additional restriction preventing address-of. That way users get to make off-heap definitions, and if they misuse them, they get comprehensible errors. LLVM IR and WebAssembly lowering is ready for these alternate-address-space allocations.

Again, I'm not sure you're getting anything at all from the address space side of this. The restrictions on these variables prevent any of the general address-space logic from applying. In a language sense, it's more like a storage class than an address space.

Regarding coroutine lowering, I can see how that can be challenging; would it be reasonable to restrict continuations to not include saved off-heap locals, for now? If there were such a local, it would be a compilation error.

I suppose you would have to.

[tlively]

In D108464#2960623, @rjmccall wrote:

+ JF, who knows something about Web Assembly, or can at least drag in the right people

In D108464#2959591, @wingo wrote:

In D108464#2957276, @wingo wrote:

Sooooo... besides the refactor, this is getting closer to where I'm going in https://lists.llvm.org/pipermail/cfe-dev/2021-July/068559.html, though still NFC. I think you can see where I would replace getASTAllocaAddressSpace with getAllocaAddressSpace(QualType Ty), and possibly (depending on the source language) avoid casting the resulting alloca to LangAS::Default. WDYT, is this sort of thing OK?

Taking this patch as perhaps a better generic discussion point, @rjmccall graciously gave some general feedback on this approach (thank you!!!):

In D108360#2957844, @rjmccall wrote:

I'm not sure that I agree with your overall plan, though:

• The WebAssembly operand stack is not a good match for an address space at the language level because it's not addressable at all. If you can't meaningfully have a pointer into the address space, then you don't really need this in the type system; it's more like a declaration modifier at best.
• Allocating local variables on the operand stack ought to be a very straightforward analysis in the backend. There's not much optimization value in trying to do it in the frontend, and it's going to be problematic for things like coroutine lowering.
• The security argument seems pretty weak, not because security isn't important but because this is not really an adequate basis for getting the tamper-proof guarantee you want. For example, LLVM passes can and do introduce its own allocas and store scalars into them sometimes. Really you need some sort of "tamper-proof" *type* which the compiler can make an end-to-end guarantee of non-tamper-ability for the values of, and while optimally this would be implemented by just keeping values on the operand stack, in the general case you will need to have some sort of strategy for keeping things in memory.

Thanks for thinking about this! Indeed I started out with the goal of not going deep into clang and if it's possible to avoid going too deeply, that would be better for everyone involved. I am starting to think however that it may be unavoidable for me at least.

So, I am focusing on WebAssembly global and local variables; the WebAssembly operand stack is an artifact of the IR-to-MC lowering and AFAICS doesn't have any bearing on what clang does -- though perhaps I am misunderstanding what you are getting at here. The issue is not to allocate locals on the operand stack, but rather to allocate them as part of the "locals" of a WebAssembly function. Cc @tlively on the WebAssembly side.

By "operand stack" I mean the innate, unaddressable stack that the WebAssembly VM maintains in order to make functions reentrant. I don't know what term the VM spec uses for it, but I believe "operand stack" is widely accepted terminology for the unaddressable stack when you've got this kind of dual-stack setup. And yes, VM "locals" would go there.

@wingo, are there cases where it is useful to declare variables as living in WebAssembly locals and not in the VM stack? I'm having trouble coming up with a case where leaving that up to the backend is not enough. We clearly need a way to prevent values from being written to main memory (AS 0), but it's not clear to me that we need a way to specifically allocate locals for them.

The main motivator is the ability to have "reference type" (externref/funcref) locals and globals at all. Reference-typed values can't be stored to linear memory. They have no size and no byte representation -- they are opaque values from the host. However, WebAssembly locals and globals can define storage locations of type externref or funcref.

I see. I think you need to think carefully about the best way to represent values of these types in LLVM IR, because it probably cannot just be "treat them as a normal value, emit code a certain way that we know how to lower, and hope nothing goes wrong". It seems to me that you probably need a new IR type for it, since normal types aren't restricted from memory and tokens can't be used as parameters or return values.

Hopefully, someone had a plan for this when they introduced that WebAssembly extension.

Yes, we had a plan :) In WebAssembly, reference types are essentially opaque pointers that cannot be dereferenced or stored into main memory. They can, however, be stored in WebAssembly globals and tables, which are modeled as LLVM global pointers and global arrays in other address spaces. At the IR level, reference types are modeled as pointers into a non-integral AS that themselves live in a non-integral AS. If the optimizer ever spills a local reference-typed value to memory, we are able to discover and correct that in the backend. I believe we are currently assuming that the optimizer will never introduce a store of a reference-typed value into a global main memory location, though.

But, if we add a generic OpenCL-like address space attribute, that would allow the user to declare some variables to be in alternate address spaces. Then we can apply the ACLE SVE semantic restrictions to these values also, and add on an additional restriction preventing address-of. That way users get to make off-heap definitions, and if they misuse them, they get comprehensible errors. LLVM IR and WebAssembly lowering is ready for these alternate-address-space allocations.

Again, I'm not sure you're getting anything at all from the address space side of this. The restrictions on these variables prevent any of the general address-space logic from applying. In a language sense, it's more like a storage class than an address space.

Using address spaces lets us model loads and stores of reference-typed values from and to globals and tables. I don't think it makes sense to present these concepts as "address spaces" to C/C++ users, but that's what we're using at the IR level.

Regarding coroutine lowering, I can see how that can be challenging; would it be reasonable to restrict continuations to not include saved off-heap locals, for now? If there were such a local, it would be a compilation error.

I suppose you would have to.

[rjmccall]

In D108464#2960791, @tlively wrote:

In D108464#2960623, @rjmccall wrote:

+ JF, who knows something about Web Assembly, or can at least drag in the right people

In D108464#2959591, @wingo wrote:

In D108464#2957276, @wingo wrote:

Sooooo... besides the refactor, this is getting closer to where I'm going in https://lists.llvm.org/pipermail/cfe-dev/2021-July/068559.html, though still NFC. I think you can see where I would replace getASTAllocaAddressSpace with getAllocaAddressSpace(QualType Ty), and possibly (depending on the source language) avoid casting the resulting alloca to LangAS::Default. WDYT, is this sort of thing OK?

Taking this patch as perhaps a better generic discussion point, @rjmccall graciously gave some general feedback on this approach (thank you!!!):

In D108360#2957844, @rjmccall wrote:

I'm not sure that I agree with your overall plan, though:

• The WebAssembly operand stack is not a good match for an address space at the language level because it's not addressable at all. If you can't meaningfully have a pointer into the address space, then you don't really need this in the type system; it's more like a declaration modifier at best.
• Allocating local variables on the operand stack ought to be a very straightforward analysis in the backend. There's not much optimization value in trying to do it in the frontend, and it's going to be problematic for things like coroutine lowering.
• The security argument seems pretty weak, not because security isn't important but because this is not really an adequate basis for getting the tamper-proof guarantee you want. For example, LLVM passes can and do introduce its own allocas and store scalars into them sometimes. Really you need some sort of "tamper-proof" *type* which the compiler can make an end-to-end guarantee of non-tamper-ability for the values of, and while optimally this would be implemented by just keeping values on the operand stack, in the general case you will need to have some sort of strategy for keeping things in memory.

Thanks for thinking about this! Indeed I started out with the goal of not going deep into clang and if it's possible to avoid going too deeply, that would be better for everyone involved. I am starting to think however that it may be unavoidable for me at least.

So, I am focusing on WebAssembly global and local variables; the WebAssembly operand stack is an artifact of the IR-to-MC lowering and AFAICS doesn't have any bearing on what clang does -- though perhaps I am misunderstanding what you are getting at here. The issue is not to allocate locals on the operand stack, but rather to allocate them as part of the "locals" of a WebAssembly function. Cc @tlively on the WebAssembly side.

By "operand stack" I mean the innate, unaddressable stack that the WebAssembly VM maintains in order to make functions reentrant. I don't know what term the VM spec uses for it, but I believe "operand stack" is widely accepted terminology for the unaddressable stack when you've got this kind of dual-stack setup. And yes, VM "locals" would go there.

@wingo, are there cases where it is useful to declare variables as living in WebAssembly locals and not in the VM stack? I'm having trouble coming up with a case where leaving that up to the backend is not enough. We clearly need a way to prevent values from being written to main memory (AS 0), but it's not clear to me that we need a way to specifically allocate locals for them.

Right, I think this is one of the key questions here. Right now this seems to be entirely type-directed: it's a special property of a couple of builtin types that you can't take the address of their objects and those objects can only live in specific places. Having it be type-directed, but only to the point of saying that certain types *must* use certain address spaces, and then imposing all the other restrictions on those types as novel restrictions on those address spaces, feels like it's adding complexity to the language concept of address spaces without much benefit.

The main motivator is the ability to have "reference type" (externref/funcref) locals and globals at all. Reference-typed values can't be stored to linear memory. They have no size and no byte representation -- they are opaque values from the host. However, WebAssembly locals and globals can define storage locations of type externref or funcref.

I see. I think you need to think carefully about the best way to represent values of these types in LLVM IR, because it probably cannot just be "treat them as a normal value, emit code a certain way that we know how to lower, and hope nothing goes wrong". It seems to me that you probably need a new IR type for it, since normal types aren't restricted from memory and tokens can't be used as parameters or return values.

Hopefully, someone had a plan for this when they introduced that WebAssembly extension.

Yes, we had a plan :) In WebAssembly, reference types are essentially opaque pointers that cannot be dereferenced or stored into main memory. They can, however, be stored in WebAssembly globals and tables, which are modeled as LLVM global pointers and global arrays in other address spaces. At the IR level, reference types are modeled as pointers into a non-integral AS that themselves live in a non-integral AS. If the optimizer ever spills a local reference-typed value to memory, we are able to discover and correct that in the backend. I believe we are currently assuming that the optimizer will never introduce a store of a reference-typed value into a global main memory location, though.

Hmm. This seems like it's abusing the inner address space to create a primitive opaque user-defined type in LLVM IR, but I don't have a compelling argument why you shouldn't do it this way. I guess my only real objection to the overall approach is vague hand-wringing about this idea of having strong representational invariants and then just working around passes that break them in the backend.

But, if we add a generic OpenCL-like address space attribute, that would allow the user to declare some variables to be in alternate address spaces. Then we can apply the ACLE SVE semantic restrictions to these values also, and add on an additional restriction preventing address-of. That way users get to make off-heap definitions, and if they misuse them, they get comprehensible errors. LLVM IR and WebAssembly lowering is ready for these alternate-address-space allocations.

Again, I'm not sure you're getting anything at all from the address space side of this. The restrictions on these variables prevent any of the general address-space logic from applying. In a language sense, it's more like a storage class than an address space.

Using address spaces lets us model loads and stores of reference-typed values from and to globals and tables. I don't think it makes sense to present these concepts as "address spaces" to C/C++ users, but that's what we're using at the IR level.

Yeah, at some point I'm willing to accept that this is your best option at the IR level, but I want to not jam this into the user-facing language unless it's really the right design approach.

John.

[wingo]

Thanks again John & Thomas for your thoughts.

In D108464#2961595, @rjmccall wrote:

In D108464#2960791, @tlively wrote:

I don't think it makes sense to present these concepts as "address spaces" to C/C++ users, but that's what we're using at the IR level.

Yeah, at some point I'm willing to accept that this is your best option at the IR level, but I want to not jam this into the user-facing language unless it's really the right design approach.

I am absolutely not married to the particular approach in this patch series and am happy to explore the design space :)

In D108464#2960791, @tlively wrote:

@wingo, are there cases where it is useful to declare variables as living in WebAssembly locals and not in the VM stack? I'm having trouble coming up with a case where leaving that up to the backend is not enough. We clearly need a way to prevent values from being written to main memory (AS 0), but it's not clear to me that we need a way to specifically allocate locals for them.

No, there are no cases that I know of. From the IR & backend POV, the point as you say is to prevent values from being written to main memory. It doesn't matter if they are in named locals or just temporaries.

The issue is that clang always lowers local variables as alloca's. Clang needs to lower them to alloca's in AS 1 for reference types. Without optimization, LLVM will lower an alloca in AS 1 to a WebAssembly local. SSA conversion in SROA could lift it to an SSA variable which may avoid the local, in some cases. So we are not specifically allocating a local for them, I agree that isn't the right way to express the requirement.

Also, being able to annotate non-reference-types with a wasm_var address space is not really part of the requirements. It certainly has no utility for variables with automatic storage duration. @sbc100 did mention that it could be useful for non-reference-typed globals, though, for ABI reasons.

In D108464#2961595, @rjmccall wrote:

Right now this seems to be entirely type-directed: it's a special property of a couple of builtin types that you can't take the address of their objects and those objects can only live in specific places. Having it be type-directed, but only to the point of saying that certain types *must* use certain address spaces, and then imposing all the other restrictions on those types as novel restrictions on those address spaces, feels like it's adding complexity to the language concept of address spaces without much benefit.

Yeah I could be getting this wrong here.

To expand a bit, it's a special property of a class of types -- currently externref and funcref but future WebAssembly specifications will include reference types like (struct i32 externref). So there is a concept to factor out here that isn't specific to just externref and funcref.

@rjmccall do you think some kind of type attribute would make more sense? If there is something already existing that I can take as a model, that would be helpful of course.

The utility we get from LangAS right now is (1) that it's a clear path to get alloca in AS 1 in codegen and (2) that it's a concept that Sema/ can query in order to detect errors and signal them to the user. I am sure there are other ways to do this though!

clang/lib/CodeGen/CGBuilder.h

public:
   // FIXME: these "default-aligned" APIs should be removed,
   // but I don't feel like fixing all the builtin code right now.
   llvm::StoreInst *CreateDefaultAlignedStore(llvm::Value *Val,
                                              llvm::Value *Addr,
                                              bool IsVolatile = false) {
     return CGBuilderBaseTy::CreateStore(Val, Addr, IsVolatile);
   }
 
-  /// Emit a load from an i1 flag variable.
-  llvm::LoadInst *CreateFlagLoad(llvm::Value *Addr,
-                                 const llvm::Twine &Name = "") {
-    assert(Addr->getType()->getPointerElementType() == getInt1Ty());
-    return CreateAlignedLoad(getInt1Ty(), Addr, CharUnits::One(), Name);
-  }
-
-  /// Emit a store to an i1 flag variable.
-  llvm::StoreInst *CreateFlagStore(bool Value, llvm::Value *Addr) {
-    assert(Addr->getType()->getPointerElementType() == getInt1Ty());
-    return CreateAlignedStore(getInt1(Value), Addr, CharUnits::One());
-  }
-
   // Temporarily use old signature; clang will be updated to an Address overload
   // in a subsequent patch.
   llvm::AtomicCmpXchgInst *
   CreateAtomicCmpXchg(llvm::Value *Ptr, llvm::Value *Cmp, llvm::Value *New,
                       llvm::AtomicOrdering SuccessOrdering,
                       llvm::AtomicOrdering FailureOrdering,
                       llvm::SyncScope::ID SSID = llvm::SyncScope::System) {
     return CGBuilderBaseTy::CreateAtomicCmpXchg(

clang/lib/CodeGen/CGCall.cpp

#endif
   // 1. Set up the arguments.
 
   // If we're using inalloca, insert the allocation after the stack save.
   // FIXME: Do this earlier rather than hacking it in here!
   Address ArgMemory = Address::invalid();
   if (llvm::StructType *ArgStruct = CallInfo.getArgStruct()) {
     const llvm::DataLayout &DL = CGM.getDataLayout();
     llvm::Instruction *IP = CallArgs.getStackBase();
-    llvm::AllocaInst *AI;
+    auto Align = CallInfo.getArgStructAlignment();
     if (IP) {
       IP = IP->getNextNode();
-      AI = new llvm::AllocaInst(ArgStruct, DL.getAllocaAddrSpace(),
-                                "argmem", IP);
+      unsigned AS = DL.getAllocaAddrSpace();
+      llvm::AllocaInst *AI = new llvm::AllocaInst(ArgStruct, AS, "argmem", IP);
+      AI->setAlignment(Align.getAsAlign());
+      ArgMemory = Address(AI, Align);
     } else {
-      AI = CreateTempAlloca(ArgStruct, "argmem");
+      LangAS AS = getASTAllocaAddressSpace();
+      ArgMemory = CreateTempAllocaInAS(ArgStruct, Align, AS, "argmem");
     }
-    auto Align = CallInfo.getArgStructAlignment();
-    AI->setAlignment(Align.getAsAlign());
+    auto *AI = cast<llvm::AllocaInst>(ArgMemory.getPointer());
     AI->setUsedWithInAlloca(true);
     assert(AI->isUsedWithInAlloca() && !AI->isStaticAlloca());
-    ArgMemory = Address(AI, Align);
   }
 
   ClangToLLVMArgMapping IRFunctionArgs(CGM.getContext(), CallInfo);
   SmallVector<llvm::Value *, 16> IRCallArgs(IRFunctionArgs.totalIRArgs());
 
   // If the call returns a temporary with struct return, create a temporary
   // alloca to hold the result, unless one is given to us.
   Address SRetPtr = Address::invalid();

clang/lib/CodeGen/CGCleanup.cpp

DominatingValue<RValue>::saved_type::save(CodeGenFunction &CGF, RValue rv) {
   if (rv.isScalar()) {
     llvm::Value *V = rv.getScalarVal();
 
     // These automatically dominate and don't need to be saved.
     if (!DominatingLLVMValue::needsSaving(V))
       return saved_type(V, ScalarLiteral);
 
     // Everything else needs an alloca.
-    Address addr =
-      CGF.CreateDefaultAlignTempAlloca(V->getType(), "saved-rvalue");
+    Address addr = CGF.CreateTempAllocaInAS(
+        V->getType(), CGF.PreferredAlignmentForIRType(V->getType()),
+        CGF.getASTAllocaAddressSpace(), "saved-rvalue");
     CGF.Builder.CreateStore(V, addr);
     return saved_type(addr.getPointer(), ScalarAddress);
   }
 
   if (rv.isComplex()) {
     CodeGenFunction::ComplexPairTy V = rv.getComplexVal();
     llvm::Type *ComplexTy =
         llvm::StructType::get(V.first->getType(), V.second->getType());
-    Address addr = CGF.CreateDefaultAlignTempAlloca(ComplexTy, "saved-complex");
+    Address addr = CGF.CreateTempAllocaInAS(
+        ComplexTy, CGF.PreferredAlignmentForIRType(ComplexTy),
+        CGF.getASTAllocaAddressSpace(), "saved-complex");
     CGF.Builder.CreateStore(V.first, CGF.Builder.CreateStructGEP(addr, 0));
     CGF.Builder.CreateStore(V.second, CGF.Builder.CreateStructGEP(addr, 1));
     return saved_type(addr.getPointer(), ComplexAddress);
   }
 
   assert(rv.isAggregate());
   Address V = rv.getAggregateAddress(); // TODO: volatile?
   if (!DominatingLLVMValue::needsSaving(V.getPointer()))
void EHScopeStack::popNullFixups() {
 
   while (BranchFixups.size() > MinSize &&
          BranchFixups.back().Destination == nullptr)
     BranchFixups.pop_back();
 }
 
 Address CodeGenFunction::createCleanupActiveFlag() {
   // Create a variable to decide whether the cleanup needs to be run.
-  Address active = CreateTempAllocaWithoutCast(
-      Builder.getInt1Ty(), CharUnits::One(), "cleanup.cond");
+  LangAS AS = getASTAllocaAddressSpace();
+  Address active = CreateTempAllocaInAS(Builder.getInt1Ty(), CharUnits::One(),
+                                        AS, "cleanup.cond");
 
   // Initialize it to false at a site that's guaranteed to be run
   // before each evaluation.
   setBeforeOutermostConditional(Builder.getFalse(), active);
 
   // Initialize it to true at the current location.
   Builder.CreateStore(Builder.getTrue(), active);
 
if (!Inst)
       continue;
 
     // Don't spill static allocas, they dominate all cleanups. These are created
     // by binding a reference to a local variable or temporary.
     auto *AI = dyn_cast<llvm::AllocaInst>(Inst);
     if (AI && AI->isStaticAlloca())
       continue;
 
-    Address Tmp =
-        CreateDefaultAlignTempAlloca(Inst->getType(), "tmp.exprcleanup");
+    Address Tmp = CreateTempAllocaInAS(
+        Inst->getType(), PreferredAlignmentForIRType(Inst->getType()),
+        getASTAllocaAddressSpace(), "tmp.exprcleanup");
 
     // Find an insertion point after Inst and spill it to the temporary.
     llvm::BasicBlock::iterator InsertBefore;
     if (auto *Invoke = dyn_cast<llvm::InvokeInst>(Inst))
       InsertBefore = Invoke->getNormalDest()->getFirstInsertionPt();
     else
       InsertBefore = std::next(Inst->getIterator());
     CGBuilderTy(CGM, &*InsertBefore).CreateStore(Inst, Tmp);
void CodeGenFunction::DeactivateCleanupBlock(EHScopeStack::stable_iterator C,
 
   // Otherwise, follow the general case.
   SetupCleanupBlockActivation(*this, C, ForDeactivation, dominatingIP);
 
   Scope.setActive(false);
 }
 
 Address CodeGenFunction::getNormalCleanupDestSlot() {
-  if (!NormalCleanupDest.isValid())
+  if (!NormalCleanupDest.isValid()) {
+    llvm::Type *Ty = Builder.getInt32Ty();
+    CharUnits Align = PreferredAlignmentForIRType(Ty);
+    LangAS AS = getASTAllocaAddressSpace();
     NormalCleanupDest =
-      CreateDefaultAlignTempAlloca(Builder.getInt32Ty(), "cleanup.dest.slot");
+        CreateTempAllocaInAS(Ty, Align, AS, "cleanup.dest.slot");
+  }
   return NormalCleanupDest;
 }
 
 /// Emits all the code to cause the given temporary to be cleaned up.
 void CodeGenFunction::EmitCXXTemporary(const CXXTemporary *Temporary,
                                        QualType TempType,
                                        Address Ptr) {
   pushDestroy(NormalAndEHCleanup, Ptr, TempType, destroyCXXObject,

clang/lib/CodeGen/CGCoroutine.cpp

struct clang::CodeGen::CGCoroData {
 
   // The promise type's 'unhandled_exception' handler, if it defines one.
   Stmt *ExceptionHandler = nullptr;
 
   // A temporary i1 alloca that stores whether 'await_resume' threw an
   // exception. If it did, 'true' is stored in this variable, and the coroutine
   // body must be skipped. If the promise type does not define an exception
   // handler, this is null.
-  llvm::Value *ResumeEHVar = nullptr;
+  Address ResumeEHVar = Address::invalid();
 
   // Stores the jump destination just before the coroutine memory is freed.
   // This is the destination that every suspend point jumps to for the cleanup
   // branch.
   CodeGenFunction::JumpDest CleanupJD;
 
   // Stores the jump destination just before the final suspend. The co_return
   // statements jumps to this point after calling return_xxx promise member.
static LValueOrRValue emitSuspendExpression(CodeGenFunction &CGF, CGCoroData &Coro,
   // Emit await_resume expression.
   CGF.EmitBlock(ReadyBlock);
 
   // Exception handling requires additional IR. If the 'await_resume' function
   // is marked as 'noexcept', we avoid generating this additional IR.
   CXXTryStmt *TryStmt = nullptr;
   if (Coro.ExceptionHandler && Kind == AwaitKind::Init &&
       memberCallExpressionCanThrow(S.getResumeExpr())) {
-    Coro.ResumeEHVar =
-        CGF.CreateTempAlloca(Builder.getInt1Ty(), Prefix + Twine("resume.eh"));
-    Builder.CreateFlagStore(true, Coro.ResumeEHVar);
+    llvm::Type *Ty = Builder.getInt1Ty();
+    CharUnits Align = CGF.PreferredAlignmentForIRType(Ty);
+    LangAS AS = CGF.getASTAllocaAddressSpace();
+    Coro.ResumeEHVar = CGF.CreateTempAllocaInAS(Ty, Align, AS, "resume.eh");
+    Builder.CreateStore(Builder.getTrue(), Coro.ResumeEHVar);
 
     auto Loc = S.getResumeExpr()->getExprLoc();
     auto *Catch = new (CGF.getContext())
         CXXCatchStmt(Loc, /*exDecl=*/nullptr, Coro.ExceptionHandler);
     auto *TryBody =
         CompoundStmt::Create(CGF.getContext(), S.getResumeExpr(), Loc, Loc);
     TryStmt = CXXTryStmt::Create(CGF.getContext(), Loc, TryBody, Catch);
     CGF.EnterCXXTryStmt(*TryStmt);
   }
 
   LValueOrRValue Res;
   if (forLValue)
     Res.LV = CGF.EmitLValue(S.getResumeExpr());
   else
     Res.RV = CGF.EmitAnyExpr(S.getResumeExpr(), aggSlot, ignoreResult);
 
   if (TryStmt) {
-    Builder.CreateFlagStore(false, Coro.ResumeEHVar);
+    Builder.CreateStore(Builder.getFalse(), Coro.ResumeEHVar);
     CGF.ExitCXXTryStmt(*TryStmt);
   }
 
   return Res;
 }
 
 RValue CodeGenFunction::EmitCoawaitExpr(const CoawaitExpr &E,
                                         AggValueSlot aggSlot,
CurCoro.Data->CleanupJD = getJumpDestInCurrentScope(RetBB);
 
     if (CurCoro.Data->ExceptionHandler) {
       // If we generated IR to record whether an exception was thrown from
       // 'await_resume', then use that IR to determine whether the coroutine
       // body should be skipped.
       // If we didn't generate the IR (perhaps because 'await_resume' was marked
       // as 'noexcept'), then we skip this check.
       BasicBlock *ContBB = nullptr;
-      if (CurCoro.Data->ResumeEHVar) {
+      if (CurCoro.Data->ResumeEHVar.isValid()) {
         BasicBlock *BodyBB = createBasicBlock("coro.resumed.body");
         ContBB = createBasicBlock("coro.resumed.cont");
-        Value *SkipBody = Builder.CreateFlagLoad(CurCoro.Data->ResumeEHVar,
-                                                 "coro.resumed.eh");
+        Value *SkipBody =
+            Builder.CreateLoad(CurCoro.Data->ResumeEHVar, "coro.resumed.eh");
         Builder.CreateCondBr(SkipBody, ContBB, BodyBB);
         EmitBlock(BodyBB);
       }
 
       auto Loc = S.getBeginLoc();
       CXXCatchStmt Catch(Loc, /*exDecl=*/nullptr,
                          CurCoro.Data->ExceptionHandler);
       auto *TryStmt =

clang/lib/CodeGen/CGDecl.cpp

void Emit(CodeGenFunction &CGF, Flags flags) override {
         flags.isForNormalCleanup() && this->useEHCleanupForArray;
 
       CGF.emitDestroy(addr, type, destroyer, useEHCleanupForArray);
     }
   };
 
   template <class Derived>
   struct DestroyNRVOVariable : EHScopeStack::Cleanup {
-    DestroyNRVOVariable(Address addr, QualType type, llvm::Value *NRVOFlag)
+    DestroyNRVOVariable(Address addr, QualType type, Address NRVOFlag)
         : NRVOFlag(NRVOFlag), Loc(addr), Ty(type) {}
 
-    llvm::Value *NRVOFlag;
+    Address NRVOFlag;
     Address Loc;
     QualType Ty;
 
     void Emit(CodeGenFunction &CGF, Flags flags) override {
       // Along the exceptions path we always execute the dtor.
-      bool NRVO = flags.isForNormalCleanup() && NRVOFlag;
+      bool NRVO = flags.isForNormalCleanup() && NRVOFlag.isValid();
 
       llvm::BasicBlock *SkipDtorBB = nullptr;
       if (NRVO) {
         // If we exited via NRVO, we skip the destructor call.
         llvm::BasicBlock *RunDtorBB = CGF.createBasicBlock("nrvo.unused");
         SkipDtorBB = CGF.createBasicBlock("nrvo.skipdtor");
-        llvm::Value *DidNRVO =
-          CGF.Builder.CreateFlagLoad(NRVOFlag, "nrvo.val");
+        llvm::Value *DidNRVO = CGF.Builder.CreateLoad(NRVOFlag, "nrvo.val");
         CGF.Builder.CreateCondBr(DidNRVO, SkipDtorBB, RunDtorBB);
         CGF.EmitBlock(RunDtorBB);
       }
 
       static_cast<Derived *>(this)->emitDestructorCall(CGF);
 
       if (NRVO) CGF.EmitBlock(SkipDtorBB);
     }
 
     virtual ~DestroyNRVOVariable() = default;
   };
 
   struct DestroyNRVOVariableCXX final
       : DestroyNRVOVariable<DestroyNRVOVariableCXX> {
     DestroyNRVOVariableCXX(Address addr, QualType type,
-                           const CXXDestructorDecl *Dtor, llvm::Value *NRVOFlag)
+                           const CXXDestructorDecl *Dtor, Address NRVOFlag)
         : DestroyNRVOVariable<DestroyNRVOVariableCXX>(addr, type, NRVOFlag),
           Dtor(Dtor) {}
 
     const CXXDestructorDecl *Dtor;
 
     void emitDestructorCall(CodeGenFunction &CGF) {
       CGF.EmitCXXDestructorCall(Dtor, Dtor_Complete,
                                 /*ForVirtualBase=*/false,
                                 /*Delegating=*/false, Loc, Ty);
     }
   };
 
   struct DestroyNRVOVariableC final
       : DestroyNRVOVariable<DestroyNRVOVariableC> {
-    DestroyNRVOVariableC(Address addr, llvm::Value *NRVOFlag, QualType Ty)
+    DestroyNRVOVariableC(Address addr, Address NRVOFlag, QualType Ty)
         : DestroyNRVOVariable<DestroyNRVOVariableC>(addr, Ty, NRVOFlag) {}
 
     void emitDestructorCall(CodeGenFunction &CGF) {
       CGF.destroyNonTrivialCStruct(CGF, Loc, Ty);
     }
   };
 
   struct CallStackRestore final : EHScopeStack::Cleanup {
if (auto *C = dyn_cast<llvm::ConstantInt>(VlaSize.NumElts))
       Dimensions.emplace_back(C, Type1D.getUnqualifiedType());
     else {
       // Generate a locally unique name for the size expression.
       Twine Name = Twine("__vla_expr") + Twine(VLAExprCounter++);
       SmallString<12> Buffer;
       StringRef NameRef = Name.toStringRef(Buffer);
       auto &Ident = getContext().Idents.getOwn(NameRef);
       VLAExprNames.push_back(&Ident);
-      auto SizeExprAddr =
-          CreateDefaultAlignTempAlloca(VlaSize.NumElts->getType(), NameRef);
+      llvm::Type *Ty = VlaSize.NumElts->getType();
+      CharUnits Align = PreferredAlignmentForIRType(Ty);
+      LangAS AS = getASTAllocaAddressSpace();
+      auto SizeExprAddr = CreateTempAllocaInAS(Ty, Align, AS, NameRef);
       Builder.CreateStore(VlaSize.NumElts, SizeExprAddr);
       Dimensions.emplace_back(SizeExprAddr.getPointer(),
                               Type1D.getUnqualifiedType());
     }
     Type1D = VlaSize.Type;
   }
 
   if (!EmitDebugInfo)
if (NRVO) {
           // applied.
           llvm::Value *Zero = Builder.getFalse();
           Address NRVOFlag =
             CreateTempAlloca(Zero->getType(), CharUnits::One(), "nrvo");
           EnsureInsertPoint();
           Builder.CreateStore(Zero, NRVOFlag);
 
           // Record the NRVO flag for this variable.
-          NRVOFlags[&D] = NRVOFlag.getPointer();
-          emission.NRVOFlag = NRVOFlag.getPointer();
+          NRVOFlags.insert(std::make_pair(&D, NRVOFlag));
+          emission.NRVOFlag = NRVOFlag;
         }
       }
     } else {
       CharUnits allocaAlignment;
       llvm::Type *allocaTy;
       if (isEscapingByRef) {
         auto &byrefInfo = getBlockByrefInfo(&D);
         allocaTy = byrefInfo.Type;
void CodeGenFunction::emitAutoVarTypeCleanup(
 
   switch (dtorKind) {
   case QualType::DK_none:
     llvm_unreachable("no cleanup for trivially-destructible variable");
 
   case QualType::DK_cxx_destructor:
     // If there's an NRVO flag on the emission, we need a different
     // cleanup.
-    if (emission.NRVOFlag) {
+    if (emission.NRVOFlag.isValid()) {
       assert(!type->isArrayType());
       CXXDestructorDecl *dtor = type->getAsCXXRecordDecl()->getDestructor();
       EHStack.pushCleanup<DestroyNRVOVariableCXX>(cleanupKind, addr, type, dtor,
                                                   emission.NRVOFlag);
       return;
     }
     break;
 
if (!var->hasAttr<ObjCPreciseLifetimeAttr>())
       destroyer = CodeGenFunction::destroyARCStrongImprecise;
     break;
 
   case QualType::DK_objc_weak_lifetime:
     break;
 
   case QualType::DK_nontrivial_c_struct:
     destroyer = CodeGenFunction::destroyNonTrivialCStruct;
-    if (emission.NRVOFlag) {
+    if (emission.NRVOFlag.isValid()) {
       assert(!type->isArrayType());
       EHStack.pushCleanup<DestroyNRVOVariableC>(cleanupKind, addr,
                                                 emission.NRVOFlag, type);
       return;
     }
     break;
   }
 

clang/lib/CodeGen/CGException.cpp

EmitAnyExprToMem(e, typedAddr, e->getType().getQualifiers(),
                    /*IsInit*/ true);
 
   // Deactivate the cleanup block.
   DeactivateCleanupBlock(cleanup,
                          cast<llvm::Instruction>(typedAddr.getPointer()));
 }
 
 Address CodeGenFunction::getExceptionSlot() {
-  if (!ExceptionSlot)
-    ExceptionSlot = CreateTempAlloca(Int8PtrTy, "exn.slot");
-  return Address(ExceptionSlot, getPointerAlign());
+  if (!ExceptionSlot.isValid()) {
+    CharUnits Align = getPointerAlign();
+    LangAS AS = getASTAllocaAddressSpace();
+    ExceptionSlot = CreateTempAllocaInAS(Int8PtrTy, Align, AS, "exn.slot");
+  }
+  return ExceptionSlot;
 }
 
 Address CodeGenFunction::getEHSelectorSlot() {
-  if (!EHSelectorSlot)
-    EHSelectorSlot = CreateTempAlloca(Int32Ty, "ehselector.slot");
-  return Address(EHSelectorSlot, CharUnits::fromQuantity(4));
+  if (!EHSelectorSlot.isValid()) {
+    CharUnits Align = CharUnits::fromQuantity(4);
+    LangAS AS = getASTAllocaAddressSpace();
+    EHSelectorSlot =
+        CreateTempAllocaInAS(Int32Ty, Align, AS, "ehselector.slot");
+  }
+  return EHSelectorSlot;
 }
 
 llvm::Value *CodeGenFunction::getExceptionFromSlot() {
   return Builder.CreateLoad(getExceptionSlot(), "exn");
 }
 
 llvm::Value *CodeGenFunction::getSelectorFromSlot() {
   return Builder.CreateLoad(getEHSelectorSlot(), "sel");
void CodeGenFunction::ExitCXXTryStmt(const CXXTryStmt &S, bool IsFnTryBlock) {
   }
 
   EmitBlock(ContBB);
   incrementProfileCounter(&S);
 }
 
 namespace {
   struct CallEndCatchForFinally final : EHScopeStack::Cleanup {
-    llvm::Value *ForEHVar;
+    Address ForEHVar;
     llvm::FunctionCallee EndCatchFn;
-    CallEndCatchForFinally(llvm::Value *ForEHVar,
-                           llvm::FunctionCallee EndCatchFn)
+    CallEndCatchForFinally(Address ForEHVar, llvm::FunctionCallee EndCatchFn)
         : ForEHVar(ForEHVar), EndCatchFn(EndCatchFn) {}
 
     void Emit(CodeGenFunction &CGF, Flags flags) override {
       llvm::BasicBlock *EndCatchBB = CGF.createBasicBlock("finally.endcatch");
       llvm::BasicBlock *CleanupContBB =
         CGF.createBasicBlock("finally.cleanup.cont");
 
       llvm::Value *ShouldEndCatch =
-        CGF.Builder.CreateFlagLoad(ForEHVar, "finally.endcatch");
+          CGF.Builder.CreateLoad(ForEHVar, "finally.endcatch");
       CGF.Builder.CreateCondBr(ShouldEndCatch, EndCatchBB, CleanupContBB);
       CGF.EmitBlock(EndCatchBB);
       CGF.EmitRuntimeCallOrInvoke(EndCatchFn); // catch-all, so might throw
       CGF.EmitBlock(CleanupContBB);
     }
   };
 
   struct PerformFinally final : EHScopeStack::Cleanup {
     const Stmt *Body;
-    llvm::Value *ForEHVar;
+    Address ForEHVar;
     llvm::FunctionCallee EndCatchFn;
     llvm::FunctionCallee RethrowFn;
-    llvm::Value *SavedExnVar;
+    Address SavedExnVar;
 
-    PerformFinally(const Stmt *Body, llvm::Value *ForEHVar,
+    PerformFinally(const Stmt *Body, Address ForEHVar,
                    llvm::FunctionCallee EndCatchFn,
-                   llvm::FunctionCallee RethrowFn, llvm::Value *SavedExnVar)
+                   llvm::FunctionCallee RethrowFn, Address SavedExnVar)
         : Body(Body), ForEHVar(ForEHVar), EndCatchFn(EndCatchFn),
           RethrowFn(RethrowFn), SavedExnVar(SavedExnVar) {}
 
     void Emit(CodeGenFunction &CGF, Flags flags) override {
       // Enter a cleanup to call the end-catch function if one was provided.
       if (EndCatchFn)
         CGF.EHStack.pushCleanup<CallEndCatchForFinally>(NormalAndEHCleanup,
                                                         ForEHVar, EndCatchFn);
void Emit(CodeGenFunction &CGF, Flags flags) override {
 
       // If the end of the finally is reachable, check whether this was
       // for EH.  If so, rethrow.
       if (CGF.HaveInsertPoint()) {
         llvm::BasicBlock *RethrowBB = CGF.createBasicBlock("finally.rethrow");
         llvm::BasicBlock *ContBB = CGF.createBasicBlock("finally.cont");
 
         llvm::Value *ShouldRethrow =
-          CGF.Builder.CreateFlagLoad(ForEHVar, "finally.shouldthrow");
+            CGF.Builder.CreateLoad(ForEHVar, "finally.shouldthrow");
         CGF.Builder.CreateCondBr(ShouldRethrow, RethrowBB, ContBB);
 
         CGF.EmitBlock(RethrowBB);
-        if (SavedExnVar) {
+        if (SavedExnVar.isValid()) {
           CGF.EmitRuntimeCallOrInvoke(RethrowFn,
-            CGF.Builder.CreateAlignedLoad(CGF.Int8PtrTy, SavedExnVar,
-                                          CGF.getPointerAlign()));
+                                      CGF.Builder.CreateLoad(SavedExnVar));
         } else {
           CGF.EmitRuntimeCallOrInvoke(RethrowFn);
         }
         CGF.Builder.CreateUnreachable();
 
         CGF.EmitBlock(ContBB);
 
         // Restore the cleanup destination.
void CodeGenFunction::FinallyInfo::enter(CodeGenFunction &CGF, const Stmt *body,
 
   // The rethrow function has one of the following two types:
   //   void (*)()
   //   void (*)(void*)
   // In the latter case we need to pass it the exception object.
   // But we can't use the exception slot because the @finally might
   // have a landing pad (which would overwrite the exception slot).
   llvm::FunctionType *rethrowFnTy = rethrowFn.getFunctionType();
-  SavedExnVar = nullptr;
-  if (rethrowFnTy->getNumParams())
-    SavedExnVar = CGF.CreateTempAlloca(CGF.Int8PtrTy, "finally.exn");
+  SavedExnVar = Address::invalid();
+  if (rethrowFnTy->getNumParams()) {
+    CharUnits Align = CGF.getPointerAlign();
+    LangAS AS = CGF.getASTAllocaAddressSpace();
+    SavedExnVar =
+        CGF.CreateTempAllocaInAS(CGF.Int8PtrTy, Align, AS, "finally.exn");
+  }
 
   // A finally block is a statement which must be executed on any edge
   // out of a given scope.  Unlike a cleanup, the finally block may
   // contain arbitrary control flow leading out of itself.  In
   // addition, finally blocks should always be executed, even if there
   // are no catch handlers higher on the stack.  Therefore, we
   // surround the protected scope with a combination of a normal
   // cleanup (to catch attempts to break out of the block via normal
   // control flow) and an EH catch-all (semantically "outside" any try
   // statement to which the finally block might have been attached).
   // The finally block itself is generated in the context of a cleanup
   // which conditionally leaves the catch-all.
 
   // Jump destination for performing the finally block on an exception
   // edge.  We'll never actually reach this block, so unreachable is
   // fine.
   RethrowDest = CGF.getJumpDestInCurrentScope(CGF.getUnreachableBlock());
 
   // Whether the finally block is being executed for EH purposes.
-  ForEHVar = CGF.CreateTempAlloca(CGF.Builder.getInt1Ty(), "finally.for-eh");
-  CGF.Builder.CreateFlagStore(false, ForEHVar);
+  llvm::Type *FlagTy = CGF.Builder.getInt1Ty();
+  ForEHVar = CGF.CreateTempAllocaInAS(
+      FlagTy, CGF.PreferredAlignmentForIRType(FlagTy),
+      CGF.getASTAllocaAddressSpace(), "finally.for-eh");
+  CGF.Builder.CreateStore(CGF.Builder.getFalse(), ForEHVar);
 
   // Enter a normal cleanup which will perform the @finally block.
   CGF.EHStack.pushCleanup<PerformFinally>(NormalCleanup, body,
                                           ForEHVar, endCatchFn,
                                           rethrowFn, SavedExnVar);
 
   // Enter a catch-all scope.
   llvm::BasicBlock *catchBB = CGF.createBasicBlock("finally.catchall");
if (catchBB->use_empty()) {
 
     // If there's a begin-catch function, call it.
     if (BeginCatchFn) {
       exn = CGF.getExceptionFromSlot();
       CGF.EmitNounwindRuntimeCall(BeginCatchFn, exn);
     }
 
     // If we need to remember the exception pointer to rethrow later, do so.
-    if (SavedExnVar) {
-      if (!exn) exn = CGF.getExceptionFromSlot();
-      CGF.Builder.CreateAlignedStore(exn, SavedExnVar, CGF.getPointerAlign());
+    if (SavedExnVar.isValid()) {
+      if (!exn)
+        exn = CGF.getExceptionFromSlot();
+      CGF.Builder.CreateStore(exn, SavedExnVar);
     }
 
     // Tell the cleanups in the finally block that we're do this for EH.
-    CGF.Builder.CreateFlagStore(true, ForEHVar);
+    CGF.Builder.CreateStore(CGF.Builder.getTrue(), ForEHVar);
 
     // Thread a jump through the finally cleanup.
     CGF.EmitBranchThroughCleanup(RethrowDest);
 
     CGF.Builder.restoreIP(savedIP);
   }
 
   // Finally, leave the @finally cleanup.

clang/lib/CodeGen/CGExpr.cpp

llvm::Value *CodeGenFunction::EmitCastToVoidPtr(llvm::Value *value) {
   llvm::PointerType *destType = Int8PtrTy;
   if (addressSpace)
     destType = llvm::Type::getInt8PtrTy(getLLVMContext(), addressSpace);
 
   if (value->getType() == destType) return value;
   return Builder.CreateBitCast(value, destType);
 }
 
-/// CreateTempAlloca - This creates a alloca and inserts it into the entry
-/// block.
-Address CodeGenFunction::CreateTempAllocaWithoutCast(llvm::Type *Ty,
-                                                     CharUnits Align,
-                                                     const Twine &Name,
-                                                     llvm::Value *ArraySize) {
-  auto Alloca = CreateTempAlloca(Ty, Name, ArraySize);
+/// CreateTempAllocaInAS - Create an alloca in \p AddressSpace with alignment \p
+/// Align.  Leave the result in \p AddressSpace.
+Address CodeGenFunction::CreateTempAllocaInAS(llvm::Type *Ty, CharUnits Align,
+                                              LangAS AddressSpace,
+  auto AS = getContext().getTargetAddressSpace(AddressSpace);
+  llvm::AllocaInst *Alloca =
+      ArraySize ? Builder.CreateAlloca(Ty, AS, ArraySize, Name)
+                : new llvm::AllocaInst(Ty, AS, ArraySize, Name, AllocaInsertPt);
   Alloca->setAlignment(Align.getAsAlign());
   return Address(Alloca, Align);
 }
 
-/// CreateTempAlloca - This creates a alloca and inserts it into the entry
-/// block. The alloca is casted to default address space if necessary.
+/// CreateTempAlloca - Create an alloca as with CreateTempAllocaInAS, then cast
+/// the result to LangAS::Default if necessary.
 Address CodeGenFunction::CreateTempAlloca(llvm::Type *Ty, CharUnits Align,
                                           const Twine &Name,
                                           llvm::Value *ArraySize,
                                           Address *AllocaAddr) {
-  auto Alloca = CreateTempAllocaWithoutCast(Ty, Align, Name, ArraySize);
+  LangAS AddressSpace = getASTAllocaAddressSpace();
+  auto Alloca = CreateTempAllocaInAS(Ty, Align, AddressSpace, Name, ArraySize);
   if (AllocaAddr)
     *AllocaAddr = Alloca;
   llvm::Value *V = Alloca.getPointer();
-  // Alloca always returns a pointer in alloca address space, which may
-  // be different from the type defined by the language. For example,
-  // in C++ the auto variables are in the default address space. Therefore
-  // cast alloca to the default address space when necessary.
-  if (getASTAllocaAddressSpace() != LangAS::Default) {
-    auto DestAddrSpace = getContext().getTargetAddressSpace(LangAS::Default);
+
+  // Alloca returns a pointer in the specified address space, which may be
+  // different from the type defined by the language. For example, in C++, auto
+  // variables are in the default address space. Therefore cast alloca to the
+  // default address space when necessary.
+  if (AddressSpace != LangAS::Default) {
     llvm::IRBuilderBase::InsertPointGuard IPG(Builder);
     // When ArraySize is nullptr, alloca is inserted at AllocaInsertPt,
     // otherwise alloca is inserted at the current insertion point of the
     // builder.
     if (!ArraySize)
       Builder.SetInsertPoint(AllocaInsertPt);
+    llvm::Type *DestTy =
+        Ty->getPointerTo(getContext().getTargetAddressSpace(LangAS::Default));
     V = getTargetHooks().performAddrSpaceCast(
-        *this, V, getASTAllocaAddressSpace(), LangAS::Default,
-        Ty->getPointerTo(DestAddrSpace), /*non-null*/ true);
+        *this, V, AddressSpace, LangAS::Default, DestTy, /*non-null*/ true);
   }
 
   return Address(V, Align);
 }
 
-/// CreateTempAlloca - This creates an alloca and inserts it into the entry
-/// block if \p ArraySize is nullptr, otherwise inserts it at the current
-/// insertion point of the builder.
-llvm::AllocaInst *CodeGenFunction::CreateTempAlloca(llvm::Type *Ty,
-                                                    const Twine &Name,
-                                                    llvm::Value *ArraySize) {
-  if (ArraySize)
-    return Builder.CreateAlloca(Ty, ArraySize, Name);
-  return new llvm::AllocaInst(Ty, CGM.getDataLayout().getAllocaAddrSpace(),
-                              ArraySize, Name, AllocaInsertPt);
-}
-
-/// CreateDefaultAlignTempAlloca - This creates an alloca with the
-/// default alignment of the corresponding LLVM type, which is *not*
-/// guaranteed to be related in any way to the expected alignment of
-/// an AST type that might have been lowered to Ty.
-Address CodeGenFunction::CreateDefaultAlignTempAlloca(llvm::Type *Ty,
-                                                      const Twine &Name) {
-  CharUnits Align =
-      CharUnits::fromQuantity(CGM.getDataLayout().getPrefTypeAlignment(Ty));
-  return CreateTempAlloca(Ty, Align, Name);
-}
-
 void CodeGenFunction::InitTempAlloca(Address Var, llvm::Value *Init) {
   auto *Alloca = Var.getPointer();
   assert(isa<llvm::AllocaInst>(Alloca) ||
          (isa<llvm::AddrSpaceCastInst>(Alloca) &&
           isa<llvm::AllocaInst>(
               cast<llvm::AddrSpaceCastInst>(Alloca)->getPointerOperand())));
 
   auto *Store = new llvm::StoreInst(Init, Alloca, /*volatile*/ false,
Result = Address(
         Builder.CreateBitCast(Result.getPointer(), VectorTy->getPointerTo()),
         Result.getAlignment());
   }
   return Result;
 }
 
 Address CodeGenFunction::CreateMemTempWithoutCast(QualType Ty, CharUnits Align,
                                                   const Twine &Name) {
-  return CreateTempAllocaWithoutCast(ConvertTypeForMem(Ty), Align, Name);
+  LangAS AS = getASTAllocaAddressSpace();
+  return CreateTempAllocaInAS(ConvertTypeForMem(Ty), Align, AS, Name);
 }
 
 Address CodeGenFunction::CreateMemTempWithoutCast(QualType Ty,
                                                   const Twine &Name) {
   return CreateMemTempWithoutCast(Ty, getContext().getTypeAlignInChars(Ty),
                                   Name);
 }
 
llvm::Value *CodeGenFunction::EmitCheckValue(llvm::Value *V) {
 
   // Integers which fit in intptr_t are zero-extended and passed directly.
   if (V->getType()->isIntegerTy() &&
       V->getType()->getIntegerBitWidth() <= TargetTy->getIntegerBitWidth())
     return Builder.CreateZExt(V, TargetTy);
 
   // Pointers are passed directly, everything else is passed by address.
   if (!V->getType()->isPointerTy()) {
-    Address Ptr = CreateDefaultAlignTempAlloca(V->getType());
+    auto Align = PreferredAlignmentForIRType(V->getType());
+    LangAS AS = getASTAllocaAddressSpace();
+    Address Ptr = CreateTempAllocaInAS(V->getType(), Align, AS);
     Builder.CreateStore(V, Ptr);
     V = Ptr.getPointer();
   }
   return Builder.CreatePtrToInt(V, TargetTy);
 }
 
 /// Emit a representation of a SourceLocation for passing to a handler
 /// in a sanitizer runtime library. The format for this data is:

clang/lib/CodeGen/CGExprCXX.cpp

void CodeGenFunction::EmitDeleteCall(const FunctionDecl *DeleteFD,
   auto ParamTypeIt = DeleteFTy->param_type_begin();
 
   // Pass the pointer itself.
   QualType ArgTy = *ParamTypeIt++;
   llvm::Value *DeletePtr = Builder.CreateBitCast(Ptr, ConvertType(ArgTy));
   DeleteArgs.add(RValue::get(DeletePtr), ArgTy);
 
   // Pass the std::destroying_delete tag if present.
-  llvm::AllocaInst *DestroyingDeleteTag = nullptr;
+  Address DestroyingDeleteTag = Address::invalid();
   if (Params.DestroyingDelete) {
     QualType DDTag = *ParamTypeIt++;
     llvm::Type *Ty = getTypes().ConvertType(DDTag);
     CharUnits Align = CGM.getNaturalTypeAlignment(DDTag);
-    DestroyingDeleteTag = CreateTempAlloca(Ty, "destroying.delete.tag");
-    DestroyingDeleteTag->setAlignment(Align.getAsAlign());
-    DeleteArgs.add(RValue::getAggregate(Address(DestroyingDeleteTag, Align)), DDTag);
+    LangAS AS = getASTAllocaAddressSpace();
+    DestroyingDeleteTag =
+        CreateTempAllocaInAS(Ty, Align, AS, "destroying.delete.tag");
+    DeleteArgs.add(RValue::getAggregate(DestroyingDeleteTag), DDTag);
   }
 
   // Pass the size if the delete function has a size_t parameter.
   if (Params.Size) {
     QualType SizeType = *ParamTypeIt++;
     CharUnits DeleteTypeSize = getContext().getTypeSizeInChars(DeleteTy);
     llvm::Value *Size = llvm::ConstantInt::get(ConvertType(SizeType),
                                                DeleteTypeSize.getQuantity());
void CodeGenFunction::EmitDeleteCall(const FunctionDecl *DeleteFD,
   assert(ParamTypeIt == DeleteFTy->param_type_end() &&
          "unknown parameter to usual delete function");
 
   // Emit the call to delete.
   EmitNewDeleteCall(*this, DeleteFD, DeleteFTy, DeleteArgs);
 
   // If call argument lowering didn't use the destroying_delete_t alloca,
   // remove it again.
-  if (DestroyingDeleteTag && DestroyingDeleteTag->use_empty())
-    DestroyingDeleteTag->eraseFromParent();
+  if (DestroyingDeleteTag.isValid()) {
+    auto *Inst = cast<llvm::Instruction>(DestroyingDeleteTag.getPointer());
+    if (Inst->use_empty())
+      Inst->eraseFromParent();
+  }
 }
 
 namespace {
   /// Calls the given 'operator delete' on a single object.
   struct CallObjectDelete final : EHScopeStack::Cleanup {
     llvm::Value *Ptr;
     const FunctionDecl *OperatorDelete;
     QualType ElementType;

clang/lib/CodeGen/CGExprScalar.cpp

case CK_BitCast: {
     //       require the element types of the vectors to be the same, we
     //       need to keep this around for bitcasts between VLAT <-> VLST where
     //       the element types of the vectors are not the same, until we figure
     //       out a better way of doing these casts.
     if ((isa<llvm::FixedVectorType>(SrcTy) &&
          isa<llvm::ScalableVectorType>(DstTy)) ||
         (isa<llvm::ScalableVectorType>(SrcTy) &&
          isa<llvm::FixedVectorType>(DstTy))) {
-      Address Addr = CGF.CreateDefaultAlignTempAlloca(SrcTy, "saved-value");
+      CharUnits Align = CGF.PreferredAlignmentForIRType(SrcTy);
+      LangAS AS = CGF.getASTAllocaAddressSpace();
+      Address Addr = CGF.CreateTempAllocaInAS(SrcTy, Align, AS, "saved-value");
       LValue LV = CGF.MakeAddrLValue(Addr, E->getType());
       CGF.EmitStoreOfScalar(Src, LV);
       Addr = Builder.CreateElementBitCast(Addr, CGF.ConvertTypeForMem(DestTy),
                                           "castFixedSve");
       LValue DestLV = CGF.MakeAddrLValue(Addr, DestTy);
       DestLV.setTBAAInfo(TBAAAccessInfo::getMayAliasInfo());
       return EmitLoadOfLValue(DestLV, CE->getExprLoc());
     }

clang/lib/CodeGen/CGGPUBuiltin.cpp

for (unsigned I = 1, NumArgs = Args.size(); I < NumArgs; ++I)
       ArgTypes.push_back(Args[I].getRValue(*this).getScalarVal()->getType());
 
     // Using llvm::StructType is correct only because printf doesn't accept
     // aggregates.  If we had to handle aggregates here, we'd have to manually
     // compute the offsets within the alloca -- we wouldn't be able to assume
     // that the alignment of the llvm type was the same as the alignment of the
     // clang type.
     llvm::Type *AllocaTy = llvm::StructType::create(ArgTypes, "printf_args");
-    llvm::Value *Alloca = CreateTempAlloca(AllocaTy);
+    CharUnits Align = PreferredAlignmentForIRType(AllocaTy);
+    LangAS AS = getASTAllocaAddressSpace();
+    Address Alloca = CreateTempAllocaInAS(AllocaTy, Align, AS);
 
     for (unsigned I = 1, NumArgs = Args.size(); I < NumArgs; ++I) {
-      llvm::Value *P = Builder.CreateStructGEP(AllocaTy, Alloca, I - 1);
+      Address P = Builder.CreateStructGEP(Alloca, I - 1);
       llvm::Value *Arg = Args[I].getRValue(*this).getScalarVal();
-      Builder.CreateAlignedStore(Arg, P, DL.getPrefTypeAlign(Arg->getType()));
+      // FIXME: Changing the following line to Builder.CreateStore(Arg, P)
+      // results in a test failure in OpenMP/nvptx_target_printf_codegen, in
+      // that a store of an i32 is expected to have alignment 4 on a 64-bit
+      // target, but using the alignment from P results in a store with
+      // alignment 8.  Could this actually be correct?
+      Builder.CreateAlignedStore(Arg, P.getPointer(),
+                                 DL.getPrefTypeAlign(Arg->getType()));
     }
-    BufferPtr = Builder.CreatePointerCast(Alloca, llvm::Type::getInt8PtrTy(Ctx));
+    BufferPtr = Builder.CreatePointerCast(Alloca.getPointer(),
+                                          llvm::Type::getInt8PtrTy(Ctx));
   }
 
   // Invoke vprintf and return.
   llvm::Function* VprintfFunc = GetVprintfDeclaration(CGM.getModule());
   return RValue::get(Builder.CreateCall(
       VprintfFunc, {Args[0].getRValue(*this).getScalarVal(), BufferPtr}));
 }
 

clang/lib/CodeGen/CGOpenMPRuntime.cpp

auto &&ElseGen = [&M, OutlinedFn, CapturedVars, RTLoc, Loc,
     // __kmpc_serialized_parallel(&Loc, GTid);
     llvm::Value *Args[] = {RTLoc, ThreadID};
     CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
                             M, OMPRTL___kmpc_serialized_parallel),
                         Args);
 
     // OutlinedFn(&GTid, &zero_bound, CapturedStruct);
     Address ThreadIDAddr = RT.emitThreadIDAddress(CGF, Loc);
-    Address ZeroAddrBound =
-        CGF.CreateDefaultAlignTempAlloca(CGF.Int32Ty,
-                                         /*Name=*/".bound.zero.addr");
+    CharUnits Align = CGF.PreferredAlignmentForIRType(CGF.Int32Ty);
+    LangAS AS = CGF.getASTAllocaAddressSpace();
+    Address ZeroAddrBound = CGF.CreateTempAllocaInAS(
+        CGF.Int32Ty, Align, AS, /*Name=*/".bound.zero.addr");
     CGF.InitTempAlloca(ZeroAddrBound, CGF.Builder.getInt32(/*C*/ 0));
     llvm::SmallVector<llvm::Value *, 16> OutlinedFnArgs;
     // ThreadId for serialized parallels is 0.
     OutlinedFnArgs.push_back(ThreadIDAddr.getPointer());
     OutlinedFnArgs.push_back(ZeroAddrBound.getPointer());
     OutlinedFnArgs.append(CapturedVars.begin(), CapturedVars.end());
 
     // Ensure we do not inline the function. This is trivially true for the ones

clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp

 void CGOpenMPRuntimeGPU::emitTeamsCall(CodeGenFunction &CGF,
                                          const OMPExecutableDirective &D,
                                          SourceLocation Loc,
                                          llvm::Function *OutlinedFn,
                                          ArrayRef<llvm::Value *> CapturedVars) {
   if (!CGF.HaveInsertPoint())
     return;
 
-  Address ZeroAddr = CGF.CreateDefaultAlignTempAlloca(CGF.Int32Ty,
-                                                      /*Name=*/".zero.addr");
+  Address ZeroAddr = CGF.CreateTempAllocaInAS(
+      CGF.Int32Ty, CGF.PreferredAlignmentForIRType(CGF.Int32Ty),
+      CGF.getASTAllocaAddressSpace(),
   CGF.InitTempAlloca(ZeroAddr, CGF.Builder.getInt32(/*C*/ 0));
   llvm::SmallVector<llvm::Value *, 16> OutlinedFnArgs;
   OutlinedFnArgs.push_back(emitThreadIDAddress(CGF, Loc).getPointer());
   OutlinedFnArgs.push_back(ZeroAddr.getPointer());
   OutlinedFnArgs.append(CapturedVars.begin(), CapturedVars.end());
   emitOutlinedFunctionCall(CGF, Loc, OutlinedFn, OutlinedFnArgs);
 }
 
if (WFn)
       ID = Bld.CreateBitOrPointerCast(WFn, CGM.Int8PtrTy);
     llvm::Value *FnPtr = Bld.CreateBitOrPointerCast(OutlinedFn, CGM.Int8PtrTy);
 
     // Create a private scope that will globalize the arguments
     // passed from the outside of the target region.
     // TODO: Is that needed?
     CodeGenFunction::OMPPrivateScope PrivateArgScope(CGF);
 
-    Address CapturedVarsAddrs = CGF.CreateDefaultAlignTempAlloca(
-        llvm::ArrayType::get(CGM.VoidPtrTy, CapturedVars.size()),
-        "captured_vars_addrs");
+    llvm::Type *VarsTy =
+        llvm::ArrayType::get(CGM.VoidPtrTy, CapturedVars.size());
+    Address CapturedVarsAddrs = CGF.CreateTempAllocaInAS(
+        VarsTy, CGF.PreferredAlignmentForIRType(VarsTy),
+        CGF.getASTAllocaAddressSpace(), "captured_vars_addrs");
     // There's something to share.
     if (!CapturedVars.empty()) {
       // Prepare for parallel region. Indicate the outlined function.
       ASTContext &Ctx = CGF.getContext();
       unsigned Idx = 0;
       for (llvm::Value *V : CapturedVars) {
         Address Dst = Bld.CreateConstArrayGEP(CapturedVarsAddrs, Idx);
         llvm::Value *PtrV;
llvm::Function *CGOpenMPRuntimeGPU::createParallelDataSharingWrapper(
 
   CodeGenFunction CGF(CGM, /*suppressNewContext=*/true);
   CGF.StartFunction(GlobalDecl(), Ctx.VoidTy, Fn, CGFI, WrapperArgs,
                     D.getBeginLoc(), D.getBeginLoc());
 
   const auto *RD = CS.getCapturedRecordDecl();
   auto CurField = RD->field_begin();
 
-  Address ZeroAddr = CGF.CreateDefaultAlignTempAlloca(CGF.Int32Ty,
-                                                      /*Name=*/".zero.addr");
+  Address ZeroAddr = CGF.CreateTempAllocaInAS(
+      CGF.Int32Ty, CGF.PreferredAlignmentForIRType(CGF.Int32Ty),
+      CGF.getASTAllocaAddressSpace(),
   CGF.InitTempAlloca(ZeroAddr, CGF.Builder.getInt32(/*C*/ 0));
   // Get the array of arguments.
   SmallVector<llvm::Value *, 8> Args;
 
   Args.emplace_back(CGF.GetAddrOfLocalVar(&WrapperArg).getPointer());
   Args.emplace_back(ZeroAddr.getPointer());
 
   CGBuilderTy &Bld = CGF.Builder;
   auto CI = CS.capture_begin();
 
   // Use global memory for data sharing.
   // Handle passing of global args to workers.
-  Address GlobalArgs =
-      CGF.CreateDefaultAlignTempAlloca(CGF.VoidPtrPtrTy, "global_args");
+  Address GlobalArgs = CGF.CreateTempAllocaInAS(
+      CGF.VoidPtrPtrTy, CGF.PreferredAlignmentForIRType(CGF.VoidPtrPtrTy),
+      CGF.getASTAllocaAddressSpace(), "global_args");
   llvm::Value *GlobalArgsPtr = GlobalArgs.getPointer();
   llvm::Value *DataSharingArgs[] = {GlobalArgsPtr};
   CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
                           CGM.getModule(), OMPRTL___kmpc_get_shared_variables),
                       DataSharingArgs);
 
   // Retrieve the shared variables from the list of references returned
   // by the runtime. Pass the variables to the outlined function.

clang/lib/CodeGen/CGStmt.cpp

if (getLangOpts().ElideConstructors && S.getNRVOCandidate() &&
             .getAddressOfLocalVariable(*this, S.getNRVOCandidate())
             .isValid())) {
     // Apply the named return value optimization for this return statement,
     // which means doing nothing: the appropriate result has already been
     // constructed into the NRVO variable.
 
     // If there is an NRVO flag for this variable, set it to 1 into indicate
     // that the cleanup code should not destroy the variable.
-    if (llvm::Value *NRVOFlag = NRVOFlags[S.getNRVOCandidate()])
-      Builder.CreateFlagStore(Builder.getTrue(), NRVOFlag);
+    const auto I = NRVOFlags.find(S.getNRVOCandidate());
+    if (I != NRVOFlags.end()) {
+      Address NRVOFlag = I->second;
+      Builder.CreateStore(Builder.getTrue(), NRVOFlag);
+    }
   } else if (!ReturnValue.isValid() || (RV && RV->getType()->isVoidType())) {
     // Make sure not to return anything, but evaluate the expression
     // for side effects.
     if (RV) {
       EmitAnyExpr(RV);
       if (auto *CE = dyn_cast<CallExpr>(RV))
         makeTailCallIfSwiftAsync(CE, Builder, CurFnInfo);
     }

clang/lib/CodeGen/CodeGenFunction.h

public:
   const CodeGen::CGBlockInfo *BlockInfo = nullptr;
   llvm::Value *BlockPointer = nullptr;
 
   llvm::DenseMap<const VarDecl *, FieldDecl *> LambdaCaptureFields;
   FieldDecl *LambdaThisCaptureField = nullptr;
 
   /// A mapping from NRVO variables to the flags used to indicate
   /// when the NRVO has been applied to this variable.
-  llvm::DenseMap<const VarDecl *, llvm::Value *> NRVOFlags;
+  llvm::DenseMap<const VarDecl *, Address> NRVOFlags;
 
   EHScopeStack EHStack;
   llvm::SmallVector<char, 256> LifetimeExtendedCleanupStack;
   llvm::SmallVector<const JumpDest *, 2> SEHTryEpilogueStack;
 
   llvm::Instruction *CurrentFuncletPad = nullptr;
 
   class CallLifetimeEnd final : public EHScopeStack::Cleanup {
public:
 
   unsigned NextCleanupDestIndex = 1;
 
   /// EHResumeBlock - Unified block containing a call to llvm.eh.resume.
   llvm::BasicBlock *EHResumeBlock = nullptr;
 
   /// The exception slot.  All landing pads write the current exception pointer
   /// into this alloca.
-  llvm::Value *ExceptionSlot = nullptr;
+  Address ExceptionSlot = Address::invalid();
 
   /// The selector slot.  Under the MandatoryCleanup model, all landing pads
   /// write the current selector value into this alloca.
-  llvm::AllocaInst *EHSelectorSlot = nullptr;
+  Address EHSelectorSlot = Address::invalid();
 
   /// A stack of exception code slots. Entering an __except block pushes a slot
   /// on the stack and leaving pops one. The __exception_code() intrinsic loads
   /// a value from the top of the stack.
   SmallVector<Address, 1> SEHCodeSlotStack;
 
   /// Value returned by __exception_info intrinsic.
   llvm::Value *SEHInfo = nullptr;
class FinallyInfo {
     /// Where the catchall's edge through the cleanup should go.
     JumpDest RethrowDest;
 
     /// A function to call to enter the catch.
     llvm::FunctionCallee BeginCatchFn;
 
     /// An i1 variable indicating whether or not the @finally is
     /// running for an exception.
-    llvm::AllocaInst *ForEHVar;
+    Address ForEHVar = Address::invalid();
 
     /// An i8* variable into which the exception pointer to rethrow
     /// has been saved.
-    llvm::AllocaInst *SavedExnVar;
+    Address SavedExnVar = Address::invalid();
 
   public:
     void enter(CodeGenFunction &CGF, const Stmt *Finally,
                llvm::FunctionCallee beginCatchFn,
                llvm::FunctionCallee endCatchFn, llvm::FunctionCallee rethrowFn);
     void exit(CodeGenFunction &CGF);
   };
 
LValue EmitLoadOfReferenceLValue(Address RefAddr, QualType RefTy,
     return EmitLoadOfReferenceLValue(RefLVal);
   }
 
   Address EmitLoadOfPointer(Address Ptr, const PointerType *PtrTy,
                             LValueBaseInfo *BaseInfo = nullptr,
                             TBAAAccessInfo *TBAAInfo = nullptr);
   LValue EmitLoadOfPointerLValue(Address Ptr, const PointerType *PtrTy);
 
-  /// CreateTempAlloca - This creates an alloca and inserts it into the entry
-  /// block if \p ArraySize is nullptr, otherwise inserts it at the current
-  /// insertion point of the builder. The caller is responsible for setting an
-  /// appropriate alignment on
-  /// the alloca.
+  /// CreateTempAllocaInAS - Create an alloca in \p AddressSpace with alignment
+  /// \p Align.
   ///
   /// \p ArraySize is the number of array elements to be allocated if it
-  ///    is not nullptr.
+  /// is not nullptr.
   ///
+  /// The alloca will be inserted into the entry block if \p ArraySize is
+  /// nullptr.  Otherwise it is inserted at the current insertion point of the
+  /// builder.
+  Address CreateTempAllocaInAS(llvm::Type *Ty, CharUnits Align,
+                               LangAS AddrSpace, const Twine &Name = "tmp",
+                               llvm::Value *ArraySize = nullptr);
+
+  /// CreateTempAlloca - Create an alloca with CreateTempAllocaInAS, then cast
+  /// the result to LangAS::Default if necessary.
+  ///
   /// LangAS::Default is the address space of pointers to local variables and
-  /// temporaries, as exposed in the source language. In certain
-  /// configurations, this is not the same as the alloca address space, and a
-  /// cast is needed to lift the pointer from the alloca AS into
-  /// LangAS::Default. This can happen when the target uses a restricted
-  /// address space for the stack but the source language requires
-  /// LangAS::Default to be a generic address space. The latter condition is
-  /// common for most programming languages; OpenCL is an exception in that
-  /// LangAS::Default is the private address space, which naturally maps
-  /// to the stack.
+  /// temporaries, as exposed in the source language. In certain configurations,
+  /// this is not the same as the alloca address space, and a cast is needed to
+  /// lift the pointer from the alloca AS into LangAS::Default. This can happen
+  /// when the target uses a restricted address space for the stack but the
+  /// source language requires LangAS::Default to be a generic address
+  /// space. The latter condition is common for most programming languages;
+  /// OpenCL is an exception in that LangAS::Default is the private address
+  /// space, which naturally maps to the stack.
   ///
   /// Because the address of a temporary is often exposed to the program in
   /// various ways, this function will perform the cast. The original alloca
   /// instruction is returned through \p Alloca if it is not nullptr.
   ///
-  /// The cast is not performaed in CreateTempAllocaWithoutCast. This is
-  /// more efficient if the caller knows that the address will not be exposed.
-  llvm::AllocaInst *CreateTempAlloca(llvm::Type *Ty, const Twine &Name = "tmp",
-                                     llvm::Value *ArraySize = nullptr);
-  Address CreateTempAlloca(llvm::Type *Ty, CharUnits align,
+  /// If the caller knows that the address will not be exposed, it is more
+  /// efficient to use CreateTempAllocaInAS instead, to avoid any unneeded
+  /// addrspace casts.
+  Address CreateTempAlloca(llvm::Type *Ty, CharUnits Align,
                            const Twine &Name = "tmp",
                            llvm::Value *ArraySize = nullptr,
                            Address *Alloca = nullptr);
-  Address CreateTempAllocaWithoutCast(llvm::Type *Ty, CharUnits align,
-                                      const Twine &Name = "tmp",
-                                      llvm::Value *ArraySize = nullptr);
 
-  /// CreateDefaultAlignedTempAlloca - This creates an alloca with the
-  /// default ABI alignment of the given LLVM type.
+  /// PreferredAlignmentForIRType - Return the preferred alignment for the IR
+  /// type \p Ty.
   ///
-  /// IMPORTANT NOTE: This is *not* generally the right alignment for
-  /// any given AST type that happens to have been lowered to the
-  /// given IR type.  This should only ever be used for function-local,
-  /// IR-driven manipulations like saving and restoring a value.  Do
-  /// not hand this address off to arbitrary IRGen routines, and especially
-  /// do not pass it as an argument to a function that might expect a
-  /// properly ABI-aligned value.
-  Address CreateDefaultAlignTempAlloca(llvm::Type *Ty,
-                                       const Twine &Name = "tmp");
+  /// IMPORTANT NOTE: This is *not* generally the right alignment for any given
+  /// AST type that happens to have been lowered to the given IR type.  This
+  /// should only ever be used for allocating function-local values used in
+  /// IR-driven manipulations like saving and restoring a value.  Do not use
+  /// this alignment for allocating arbitrary IRGen routines, and especially do
+  /// not use it to allocate values that might be passed to functions that
+  /// expect a properly ABI-aligned value.
+  CharUnits PreferredAlignmentForIRType(llvm::Type *Ty) {
+    return CharUnits::fromQuantity(
+        CGM.getDataLayout().getPrefTypeAlignment(Ty));
+  }
 
   /// InitTempAlloca - Provide an initial value for the given alloca which
   /// will be observable at all locations in the function.
   ///
   /// The address should be something that was returned from one of
   /// the CreateTempAlloca or CreateMemTemp routines, and the
   /// initializer must be valid in the entry block (i.e. it must
   /// either be a constant or an argument value).
class AutoVarEmission {
     const VarDecl *Variable;
 
     /// The address of the alloca for languages with explicit address space
     /// (e.g. OpenCL) or alloca casted to generic pointer for address space
     /// agnostic languages (e.g. C++). Invalid if the variable was emitted
     /// as a global constant.
     Address Addr;
 
-    llvm::Value *NRVOFlag;
+    Address NRVOFlag;
 
     /// True if the variable is a __block variable that is captured by an
     /// escaping block.
     bool IsEscapingByRef;
 
     /// True if the variable is of aggregate type and has a constant
     /// initializer.
     bool IsConstantAggregate;
 
     /// Non-null if we should use lifetime annotations.
     llvm::Value *SizeForLifetimeMarkers;
 
     /// Address with original alloca instruction. Invalid if the variable was
     /// emitted as a global constant.
     Address AllocaAddr;
 
     struct Invalid {};
     AutoVarEmission(Invalid)
         : Variable(nullptr), Addr(Address::invalid()),
-          AllocaAddr(Address::invalid()) {}
+          NRVOFlag(Address::invalid()), AllocaAddr(Address::invalid()) {}
 
     AutoVarEmission(const VarDecl &variable)
-        : Variable(&variable), Addr(Address::invalid()), NRVOFlag(nullptr),
-          IsEscapingByRef(false), IsConstantAggregate(false),
-          SizeForLifetimeMarkers(nullptr), AllocaAddr(Address::invalid()) {}
+        : Variable(&variable), Addr(Address::invalid()),
+          NRVOFlag(Address::invalid()), IsEscapingByRef(false),
+          IsConstantAggregate(false), SizeForLifetimeMarkers(nullptr),
+          AllocaAddr(Address::invalid()) {}
 
     bool wasEmittedAsGlobal() const { return !Addr.isValid(); }
 
   public:
     static AutoVarEmission invalid() { return AutoVarEmission(Invalid()); }
 
     bool useLifetimeMarkers() const {
       return SizeForLifetimeMarkers != nullptr;

clang/lib/CodeGen/CodeGenFunction.cpp

#undef SANITIZER
 
   ReturnBlock = getJumpDestInCurrentScope("return");
 
   Builder.SetInsertPoint(EntryBB);
 
   // If we're checking the return value, allocate space for a pointer to a
   // precise source location of the checked return statement.
   if (requiresReturnValueCheck()) {
-    ReturnLocation = CreateDefaultAlignTempAlloca(Int8PtrTy, "return.sloc.ptr");
+    CharUnits Align = PreferredAlignmentForIRType(Int8PtrTy);
+    LangAS AS = getASTAllocaAddressSpace();
+    ReturnLocation =
+        CreateTempAllocaInAS(Int8PtrTy, Align, AS, "return.sloc.ptr");
     InitTempAlloca(ReturnLocation, llvm::ConstantPointerNull::get(Int8PtrTy));
   }
 
   // Emit subprogram debug descriptor.
   if (CGDebugInfo *DI = getDebugInfo()) {
     // Reconstruct the type from the argument list so that implicit parameters,
     // such as 'this' and 'vtt', show up in the debug info. Preserve the calling
     // convention.
#undef SANITIZER
   } else if (CurFnInfo->getReturnInfo().getKind() == ABIArgInfo::Indirect) {
     // Indirect return; emit returned value directly into sret slot.
     // This reduces code size, and affects correctness in C++.
     auto AI = CurFn->arg_begin();
     if (CurFnInfo->getReturnInfo().isSRetAfterThis())
       ++AI;
     ReturnValue = Address(&*AI, CurFnInfo->getReturnInfo().getIndirectAlign());
     if (!CurFnInfo->getReturnInfo().getIndirectByVal()) {
+      CharUnits Align = PreferredAlignmentForIRType(Int8PtrTy);
+      LangAS AS = getASTAllocaAddressSpace();
       ReturnValuePointer =
-          CreateDefaultAlignTempAlloca(Int8PtrTy, "result.ptr");
+          CreateTempAllocaInAS(Int8PtrTy, Align, AS, "result.ptr");
       Builder.CreateStore(Builder.CreatePointerBitCastOrAddrSpaceCast(
                               ReturnValue.getPointer(), Int8PtrTy),
                           ReturnValuePointer);
     }
   } else if (CurFnInfo->getReturnInfo().getKind() == ABIArgInfo::InAlloca &&
              !hasScalarEvaluationKind(CurFnInfo->getReturnType())) {
     // Load the sret pointer from the argument struct and return into that.
     unsigned Idx = CurFnInfo->getReturnInfo().getInAllocaFieldIndex();