Allow DataLayout to specify addrspace for allocas.

[arsenm]

Summary

LLVM makes several assumptions about address space 0. However,
alloca is presently constrained to always return this address space.
There's no real way to avoid using alloca, so without this
there is no way to opt out of these assumptions.

The problematic assumptions include:

• That the pointer size used for the stack is the same size as the code size pointer, which is also the maximum sized pointer.

• That 0 is an invalid, non-dereferencable pointer value.

These are problems for AMDGPU because alloca is used to
implement the private address space, which uses a 32-bit
index as the pointer value. Other pointers are 64-bit
and behave more like LLVM's notion of generic address
space. By changing the address space used for allocas,
we can change our generic pointer type to be LLVM's generic
pointer type which does have similar properties.

[arsenm]

Print as , addrspace(num), but require it match the datalayout

[efriedma]

include/llvm/IR/IRBuilder.h

1106

I'm not sure this overload is a good idea; it saves a little typing, but it hides the reason we need the datalayout.

[arsenm]

include/llvm/IR/IRBuilder.h

1106

I'm not sure that's a problem here. This is the most common version, and the point of IRBuilder is to be the convenient way of creating IR. A DataLayout argument is less error prone than an unsigned argument, where it easier to accidentally pass the alignment or something else.

It would also be able to hide this more by getting the parent module's DataLayout from the insertion block (which is what the CHERI patches did). However there are already functions using DataLayout input arguments, so I figured there might be some reason to pass it in rather than doing that.

[efriedma]

include/llvm/IR/IRBuilder.h

1106

I guess. The part that concerns me is that the type of the result might not be what someone expects if they aren't paying attention, and the resulting assertion failure is a bit unfriendly... but I guess we can go with this for now, and see if it ends up being confusing in practice.

Getting the DataLayout from the insertion block doesn't work for all methods on IRBuilder; an IRBuilder isn't guaranteed to always have an insertion point. This can be used, for example, if you're building a constant. I guess for alloca in particular, it would be safe to assume we have an insertion point.

[arsenm]

ping

[efriedma]

LGTM except for a minor comment about the text in LangRef.

docs/LangRef.rst

1816

"the address space of the result for the stack" is a little unclear.

[arsenm]

docs/LangRef.rst

1816

I wasn't sure about referring to the stack at all. Should I rename everything added that says stack with alloca (e.g DL.getStackAddrSpace() -> DL.getAllocaAddrSpace()?)

[efriedma]

docs/LangRef.rst

1816

Well, in many contexts, it isn't precisely clear what the stack refers to... and there's a bunch of other stuff conventionally stored on the stack which could be stored elsewhere. Maybe better to just call it the address-space of allocas, I guess?

Even assuming the meaning of "the stack" is clear, the sentence is missing something: it doesn't mention allocation anywhere.

[arsenm]

Change LangRef description, don't call stack

[efriedma]

LGTM, but I'd to see a second approval for an IR change like this.

[rjmccall]

What else in LLVM actually hard-codes 0 as the assumed address space? Intrinsic function declarations? Optimizations that create new globals (if there are any?) Wouldn't it be much less invasive to change those and leave the assumption that the stack is in addrspace 0?

[arsenm]

In D31042#711596, @rjmccall wrote:

What else in LLVM actually hard-codes 0 as the assumed address space? Intrinsic function declarations? Optimizations that create new globals (if there are any?) Wouldn't it be much less invasive to change those and leave the assumption that the stack is in addrspace 0?

I'm not sure what you are suggesting. The concept of assumed address space doesn't make any sense on AMDGPU. Everything has an address space. This isn't changing how address space 0 is treated, and is specifically avoiding doing so. AMDGPU does want the assumed address space 0 properties, just for some of the other address spaces that are not the stack address space, which is one of the advantages of making this change. The problem isn't really places hard coding 0 as the address space, it's the properties of the stack address space do not behave like LLVM's notion of the generic address space as I mentioned in the RFC.

As far as 0 being a valid stack pointer, we currently have a workaround of not allocating user objects there. The bigger issues in my mind is that the pointer size for the stack does not match the code pointer size, which is a problematic area which keeps popping up. Another is it is not valid to addrspacecast any other address space to the stack address space. One partial alternative would be to introduce a code pointer size to the datalayout or code address space or something along those lines. I think that would still leave a conceptual gap between AMDGPU's stack and alloca, and be more of a hassle. (For example you still cannot addrspacecast any pointer to the address space of the stack).

There are some intrinsics which hard code 0, some of which need to be fixed (such as lifetime intrinsics which D31041 and D31043 take care of) but other than that one that's mostly a different question. There is a jump table optimization which creates new globals (which is disabled if BR_JT is illegal). Moving AMDGPU's flat address space to match LLVM's generic address space would also have the advantage of allowing legal addrspacecasts to address space 0 for some of these intrinsics.

[rjmccall]

In D31042#711611, @arsenm wrote:

In D31042#711596, @rjmccall wrote:

What else in LLVM actually hard-codes 0 as the assumed address space? Intrinsic function declarations? Optimizations that create new globals (if there are any?) Wouldn't it be much less invasive to change those and leave the assumption that the stack is in addrspace 0?

I'm not sure what you are suggesting. The concept of assumed address space doesn't make any sense on AMDGPU. Everything has an address space. This isn't changing how address space 0 is treated, and is specifically avoiding doing so. AMDGPU does want the assumed address space 0 properties, just for some of the other address spaces that are not the stack address space, which is one of the advantages of making this change. The problem isn't really places hard coding 0 as the address space, it's the properties of the stack address space do not behave like LLVM's notion of the generic address space as I mentioned in the RFC.

As far as 0 being a valid stack pointer, we currently have a workaround of not allocating user objects there. The bigger issues in my mind is that the pointer size for the stack does not match the code pointer size, which is a problematic area which keeps popping up. Another is it is not valid to addrspacecast any other address space to the stack address space. One partial alternative would be to introduce a code pointer size to the datalayout or code address space or something along those lines. I think that would still leave a conceptual gap between AMDGPU's stack and alloca, and be more of a hassle. (For example you still cannot addrspacecast any pointer to the address space of the stack).

There are some intrinsics which hard code 0, some of which need to be fixed (such as lifetime intrinsics which D31041 and D31043 take care of) but other than that one that's mostly a different question. There is a jump table optimization which creates new globals (which is disabled if BR_JT is illegal). Moving AMDGPU's flat address space to match LLVM's generic address space would also have the advantage of allowing legal addrspacecasts to address space 0 for some of these intrinsics.

You keep talking about LLVM having a "generic address space". I'm trying to figure out what you mean by that, because in LLVM, global values can be declared in an arbitrary address space, pointer values all carry an address space, instructions work on arbitrary address spaces, the memory intrinsics are generally overloaded by pointer address spaces, etc. And in general, the idea that there exists a generic address space which subsumes every other address space is just wrong. As an example, on x86-64 we use address spaces as a way of requesting segmented addressing; even if the memory there was addressable in the standard segment (which is not guaranteed), AFAIK there's no way to actually perform that conversion.

Given that, there's nothing wrong with LLVM deciding to hard-code a specific address space index as the address space of the stack. That's worth doing if it significantly simplifies the IR, e.g. by not requiring a data layout to be passed to CreateAlloca. It just means that frontends that *do* assume there's a generic address space may need to insert addrspace conversions in some cases.

it's okay to call out some specific address space, like addrspace 0, and say that it's always the address space of the stack.

I certainly hope that nothing in LLVM is ever implicitly inserting addrspace conversions or assuming anything about them without target-specific information. In general, it is not true that there exists an address space that subsumes all others.

[rjmccall]

In D31042#711627, @rjmccall wrote:

In D31042#711611, @arsenm wrote:

In D31042#711596, @rjmccall wrote:

What else in LLVM actually hard-codes 0 as the assumed address space? Intrinsic function declarations? Optimizations that create new globals (if there are any?) Wouldn't it be much less invasive to change those and leave the assumption that the stack is in addrspace 0?

I'm not sure what you are suggesting. The concept of assumed address space doesn't make any sense on AMDGPU. Everything has an address space. This isn't changing how address space 0 is treated, and is specifically avoiding doing so. AMDGPU does want the assumed address space 0 properties, just for some of the other address spaces that are not the stack address space, which is one of the advantages of making this change. The problem isn't really places hard coding 0 as the address space, it's the properties of the stack address space do not behave like LLVM's notion of the generic address space as I mentioned in the RFC.

As far as 0 being a valid stack pointer, we currently have a workaround of not allocating user objects there. The bigger issues in my mind is that the pointer size for the stack does not match the code pointer size, which is a problematic area which keeps popping up. Another is it is not valid to addrspacecast any other address space to the stack address space. One partial alternative would be to introduce a code pointer size to the datalayout or code address space or something along those lines. I think that would still leave a conceptual gap between AMDGPU's stack and alloca, and be more of a hassle. (For example you still cannot addrspacecast any pointer to the address space of the stack).

There are some intrinsics which hard code 0, some of which need to be fixed (such as lifetime intrinsics which D31041 and D31043 take care of) but other than that one that's mostly a different question. There is a jump table optimization which creates new globals (which is disabled if BR_JT is illegal). Moving AMDGPU's flat address space to match LLVM's generic address space would also have the advantage of allowing legal addrspacecasts to address space 0 for some of these intrinsics.

You keep talking about LLVM having a "generic address space". I'm trying to figure out what you mean by that, because in LLVM, global values can be declared in an arbitrary address space, pointer values all carry an address space, instructions work on arbitrary address spaces, the memory intrinsics are generally overloaded by pointer address spaces, etc. And in general, the idea that there exists a generic address space which subsumes every other address space is just wrong. As an example, on x86-64 we use address spaces as a way of requesting segmented addressing; even if the memory there was addressable in the standard segment (which is not guaranteed), AFAIK there's no way to actually perform that conversion.

Given that, there's nothing wrong with LLVM deciding to hard-code a specific address space index as the address space of the stack. That's worth doing if it significantly simplifies the IR, e.g. by not requiring a data layout to be passed to CreateAlloca. It just means that frontends that *do* assume there's a generic address space may need to insert addrspace conversions in some cases.

it's okay to call out some specific address space, like addrspace 0, and say that it's always the address space of the stack.

I certainly hope that nothing in LLVM is ever implicitly inserting addrspace conversions or assuming anything about them without target-specific information. In general, it is not true that there exists an address space that subsumes all others.

I'd meant to edit out these last two paragraphs; alas, I'm making these comments through Phabricator, which uses a very small text edit box, and I just failed to see that there was more text below. That's why they seem to be redundant with the earlier comments, but at least they're not contradictory.

[arsenm]

In D31042#711627, @rjmccall wrote:

You keep talking about LLVM having a "generic address space". I'm trying to figure out what you mean by that, because in LLVM, global values can be declared in an arbitrary address space, pointer values all carry an address space, instructions work on arbitrary address spaces, the memory intrinsics are generally overloaded by pointer address spaces, etc. And in general, the idea that there exists a generic address space which subsumes every other address space is just wrong. As an example, on x86-64 we use address spaces as a way of requesting segmented addressing; even if the memory there was addressable in the standard segment (which is not guaranteed), AFAIK there's no way to actually perform that conversion.

Given that, there's nothing wrong with LLVM deciding to hard-code a specific address space index as the address space of the stack. That's worth doing if it significantly simplifies the IR, e.g. by not requiring a data layout to be passed to CreateAlloca. It just means that frontends that *do* assume there's a generic address space may need to insert addrspace conversions in some cases.

it's okay to call out some specific address space, like addrspace 0, and say that it's always the address space of the stack.

I certainly hope that nothing in LLVM is ever implicitly inserting addrspace conversions or assuming anything about them without target-specific information. In general, it is not true that there exists an address space that subsumes all others.

LLVM treats 0 as the "generic" address space. This is how the LangRef refers to it. In the AMDGPU sense this is a pointer that can be used to access any memory. For AMDGPU, globals declared without an address space are invalid. All objects must have a definite address space.

There have been bugs in the past from random places inserting addrspacecasts but I don't think I've seen one recently

[rjmccall]

In D31042#711648, @arsenm wrote:

LLVM treats 0 as the "generic" address space. This is how the LangRef refers to it.

You're revising LangRef here one way or the other. I'm suggesting that a better revision would be to stop saying that address space 0 is the "generic" address space (because there is no such thing), and instead say that it is the address space of the stack. Is there anything actually preventing you from doing that?

[arsenm]

In D31042#711661, @rjmccall wrote:

In D31042#711648, @arsenm wrote:

LLVM treats 0 as the "generic" address space. This is how the LangRef refers to it.

You're revising LangRef here one way or the other. I'm suggesting that a better revision would be to stop saying that address space 0 is the "generic" address space (because there is no such thing), and instead say that it is the address space of the stack. Is there anything actually preventing you from doing that?

Yes. The pointer size of the stack smaller than the pointer size for code or most other memory. 0 is a valid, dereferencable pointer for the stack address space

[rjmccall]

In D31042#711662, @arsenm wrote:

In D31042#711661, @rjmccall wrote:

In D31042#711648, @arsenm wrote:

LLVM treats 0 as the "generic" address space. This is how the LangRef refers to it.

You're revising LangRef here one way or the other. I'm suggesting that a better revision would be to stop saying that address space 0 is the "generic" address space (because there is no such thing), and instead say that it is the address space of the stack. Is there anything actually preventing you from doing that?

Yes. The pointer size of the stack smaller than the pointer size for code or most other memory. 0 is a valid, dereferencable pointer for the stack address space

I don't see why the pointer size has anything to do with it. In the absence of target-specific information, LLVM is not allowed to assume that one address space is subsumed by another address space at all. It doesn't matter if one of them is address space 0.

The special treatment of zero pointers is an interesting point, but again, I feel like that transformation could be made target-dependent *much* more easily than this. This is an incredibly invasive change.

[arsenm]

In D31042#711771, @rjmccall wrote:

In D31042#711662, @arsenm wrote:

In D31042#711661, @rjmccall wrote:

In D31042#711648, @arsenm wrote:

LLVM treats 0 as the "generic" address space. This is how the LangRef refers to it.

You're revising LangRef here one way or the other. I'm suggesting that a better revision would be to stop saying that address space 0 is the "generic" address space (because there is no such thing), and instead say that it is the address space of the stack. Is there anything actually preventing you from doing that?

Yes. The pointer size of the stack smaller than the pointer size for code or most other memory. 0 is a valid, dereferencable pointer for the stack address space

I don't see why the pointer size has anything to do with it. In the absence of target-specific information, LLVM is not allowed to assume that one address space is subsumed by another address space at all. It doesn't matter if one of them is address space 0.

The special treatment of zero pointers is an interesting point, but again, I feel like that transformation could be made target-dependent *much* more easily than this. This is an incredibly invasive change.

I think being able to opt out of any assumptions is CHERI's use case for this patch, but I don't work on that so I'm not sure.

The relationships between different address spaces is only indirectly related to adding an address space to alloca, and is more a useful side effect. I think LLVM's assumptions about pointers and address space 0 (and worse so, clang) are more widespread than you think. There is no concept of a code pointer size. This is one area where address space 0 is treated as a lack of an address space. The code size and stack pointer do not have the same size for AMDGPU, and it is important to be able to distinguish these. There is a lot of code that assumes the default address space size is "the one true pointer size". You can't pass around a 64-bit function pointer properly if it's required to have the same properties as a 32-bit stack pointer. It isn't really OK to start treating function pointer types differently than other types. Fixing that is an uphill battle in the frontend and backend. It is less work to align the AMDGPU generic address space notion with LLVM's "default" address space and the assumptions made about it than to fix everything else. Adding address spaces to functions would be a much more invasive change. I don't think adding this DataLayout argument is very invasive, and was less work than I anticipated (The IRBuilder changes could also be avoided by getting the DataLayout from the insertion block's parent module). There is no big semantic change, just simple bookkeeping updates.

The pressing reason for this is C++ GPU support. Clang's handling of address spaces is more problematic. The assumptions about address space 0 are more widespread and tied to LLVM's address space 0. The work to start annotating every pointer there as required would be a much more invasive change.

[rjmccall]

The relationships between different address spaces is only indirectly related to adding an address space to alloca, and is more a useful side effect. I think LLVM's assumptions about pointers and address space 0 (and worse so, clang) are more widespread than you think. There is no concept of a code pointer size.

Well, address space is carried by PointerType, so it isn't true that LLVM IR has no concept of code pointer sizes. But I hadn't realized that Function doesn't have a way to actually set an address space on creation; I can see how that would make that change more involved. On the other hand, it seems like something that could easily just be added as a default argument to a couple of APIs; I don't see why *that's* seen as an invasive change but adding a new mandatory argument to the interfaces for creating one of the most common instruction classes is not.

This is one area where address space 0 is treated as a lack of an address space. The code size and stack pointer do not have the same size for AMDGPU, and it is important to be able to distinguish these.

Yes, I did understand that the first time it was explained.

There is a lot of code that assumes the default address space size is "the one true pointer size".

Are you suggesting that LLVM's address space 0 must always be at least as large as every other address space? Where is that assumption made? Again, that seems like something that should be fixed independent of what we decide here.

You can't pass around a 64-bit function pointer properly if it's required to have the same properties as a 32-bit stack pointer. It isn't really OK to start treating function pointer types differently than other types.

Of course. I am not suggesting that we should do that.

I don't think adding this DataLayout argument is very invasive, and was less work than I anticipated

That's because you aren't changing any of the frontends, most of which are not in-tree. I understand that LLVM does not promise to keep the C++ API stable, but there's surely *some* responsibility to not break every frontend for features they don't care about. At the very least, yes, you should be picking up this value from the Module instead of requiring it to be passed in.

The pressing reason for this is C++ GPU support. Clang's handling of address spaces is more problematic. The assumptions about address space 0 are more widespread and tied to LLVM's address space 0.

Clang's assumptions about address space 0 are pervasive because it represents an actual concept in the source language: C has a generic address space. My point is that LLVM generally should not.

I don't think the assumptions in Clang that the Clang AST address space 0 equals LLVM's address space 0 are nearly as pervasive as you think. At any rate, since you require stack allocations to be in a different LLVM address space from the generic address space, but (per C) taking the address of a local variable yields a pointer in the generic address space, you're going to have to teach Clang about these differences between address spaces anyway, because it'll have to insert address-space conversions in a number of places.

[arsenm]

In D31042#711863, @rjmccall wrote:

There is a lot of code that assumes the default address space size is "the one true pointer size".

Are you suggesting that LLVM's address space 0 must always be at least as large as every other address space? Where is that assumption made? Again, that seems like something that should be fixed independent of what we decide here.

Not necessarily, it's just an occasional sticking point in some remaining that code isn't aware of multiple pointer sizes. One area was in debug info. Patches to fix that were rejected a few times in the past. If you grep for argumentless getPointerSizes in CodeGen+MC, most of them seem to be for debug info. I have another patch I've been holding on to which works around the DAG picking the wrong value type for pointers to function types. Pretty much everything in the DAG for dealing specifically with pointer types is unworkable with different sized pointers, but most of that is possible to avoid using.

You can't pass around a 64-bit function pointer properly if it's required to have the same properties as a 32-bit stack pointer. It isn't really OK to start treating function pointer types differently than other types.

Of course. I am not suggesting that we should do that.

I don't think adding this DataLayout argument is very invasive, and was less work than I anticipated

That's because you aren't changing any of the frontends, most of which are not in-tree. I understand that LLVM does not promise to keep the C++ API stable, but there's surely *some* responsibility to not break every frontend for features they don't care about. At the very least, yes, you should be picking up this value from the Module instead of requiring it to be passed in.

So your objection is just to the API change? I can change that if you insist. The actual mechanical adding of the argument isn't the source of the work. As someone who has maintained out of tree uses, this class of change is the least irritating. Most of the clang IR gen work is in updating the downstream users which use hardcoded pointer types, which users not using this don't need to care about it.

The pressing reason for this is C++ GPU support. Clang's handling of address spaces is more problematic. The assumptions about address space 0 are more widespread and tied to LLVM's address space 0.

Clang's assumptions about address space 0 are pervasive because it represents an actual concept in the source language: C has a generic address space. My point is that LLVM generally should not.

I don't think the assumptions in Clang that the Clang AST address space 0 equals LLVM's address space 0 are nearly as pervasive as you think. At any rate, since you require stack allocations to be in a different LLVM address space from the generic address space, but (per C) taking the address of a local variable yields a pointer in the generic address space, you're going to have to teach Clang about these differences between address spaces anyway, because it'll have to insert address-space conversions in a number of places.

Yes, I expect there will be many addrspacecasts inserted from the alloca, which then the InferAddressSpaces pass would hopefully optimize away.

[rjmccall]

In D31042#711913, @arsenm wrote:

In D31042#711863, @rjmccall wrote:

I don't think the assumptions in Clang that the Clang AST address space 0 equals LLVM's address space 0 are nearly as pervasive as you think. At any rate, since you require stack allocations to be in a different LLVM address space from the generic address space, but (per C) taking the address of a local variable yields a pointer in the generic address space, you're going to have to teach Clang about these differences between address spaces anyway, because it'll have to insert address-space conversions in a number of places.

Yes, I expect there will be many addrspacecasts inserted from the alloca, which then the InferAddressSpaces pass would hopefully optimize away.

Okay. So, to summarize:

In amdgpu OpenCL, at source level, sizeof(void private *) == sizeof(void {global,local,constant} *) == 8. However, at an implementation level, stack addresses are actually 32-bit, and you want that to be modeled correctly in the IR. Therefore, allocas need to return a value in a 32-bit address space, which the frontend will immediately widen to a 64-bit address space in order to match the rules for __private. In order to maintain performance, you have a pass that recognizes when an address value is the result of that kind of widening and re-narrows it to the 32-bit stack address space.

Because LLVM currently makes some unfortunate assumptions about address space 0, this stack address space cannot be address space 0. These assumptions include:
(1) As far as debug info is concerned, the size of a pointer is the size of a pointer in address space 0. (Is this assumption why sizeof(void private *) == 8 despite private being clearly intended in the OpenCL spec to be the address space of stack pointers?)
(2) Functions are currently always defined in address space 0. A function pointer in your implementation must be in a 64-bit address space. (My understanding was that OpenCL generally forbade function pointers, so I'm not sure if this is actually important?)
(3) Some optimizations assume that 0 is an invalid address in address space 0, but in fact it's a valid stack address.

It's entirely possible that we should, indeed, change all of these assumptions. For the time being, you feel that's not a reasonable fight, and so you would like to allow IR to use an explicitly non-0 address space for the stack.

I think I'm prepared to accept your argument.

I don't think adding this DataLayout argument is very invasive, and was less work than I anticipated

That's because you aren't changing any of the frontends, most of which are not in-tree. I understand that LLVM does not promise to keep the C++ API stable, but there's surely *some* responsibility to not break every frontend for features they don't care about. At the very least, yes, you should be picking up this value from the Module instead of requiring it to be passed in.

So your objection is just to the API change? I can change that if you insist. The actual mechanical adding of the argument isn't the source of the work. As someone who has maintained out of tree uses, this class of change is the least irritating. Most of the clang IR gen work is in updating the downstream users which use hardcoded pointer types, which users not using this don't need to care about it.

Yes, I would like you to avoid changing the IRBuilder API if you can. I can see why you need to change the AllocaInst constructors, though.

Do you actually need this to be printed/parsed in .ll files? It's globally consistent across all alloca instructions, and the data layout is always parsed before anything else in the file. It's not the end of the world, just seems odd to imply that this is actually something that can be different on different instructions.

[efriedma]

Do you actually need this to be printed/parsed in .ll files? It's globally consistent across all alloca instructions, and the data layout is always parsed before anything else in the file. It's not the end of the world, just seems odd to imply that this is actually something that can be different on different instructions.

No, strictly speaking it isn't necessary: every module has a datalayout, so we can compute the type from that. I requested it to make debugging more straightforward. Without it, there isn't any obvious indication if you dump() an instruction or look at an IR file that the address-space of the result is non-zero.

[rjmccall]

In D31042#712292, @efriedma wrote:

Do you actually need this to be printed/parsed in .ll files? It's globally consistent across all alloca instructions, and the data layout is always parsed before anything else in the file. It's not the end of the world, just seems odd to imply that this is actually something that can be different on different instructions.

No, strictly speaking it isn't necessary: every module has a datalayout, so we can compute the type from that. I requested it to make debugging more straightforward. Without it, there isn't any obvious indication if you dump() an instruction or look at an IR file that the address-space of the result is non-zero.

Well, since IR assembly is primarily a debugging / testing aid, and you won't get similar test output anyway because the address space will show up in every use of the instruction, I suppose it's fine.

[arsenm]

In D31042#712124, @rjmccall wrote:

In D31042#711913, @arsenm wrote:

In D31042#711863, @rjmccall wrote:

I don't think the assumptions in Clang that the Clang AST address space 0 equals LLVM's address space 0 are nearly as pervasive as you think. At any rate, since you require stack allocations to be in a different LLVM address space from the generic address space, but (per C) taking the address of a local variable yields a pointer in the generic address space, you're going to have to teach Clang about these differences between address spaces anyway, because it'll have to insert address-space conversions in a number of places.

Yes, I expect there will be many addrspacecasts inserted from the alloca, which then the InferAddressSpaces pass would hopefully optimize away.

Okay. So, to summarize:

In amdgpu OpenCL, at source level, sizeof(void private *) == sizeof(void {global,local,constant} *) == 8. However, at an implementation level, stack addresses are actually 32-bit, and you want that to be modeled correctly in the IR. Therefore, allocas need to return a value in a 32-bit address space, which the frontend will immediately widen to a 64-bit address space in order to match the rules for __private. In order to maintain performance, you have a pass that recognizes when an address value is the result of that kind of widening and re-narrows it to the 32-bit stack address space.

This is mostly accurate except for the detail about the cast. For OpenCL 2.0 or C++ the alloca pointer will most likely need to be addrspacecasted to the OpenCL generic address space. Private pointers in memory may need to be zero extended to 64-bit in memory. For OpenCL 1.x there is no generic address space, and the old subtargets don't have the necessary hardware features to support this.This is one of reasons I don't want to use the same workaround NVPTX uses for this problem. There would be a pointer that if not optimized out would require creative codegen work to artificially support it on those targets.

Because LLVM currently makes some unfortunate assumptions about address space 0, this stack address space cannot be address space 0. These assumptions include:
(1) As far as debug info is concerned, the size of a pointer is the size of a pointer in address space 0. (Is this assumption why sizeof(void private *) == 8 despite private being clearly intended in the OpenCL spec to be the address space of stack pointers?)

This is a point that the spec is vague on. While the spec explicitly allows different size pointers for different address spaces, there isn't really any detailed description of what this entails. This wasn't addressed at all by the SPIR spec, which just defined a single 64-bit datalayout string. sizeof really indicates the in-memory representation's size, so by using a DataLayout with 32-bit pointers with an ABI alignment of 64-bit, we get the necessary codegen properties of a 32-bit pointer while maintaining the required ABI and single sizeof().

(2) Functions are currently always defined in address space 0. A function pointer in your implementation must be in a 64-bit address space. (My understanding was that OpenCL generally forbade function pointers, so I'm not sure if this is actually important?)

It is true that OpenCL doesn't allow function pointers at the source level. I'm working on implementing calls without inlining everything now, and to be able to correctly lower LLVM IR for them requires the DAG picking the right MVT when emitting the global address so it still comes up there. Function pointer support is also helpful for implementing other languages being worked on like C++ and python. I believe recent version of CUDA support them as well.

(3) Some optimizations assume that 0 is an invalid address in address space 0, but in fact it's a valid stack address.

It's entirely possible that we should, indeed, change all of these assumptions. For the time being, you feel that's not a reasonable fight, and so you would like to allow IR to use an explicitly non-0 address space for the stack.

Yes. Some day I would like to be able to apply some of these assumptions to other non-0 address spaces as well (i.e. 0 is null in some address spaces, but not others) but I think that is a larger task.

I think I'm prepared to accept your argument.

I don't think adding this DataLayout argument is very invasive, and was less work than I anticipated

That's because you aren't changing any of the frontends, most of which are not in-tree. I understand that LLVM does not promise to keep the C++ API stable, but there's surely *some* responsibility to not break every frontend for features they don't care about. At the very least, yes, you should be picking up this value from the Module instead of requiring it to be passed in.

So your objection is just to the API change? I can change that if you insist. The actual mechanical adding of the argument isn't the source of the work. As someone who has maintained out of tree uses, this class of change is the least irritating. Most of the clang IR gen work is in updating the downstream users which use hardcoded pointer types, which users not using this don't need to care about it.

Yes, I would like you to avoid changing the IRBuilder API if you can. I can see why you need to change the AllocaInst constructors, though.

Do you actually need this to be printed/parsed in .ll files? It's globally consistent across all alloca instructions, and the data layout is always parsed before anything else in the file. It's not the end of the world, just seems odd to imply that this is actually something that can be different on different instructions.

No, this was just a request in the RFC thread to make it more clear. I can maybe envision a use case in the future for having a different address space on individual allocas, but that is not a concern I have today.

[arsenm]

Don't change IRBuilder API

[rjmccall]

In D31042#712324, @arsenm wrote:

In D31042#712124, @rjmccall wrote:

In D31042#711913, @arsenm wrote:

In D31042#711863, @rjmccall wrote:

I don't think the assumptions in Clang that the Clang AST address space 0 equals LLVM's address space 0 are nearly as pervasive as you think. At any rate, since you require stack allocations to be in a different LLVM address space from the generic address space, but (per C) taking the address of a local variable yields a pointer in the generic address space, you're going to have to teach Clang about these differences between address spaces anyway, because it'll have to insert address-space conversions in a number of places.

Yes, I expect there will be many addrspacecasts inserted from the alloca, which then the InferAddressSpaces pass would hopefully optimize away.

Okay. So, to summarize:

In amdgpu OpenCL, at source level, sizeof(void private *) == sizeof(void {global,local,constant} *) == 8. However, at an implementation level, stack addresses are actually 32-bit, and you want that to be modeled correctly in the IR. Therefore, allocas need to return a value in a 32-bit address space, which the frontend will immediately widen to a 64-bit address space in order to match the rules for __private. In order to maintain performance, you have a pass that recognizes when an address value is the result of that kind of widening and re-narrows it to the 32-bit stack address space.

This is mostly accurate except for the detail about the cast. For OpenCL 2.0 or C++ the alloca pointer will most likely need to be addrspacecasted to the OpenCL generic address space.

It looks to me like the language spec says that the address of a local variable is in the __private address space, but that a pointer in any address space can be promoted into the generic address space. Clang's AST should call out that second promotion as a separate ImplicitCastExpr. Of course, you may decide that you want to peephole it into a single addrspacecast instruction.

Private pointers in memory may need to be zero extended to 64-bit in memory. For OpenCL 1.x there is no generic address space, and the old subtargets don't have the necessary hardware features to support this.

By "this", you mean an efficiently-accessible generic address space? Presumably because the different memory-access operations work on specific address spaces and so, as you say, you would need creative codegen to figure out which case the pointer originally belonged to. That makes sense.

Because LLVM currently makes some unfortunate assumptions about address space 0, this stack address space cannot be address space 0. These assumptions include:
(1) As far as debug info is concerned, the size of a pointer is the size of a pointer in address space 0. (Is this assumption why sizeof(void private *) == 8 despite private being clearly intended in the OpenCL spec to be the address space of stack pointers?)

This is a point that the spec is vague on. While the spec explicitly allows different size pointers for different address spaces, there isn't really any detailed description of what this entails.

I don't see what the problem is. The spec doesn't allow pointers to be converted between address spaces at all (except for the promotion into the generic address space in OpenCL 2.0). Pointers into different address spaces are just completely different types.

I mean, in general, the OpenCL spec is rather hilariously poorly-drafted, and as a compiler programmer you have to read between the lines to figure out how things are supposed to be generalized. The old 1.1 spec used to say that "variables declared as pointers are assumed to point to the __private address space if an address space qualifier is not specified", and everyone just assumes that that's meant to be a general rule for pointer types, not something that only applies to variables and only at the outermost level of pointer. Similarly, you have to read the paragraph about assignability as a general conversion rule, and you have to guess that probably when the 2.0 specification says that "[c]asting a pointer to address space A to a pointer to address space B is illegal if A and B are named address spaces and A is not the same as B", that it's okay to both cast into *and out of* the generic address space, and you have to guess at what the semantics of the latter are supposed to be. The best guidance I can offer is to instead read the Embedded C specification (WG14 N1169 might be the most version), which goes into address spaces somewhat more rigorously; it's what I've always used as a baseline for understanding what OpenCL is trying to do. From this point of view, the four OpenCL 1.0 address spaces are disjoint, and OpenCL 2.0's generic address space is simply one that is a superset of all the others. Embedded C allows casts between address spaces, but the cast has undefined behavior if the pointer doesn't actually point into the destination address space. Since the four named address spaces are formally disjoint, casts directly between them are always invalid, and so OpenCL's rule that they're just forbidden makes sense. Anyway, that's what I would suggest as an approach for understanding OpenCL.

This wasn't addressed at all by the SPIR spec, which just defined a single 64-bit datalayout string. sizeof really indicates the in-memory representation's size, so by using a DataLayout with 32-bit pointers with an ABI alignment of 64-bit, we get the necessary codegen properties of a 32-bit pointer while maintaining the required ABI and single sizeof().

Well, SPIR is an abstract representation. If you were emitting SPIR, you wouldn't want any of these assumptions about stack addressing to leak into it. If you're lowering from SPIR, of course, you just have to invent an ABI rule for how your 32-bit pointers are passed around as 64-bit quantities, and then you globally rewrite the IR to implement that.

(2) Functions are currently always defined in address space 0. A function pointer in your implementation must be in a 64-bit address space. (My understanding was that OpenCL generally forbade function pointers, so I'm not sure if this is actually important?)

It is true that OpenCL doesn't allow function pointers at the source level. I'm working on implementing calls without inlining everything now, and to be able to correctly lower LLVM IR for them requires the DAG picking the right MVT when emitting the global address so it still comes up there. Function pointer support is also helpful for implementing other languages being worked on like C++ and python. I believe recent version of CUDA support them as well.

Well, if you don't have real function pointers, and therefore the only use of functions is in direct calls, it feels like you can always hack around pointer-size problems in your backend pretty easily. If the source language has real function pointers, it also has to decide what address space they're in — it could be a dedicated address space for functions, which would be legal under C rules, but which would take some effort to support in Clang.

(3) Some optimizations assume that 0 is an invalid address in address space 0, but in fact it's a valid stack address.

It's entirely possible that we should, indeed, change all of these assumptions. For the time being, you feel that's not a reasonable fight, and so you would like to allow IR to use an explicitly non-0 address space for the stack.

Yes. Some day I would like to be able to apply some of these assumptions to other non-0 address spaces as well (i.e. 0 is null in some address spaces, but not others) but I think that is a larger task.

Yes, that seems like a good goal. In fact, it would be nice if the assumptions about address space 0 were just a default behavior that could be overridden by the data layout.

[arsenm]

In D31042#712484, @rjmccall wrote:

In D31042#712324, @arsenm wrote:

In D31042#712124, @rjmccall wrote:

In D31042#711913, @arsenm wrote:

In D31042#711863, @rjmccall wrote:

I don't think the assumptions in Clang that the Clang AST address space 0 equals LLVM's address space 0 are nearly as pervasive as you think. At any rate, since you require stack allocations to be in a different LLVM address space from the generic address space, but (per C) taking the address of a local variable yields a pointer in the generic address space, you're going to have to teach Clang about these differences between address spaces anyway, because it'll have to insert address-space conversions in a number of places.

Yes, I expect there will be many addrspacecasts inserted from the alloca, which then the InferAddressSpaces pass would hopefully optimize away.

Okay. So, to summarize:

In amdgpu OpenCL, at source level, sizeof(void private *) == sizeof(void {global,local,constant} *) == 8. However, at an implementation level, stack addresses are actually 32-bit, and you want that to be modeled correctly in the IR. Therefore, allocas need to return a value in a 32-bit address space, which the frontend will immediately widen to a 64-bit address space in order to match the rules for __private. In order to maintain performance, you have a pass that recognizes when an address value is the result of that kind of widening and re-narrows it to the 32-bit stack address space.

This is mostly accurate except for the detail about the cast. For OpenCL 2.0 or C++ the alloca pointer will most likely need to be addrspacecasted to the OpenCL generic address space.

It looks to me like the language spec says that the address of a local variable is in the __private address space, but that a pointer in any address space can be promoted into the generic address space. Clang's AST should call out that second promotion as a separate ImplicitCastExpr. Of course, you may decide that you want to peephole it into a single addrspacecast instruction.

Private pointers in memory may need to be zero extended to 64-bit in memory. For OpenCL 1.x there is no generic address space, and the old subtargets don't have the necessary hardware features to support this.

By "this", you mean an efficiently-accessible generic address space? Presumably because the different memory-access operations work on specific address spaces and so, as you say, you would need creative codegen to figure out which case the pointer originally belonged to. That makes sense.

Yes

Because LLVM currently makes some unfortunate assumptions about address space 0, this stack address space cannot be address space 0. These assumptions include:
(1) As far as debug info is concerned, the size of a pointer is the size of a pointer in address space 0. (Is this assumption why sizeof(void private *) == 8 despite private being clearly intended in the OpenCL spec to be the address space of stack pointers?)

This is a point that the spec is vague on. While the spec explicitly allows different size pointers for different address spaces, there isn't really any detailed description of what this entails.

I don't see what the problem is. The spec doesn't allow pointers to be converted between address spaces at all (except for the promotion into the generic address space in OpenCL 2.0). Pointers into different address spaces are just completely different types.

The problem is mostly what happens if the host and device pointer size don't match? Struct layouts etc. still need to be compatible for whatever memory buffer was passed into the kernel. There isn't much practical reason to do it, but you could have a struct with private pointer members in it changing the offsets of the other items, not that you could do anything valid with the contents.

[rjmccall]

In D31042#712613, @arsenm wrote:

In D31042#712484, @rjmccall wrote:

Because LLVM currently makes some unfortunate assumptions about address space 0, this stack address space cannot be address space 0. These assumptions include:
(1) As far as debug info is concerned, the size of a pointer is the size of a pointer in address space 0. (Is this assumption why sizeof(void private *) == 8 despite private being clearly intended in the OpenCL spec to be the address space of stack pointers?)

This is a point that the spec is vague on. While the spec explicitly allows different size pointers for different address spaces, there isn't really any detailed description of what this entails.

I don't see what the problem is. The spec doesn't allow pointers to be converted between address spaces at all (except for the promotion into the generic address space in OpenCL 2.0). Pointers into different address spaces are just completely different types.

The problem is mostly what happens if the host and device pointer size don't match? Struct layouts etc. still need to be compatible for whatever memory buffer was passed into the kernel. There isn't much practical reason to do it, but you could have a struct with private pointer members in it changing the offsets of the other items, not that you could do anything valid with the contents.

The easy lowering would be to leave a 64-bit private address space around but consider it "illegal". Address space conversions between it and the 32-bit private address space would simply be zexts and truncs. But your 8-byte-aligned pointer idea works, too, assuming it doesn't give LLVM too many fits to have an over-aligned primitive type.

The more complex lowering would be to rewrite all the struct types to replace T addrspace(private64)* with { T addrspace(private32)*, i32 }. There's actually relatively few IR constructs that work with struct types — GEP, insertvalue/extractvalue, constant aggregates, I think that's it. The IR type would change alignment, but only by getting weaker, and memory accesses should all have explicit alignments anyway.

Anyway, I guess we're getting a bit afield.

John.

[arsenm]

Any more comments for the patch?

[rjmccall]

Not from me.

[arsenm]

Is this good to commit?

[efriedma]

Yes, I think you've addressed everything.

[arsenm]

r299888

docs/LangRef.rst

``e``
     location.
 ``S<size>``
     Specifies the natural alignment of the stack in bits. Alignment
     promotion of stack variables is limited to the natural stack
     alignment to avoid dynamic stack realignment. The stack alignment
     must be a multiple of 8-bits. If omitted, the natural stack
     alignment defaults to "unspecified", which does not prevent any
     alignment promotions.
+``A<address space>``
+    Specifies the address space of  objects created by '``alloca``'.
+    Defaults to the default address space of 0.
 ``p[n]:<size>:<abi>:<pref>``
     This specifies the *size* of a pointer and its ``<abi>`` and
     ``<pref>``\erred alignments for address space ``n``. All sizes are in
     bits. The address space, ``n``, is optional, and if not specified,
     denotes the default address space 0. The value of ``n`` must be
     in the range [1,2^23).
 ``i<size>:<abi>:<pref>``
     This specifies the alignment for an integer type of a given bit

 '``alloca``' Instruction
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 
 ::
 
-      <result> = alloca [inalloca] <type> [, <ty> <NumElements>] [, align <alignment>]     ; yields type*:result
+      <result> = alloca [inalloca] <type> [, <ty> <NumElements>] [, align <alignment>] [, addrspace(<num>)]     ; yields type addrspace(num)*:result
 
 Overview:
 """""""""
 
 The '``alloca``' instruction allocates memory on the stack frame of the
 currently executing function, to be automatically released when this
 function returns to its caller. The object is always allocated in the
-generic address space (address space zero).
+address space for allocas indicated in the datalayout.
 
 Arguments:
 """"""""""
 
 The '``alloca``' instruction allocates ``sizeof(<type>)*NumElements``
 bytes of memory on the runtime stack, returning a pointer of the
 appropriate type to the program. If "NumElements" is specified, it is
 the number of elements allocated, otherwise "NumElements" is defaulted

include/llvm/IR/DataLayout.h

 /// The target data layout string is specified *by the target* - a frontend
 /// generating LLVM IR is required to generate the right target data for the
 /// target being codegen'd to.
 class DataLayout {
 private:
   /// Defaults to false.
   bool BigEndian;
 
+  unsigned AllocaAddrSpace;
   unsigned StackNaturalAlign;
 
   enum ManglingModeT {
     MM_None,
     MM_ELF,
     MM_MachO,
     MM_WinCOFF,
     MM_WinCOFFX86,
public:
   void init(const Module *M);
 
   DataLayout(const DataLayout &DL) : LayoutMap(nullptr) { *this = DL; }
 
   DataLayout &operator=(const DataLayout &DL) {
     clear();
     StringRepresentation = DL.StringRepresentation;
     BigEndian = DL.isBigEndian();
+    AllocaAddrSpace = DL.AllocaAddrSpace;
     StackNaturalAlign = DL.StackNaturalAlign;
     ManglingMode = DL.ManglingMode;
     LegalIntWidths = DL.LegalIntWidths;
     Alignments = DL.Alignments;
     Pointers = DL.Pointers;
     NonIntegralAddressSpaces = DL.NonIntegralAddressSpaces;
     return *this;
   }
public:
   bool isIllegalInteger(uint64_t Width) const { return !isLegalInteger(Width); }
 
   /// Returns true if the given alignment exceeds the natural stack alignment.
   bool exceedsNaturalStackAlignment(unsigned Align) const {
     return (StackNaturalAlign != 0) && (Align > StackNaturalAlign);
   }
 
   unsigned getStackAlignment() const { return StackNaturalAlign; }
+  unsigned getAllocaAddrSpace() const { return AllocaAddrSpace; }
 
   bool hasMicrosoftFastStdCallMangling() const {
     return ManglingMode == MM_WinCOFFX86;
   }
 
   bool hasLinkerPrivateGlobalPrefix() const { return ManglingMode == MM_MachO; }
 
   StringRef getLinkerPrivateGlobalPrefix() const {

include/llvm/IR/IRBuilder.h

 #include "llvm/IR/DerivedTypes.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/InstrTypes.h"
 #include "llvm/IR/Instruction.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
 #include "llvm/IR/Operator.h"
 #include "llvm/IR/Type.h"
 #include "llvm/IR/Value.h"
 #include "llvm/IR/ValueHandle.h"
 #include "llvm/Support/AtomicOrdering.h"
 #include "llvm/Support/CBindingWrapping.h"
 #include "llvm/Support/Casting.h"
 #include "llvm-c/Types.h"
if (Constant *VC = dyn_cast<Constant>(V))
       return Insert(Folder.CreateNot(VC), Name);
     return Insert(BinaryOperator::CreateNot(V), Name);
   }
 
   //===--------------------------------------------------------------------===//
   // Instruction creation methods: Memory Instructions
   //===--------------------------------------------------------------------===//
 
+  AllocaInst *CreateAlloca(Type *Ty, unsigned AddrSpace,
+                           Value *ArraySize = nullptr, const Twine &Name = "") {
+    return Insert(new AllocaInst(Ty, AddrSpace, ArraySize), Name);
+  }
+
   AllocaInst *CreateAlloca(Type *Ty, Value *ArraySize = nullptr,
                            const Twine &Name = "") {
-    return Insert(new AllocaInst(Ty, ArraySize), Name);
+    const DataLayout &DL = BB->getParent()->getParent()->getDataLayout();
+    return Insert(new AllocaInst(Ty, DL.getAllocaAddrSpace(), ArraySize), Name);
   }
   // \brief Provided to resolve 'CreateLoad(Ptr, "...")' correctly, instead of
   // converting the string to 'bool' for the isVolatile parameter.
   LoadInst *CreateLoad(Value *Ptr, const char *Name) {
     return Insert(new LoadInst(Ptr), Name);
   }
   LoadInst *CreateLoad(Value *Ptr, const Twine &Name = "") {
     return Insert(new LoadInst(Ptr), Name);
   }
   LoadInst *CreateLoad(Type *Ty, Value *Ptr, const Twine &Name = "") {
     return Insert(new LoadInst(Ty, Ptr), Name);
   }
   LoadInst *CreateLoad(Value *Ptr, bool isVolatile, const Twine &Name = "") {

include/llvm/IR/Instructions.h

 
 protected:
   // Note: Instruction needs to be a friend here to call cloneImpl.
   friend class Instruction;
 
   AllocaInst *cloneImpl() const;
 
 public:
-  explicit AllocaInst(Type *Ty, Value *ArraySize = nullptr,
+  explicit AllocaInst(Type *Ty, unsigned AddrSpace,
+                      Value *ArraySize = nullptr,
                       const Twine &Name = "",
                       Instruction *InsertBefore = nullptr);
-  AllocaInst(Type *Ty, Value *ArraySize,
+  AllocaInst(Type *Ty, unsigned AddrSpace, Value *ArraySize,
              const Twine &Name, BasicBlock *InsertAtEnd);
 
-  AllocaInst(Type *Ty, const Twine &Name, Instruction *InsertBefore = nullptr);
-  AllocaInst(Type *Ty, const Twine &Name, BasicBlock *InsertAtEnd);
+  AllocaInst(Type *Ty, unsigned AddrSpace,
+             const Twine &Name, Instruction *InsertBefore = nullptr);
+  AllocaInst(Type *Ty, unsigned AddrSpace,
+             const Twine &Name, BasicBlock *InsertAtEnd);
 
-  AllocaInst(Type *Ty, Value *ArraySize, unsigned Align,
+  AllocaInst(Type *Ty, unsigned AddrSpace, Value *ArraySize, unsigned Align,
              const Twine &Name = "", Instruction *InsertBefore = nullptr);
-  AllocaInst(Type *Ty, Value *ArraySize, unsigned Align,
+  AllocaInst(Type *Ty, unsigned AddrSpace, Value *ArraySize, unsigned Align,
              const Twine &Name, BasicBlock *InsertAtEnd);
 
   // Out of line virtual method, so the vtable, etc. has a home.
   ~AllocaInst() override;
 
   /// Return true if there is an allocation size parameter to the allocation
   /// instruction that is not 1.
   bool isArrayAllocation() const;

lib/AsmParser/LLParser.h

private:
     bool ParseOptionalCallingConv(unsigned &CC);
     bool ParseOptionalAlignment(unsigned &Alignment);
     bool ParseOptionalDerefAttrBytes(lltok::Kind AttrKind, uint64_t &Bytes);
     bool ParseScopeAndOrdering(bool isAtomic, SynchronizationScope &Scope,
                                AtomicOrdering &Ordering);
     bool ParseOrdering(AtomicOrdering &Ordering);
     bool ParseOptionalStackAlignment(unsigned &Alignment);
     bool ParseOptionalCommaAlign(unsigned &Alignment, bool &AteExtraComma);
+    bool ParseOptionalCommaAddrSpace(unsigned &AddrSpace, LocTy &Loc,
+                                     bool &AteExtraComma);
     bool ParseOptionalCommaInAlloca(bool &IsInAlloca);
     bool parseAllocSizeArguments(unsigned &ElemSizeArg,
                                  Optional<unsigned> &HowManyArg);
     bool ParseIndexList(SmallVectorImpl<unsigned> &Indices,
                         bool &AteExtraComma);
     bool ParseIndexList(SmallVectorImpl<unsigned> &Indices) {
       bool AteExtraComma;
       if (ParseIndexList(Indices, AteExtraComma)) return true;

lib/AsmParser/LLParser.cpp

if (Lex.getKind() != lltok::kw_align)
       return Error(Lex.getLoc(), "expected metadata or 'align'");
 
     if (ParseOptionalAlignment(Alignment)) return true;
   }
 
   return false;
 }
 
+/// ParseOptionalCommaAddrSpace
+///   ::=
+///   ::= ',' addrspace(1)
+///
+/// This returns with AteExtraComma set to true if it ate an excess comma at the
+/// end.
+bool LLParser::ParseOptionalCommaAddrSpace(unsigned &AddrSpace,
+                                           LocTy &Loc,
+                                           bool &AteExtraComma) {
+  AteExtraComma = false;
+  while (EatIfPresent(lltok::comma)) {
+    // Metadata at the end is an early exit.
+    if (Lex.getKind() == lltok::MetadataVar) {
+      AteExtraComma = true;
+      return false;
+    }
+
+    Loc = Lex.getLoc();
+    if (Lex.getKind() != lltok::kw_addrspace)
+      return Error(Lex.getLoc(), "expected metadata or 'addrspace'");
+
+    if (ParseOptionalAddrSpace(AddrSpace))
+      return true;
+  }
+
+  return false;
+}
+
 bool LLParser::parseAllocSizeArguments(unsigned &BaseSizeArg,
                                        Optional<unsigned> &HowManyArg) {
   Lex.Lex();
 
   auto StartParen = Lex.getLoc();
   if (!EatIfPresent(lltok::lparen))
     return Error(StartParen, "expected '('");
 

 // Memory Instructions.
 //===----------------------------------------------------------------------===//
 
 /// ParseAlloc
 ///   ::= 'alloca' 'inalloca'? 'swifterror'? Type (',' TypeAndValue)?
 ///       (',' 'align' i32)?
 int LLParser::ParseAlloc(Instruction *&Inst, PerFunctionState &PFS) {
   Value *Size = nullptr;
-  LocTy SizeLoc, TyLoc;
+  LocTy SizeLoc, TyLoc, ASLoc;
   unsigned Alignment = 0;
+  unsigned AddrSpace = 0;
   Type *Ty = nullptr;
 
   bool IsInAlloca = EatIfPresent(lltok::kw_inalloca);
   bool IsSwiftError = EatIfPresent(lltok::kw_swifterror);
 
   if (ParseType(Ty, TyLoc)) return true;
 
   if (Ty->isFunctionTy() || !PointerType::isValidElementType(Ty))
     return Error(TyLoc, "invalid type for alloca");
 
   bool AteExtraComma = false;
   if (EatIfPresent(lltok::comma)) {
     if (Lex.getKind() == lltok::kw_align) {
-      if (ParseOptionalAlignment(Alignment)) return true;
+      if (ParseOptionalAlignment(Alignment))
+        return true;
+      if (ParseOptionalCommaAddrSpace(AddrSpace, ASLoc, AteExtraComma))
+        return true;
+    } else if (Lex.getKind() == lltok::kw_addrspace) {
+      ASLoc = Lex.getLoc();
+      if (ParseOptionalAddrSpace(AddrSpace))
+        return true;
     } else if (Lex.getKind() == lltok::MetadataVar) {
       AteExtraComma = true;
     } else {
       if (ParseTypeAndValue(Size, SizeLoc, PFS) ||
-          ParseOptionalCommaAlign(Alignment, AteExtraComma))
+          ParseOptionalCommaAlign(Alignment, AteExtraComma) ||
+          (!AteExtraComma &&
+           ParseOptionalCommaAddrSpace(AddrSpace, ASLoc, AteExtraComma)))
         return true;
     }
   }
 
   if (Size && !Size->getType()->isIntegerTy())
     return Error(SizeLoc, "element count must have integer type");
 
-  AllocaInst *AI = new AllocaInst(Ty, Size, Alignment);
+  const DataLayout &DL = M->getDataLayout();
+  unsigned AS = DL.getAllocaAddrSpace();
+  if (AS != AddrSpace) {
+    // TODO: In the future it should be possible to specify addrspace per-alloca.
+    return Error(ASLoc, "address space must match datalayout");
+  }
+
+  AllocaInst *AI = new AllocaInst(Ty, AS, Size, Alignment);
   AI->setUsedWithInAlloca(IsInAlloca);
   AI->setSwiftError(IsSwiftError);
   Inst = AI;
   return AteExtraComma ? InstExtraComma : InstNormal;
 }
 
 /// ParseLoad
 ///   ::= 'load' 'volatile'? TypeAndValue (',' 'align' i32)?

lib/Bitcode/Reader/BitcodeReader.cpp

case bitc::FUNC_CODE_INST_ALLOCA: { // ALLOCA: [instty, opty, op, align]
       Type *OpTy = getTypeByID(Record[1]);
       Value *Size = getFnValueByID(Record[2], OpTy);
       unsigned Align;
       if (Error Err = parseAlignmentValue(AlignRecord & ~FlagMask, Align)) {
         return Err;
       }
       if (!Ty || !Size)
         return error("Invalid record");
-      AllocaInst *AI = new AllocaInst(Ty, Size, Align);
+
+      // FIXME: Make this an optional field.
+      const DataLayout &DL = TheModule->getDataLayout();
+      unsigned AS = DL.getAllocaAddrSpace();
+
+      AllocaInst *AI = new AllocaInst(Ty, AS, Size, Align);
       AI->setUsedWithInAlloca(InAlloca);
       AI->setSwiftError(SwiftError);
       I = AI;
       InstructionList.push_back(I);
       break;
     }
     case bitc::FUNC_CODE_INST_LOAD: { // LOAD: [opty, op, align, vol]
       unsigned OpNum = 0;

lib/CodeGen/SjLjEHPrepare.cpp

Value *SjLjEHPrepare::setupFunctionContext(Function &F,
                                            ArrayRef<LandingPadInst *> LPads) {
   BasicBlock *EntryBB = &F.front();
 
   // Create an alloca for the incoming jump buffer ptr and the new jump buffer
   // that needs to be restored on all exits from the function. This is an alloca
   // because the value needs to be added to the global context list.
   auto &DL = F.getParent()->getDataLayout();
   unsigned Align = DL.getPrefTypeAlignment(FunctionContextTy);
-  FuncCtx = new AllocaInst(FunctionContextTy, nullptr, Align, "fn_context",
-                           &EntryBB->front());
+  FuncCtx = new AllocaInst(FunctionContextTy, DL.getAllocaAddrSpace(),
+                           nullptr, Align, "fn_context", &EntryBB->front());
 
   // Fill in the function context structure.
   for (LandingPadInst *LPI : LPads) {
     IRBuilder<> Builder(LPI->getParent(),
                         LPI->getParent()->getFirstInsertionPt());
 
     // Reference the __data field.
     Value *FCData =

lib/CodeGen/WinEHPrepare.cpp

private:
   void cloneCommonBlocks(Function &F);
   void removeImplausibleInstructions(Function &F);
   void cleanupPreparedFunclets(Function &F);
   void verifyPreparedFunclets(Function &F);
 
   // All fields are reset by runOnFunction.
   EHPersonality Personality = EHPersonality::Unknown;
 
+  const DataLayout *DL = nullptr;
   DenseMap<BasicBlock *, ColorVector> BlockColors;
   MapVector<BasicBlock *, std::vector<BasicBlock *>> FuncletBlocks;
 };
 
 } // end anonymous namespace
 
 char WinEHPrepare::ID = 0;
 INITIALIZE_TM_PASS(WinEHPrepare, "winehprepare", "Prepare Windows exceptions",
bool WinEHPrepare::runOnFunction(Function &Fn) {
 
   // Classify the personality to see what kind of preparation we need.
   Personality = classifyEHPersonality(Fn.getPersonalityFn());
 
   // Do nothing if this is not a funclet-based personality.
   if (!isFuncletEHPersonality(Personality))
     return false;
 
+  DL = &Fn.getParent()->getDataLayout();
   return prepareExplicitEH(Fn);
 }
 
 bool WinEHPrepare::doFinalization(Module &M) { return false; }
 
 void WinEHPrepare::getAnalysisUsage(AnalysisUsage &AU) const {}
 
 static int addUnwindMapEntry(WinEHFuncInfo &FuncInfo, int ToState,

 AllocaInst *WinEHPrepare::insertPHILoads(PHINode *PN, Function &F) {
   BasicBlock *PHIBlock = PN->getParent();
   AllocaInst *SpillSlot = nullptr;
   Instruction *EHPad = PHIBlock->getFirstNonPHI();
 
   if (!isa<TerminatorInst>(EHPad)) {
     // If the EHPad isn't a terminator, then we can insert a load in this block
     // that will dominate all uses.
-    SpillSlot = new AllocaInst(PN->getType(), nullptr,
+    SpillSlot = new AllocaInst(PN->getType(), DL->getAllocaAddrSpace(), nullptr,
                                Twine(PN->getName(), ".wineh.spillslot"),
                                &F.getEntryBlock().front());
     Value *V = new LoadInst(SpillSlot, Twine(PN->getName(), ".wineh.reload"),
                             &*PHIBlock->getFirstInsertionPt());
     PN->replaceAllUsesWith(V);
     return SpillSlot;
   }
 
void WinEHPrepare::insertPHIStore(
   new StoreInst(PredVal, SpillSlot, PredBlock->getTerminator());
 }
 
 void WinEHPrepare::replaceUseWithLoad(Value *V, Use &U, AllocaInst *&SpillSlot,
                                       DenseMap<BasicBlock *, Value *> &Loads,
                                       Function &F) {
   // Lazilly create the spill slot.
   if (!SpillSlot)
-    SpillSlot = new AllocaInst(V->getType(), nullptr,
+    SpillSlot = new AllocaInst(V->getType(), DL->getAllocaAddrSpace(), nullptr,
                                Twine(V->getName(), ".wineh.spillslot"),
                                &F.getEntryBlock().front());
 
   auto *UsingInst = cast<Instruction>(U.getUser());
   if (auto *UsingPHI = dyn_cast<PHINode>(UsingInst)) {
     // If this is a PHI node, we can't insert a load of the value before
     // the use.  Instead insert the load in the predecessor block
     // corresponding to the incoming value.

lib/IR/AsmWriter.cpp

if (isa<BranchInst>(I) && cast<BranchInst>(I).isConditional()) {
     if (!AI->getArraySize() || AI->isArrayAllocation() ||
         !AI->getArraySize()->getType()->isIntegerTy(32)) {
       Out << ", ";
       writeOperand(AI->getArraySize(), true);
     }
     if (AI->getAlignment()) {
       Out << ", align " << AI->getAlignment();
     }
+
+    unsigned AddrSpace = AI->getType()->getAddressSpace();
+    if (AddrSpace != 0) {
+      Out << ", addrspace(" << AddrSpace << ')';
+    }
+
   } else if (isa<CastInst>(I)) {
     if (Operand) {
       Out << ' ';
       writeOperand(Operand, true);   // Work with broken code
     }
     Out << " to ";
     TypePrinter.print(I.getType(), Out);
   } else if (isa<VAArgInst>(I)) {

lib/IR/DataLayout.cpp

static const LayoutAlignElem DefaultAlignments[] = {
   { AGGREGATE_ALIGN, 0, 0, 8 }   // struct
 };
 
 void DataLayout::reset(StringRef Desc) {
   clear();
 
   LayoutMap = nullptr;
   BigEndian = false;
+  AllocaAddrSpace = 0;
   StackNaturalAlign = 0;
   ManglingMode = MM_None;
   NonIntegralAddressSpaces.clear();
 
   // Default alignments
   for (const LayoutAlignElem &E : DefaultAlignments) {
     setAlignment((AlignTypeEnum)E.AlignType, E.ABIAlign, E.PrefAlign,
                  E.TypeBitWidth);
case 'n':  // Native integer types.
           break;
         Split = split(Rest, ':');
       }
       break;
     case 'S': { // Stack natural alignment.
       StackNaturalAlign = inBytes(getInt(Tok));
       break;
     }
+    case 'A': { // Default stack/alloca address space.
+      AllocaAddrSpace = getInt(Tok);
+      if (!isUInt<24>(AllocaAddrSpace))
+        report_fatal_error("Invalid address space, must be a 24bit integer");
+      break;
+    }
     case 'm':
       if (!Tok.empty())
         report_fatal_error("Unexpected trailing characters after mangling specifier in datalayout string");
       if (Rest.empty())
         report_fatal_error("Expected mangling specifier in datalayout string");
       if (Rest.size() > 1)
         report_fatal_error("Unknown mangling specifier in datalayout string");
       switch(Rest[0]) {

 DataLayout::DataLayout(const Module *M) : LayoutMap(nullptr) {
   init(M);
 }
 
 void DataLayout::init(const Module *M) { *this = M->getDataLayout(); }
 
 bool DataLayout::operator==(const DataLayout &Other) const {
   bool Ret = BigEndian == Other.BigEndian &&
+             AllocaAddrSpace == Other.AllocaAddrSpace &&
              StackNaturalAlign == Other.StackNaturalAlign &&
              ManglingMode == Other.ManglingMode &&
              LegalIntWidths == Other.LegalIntWidths &&
              Alignments == Other.Alignments && Pointers == Other.Pointers;
   // Note: getStringRepresentation() might differs, it is not canonicalized
   return Ret;
 }
 

lib/IR/Instructions.cpp

else {
     assert(!isa<BasicBlock>(Amt) &&
            "Passed basic block into allocation size parameter! Use other ctor");
     assert(Amt->getType()->isIntegerTy() &&
            "Allocation array size is not an integer!");
   }
   return Amt;
 }
 
-AllocaInst::AllocaInst(Type *Ty, const Twine &Name, Instruction *InsertBefore)
-    : AllocaInst(Ty, /*ArraySize=*/nullptr, Name, InsertBefore) {}
-
-AllocaInst::AllocaInst(Type *Ty, const Twine &Name, BasicBlock *InsertAtEnd)
-    : AllocaInst(Ty, /*ArraySize=*/nullptr, Name, InsertAtEnd) {}
-
-AllocaInst::AllocaInst(Type *Ty, Value *ArraySize, const Twine &Name,
+AllocaInst::AllocaInst(Type *Ty, unsigned AddrSpace, const Twine &Name,
                        Instruction *InsertBefore)
-    : AllocaInst(Ty, ArraySize, /*Align=*/0, Name, InsertBefore) {}
+  : AllocaInst(Ty, AddrSpace, /*ArraySize=*/nullptr, Name, InsertBefore) {}
 
-AllocaInst::AllocaInst(Type *Ty, Value *ArraySize, const Twine &Name,
+AllocaInst::AllocaInst(Type *Ty, unsigned AddrSpace, const Twine &Name,
                        BasicBlock *InsertAtEnd)
-    : AllocaInst(Ty, ArraySize, /*Align=*/0, Name, InsertAtEnd) {}
+  : AllocaInst(Ty, AddrSpace, /*ArraySize=*/nullptr, Name, InsertAtEnd) {}
 
-AllocaInst::AllocaInst(Type *Ty, Value *ArraySize, unsigned Align,
+AllocaInst::AllocaInst(Type *Ty, unsigned AddrSpace, Value *ArraySize,
                        const Twine &Name, Instruction *InsertBefore)
-    : UnaryInstruction(PointerType::getUnqual(Ty), Alloca,
-                       getAISize(Ty->getContext(), ArraySize), InsertBefore),
-      AllocatedType(Ty) {
+  : AllocaInst(Ty, AddrSpace, ArraySize, /*Align=*/0, Name, InsertBefore) {}
+
+AllocaInst::AllocaInst(Type *Ty, unsigned AddrSpace, Value *ArraySize,
+                       const Twine &Name, BasicBlock *InsertAtEnd)
+  : AllocaInst(Ty, AddrSpace, ArraySize, /*Align=*/0, Name, InsertAtEnd) {}
+
+AllocaInst::AllocaInst(Type *Ty, unsigned AddrSpace, Value *ArraySize,
+                       unsigned Align, const Twine &Name,
+                       Instruction *InsertBefore)
+  : UnaryInstruction(PointerType::get(Ty, AddrSpace), Alloca,
   setAlignment(Align);
   assert(!Ty->isVoidTy() && "Cannot allocate void!");
   setName(Name);
 }
 
-AllocaInst::AllocaInst(Type *Ty, Value *ArraySize, unsigned Align,
-                       const Twine &Name, BasicBlock *InsertAtEnd)
-    : UnaryInstruction(PointerType::getUnqual(Ty), Alloca,
-                       getAISize(Ty->getContext(), ArraySize), InsertAtEnd),
+AllocaInst::AllocaInst(Type *Ty, unsigned AddrSpace, Value *ArraySize,
+                       unsigned Align, const Twine &Name,
+                       BasicBlock *InsertAtEnd)
+  : UnaryInstruction(PointerType::get(Ty, AddrSpace), Alloca,
       AllocatedType(Ty) {
   setAlignment(Align);
   assert(!Ty->isVoidTy() && "Cannot allocate void!");
   setName(Name);
 }
 
 // Out of line virtual method, so the vtable, etc has a home.
 AllocaInst::~AllocaInst() {

 }
 
 InsertValueInst *InsertValueInst::cloneImpl() const {
   return new InsertValueInst(*this);
 }
 
 AllocaInst *AllocaInst::cloneImpl() const {
   AllocaInst *Result = new AllocaInst(getAllocatedType(),
+                                      getType()->getAddressSpace(),
                                       (Value *)getOperand(0), getAlignment());
   Result->setUsedWithInAlloca(isUsedWithInAlloca());
   Result->setSwiftError(isSwiftError());
   return Result;
 }
 
 LoadInst *LoadInst::cloneImpl() const {
   return new LoadInst(getOperand(0), Twine(), isVolatile(),

lib/IR/Verifier.cpp

for (const User *U : SwiftErrorVal->users()) {
     if (auto II = dyn_cast<InvokeInst>(U))
       verifySwiftErrorCallSite(const_cast<InvokeInst*>(II), SwiftErrorVal);
   }
 }
 
 void Verifier::visitAllocaInst(AllocaInst &AI) {
   SmallPtrSet<Type*, 4> Visited;
   PointerType *PTy = AI.getType();
-  Assert(PTy->getAddressSpace() == 0,
-         "Allocation instruction pointer not in the generic address space!",
+  // TODO: Relax this restriction?
+  Assert(PTy->getAddressSpace() == DL.getAllocaAddrSpace(),
+         "Allocation instruction pointer not in the stack address space!",
          &AI);
   Assert(AI.getAllocatedType()->isSized(&Visited),
          "Cannot allocate unsized type", &AI);
   Assert(AI.getArraySize()->getType()->isIntegerTy(),
          "Alloca array size must have integer type", &AI);
   Assert(AI.getAlignment() <= Value::MaximumAlignment,
          "huge alignment values are unsupported", &AI);
 

lib/Target/NVPTX/NVPTXLowerArgs.cpp

 void NVPTXLowerArgs::handleByValParam(Argument *Arg) {
   Function *Func = Arg->getParent();
   Instruction *FirstInst = &(Func->getEntryBlock().front());
   PointerType *PType = dyn_cast<PointerType>(Arg->getType());
 
   assert(PType && "Expecting pointer type in handleByValParam");
 
   Type *StructType = PType->getElementType();
-  AllocaInst *AllocA = new AllocaInst(StructType, Arg->getName(), FirstInst);
+  unsigned AS = Func->getParent()->getDataLayout().getAllocaAddrSpace();
+  AllocaInst *AllocA = new AllocaInst(StructType, AS, Arg->getName(), FirstInst);
   // Set the alignment to alignment of the byval parameter. This is because,
   // later load/stores assume that alignment, and we are going to replace
   // the use of the byval parameter with this alloca instruction.
   AllocA->setAlignment(Func->getParamAlignment(Arg->getArgNo() + 1));
   Arg->replaceAllUsesWith(AllocA);
 
   Value *ArgInParam = new AddrSpaceCastInst(
       Arg, PointerType::get(StructType, ADDRESS_SPACE_PARAM), Arg->getName(),

lib/Transforms/Coroutines/CoroElide.cpp

for (auto *CA : CoroAllocs) {
     CA->replaceAllUsesWith(False);
     CA->eraseFromParent();
   }
 
   // FIXME: Design how to transmit alignment information for every alloca that
   // is spilled into the coroutine frame and recreate the alignment information
   // here. Possibly we will need to do a mini SROA here and break the coroutine
   // frame into individual AllocaInst recreating the original alignment.
-  auto *Frame = new AllocaInst(FrameTy, "", InsertPt);
+  const DataLayout &DL = F->getParent()->getDataLayout();
+  auto *Frame = new AllocaInst(FrameTy, DL.getAllocaAddrSpace(), "", InsertPt);
   auto *FrameVoidPtr =
       new BitCastInst(Frame, Type::getInt8PtrTy(C), "vFrame", InsertPt);
 
   for (auto *CB : CoroBegins) {
     CB->replaceAllUsesWith(FrameVoidPtr);
     CB->eraseFromParent();
   }
 

lib/Transforms/IPO/ArgumentPromotion.cpp

if (!Call->use_empty()) {
       New->takeName(Call);
     }
 
     // Finally, remove the old call from the program, reducing the use-count of
     // F.
     Call->eraseFromParent();
   }
 
+  const DataLayout &DL = F->getParent()->getDataLayout();
+
   // Since we have now created the new function, splice the body of the old
   // function right into the new function, leaving the old rotting hulk of the
   // function empty.
   NF->getBasicBlockList().splice(NF->begin(), F->getBasicBlockList());
 
   // Loop over the argument list, transferring uses of the old arguments over to
   // the new arguments, also transferring over the names as well.
   //
for (Function::arg_iterator I = F->arg_begin(), E = F->arg_end(),
 
     if (ByValArgsToTransform.count(&*I)) {
       // In the callee, we create an alloca, and store each of the new incoming
       // arguments into the alloca.
       Instruction *InsertPt = &NF->begin()->front();
 
       // Just add all the struct element types.
       Type *AgTy = cast<PointerType>(I->getType())->getElementType();
-      Value *TheAlloca = new AllocaInst(AgTy, nullptr, "", InsertPt);
+      Value *TheAlloca = new AllocaInst(AgTy, DL.getAllocaAddrSpace(), nullptr,
+                                        "", InsertPt);
       StructType *STy = cast<StructType>(AgTy);
       Value *Idxs[2] = {ConstantInt::get(Type::getInt32Ty(F->getContext()), 0),
                         nullptr};
 
       for (unsigned i = 0, e = STy->getNumElements(); i != e; ++i) {
         Idxs[1] = ConstantInt::get(Type::getInt32Ty(F->getContext()), i);
         Value *Idx = GetElementPtrInst::Create(
             AgTy, TheAlloca, Idxs, TheAlloca->getName() + "." + Twine(i),

lib/Transforms/IPO/GlobalOpt.cpp

if (!GS.HasMultipleAccessingFunctions &&
       GS.AccessingFunction &&
       GV->getValueType()->isSingleValueType() &&
       GV->getType()->getAddressSpace() == 0 &&
       !GV->isExternallyInitialized() &&
       allNonInstructionUsersCanBeMadeInstructions(GV) &&
       GS.AccessingFunction->doesNotRecurse() &&
       isPointerValueDeadOnEntryToFunction(GS.AccessingFunction, GV,
                                           LookupDomTree)) {
+    const DataLayout &DL = GV->getParent()->getDataLayout();
+
     DEBUG(dbgs() << "LOCALIZING GLOBAL: " << *GV << "\n");
     Instruction &FirstI = const_cast<Instruction&>(*GS.AccessingFunction
                                                    ->getEntryBlock().begin());
     Type *ElemTy = GV->getValueType();
     // FIXME: Pass Global's alignment when globals have alignment
-    AllocaInst *Alloca = new AllocaInst(ElemTy, nullptr,
+    AllocaInst *Alloca = new AllocaInst(ElemTy, DL.getAllocaAddrSpace(), nullptr,
                                         GV->getName(), &FirstI);
     if (!isa<UndefValue>(GV->getInitializer()))
       new StoreInst(GV->getInitializer(), Alloca, &FirstI);
 
     makeAllConstantUsesInstructions(GV);
 
     GV->replaceAllUsesWith(Alloca);
     GV->eraseFromParent();

lib/Transforms/Instrumentation/DataFlowSanitizer.cpp

void storeShadow(Value *Addr, uint64_t Size, uint64_t Align, Value *Shadow,
                    Instruction *Pos);
 };
 
 class DFSanVisitor : public InstVisitor<DFSanVisitor> {
  public:
   DFSanFunction &DFSF;
   DFSanVisitor(DFSanFunction &DFSF) : DFSF(DFSF) {}
 
+  const DataLayout &getDataLayout() const {
+    return DFSF.F->getParent()->getDataLayout();
+  }
+
   void visitOperandShadowInst(Instruction &I);
 
   void visitBinaryOperator(BinaryOperator &BO);
   void visitCastInst(CastInst &CI);
   void visitCmpInst(CmpInst &CI);
   void visitGetElementPtrInst(GetElementPtrInst &GEPI);
   void visitLoadInst(LoadInst &LI);
   void visitStoreInst(StoreInst &SI);
case DataFlowSanitizer::WK_Custom: {
         i = CS.arg_begin();
         for (unsigned n = FT->getNumParams(); n != 0; ++i, --n)
           Args.push_back(DFSF.getShadow(*i));
 
         if (FT->isVarArg()) {
           auto *LabelVATy = ArrayType::get(DFSF.DFS.ShadowTy,
                                            CS.arg_size() - FT->getNumParams());
           auto *LabelVAAlloca = new AllocaInst(
-              LabelVATy, "labelva", &DFSF.F->getEntryBlock().front());
+              LabelVATy, getDataLayout().getAllocaAddrSpace(),
+              "labelva", &DFSF.F->getEntryBlock().front());
 
           for (unsigned n = 0; i != CS.arg_end(); ++i, ++n) {
             auto LabelVAPtr = IRB.CreateStructGEP(LabelVATy, LabelVAAlloca, n);
             IRB.CreateStore(DFSF.getShadow(*i), LabelVAPtr);
           }
 
           Args.push_back(IRB.CreateStructGEP(LabelVATy, LabelVAAlloca, 0));
         }
 
         if (!FT->getReturnType()->isVoidTy()) {
           if (!DFSF.LabelReturnAlloca) {
             DFSF.LabelReturnAlloca =
-                new AllocaInst(DFSF.DFS.ShadowTy, "labelreturn",
-                               &DFSF.F->getEntryBlock().front());
+              new AllocaInst(DFSF.DFS.ShadowTy,
+                             getDataLayout().getAllocaAddrSpace(),
+                             "labelreturn", &DFSF.F->getEntryBlock().front());
           }
           Args.push_back(DFSF.LabelReturnAlloca);
         }
 
         for (i = CS.arg_begin() + FT->getNumParams(); i != CS.arg_end(); ++i)
           Args.push_back(*i);
 
         CallInst *CustomCI = IRB.CreateCall(CustomF, Args);
if (DFSF.DFS.getInstrumentedABI() == DataFlowSanitizer::IA_Args) {
     i = CS.arg_begin();
     for (unsigned n = FT->getNumParams(); n != 0; ++i, --n)
       Args.push_back(DFSF.getShadow(*i));
 
     if (FT->isVarArg()) {
       unsigned VarArgSize = CS.arg_size() - FT->getNumParams();
       ArrayType *VarArgArrayTy = ArrayType::get(DFSF.DFS.ShadowTy, VarArgSize);
       AllocaInst *VarArgShadow =
-          new AllocaInst(VarArgArrayTy, "", &DFSF.F->getEntryBlock().front());
+        new AllocaInst(VarArgArrayTy, getDataLayout().getAllocaAddrSpace(),
+                       "", &DFSF.F->getEntryBlock().front());
       Args.push_back(IRB.CreateConstGEP2_32(VarArgArrayTy, VarArgShadow, 0, 0));
       for (unsigned n = 0; i != e; ++i, ++n) {
         IRB.CreateStore(
             DFSF.getShadow(*i),
             IRB.CreateConstGEP2_32(VarArgArrayTy, VarArgShadow, 0, n));
         Args.push_back(*i);
       }
     }

lib/Transforms/Instrumentation/MemorySanitizer.cpp

void visitVACopyInst(VACopyInst &I) override {
 
     // Unpoison the whole __va_list_tag.
     // FIXME: magic ABI constants.
     IRB.CreateMemSet(ShadowPtr, Constant::getNullValue(IRB.getInt8Ty()),
                      /* size */24, /* alignment */8, false);
   }
 
   void finalizeInstrumentation() override {
+    const DataLayout &DL = F.getParent()->getDataLayout();
+
     assert(!VAArgOverflowSize && !VAArgTLSCopy &&
            "finalizeInstrumentation called twice");
     if (!VAStartInstrumentationList.empty()) {
       // If there is a va_start in this function, make a backup copy of
       // va_arg_tls somewhere in the function entry block.
       IRBuilder<> IRB(F.getEntryBlock().getFirstNonPHI());
       VAArgOverflowSize = IRB.CreateLoad(MS.VAArgOverflowSizeTLS);
       Value *CopySize =
Value* getVAField32(IRBuilder<> &IRB, Value *VAListTag, int offset) {
     Value *SaveArea32 = IRB.CreateLoad(SaveAreaPtr);
     return IRB.CreateSExt(SaveArea32, MS.IntptrTy);
   }
 
   void finalizeInstrumentation() override {
     assert(!VAArgOverflowSize && !VAArgTLSCopy &&
            "finalizeInstrumentation called twice");
     if (!VAStartInstrumentationList.empty()) {
+      const DataLayout &DL = F.getParent()->getDataLayout();
       // If there is a va_start in this function, make a backup copy of
       // va_arg_tls somewhere in the function entry block.
       IRBuilder<> IRB(F.getEntryBlock().getFirstNonPHI());
       VAArgOverflowSize = IRB.CreateLoad(MS.VAArgOverflowSizeTLS);
       Value *CopySize =
         IRB.CreateAdd(ConstantInt::get(MS.IntptrTy, AArch64VAEndOffset),
                       VAArgOverflowSize);
       VAArgTLSCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);

lib/Transforms/Scalar/RewriteStatepointsForGC.cpp

#endif
   DenseMap<Value *, Value *> AllocaMap;
   SmallVector<AllocaInst *, 200> PromotableAllocas;
   // Used later to chack that we have enough allocas to store all values
   std::size_t NumRematerializedValues = 0;
   PromotableAllocas.reserve(Live.size());
 
   // Emit alloca for "LiveValue" and record it in "allocaMap" and
   // "PromotableAllocas"
+  const DataLayout &DL = F.getParent()->getDataLayout();
   auto emitAllocaFor = [&](Value *LiveValue) {
-    AllocaInst *Alloca = new AllocaInst(LiveValue->getType(), "",
+    AllocaInst *Alloca = new AllocaInst(LiveValue->getType(),
+                                        DL.getAllocaAddrSpace(), "",
                                         F.getEntryBlock().getFirstNonPHI());
     AllocaMap[LiveValue] = Alloca;
     PromotableAllocas.push_back(Alloca);
   };
 
   // Emit alloca for each live gc pointer
   for (Value *V : Live)
     emitAllocaFor(V);

lib/Transforms/Scalar/SROA.cpp

if (LastSROAPrefix != StringRef::npos) {
           OldName = OldName.substr(OffsetEnd + 1);
       }
     }
     // Strip any SROA suffixes as well.
     OldName = OldName.substr(0, OldName.find(".sroa_"));
 #endif
 
     return getAdjustedPtr(IRB, DL, &NewAI,
-                          APInt(DL.getPointerSizeInBits(), Offset), PointerTy,
+                          APInt(DL.getPointerTypeSizeInBits(PointerTy), Offset),
+                          PointerTy,
 #ifndef NDEBUG
                           Twine(OldName) + "."
 #else
                           Twine()
 #endif
                           );
   }
 
Value *rewriteIntegerLoad(LoadInst &LI) {
     return V;
   }
 
   bool visitLoadInst(LoadInst &LI) {
     DEBUG(dbgs() << "    original: " << LI << "\n");
     Value *OldOp = LI.getOperand(0);
     assert(OldOp == OldPtr);
 
+    unsigned AS = LI.getPointerAddressSpace();
+
     Type *TargetTy = IsSplit ? Type::getIntNTy(LI.getContext(), SliceSize * 8)
                              : LI.getType();
     const bool IsLoadPastEnd = DL.getTypeStoreSize(TargetTy) > SliceSize;
     bool IsPtrAdjusted = false;
     Value *V;
     if (VecTy) {
       V = rewriteVectorizedLoadInst();
     } else if (IntTy && LI.getType()->isIntegerTy()) {
if (VecTy) {
         if (auto *TITy = dyn_cast<IntegerType>(TargetTy))
           if (AITy->getBitWidth() < TITy->getBitWidth()) {
             V = IRB.CreateZExt(V, TITy, "load.ext");
             if (DL.isBigEndian())
               V = IRB.CreateShl(V, TITy->getBitWidth() - AITy->getBitWidth(),
                                 "endian_shift");
           }
     } else {
-      Type *LTy = TargetTy->getPointerTo();
+      Type *LTy = TargetTy->getPointerTo(AS);
       LoadInst *NewLI = IRB.CreateAlignedLoad(getNewAllocaSlicePtr(IRB, LTy),
                                               getSliceAlign(TargetTy),
                                               LI.isVolatile(), LI.getName());
       if (LI.isVolatile())
         NewLI->setAtomic(LI.getOrdering(), LI.getSynchScope());
 
       V = NewLI;
       IsPtrAdjusted = true;
if (IsSplit) {
              "Non-byte-multiple bit width");
       // Move the insertion point just past the load so that we can refer to it.
       IRB.SetInsertPoint(&*std::next(BasicBlock::iterator(&LI)));
       // Create a placeholder value with the same type as LI to use as the
       // basis for the new value. This allows us to replace the uses of LI with
       // the computed value, and then replace the placeholder with LI, leaving
       // LI only used for this computation.
       Value *Placeholder =
-          new LoadInst(UndefValue::get(LI.getType()->getPointerTo()));
+          new LoadInst(UndefValue::get(LI.getType()->getPointerTo(AS)));
       V = insertInteger(DL, IRB, Placeholder, V, NewBeginOffset - BeginOffset,
                         "insert");
       LI.replaceAllUsesWith(V);
       Placeholder->replaceAllUsesWith(&LI);
       delete Placeholder;
     } else {
       LI.replaceAllUsesWith(V);
     }
if (NewBeginOffset == NewAllocaBeginOffset &&
                                  "endian_shift");
             V = IRB.CreateTrunc(V, AITy, "load.trunc");
           }
 
       V = convertValue(DL, IRB, V, NewAllocaTy);
       NewSI = IRB.CreateAlignedStore(V, &NewAI, NewAI.getAlignment(),
                                      SI.isVolatile());
     } else {
-      Value *NewPtr = getNewAllocaSlicePtr(IRB, V->getType()->getPointerTo());
+      unsigned AS = SI.getPointerAddressSpace();
+      Value *NewPtr = getNewAllocaSlicePtr(IRB, V->getType()->getPointerTo(AS));
       NewSI = IRB.CreateAlignedStore(V, NewPtr, getSliceAlign(V->getType()),
                                      SI.isVolatile());
     }
     NewSI->copyMetadata(SI, LLVMContext::MD_mem_parallel_loop_access);
     if (SI.isVolatile())
       NewSI->setAtomic(SI.getOrdering(), SI.getSynchScope());
     Pass.DeadInsts.insert(&SI);
     deleteIfTriviallyDead(OldOp);
if (!Alignment) {
       Alignment = DL.getABITypeAlignment(AI.getAllocatedType());
     }
     Alignment = MinAlign(Alignment, P.beginOffset());
     // If we will get at least this much alignment from the type alone, leave
     // the alloca's alignment unconstrained.
     if (Alignment <= DL.getABITypeAlignment(SliceTy))
       Alignment = 0;
     NewAI = new AllocaInst(
-        SliceTy, nullptr, Alignment,
+      SliceTy, AI.getType()->getAddressSpace(), nullptr, Alignment,
         AI.getName() + ".sroa." + Twine(P.begin() - AS.begin()), &AI);
     ++NumNewAllocas;
   }
 
   DEBUG(dbgs() << "Rewriting alloca partition "
                << "[" << P.beginOffset() << "," << P.endOffset()
                << ") to: " << *NewAI << "\n");
 

lib/Transforms/Utils/CodeExtractor.cpp

 /// the call instruction, splitting any PHI nodes in the header block as
 /// necessary.
 void CodeExtractor::
 emitCallAndSwitchStatement(Function *newFunction, BasicBlock *codeReplacer,
                            ValueSet &inputs, ValueSet &outputs) {
   // Emit a call to the new function, passing in: *pointer to struct (if
   // aggregating parameters), or plan inputs and allocated memory for outputs
   std::vector<Value*> params, StructValues, ReloadOutputs, Reloads;
-  
-  LLVMContext &Context = newFunction->getContext();
+  Module *M = newFunction->getParent();
+  LLVMContext &Context = M->getContext();
+  const DataLayout &DL = M->getDataLayout();
 
   // Add inputs as params, or to be filled into the struct
   for (Value *input : inputs)
     if (AggregateArgs)
       StructValues.push_back(input);
     else
       params.push_back(input);
 
   // Create allocas for the outputs
   for (Value *output : outputs) {
     if (AggregateArgs) {
       StructValues.push_back(output);
     } else {
       AllocaInst *alloca =
-          new AllocaInst(output->getType(), nullptr, output->getName() + ".loc",
-                         &codeReplacer->getParent()->front().front());
+        new AllocaInst(output->getType(), DL.getAllocaAddrSpace(),
+                       nullptr, output->getName() + ".loc",
       ReloadOutputs.push_back(alloca);
       params.push_back(alloca);
     }
   }
 
   StructType *StructArgTy = nullptr;
   AllocaInst *Struct = nullptr;
   if (AggregateArgs && (inputs.size() + outputs.size() > 0)) {
     std::vector<Type*> ArgTypes;
     for (ValueSet::iterator v = StructValues.begin(),
            ve = StructValues.end(); v != ve; ++v)
       ArgTypes.push_back((*v)->getType());
 
     // Allocate a struct at the beginning of this function
     StructArgTy = StructType::get(newFunction->getContext(), ArgTypes);
-    Struct = new AllocaInst(StructArgTy, nullptr, "structArg",
+    Struct = new AllocaInst(StructArgTy, DL.getAllocaAddrSpace(), nullptr,
+                            "structArg",
                             &codeReplacer->getParent()->front().front());
     params.push_back(Struct);
 
     for (unsigned i = 0, e = inputs.size(); i != e; ++i) {
       Value *Idx[2];
       Idx[0] = Constant::getNullValue(Type::getInt32Ty(Context));
       Idx[1] = ConstantInt::get(Type::getInt32Ty(Context), i);
       GetElementPtrInst *GEP = GetElementPtrInst::Create(

lib/Transforms/Utils/DemoteRegToStack.cpp

 /// the alloca inserted to create a stack slot for I.
 AllocaInst *llvm::DemoteRegToStack(Instruction &I, bool VolatileLoads,
                                    Instruction *AllocaPoint) {
   if (I.use_empty()) {
     I.eraseFromParent();
     return nullptr;
   }
 
+  Function *F = I.getParent()->getParent();
+  const DataLayout &DL = F->getParent()->getDataLayout();
+
   // Create a stack slot to hold the value.
   AllocaInst *Slot;
   if (AllocaPoint) {
-    Slot = new AllocaInst(I.getType(), nullptr,
+    Slot = new AllocaInst(I.getType(), DL.getAllocaAddrSpace(), nullptr,
                           I.getName()+".reg2mem", AllocaPoint);
   } else {
-    Function *F = I.getParent()->getParent();
-    Slot = new AllocaInst(I.getType(), nullptr, I.getName() + ".reg2mem",
-                          &F->getEntryBlock().front());
+    Slot = new AllocaInst(I.getType(), DL.getAllocaAddrSpace(), nullptr,
+                          I.getName() + ".reg2mem", &F->getEntryBlock().front());
   }
 
   // We cannot demote invoke instructions to the stack if their normal edge
   // is critical. Therefore, split the critical edge and create a basic block
   // into which the store can be inserted.
   if (InvokeInst *II = dyn_cast<InvokeInst>(&I)) {
     if (!II->getNormalDest()->getSinglePredecessor()) {
       unsigned SuccNum = GetSuccessorNumber(II->getParent(), II->getNormalDest());

 /// node and replaces it with a slot in the stack frame allocated via alloca.
 /// The PHI node is deleted. It returns the pointer to the alloca inserted.
 AllocaInst *llvm::DemotePHIToStack(PHINode *P, Instruction *AllocaPoint) {
   if (P->use_empty()) {
     P->eraseFromParent();
     return nullptr;
   }
 
+  const DataLayout &DL = P->getModule()->getDataLayout();
+
   // Create a stack slot to hold the value.
   AllocaInst *Slot;
   if (AllocaPoint) {
-    Slot = new AllocaInst(P->getType(), nullptr,
+    Slot = new AllocaInst(P->getType(), DL.getAllocaAddrSpace(), nullptr,
                           P->getName()+".reg2mem", AllocaPoint);
   } else {
     Function *F = P->getParent()->getParent();
-    Slot = new AllocaInst(P->getType(), nullptr, P->getName() + ".reg2mem",
+    Slot = new AllocaInst(P->getType(), DL.getAllocaAddrSpace(), nullptr,
+                          P->getName() + ".reg2mem",
                           &F->getEntryBlock().front());
   }
 
   // Iterate over each operand inserting a store in each predecessor.
   for (unsigned i = 0, e = P->getNumIncomingValues(); i < e; ++i) {
     if (InvokeInst *II = dyn_cast<InvokeInst>(P->getIncomingValue(i))) {
       assert(II->getParent() != P->getIncomingBlock(i) &&
              "Invoke edge not supported yet"); (void)II;

lib/Transforms/Utils/InlineFunction.cpp

 static Value *HandleByValArgument(Value *Arg, Instruction *TheCall,
                                   const Function *CalledFunc,
                                   InlineFunctionInfo &IFI,
                                   unsigned ByValAlignment) {
   PointerType *ArgTy = cast<PointerType>(Arg->getType());
   Type *AggTy = ArgTy->getElementType();
 
   Function *Caller = TheCall->getFunction();
+  const DataLayout &DL = Caller->getParent()->getDataLayout();
 
   // If the called function is readonly, then it could not mutate the caller's
   // copy of the byval'd memory.  In this case, it is safe to elide the copy and
   // temporary.
   if (CalledFunc->onlyReadsMemory()) {
     // If the byval argument has a specified alignment that is greater than the
     // passed in pointer, then we either have to round up the input pointer or
     // give up on this transformation.
     if (ByValAlignment <= 1)  // 0 = unspecified, 1 = no particular alignment.
       return Arg;
 
     AssumptionCache *AC =
         IFI.GetAssumptionCache ? &(*IFI.GetAssumptionCache)(*Caller) : nullptr;
-    const DataLayout &DL = Caller->getParent()->getDataLayout();
 
     // If the pointer is already known to be sufficiently aligned, or if we can
     // round it up to a larger alignment, then we don't need a temporary.
     if (getOrEnforceKnownAlignment(Arg, ByValAlignment, DL, TheCall, AC) >=
         ByValAlignment)
       return Arg;
-    
     // Otherwise, we have to make a memcpy to get a safe alignment.  This is bad
     // for code quality, but rarely happens and is required for correctness.
   }
 
   // Create the alloca.  If we have DataLayout, use nice alignment.
-  unsigned Align =
-      Caller->getParent()->getDataLayout().getPrefTypeAlignment(AggTy);
+  unsigned Align = DL.getPrefTypeAlignment(AggTy);
 
   // If the byval had an alignment specified, we *must* use at least that
   // alignment, as it is required by the byval argument (and uses of the
   // pointer inside the callee).
   Align = std::max(Align, ByValAlignment);
-  
-  Value *NewAlloca = new AllocaInst(AggTy, nullptr, Align, Arg->getName(), 
+  Value *NewAlloca = new AllocaInst(AggTy, DL.getAllocaAddrSpace(),
+                                    nullptr, Align, Arg->getName(),
                                     &*Caller->begin()->begin());
   IFI.StaticAllocas.push_back(cast<AllocaInst>(NewAlloca));
-  
   // Uses of the argument in the function should use our new alloca
   // instead.
   return NewAlloca;
 }
 
 // Check whether this Value is used by a lifetime intrinsic.
 static bool isUsedByLifetimeMarker(Value *V) {
   for (User *U : V->users()) {

test/Assembler/alloca-addrspace-parse-error-0.ll

+; RUN: not llvm-as < %s 2>&1 | FileCheck %s
+
+target datalayout = "A1"
+
+; CHECK: :8:3: error: expected metadata after comma
+define void @use_alloca() {
+  %alloca = alloca i32, addrspace(1),
+  ret void
+}
+
+!0 = !{}

test/Assembler/alloca-addrspace-parse-error-1.ll

+; RUN: not llvm-as < %s 2>&1 | FileCheck %s
+
+target datalayout = "A1"
+
+; addrspace and align in wrong order
+; CHECK: :8:39: error: expected metadata after comma
+define void @use_alloca() {
+  %alloca = alloca i32, addrspace(1), align 4
+  ret void
+}
+
+!0 = !{}

test/Assembler/alloca-addrspace0.ll

+; RUN: llvm-as < %s | llvm-dis | FileCheck %s
+
+target datalayout = "A0"
+; CHECK: target datalayout = "A0"
+
+
+; CHECK: %alloca_scalar_no_align = alloca i32
+; CHECK-NEXT: %alloca_scalar_align4 = alloca i32, align 4
+; CHECK-NEXT: %alloca_scalar_no_align_metadata = alloca i32, !foo !0
+; CHECK-NEXT: %alloca_scalar_align4_metadata = alloca i32, align 4, !foo !0
+; CHECK-NEXT: %alloca_inalloca_scalar_no_align = alloca inalloca i32
+; CHECK-NEXT: %alloca_inalloca_scalar_align4_metadata = alloca inalloca i32, align 4, !foo !0
+define void @use_alloca() {
+  %alloca_scalar_no_align = alloca i32, addrspace(0)
+  %alloca_scalar_align4 = alloca i32, align 4, addrspace(0)
+  %alloca_scalar_no_align_metadata = alloca i32, addrspace(0), !foo !0
+  %alloca_scalar_align4_metadata = alloca i32, align 4, addrspace(0), !foo !0
+  %alloca_inalloca_scalar_no_align = alloca inalloca i32, addrspace(0)
+  %alloca_inalloca_scalar_align4_metadata = alloca inalloca i32, align 4, addrspace(0), !foo !0
+
+  ret void
+}
+
+!0 = !{}

test/Assembler/datalayout-alloca-addrspace-mismatch-0.ll

+; RUN: not llvm-as < %s 2>&1 | FileCheck %s
+
+target datalayout = "A1"
+
+; CHECK: :7:41: error: address space must match datalayout
+define void @use_alloca() {
+  %alloca_scalar_no_align = alloca i32, addrspace(2)
+  ret void
+}

test/Assembler/datalayout-alloca-addrspace-mismatch-1.ll

+; RUN: not llvm-as < %s 2>&1 | FileCheck %s
+
+target datalayout = "A1"
+
+; CHECK: :7:50: error: address space must match datalayout
+define void @use_alloca() {
+  %alloca_scalar_no_align = alloca i32, align 4, addrspace(2)
+  ret void
+}

test/Assembler/datalayout-alloca-addrspace-mismatch-2.ll

+; RUN: not llvm-as < %s 2>&1 | FileCheck %s
+
+target datalayout = "A1"
+
+; CHECK: :7:50: error: address space must match datalayout
+define void @use_alloca() {
+  %alloca_scalar_no_align = alloca i32, align 4, addrspace(2), !foo !0
+  ret void
+}
+
+!0 = !{}

test/Assembler/datalayout-alloca-addrspace.ll

+; RUN: llvm-as < %s | llvm-dis | FileCheck %s
+
+target datalayout = "A1"
+; CHECK: target datalayout = "A1"
+
+; CHECK: %alloca_scalar_no_align = alloca i32, addrspace(1)
+; CHECK-NEXT: %alloca_scalar_align4 = alloca i32, align 4, addrspace(1)
+; CHECK-NEXT: %alloca_scalar_no_align_metadata = alloca i32, addrspace(1), !foo !0
+; CHECK-NEXT: %alloca_scalar_align4_metadata = alloca i32, align 4, addrspace(1), !foo !0
+; CHECK-NEXT: %alloca_inalloca_scalar_no_align = alloca inalloca i32, addrspace(1)
+; CHECK-NEXT: %alloca_inalloca_scalar_align4_metadata = alloca inalloca i32, align 4, addrspace(1), !foo !0
+define void @use_alloca() {
+  %alloca_scalar_no_align = alloca i32, addrspace(1)
+  %alloca_scalar_align4 = alloca i32, align 4, addrspace(1)
+  %alloca_scalar_no_align_metadata = alloca i32, addrspace(1), !foo !0
+  %alloca_scalar_align4_metadata = alloca i32, align 4, addrspace(1), !foo !0
+  %alloca_inalloca_scalar_no_align = alloca inalloca i32, addrspace(1)
+  %alloca_inalloca_scalar_align4_metadata = alloca inalloca i32, align 4, addrspace(1), !foo !0
+
+  ret void
+}
+
+!0 = !{}

test/Assembler/invalid-datalayout-alloca-addrspace.ll

+; RUN: not llvm-as < %s 2>&1 | FileCheck %s
+
+target datalayout = "A16777216"
+; CHECK: Invalid address space, must be a 24bit integer

test/Transforms/SROA/alloca-address-space.ll

+; RUN: opt < %s -sroa -S | FileCheck %s
+target datalayout = "e-p:64:64:64-p1:16:16:16-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:32:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-n8:16:32:64-A2"
+
+declare void @llvm.memcpy.p2i8.p2i8.i32(i8 addrspace(2)* nocapture, i8 addrspace(2)* nocapture readonly, i32, i32, i1)
+declare void @llvm.memcpy.p1i8.p2i8.i32(i8 addrspace(1)* nocapture, i8 addrspace(2)* nocapture readonly, i32, i32, i1)
+declare void @llvm.memcpy.p2i8.p1i8.i32(i8 addrspace(2)* nocapture, i8 addrspace(1)* nocapture readonly, i32, i32, i1)
+declare void @llvm.memcpy.p1i8.p1i8.i32(i8 addrspace(1)* nocapture, i8 addrspace(1)* nocapture readonly, i32, i32, i1)
+
+
+
+; CHECK-LABEL: @test_address_space_1_1(
+; CHECK: load <2 x i64>, <2 x i64> addrspace(1)* %a, align 2
+; CHECK: store <2 x i64> {{.*}}, <2 x i64> addrspace(1)* {{.*}}, align 2
+; CHECK: ret void
+define void @test_address_space_1_1(<2 x i64> addrspace(1)* %a, i16 addrspace(1)* %b) {
+  %aa = alloca <2 x i64>, align 16, addrspace(2)
+  %aptr = bitcast <2 x i64> addrspace(1)* %a to i8 addrspace(1)*
+  %aaptr = bitcast <2 x i64> addrspace(2)* %aa to i8 addrspace(2)*
+  call void @llvm.memcpy.p2i8.p1i8.i32(i8 addrspace(2)* %aaptr, i8 addrspace(1)* %aptr, i32 16, i32 2, i1 false)
+  %bptr = bitcast i16 addrspace(1)* %b to i8 addrspace(1)*
+  call void @llvm.memcpy.p1i8.p2i8.i32(i8 addrspace(1)* %bptr, i8 addrspace(2)* %aaptr, i32 16, i32 2, i1 false)
+  ret void
+}
+
+; CHECK-LABEL: @test_address_space_1_0(
+; CHECK: load <2 x i64>, <2 x i64> addrspace(1)* %a, align 2
+; CHECK: store <2 x i64> {{.*}}, <2 x i64> addrspace(2)* {{.*}}, align 2
+; CHECK: ret void
+define void @test_address_space_1_0(<2 x i64> addrspace(1)* %a, i16 addrspace(2)* %b) {
+  %aa = alloca <2 x i64>, align 16, addrspace(2)
+  %aptr = bitcast <2 x i64> addrspace(1)* %a to i8 addrspace(1)*
+  %aaptr = bitcast <2 x i64> addrspace(2)* %aa to i8 addrspace(2)*
+  call void @llvm.memcpy.p2i8.p1i8.i32(i8 addrspace(2)* %aaptr, i8 addrspace(1)* %aptr, i32 16, i32 2, i1 false)
+  %bptr = bitcast i16 addrspace(2)* %b to i8 addrspace(2)*
+  call void @llvm.memcpy.p2i8.p2i8.i32(i8 addrspace(2)* %bptr, i8 addrspace(2)* %aaptr, i32 16, i32 2, i1 false)
+  ret void
+}
+
+; CHECK-LABEL: @test_address_space_0_1(
+; CHECK: load <2 x i64>, <2 x i64> addrspace(2)* %a, align 2
+; CHECK: store <2 x i64> {{.*}}, <2 x i64> addrspace(1)* {{.*}}, align 2
+; CHECK: ret void
+define void @test_address_space_0_1(<2 x i64> addrspace(2)* %a, i16 addrspace(1)* %b) {
+  %aa = alloca <2 x i64>, align 16, addrspace(2)
+  %aptr = bitcast <2 x i64> addrspace(2)* %a to i8 addrspace(2)*
+  %aaptr = bitcast <2 x i64> addrspace(2)* %aa to i8 addrspace(2)*
+  call void @llvm.memcpy.p2i8.p2i8.i32(i8 addrspace(2)* %aaptr, i8 addrspace(2)* %aptr, i32 16, i32 2, i1 false)
+  %bptr = bitcast i16 addrspace(1)* %b to i8 addrspace(1)*
+  call void @llvm.memcpy.p1i8.p2i8.i32(i8 addrspace(1)* %bptr, i8 addrspace(2)* %aaptr, i32 16, i32 2, i1 false)
+  ret void
+}
+
+%struct.struct_test_27.0.13 = type { i32, float, i64, i8, [4 x i32] }
+
+; CHECK-LABEL: @copy_struct(
+; CHECK-NOT: memcpy
+define void @copy_struct([5 x i64] %in.coerce) {
+for.end:
+  %in = alloca %struct.struct_test_27.0.13, align 8, addrspace(2)
+  %0 = bitcast %struct.struct_test_27.0.13 addrspace(2)* %in to [5 x i64] addrspace(2)*
+  store [5 x i64] %in.coerce, [5 x i64] addrspace(2)* %0, align 8
+  %scevgep9 = getelementptr %struct.struct_test_27.0.13, %struct.struct_test_27.0.13 addrspace(2)* %in, i32 0, i32 4, i32 0
+  %scevgep910 = bitcast i32 addrspace(2)* %scevgep9 to i8 addrspace(2)*
+  call void @llvm.memcpy.p1i8.p2i8.i32(i8 addrspace(1)* undef, i8 addrspace(2)* %scevgep910, i32 16, i32 4, i1 false)
+  ret void
+}
+
+%union.anon = type { i32* }
+
+@g = common global i32 0, align 4
+@l = common addrspace(3) global i32 0, align 4
+
+; Make sure an illegal bitcast isn't introduced
+; CHECK-LABEL: @pr27557(
+; CHECK: %[[CAST:.*]] = bitcast i32* addrspace(2)* {{.*}} to i32 addrspace(3)* addrspace(2)*
+; CHECK: store i32 addrspace(3)* @l, i32 addrspace(3)* addrspace(2)* %[[CAST]]
+define void @pr27557() {
+  %1 = alloca %union.anon, align 8, addrspace(2)
+  %2 = bitcast %union.anon addrspace(2)* %1 to i32* addrspace(2)*
+  store i32* @g, i32* addrspace(2)* %2, align 8
+  %3 = bitcast %union.anon addrspace(2)* %1 to i32 addrspace(3)* addrspace(2)*
+  store i32 addrspace(3)* @l, i32 addrspace(3)* addrspace(2)* %3, align 8
+  ret void
+}

tools/llvm-stress/llvm-stress.cpp

struct ConstModifier: public Modifier {
   }
 };
 
 struct AllocaModifier: public Modifier {
   AllocaModifier(BasicBlock *BB, PieceTable *PT, Random *R):Modifier(BB, PT, R){}
 
   void Act() override {
     Type *Tp = pickType();
-    PT->push_back(new AllocaInst(Tp, "A", BB->getFirstNonPHI()));
+    const DataLayout &DL = BB->getModule()->getDataLayout();
+    PT->push_back(new AllocaInst(Tp, DL.getAllocaAddrSpace(),
+                                 "A", BB->getFirstNonPHI()));
   }
 };
 
 struct ExtractElementModifier: public Modifier {
   ExtractElementModifier(BasicBlock *BB, PieceTable *PT, Random *R):
     Modifier(BB, PT, R) {}
 
   void Act() override {

unittests/Analysis/ScalarEvolutionTest.cpp

TEST_F(ScalarEvolutionsTest, ExpandPtrTypeSCEV) {
   //   %bitcast1 = bitcast i32* %gep0 to i8*
   //   %gep1 = getelementptr i8, i8* %bitcast1, i32 1
   //   %gep2 = getelementptr i8, i8* undef, i32 1
   //   %cmp = icmp ult i8* undef, %bitcast1
   //   %select = select i1 %cmp, i8* %gep1, i8* %gep2
   //   %bitcast2 = bitcast i8* %select to i32*
   //   br i1 undef, label %loop, label %exit
 
+  const DataLayout &DL = F->getParent()->getDataLayout();
   BranchInst *Br = BranchInst::Create(
       LoopBB, ExitBB, UndefValue::get(Type::getInt1Ty(Context)), LoopBB);
-  AllocaInst *Alloca = new AllocaInst(I32Ty, "alloca", Br);
+  AllocaInst *Alloca = new AllocaInst(I32Ty, DL.getAllocaAddrSpace(),
+                                      "alloca", Br);
   ConstantInt *Ci32 = ConstantInt::get(Context, APInt(32, 1));
   GetElementPtrInst *Gep0 =
       GetElementPtrInst::Create(I32Ty, Alloca, Ci32, "gep0", Br);
   CastInst *CastA =
       CastInst::CreateBitOrPointerCast(Gep0, I8PtrTy, "bitcast1", Br);
   GetElementPtrInst *Gep1 =
       GetElementPtrInst::Create(I8Ty, CastA, Ci32, "gep1", Br);
   GetElementPtrInst *Gep2 = GetElementPtrInst::Create(