This is an archive of the discontinued LLVM Phabricator instance.

ext_address_space attribute
ClosedPublic

Authored by agozillon on May 29 2017, 11:03 PM.

Details

Summary

The addition of a new attribute ext_address_space that supplements the existing address_space attribute by allowing dependent integers to be used in conjunction with address spaces. It takes a lot of inspiration from the existing ext_vector_type attribute which similarly allows the acceptance of dependent integer values. This addition allows address spaces to be used in conjunction with templates like the following basic function:

template <unsigned Nv>
int get_as(int attribute((ext_address_space(Nv))) *) {

return Nv;

}

This modification requires the addition of a new attribute ext_address_space and a new type DependentExtAddressSpaceType, alongside the relevant boiler plate code required for these new additions. Such as case statements for template deduction and name mangling. The ext_ part of the attribute name is following the naming convention of ext_vector_type, as it currently extends the existing address_space attributes functionality. The DependentExtAddressSpaceType is a new type for storing the dependent address space integer passed to it and the type it's to be attached to. DependentExtAddressSpaceType is based on ext_vector_type's DependentSizedExtVectorType, so future modifications to this type could also be considered for DependentExtAddressSpaceType. One large example is the addition of a more specific type location class.

There is one difference between the ext_vector_type and ext_address_space attributes and it's that ext_address_space can be used outside of using and typedef declarations.

The handling of the ext_address_space attribute inside SemaType.cpp is similar to the address_space attribute and aims to emit the same diagnostic errors and functionality that it does. In fact it would be possible to integrate the both of them. However, I did not wish to during my implementation of this functionality in-case it had adverse effects.

Diff Detail

Event Timeline

agozillon created this revision.May 29 2017, 11:03 PM
agozillon edited reviewers, added: doug.gregor; removed: rjmccall.May 29 2017, 11:06 PM
rjmccall added a subscriber: rjmccall.

ext_vector_type is a different attribute from vector_type because the argument has different semantics (it's the element count instead of the vector size) and the resulting vector type has different type-checking behavior (e.g. with member expressions). These do not seem to be true of ext_address_space. You should use the same attribute spelling and just allow the argument expression to be value-dependent.

I'm uncertain about this representation. Address-space qualifiers are still fundamentally qualifiers, but this turns dependent address spaces into something very different from a non-dependent address space. We could make Qualifiers / ExtQuals capable of storing dependent address-space expressions. That would at least ensure that getUnqualifiedType() automatically did the right thing with them.

include/clang/AST/Type.h
1713

It is better style to just wrap the comment here. The comment is also needlessly redundant; I would just say "value-dependent address space qualifier".

include/clang/Serialization/ASTBitCodes.h
937

Reformatting this block doesn't seem completely unreasonable, because the indentation is not completely consistent, but if you want to do it, please do it as a separate commit.

Anastasia edited edge metadata.May 31 2017, 8:10 AM

Btw, in OpenCL C++ similar issue is solved using storage classes (similar to smart pointers), see https://www.khronos.org/registry/OpenCL/specs/opencl-2.2-cplusplus.pdf#40 s2.5 for more details. Wondering if this approach could help here too although I could imagine the issue might be with the fact that any arbitrary number is used for AS in C/C++ instead of a known set as in OpenCL.

agozillon updated this revision to Diff 101511.Jun 5 2017, 10:49 PM

Modifications based on the Type.h and ASTBitcodes.h comments made by @rjmccall namely modifying the isDependentExtAddressSpaceType() comment and reverting the formatting performed on the ASTBitcodes.h TypeCode enumerator.

I would still like more people's input on whether a Type node is the right way to represent this vs. something on ExtQuals. I guess I am leaning towards saying that it should be on ExtQuals unless we think there's something deeply problematic about having a dependent type qualifier.

include/clang/AST/ASTContext.h
1283

Why is this called PointerType? An address space qualifier goes on the pointee type.

That question applies to a lot of this patch.

include/clang/AST/TypeLoc.h
1696

If we leave this as a Type node, would you mind just doing this? Most of the other FIXMEs here date from when we added TypeLoc originally; it would be nice to avoid accumulating new ones.

include/clang/Sema/ExternalSemaSource.h
146 ↗(On Diff #101511)

You don't actually seem to have done anything with this idea; you're just accumulating them.

It might be reasonable to teach the type printer to be more reluctant about looking through address space typedefs, like how it's reluctant to look through vector typedefs. But you'd have to be careful, because the typedef can also accumulate sugar on the underlying type.

lib/AST/ASTContext.cpp
3133

The code that works with qualifiers often strips qualifiers and then re-applies them. I feel it's important to deviate from the model of DependentlySizedExtVectorType here and make an effort to produce the same Type object when the expression and type match exactly.

3144

Please hoist this to where you called getCanonicalType earlier.

lib/AST/ItaniumMangle.cpp
2976

Did you just make up this mangling? You should at least make an issue for it on the C++ ABI:

https://github.com/itanium-cxx-abi/cxx-abi
lib/Sema/SemaTemplateDeduction.cpp
1829

Shouldn't this be ">="? And I think you don't need the 0 +.

lib/Sema/SemaType.cpp
5390

The outer check is stronger than the inner check. I think you should just be doing getAs<DependentExtAddressSpaceType>().

5592

This definitely need to be a getAs<>. It's not okay to stack up these qualifiers no matter how you've done it.

Or you could split the type and see if the Qualifiers contains a dependent address space qualifier.

Okay, well, nobody else is cooperating and sending feedback on the design, and it's not dissimilar from the design we already use for AtomicType, so I guess we can commit to it unless Richard wants to weigh in.

agozillon marked 2 inline comments as done.Aug 9 2017, 11:55 AM

Btw, in OpenCL C++ similar issue is solved using storage classes (similar to smart pointers), see https://www.khronos.org/registry/OpenCL/specs/opencl-2.2-cplusplus.pdf#40 s2.5 for more details. Wondering if this approach could help here too although I could imagine the issue might be with the fact that any arbitrary number is used for AS in C/C++ instead of a known set as in OpenCL.

lib/AST/ASTContext.cpp
3133

In this case if you mean just returning the result of the FindNodeOrInsertPos call if it finds an existing node, I'm unsure it will work. I did test to make sure however.

The main issue in that case is that the SourceLocation is tied to the type, which is needed to print out diagnostic errors if something goes wrong during instantiation. Returning the Node will in most cases return a type with the incorrect SourceLocation which can lead to some oddly placed error messages.

agozillon updated this revision to Diff 110695.Aug 11 2017, 6:10 AM

Tried to address all of the points made in the last review, a few should be ticked off the list. However, a couple will most definitely need revisited as I need a little more direction on them I shall leave these points unchecked for now (mostly the mangling and the one revision I have replied to already). It is possible that some of the revisions I check off will be considered incorrect, so it's fine to reopen them with a little more input!

The largest change was the addition of a new TypeLoc class in relation to the DependentExtAddressSpaceType. Most of the others were trivial modifications, other than the Mangling which is still a work in progress.

agozillon marked 6 inline comments as done.Aug 11 2017, 6:35 AM
agozillon added inline comments.
lib/AST/ItaniumMangle.cpp
2976

A little more direction on what to do with the qualifiers in this Mangling would be appreciated. I was under the incorrect assumption that passing the expression and type to there own mangling variation would handle the qualifiers.

rjmccall added inline comments.Aug 11 2017, 10:40 AM
lib/AST/ItaniumMangle.cpp
2976

Since your type is really just a qualifier, and it's not naturally ordered with other qualifiers (i.e. 'attribute((address_space(N))) const' is the same thing as 'const attribute((address_space(N)))'), you have to actually think about how other qualifiers will be canonically represented vis-a-vis your node. That is, will you always put other qualifiers "outside" your node or "inside" it? This is a much bigger issue than just mangling and really affects basically everything in the compiler that deals with qualifiers.

  • However you decide that, the ABI then has a requirement about the order in which unordered qualifiers are mangled, with the goal of ensuring that they are always mangled in the same order instead of randomly jumbled about based on how the type was written.

These reasons are part of why I was arguing that you might want to store the expression on ExtQuals instead of making it a Type node — the idea being that a type can have some sort of address-space qualifier, and that qualifier might be dependent (an Expr*) or not (an unsigned). The main difficulty with doing that, I think, is that you'll be introducing the concept of a dependent type qualifier, which means that things like Type::isDependentType() will need to be pulled up to QualType and you'll have to go touch a lot of code. But I think on balance that might save you a lot of grief, and I'm sure we'll eventually find other uses for dependent qualifiers.

agozillon updated this revision to Diff 113592.Sep 1 2017, 2:37 PM

I have made modifications to the DependentExtAddressSpaceType mangleType function to follow the ABI specification. I am unsure if it's entirely correct yet (as such I would greatly appreciate some verification), but I believe it's moving in the right direction.

At the moment I believe the output from the mangling for a type of: const volatile int attribute((ext_address_space(N))) *, where the N is a dependent variable that is constant would be - PU2ASIKT_EVKi (however, please do correct me if my understanding is incorrect).

rjmccall added inline comments.Sep 1 2017, 2:55 PM
lib/AST/ItaniumMangle.cpp
2982

Mangling the qualifiers on the address-space expression's type is not correct. Just mangle the expression.

agozillon updated this revision to Diff 113655.Sep 2 2017, 11:28 AM

Removal of the Address Space Expr qualifier mangling that was not required, the mangling should now be correct I believe.

I think I have addressed all of the currently marked review points.

agozillon marked 3 inline comments as done.Sep 2 2017, 11:32 AM

I have marked the mangling review point as "Done" as I believe it is now addressed, however if it still needs further modification I am of course happy to re-open the review point and make amends.

rjmccall added inline comments.Sep 7 2017, 11:31 AM
include/clang/AST/Type.h
2802

I thought we had agreed that this should be just be the normal address_space attribute, but with a dependent expression.

lib/AST/ItaniumMangle.cpp
2980

I think the only reasonable way to get qualifier-ordering correctness here is going to be to make mangleQualifiers take an optional DepenentAddressSpaceType* and, if it's present, mangle it in the appropriate position.

You should call split() on T->getPointeeType() and make sure that the recursive call here mangles the unqualified type.

lib/Sema/SemaType.cpp
5542

Type-dependence implies value-dependence.

rjmccall added inline comments.Sep 7 2017, 11:48 AM
lib/Sema/SemaType.cpp
5542

Oh, but you actually need to build a DependentAddressSpaceType if the expression is instantiation-dependent, not just value-dependent. IIRC, the test case would be something like:

address_space(true ? 0 : some_expression_that_might_not_instantiate)
agozillon added inline comments.Sep 8 2017, 11:09 AM
include/clang/AST/Type.h
2802

By this do you mean remove the ext_address_space notation and just merge the functionality with the current address_space attribute? If so I shall do this, I was a little unsure on what we had decided previously so I apologies for not modifying it sooner.

rjmccall added inline comments.Sep 8 2017, 11:24 AM
include/clang/AST/Type.h
2802

Yes, that's what I mean. There's no semantic difference, it's a pure generalization of the current attribute.

agozillon updated this revision to Diff 114530.Sep 10 2017, 5:31 PM

Merged old ext_address_space functionality into address_space. So now address_space can work with template parameters.

I made modifications to the mangling again, I am a little unsure where in the mangleQualifiers the dependent mangling should go however (so it will most likely need further tweaking). e.g. before or after the non-dependent AS mangling.

I also modified the if statement to check for value dependency and instantiation dependency, rather than value and type dependency.

rjmccall added inline comments.Sep 11 2017, 5:07 PM
include/clang/AST/RecursiveASTVisitor.h
994

Is there actually a valid case where this expression is null?

include/clang/AST/Type.h
2749

The second sentence in this comment isn't quite how I'd put it. How about:

Non-dependent address spaces are not represented with a special Type subclass; they
are stored on an ExtQuals node as part of a QualType.
2766

Indentation.

include/clang/AST/TypeLoc.h
1682

Might as well have the ~~~~ cover the entire attribute name, just to avoid confusion.

1692

Indentation so that this lines up with the 11 again.

1703

Indentation.

include/clang/Sema/Sema.h
1373

Indentation.

lib/AST/ASTContext.cpp
2311

Capitalization, ending period.

2317

I'd just drop this comment.

2320

Capitalization.

3126

Please include an assertion that the expression is instantiation-dependent.

3133

You'd just be hashing by Expr* identity here, so it's likely that the source locations would be the same.

It would not be correct to try to hash by Expr structural equality.

lib/AST/ItaniumMangle.cpp
2203

This is the right place to put this in this function. It can't be combined with a non-dependent address space, and it comes alphabetically after all the others (because they start with underscores).

The more important thing is that you need to make sure that all the places which call mangleQualifiers also try to destructure a DependentAddressSpaceType at the same time. This might mean that the normal path for mangling a DependentAddressSpaceType never gets called.

lib/Sema/SemaDecl.cpp
14350

T->isDependentAddressSpaceType()

14352

T->getBaseElementTypeUnsafe()->isDependentAddressSpaceType()

lib/Sema/SemaType.cpp
5410

This doesn't seem right. The nested TypeLoc in a DependentAddressSpaceType should be for the underlying type; you shouldn't be separately allocating anything.

5503

Why did you remove the "Type Attribute Processing" section header?

5507

Value dependence implies instantiation dependence.

5642

Please remove dead code instead of just commenting it out.

agozillon updated this revision to Diff 115535.Sep 16 2017, 6:16 AM
agozillon marked 25 inline comments as done.

With these patch modifications I believe I corrected all of the incorrect indentation and commenting notified from the last set of review points. I also brought back the Type Attribute Processing header which I accidentally removed (sorry!). I also marked off all of the smaller review points that I modified to reflect the modifications made.

The first of the larger modifications I made is that I modified the TypeLoc code (@5410~) in SemaType.cpp to perform more like AtomicTypeLoc and AttributedTypeLoc, which I believe is the correct way to deal with DependentAddressSpaceTypeLoc in this scenario (correct me if I am wrong however).

I also added destructuring of DependentAddressSpaceType's in relevant places where mangleQualifiers gets invoked inside ItaniumMangle.cpp. The two locations where I don't destructure a DependentAddressSpaceType before mangleQualifiers is invoked are in the FunctionProtoType overload and the mangleNestedName's function. In both cases because they're dealing with types that are not DependentAddressSpaceType's and they're also only looking for CVR qualifiers (no address space qualifiers or vendor extensions). I believe this is correct, but as always feel free to correct me.

I also modified getDependentAddressSpaceType inside ASTContext.cpp to try and return the same Type when the Type and Expr are the same, however I think it still needs some work. I am unsure in this case what's meant by hashing by Expr* rather than Expr structural equality. In this case would it mean ID.AddPointer(AddrSpaceExpr) instead of AddrSpaceExpr->Profile(ID, Context, true) inside the DependentAddressSpaceType profile function? The former hashing by a pointer and the latter hashing by strucutre?

Thank you for your time.

include/clang/AST/RecursiveASTVisitor.h
994

I don't believe there is, any time a DependentAddressSpaceType is instantiated their should be an Expr to go with it.

lib/Sema/SemaType.cpp
5642

Sorry, I forgot to remove this before modifying the patch! Thank you for pointing it out.

The first of the larger modifications I made is that I modified the TypeLoc code (@5410~) in SemaType.cpp to perform more like AtomicTypeLoc and AttributedTypeLoc, which I believe is the correct way to deal with DependentAddressSpaceTypeLoc in this scenario (correct me if I am wrong however).

That looks right, thanks.

I also added destructuring of DependentAddressSpaceType's in relevant places where mangleQualifiers gets invoked inside ItaniumMangle.cpp. The two locations where I don't destructure a DependentAddressSpaceType before mangleQualifiers is invoked are in the FunctionProtoType overload and the mangleNestedName's function. In both cases because they're dealing with types that are not DependentAddressSpaceType's and they're also only looking for CVR qualifiers (no address space qualifiers or vendor extensions). I believe this is correct, but as always feel free to correct me.

Yes, those are more restricted cases where this qualifier is not possible.

I also modified getDependentAddressSpaceType inside ASTContext.cpp to try and return the same Type when the Type and Expr are the same, however I think it still needs some work. I am unsure in this case what's meant by hashing by Expr* rather than Expr structural equality. In this case would it mean ID.AddPointer(AddrSpaceExpr) instead of AddrSpaceExpr->Profile(ID, Context, true) inside the DependentAddressSpaceType profile function? The former hashing by a pointer and the latter hashing by strucutre?

Yes, exactly. Essentially we're saying that types written in different places in the source code are always different types.

...but when I really stop to think about it, that doesn't actually seem like the right rule. That's the rule we use for VariableArrayType, because that C99 says that different VLA types are always different, which makes sense since the expression may actually be different in different contexts. But it's probably not the right rule for this, because the expression isn't really dynamic like a VLA bound, it's just dependent. Comparing expressions by equivalence rather than source equality is the rule used in order to test whether e.g. a function template is a redeclaration of another. It's the rule used by DependentSizedArrayType, and it's almost certainly the right rule to use here, too. I suggest following the basic model used by getDependentSizedArrayType and its corresponding Profile methods.

Thank you for your time.

The main modifications with this change is to make the getDependentAddressSpaceType function similarly to getDependentSizedArrayType. Alongside modifying the DependentAddressSpaceType profile to hash by Expr structure similarly to DependentSizedArrayType.

The getDependentAddressSpaceType function couldn't be made to function exactly the same, mainly as getDependentSizedArrayType moves the qualifiers from its internal type (elementType) onto itself. Which doesn't work in the case of DependentAddressSpaceType, I believe. There was also a case for a scenario where the sizeExpr was null, however the AddrSpaceExpr should never be null as far as I'm aware so the case would never be invoked. That's the two main modifications that separate the two types get functions. However, if the function is still incorrect I am as always happy to make further changes.

I think this is the last modification asked for at the moment.

Thanks again.

Looks great. A few test requests.

test/Sema/address_space-dependent.cpp
1 ↗(On Diff #115600)

This file belongs in test/SemaTemplate/.

Please add a test to verify that two identical template function declarations expressed using the same dependent address space are treated as redeclarations of each other. I think this should work:

template <unsigned A> int __attribute__((address_space(A))) *same_template();
template <unsigned B> int __attribute__((address_space(B))) *same_template();
void test_same_template() { (void) same_template<0>(); }

template <unsigned A> int __attribute__((address_space(A))) *different_template();
template <unsigned B> int __attribute__((address_space(B+1))) *different_template();
void test_same_template() { (void) different_template<0>(); } // should give an ambiguity error

Please also add tests that class template partial specializations can successfully deduce an address space, e.g.:

template <class T> struct partial_spec;
template <class T, unsigned AS> struct partial_spec<__attribute__((address_space(AS))) T *> {};
partial_spec<__attribute__((address_space(0))) int *> test_partial_spec = {};
agozillon updated this revision to Diff 116463.Sep 23 2017, 8:11 AM

Adding the new mentioned tests and moving the test file to the correct specified folder.

agozillon marked 6 inline comments as done.Sep 23 2017, 8:42 AM

Setting older review points to done and setting the newer Test review point to done as I have now added the new tests and moved it to the appropriate folder.

rjmccall accepted this revision.Sep 25 2017, 10:25 AM

LGTM, thanks!

This revision is now accepted and ready to land.Sep 25 2017, 10:25 AM
agozillon added a comment.EditedSep 25 2017, 2:02 PM

LGTM, thanks!

Sorry to bother you again. As I am quite new to the Clang/LLVM review process I was wondering what the next steps after a revisions acceptance are and if I have any further part to play? I did have a read through some of the documentation on reviews but I unfortunately couldn't find any further information.

LGTM, thanks!

Sorry to bother you again. As I am quite new to the Clang/LLVM review process I was wondering what the next steps after a revisions acceptance are and if I have any further part to play? I did have a read through some of the documentation on reviews but I unfortunately couldn't find any further information.

Once the patch is accepted, and you've tested it locally to your satisfaction, you can go ahead and commit. It's then your responsibility to pay attention to any build-bot failures — which you should get email notification of — for a day or so to check that they don't seem related to your patch.

Do you have commit access?

LGTM, thanks!

Sorry to bother you again. As I am quite new to the Clang/LLVM review process I was wondering what the next steps after a revisions acceptance are and if I have any further part to play? I did have a read through some of the documentation on reviews but I unfortunately couldn't find any further information.

Once the patch is accepted, and you've tested it locally to your satisfaction, you can go ahead and commit. It's then your responsibility to pay attention to any build-bot failures — which you should get email notification of — for a day or so to check that they don't seem related to your patch.

Do you have commit access?

I unfortunately do not have commit access at the moment, I wasn't sure if I would be able to get commit access as a first time contributor. Would applying for commit access be possible at this point?

LGTM, thanks!

Sorry to bother you again. As I am quite new to the Clang/LLVM review process I was wondering what the next steps after a revisions acceptance are and if I have any further part to play? I did have a read through some of the documentation on reviews but I unfortunately couldn't find any further information.

Once the patch is accepted, and you've tested it locally to your satisfaction, you can go ahead and commit. It's then your responsibility to pay attention to any build-bot failures — which you should get email notification of — for a day or so to check that they don't seem related to your patch.

Do you have commit access?

I unfortunately do not have commit access at the moment, I wasn't sure if I would be able to get commit access as a first time contributor. Would applying for commit access be possible at this point?

If you intend to stick around and keep contributing patches, go ahead and ask for commit access.

John.

LGTM, thanks!

Sorry to bother you again. As I am quite new to the Clang/LLVM review process I was wondering what the next steps after a revisions acceptance are and if I have any further part to play? I did have a read through some of the documentation on reviews but I unfortunately couldn't find any further information.

Once the patch is accepted, and you've tested it locally to your satisfaction, you can go ahead and commit. It's then your responsibility to pay attention to any build-bot failures — which you should get email notification of — for a day or so to check that they don't seem related to your patch.

Do you have commit access?

I unfortunately do not have commit access at the moment, I wasn't sure if I would be able to get commit access as a first time contributor. Would applying for commit access be possible at this point?

If you intend to stick around and keep contributing patches, go ahead and ask for commit access.

John.

I have now gained commit access and committed the modifications from this review: https://github.com/llvm-mirror/clang/commit/d27ee8f21eebc00294112a46121537238cdda59f

Thank you very much for your time, help and patience over the course of this review John.

This revision was automatically updated to reflect the committed changes.