[Matrix] Implement matrix index expressions ([][]).

[fhahn]

Summary

This patch implements matrix index expressions
(matrix[RowIdx][ColumnIdx]).

It does so by introducing a new MatrixSubscriptExpr(Base, RowIdx, ColumnIdx).
MatrixSubscriptExprs are built in 2 steps in ActOnMatrixSubscriptExpr. First,
if the base of a subscript is of matrix type, we create a incomplete
MatrixSubscriptExpr(base, idx, nullptr). Second, if the base is an incomplete
MatrixSubscriptExpr, we create a complete
MatrixSubscriptExpr(base->getBase(), base->getRowIdx(), idx)

Similar to vector elements, it is not possible to take the address of
a MatrixSubscriptExpr.
For CodeGen, a new MatrixElt type is added to LValue, which is very
similar to VectorElt. The only difference is that we may need to cast
the type of the base from an array to a vector type when accessing it.

[fhahn]

Use placeholder type for incomplete matrix index expressions, as suggested by @rjmccall

[fhahn]

Rebase on top of the latest version of D72281. Refactor code to re-use new address conversion helper.

[fhahn]

Update to support non-constant-integer-expressions as indices.

[rjmccall]

Sorry for the slow review; I'm getting crushed with review requests.

clang/lib/AST/Expr.cpp

3859

You need to IgnoreParens() here.

clang/lib/AST/ExprConstant.cpp

7768

This can also occur in the other evaluators due to indexing into an r-value.

clang/lib/Sema/SemaExpr.cpp

5326

It should have the value kind of its base. get_matrix_rvalue()[0][0] is still an r-value, so you shouldn't allow it to be assigned to. Needs tests.

I'm fine with re-using OK_VectorComponent here, but you should (1) rename it to OK_VectorOrMatrixComponent and (2) take the opportunity to audit the places that use it to make sure they do something sensible. I expect you'll need to at least update some diagnostics about e.g. disallowing taking the address of the expression.

I think you should build a new expression node instead of reusing ArraySubscriptExpr, since basically none of the code that looks for an ArraySubscriptExpr will do the right thing for your operation. That will also allow you to avoid the "is this actually enabled" check, since you'll only see this node when your feature is enabled. If you're feeling generous, you could move vector subscripting to this new node as well and leave ArraySubscriptExpr exclusively for the standard (or dependent) cases.

[fhahn]

In D76791#2048387, @rjmccall wrote:

Sorry for the slow review; I'm getting crushed with review requests.

Thanks John! I've updated the patch and introduced a new MatrixSubscriptExpr. I think it simplifies the code in a few places and makes things clearer.

[fhahn]

clang/lib/AST/Expr.cpp

3859

This code is now gone.

clang/lib/AST/ExprConstant.cpp

7768

This code is not required any longer.

clang/lib/Sema/SemaExpr.cpp

5326

It should have the value kind of its base. get_matrix_rvalue()[0][0] is still an r-value, so you shouldn't allow it to be assigned to. Needs tests.

added tests to Sema/matrix-type-operators.s and SemaCXX/matrix-type-operators.cpp.

I'm fine with re-using OK_VectorComponent here, but you should (1) rename it to OK_VectorOrMatrixComponent and (2) take the opportunity to audit the places that use it to make sure they do something sensible. I expect you'll need to at least update some diagnostics about e.g. disallowing taking the address of the expression.

I've introduced a new OK_MatrixElement, because I don't think there is enough information to distinguish between vectors/matrixes where it is used. Also added a reinterpret_cast test.

I think you should build a new expression node instead of reusing ArraySubscriptExpr, since basically none of the code that looks for an ArraySubscriptExpr will do the right thing for your operation. That will also allow you to avoid the "is this actually enabled" check, since you'll only see this node when your feature is enabled. If you're feeling generous, you could move vector subscripting to this new node as well and leave ArraySubscriptExpr exclusively for the standard (or dependent) cases.

I've added a new MatrixSubscriptExpr. That helps to simplify the code in a couple of places. ActOnArraySubscriptExpr calls ActOnMatrixSubscriptExpr if we can identify the base as matrix type or MatrixSubscriptExpr. I initially tried to move the ActOnMatrixSubscriptExpr call to the parse step, but that does not work in the presence of dependent base types I think. Until we instantiate the dependent types, there's no way to distinguish between ArraySubscriptExpr/MatrixSubscriptExpr I think and it is best to treat them as unanalyzed ArraySubscriptExpr until then.

[fhahn]

Add clarifying comment to ActOnMatrixSubscriptExpr calls in ActOnArraySubscriptExpr.

[rjmccall]

clang/include/clang/AST/Expr.h

2653

Oh, that's interesting. So you've changed this to flatten the component expressions? I think that might be inconsistent with our usual source-preservation goals unless you intend to restrict the intermediate base expression to be an immediate subscript. That is, this is okay if you're going to require the user to write matrix[i][j] and forbid (matrix[i])[j], but if you intend to allow the latter, you should preserve that structure here. You can do that while still providing this API; you just have to implement getBase() etc. by looking through parens, and you should have an accessor which returns the syntactic base expression.

What expression node do you use for the intermediate subscript expression? You should talk about this in the doc comment.

clang/include/clang/Basic/Specifiers.h

159 ↗

(On Diff #265750)

redundancy

clang/include/clang/Sema/Sema.h

4907 ↗

(On Diff #265750)

It'd be more conventional to call it BuildMatrixSubscriptExpr. You can do all the semantic checks here and make ActOnArraySubscriptExpr handle the syntactic checks. This is all assuming that you intend to impose the syntactic restriction discussed above.

clang/lib/AST/ItaniumMangle.cpp

4238 ↗

(On Diff #265750)

This is simple, you should just mangle them syntactically. 'ixix' <base expression> <row expression> <column expression>. Test case is

using double4x4 = double __attribute__((matrix_type(4,4)));

template <class R, class C>
auto matrix_subscript(double4x4 m, R r, C c) -> decltype(m[r][c]) {}

double test_matrix_subscript(double 4x4 m) { return m[3][2]; }

clang/lib/Sema/SemaExpr.cpp

4556

You're not handling the case of a parenthesized MatrixSubscriptExpr. If that should be an error, that's a reasonable language rule, but it should probably be a better error than whatever you'll get by default.

And you might be specifically and problematically avoiding an error because of the special treatment of IncompleteMatrixIdx below. I'd just pull that up here and emit an error.

Doing this before the C++2a comma-index check is intentional?

[fhahn]

Address John's latest comments, thanks!

clang/include/clang/AST/Expr.h

2653

Oh, that's interesting. So you've changed this to flatten the component expressions? I think that might be inconsistent with our usual source-preservation goals unless you intend to restrict the intermediate base expression to be an immediate subscript. That is, this is okay if you're going to require the user to write matrix[i][j] and forbid (matrix[i])[j], but if you intend to allow the latter, you should preserve that structure here. You can do that while still providing this API; you just have to implement getBase() etc. by looking through parens, and you should have an accessor which returns the syntactic base expression.

My intuition was that the matrix subscript operator would be a single operator with 3 arguments. Allowing things like (matrix[I])[j] has potential for confusion I think, as there is no way to create an expression just referencing a row/column on its own. (responded in more details at the SemaExpr.cpp comment).

What expression node do you use for the intermediate subscript expression? You should talk about this in the doc comment.

Intermediate subscript expressions are represented as 'incomplete' MatrixSubscriptExpr (type is MatrixIncompleteIdx, ColumnIdx is nullptr). I've updated the comment.

clang/include/clang/Basic/Specifiers.h

159 ↗

(On Diff #265750)

I suppose no comment is sufficient?

clang/include/clang/Sema/Sema.h

4907 ↗

(On Diff #265750)

Thanks, I wasn't sure about the actual distinction. I've renamed it to BuildMatrixSubscriptExpr now.

clang/lib/AST/ItaniumMangle.cpp

4238 ↗

(On Diff #265750)

Thanks, I've updated the code as suggested and added the test. I had to adjust the test slightly (adding a call to matrix_subscript).

This test is quite interesting, because it accidentally another issue. Matrix element subscripts are treated as Lvalues currently, but the code currently does not support taking the address of matrix elements. decltype(m[r][c]) will be & double and if you add return m[r][c]; the issue is exposed.

I am not sure how to best address this, short of computing the address of the element in memory directly. Do you have any suggestions?

I think for some vector types, we currently mis-compile here as well. For example,

using double4 = double __attribute__((ext_vector_type(4)));

template <class R>
auto matrix_subscript(const double4 m, R r) -> decltype(m[r]) { return m[r]; }

produces the IR below. Note the ret double* %ref.tmp, where %ref.tmp is an alloca.

define linkonce_odr nonnull align 8 dereferenceable(8) double* @_Z16matrix_subscriptIiEDTixfpK_fp0_EDv4_dT_(<4 x double>* byval(<4 x double>) align 16 %0, i32 %r) #0 {
entry:
  %m.addr = alloca <4 x double>, align 16
  %r.addr = alloca i32, align 4
  %ref.tmp = alloca double, align 8
  %m = load <4 x double>, <4 x double>* %0, align 16
  store <4 x double> %m, <4 x double>* %m.addr, align 16
  store i32 %r, i32* %r.addr, align 4
  %1 = load <4 x double>, <4 x double>* %m.addr, align 16
  %2 = load i32, i32* %r.addr, align 4
  %vecext = extractelement <4 x double> %1, i32 %2
  store double %vecext, double* %ref.tmp, align 8
  ret double* %ref.tmp
}

clang/lib/Sema/SemaExpr.cpp

4556

You're not handling the case of a parenthesized MatrixSubscriptExpr. If that should be an error, that's a reasonable language rule, but it should probably be a better error than whatever you'll get by default.

Yes the way I think about the matrix subscript expression is in terms of a single operator with 3 arguments (base[RowIdx][ColumnIdx]), so something like (a[i])[j] should not be allowed, similar to things like a([I]) not being allowed.

And you might be specifically and problematically avoiding an error because of the special treatment of IncompleteMatrixIdx below. I'd just pull that up here and emit an error.

I've moved the handling of IncompleteMatrixIdx here (the previous position was due to the first version of the patch).

Doing this before the C++2a comma-index check is intentional?

Not really. I've moved it after the check.

[rjmccall]

clang/include/clang/AST/Expr.h

2653

My intuition was that the matrix subscript operator would be a single operator with 3 arguments. Allowing things like (matrix[I])[j] has potential for confusion I think, as there is no way to create an expression just referencing a row/column on its own. (responded in more details at the SemaExpr.cpp comment).

I think it's a perfectly reasonable language design to disallow parentheses here; it just wasn't obvious at first that that was what you were intending to do.

Intermediate subscript expressions are represented as 'incomplete' MatrixSubscriptExpr (type is MatrixIncompleteIdx, ColumnIdx is nullptr). I've updated the comment.

Thanks. In that case, there should be a way to more directly query whether a particular expression is complete or not.

clang/include/clang/Basic/Specifiers.h

159 ↗

(On Diff #265750)

A comment is a good idea; I was just pointing out that you'd written "a matrix element of a matrix".

If you're intending to allow more complex matrix components in the future (e.g. row/column projections), consider going ahead and calling this OK_MatrixComponent.

clang/lib/AST/ItaniumMangle.cpp

4238 ↗

(On Diff #265750)

Thanks, I've updated the code as suggested and added the test. I had to adjust the test slightly (adding a call to matrix_subscript).

Er, yes, I did mean to include one of those. :)

I am not sure how to best address this, short of computing the address of the element in memory directly. Do you have any suggestions?

Well, there's two levels of this.

First, you *could* make matrix elements ordinary l-values — it's not like you have any interest in ever not storing them separately, right? Vector components use a special l-value kind specifically to prevent addressability, probably for reasons related to the swizzle projections. Although, saying that, I'm not sure you can assign to the swizzle projections, in which case I'm not really sure why vectors need a special object kind except to forbid taking the address for its own sake. The only problem I can see with allowing element l-values to be ordinary l-values is that the code coming out of IRGen after you assign to one isn't going to be quite so tidily mem2reg'able as it is with the vector-style code generation. Of course, you could peephole that in IRGen if it's important. And if you want e.g. projected column matrices to be non-addressable in the future (which I assume you do?), you can always add a special object kind for those when you add them.

If you do want to restrict taking the address of an matrix component l-value, that should be a straightforward extra restriction in the places where we already forbid taking the address of a bit-field. You'll need the same restriction in reference-binding, of course. You could make decltype not infer a reference type for your l-values if you want, although I believe decltype does infer a reference type for bit-fields.

Your example is unfortunately not a miscompile, it's just a combination of terrible language features. The returned reference is bound to a materialized temporary because vector elements aren't addressable. You wouldn't be able to do that if m weren't const because the return type would become double &, which can't bind to a temporary.

clang/lib/Sema/SemaExpr.cpp

4556

Not really. I've moved it after the check.

Okay. However, you should consider going ahead and making it an error for matrix subscripts, just in case you want that syntax to mean something else in the future. It will also let you immediately catch people trying to use m[0,0] instead of m[0][0].

4565

This change is no longer necessary.

4633

Checking dependence is actually just as cheap as checking for C++, there's no real need to gate. But you need to check for placeholder types in the index operands before doing these type checks. The best test case here is an Objective-C property reference, something like:

__attribute__((objc_root_class))
@interface IntValue
@property int value;
@end

double test(double4x4 m, IntValue *iv) {
  return m[iv.value][iv.value];
}

Also, I would suggest not doing any checking on incomplete matrix expressions; just notice that it's still incomplete, build the expression, and return. You can do the checking once when you have all the operands together.

[fhahn]

Addressed latest round of comments, thanks!

Changes include:

• OK_MatrixComponent now behaves like OK_VectorComponent with respect to taking address/reference.
• Look through non-overload placeholder expressions.
• Move matrix related syntactic checks to new ActOnMatrixSubscriptExpr.
• Add MatrixSubscriptExpr::isIncomplete & use it at a few places in assertions.
• Update CreateBuiltinMatrixType to only validate indices once, on the complete expression.
• Do not allow , in matrix subscript expressions.

[fhahn]

clang/include/clang/AST/Expr.h

2653

Thanks. In that case, there should be a way to more directly query whether a particular expression is complete or not.

Done, I've added an isIncomplete method. I also added assertions using it in a few places.

clang/include/clang/Basic/Specifiers.h

159 ↗

(On Diff #265750)

I've renamed it to OK_MatrixComponent and added a comment that a matrix component is a single element of a matrix.

clang/lib/AST/ItaniumMangle.cpp

4238 ↗

(On Diff #265750)

If you do want to restrict taking the address of an matrix component l-value, that should be a straightforward extra restriction in the places where we already forbid taking the address of a bit-field. You'll need the same restriction in reference-binding, of course. You could make decltype not infer a reference type for your l-values if you want, although I believe decltype does infer a reference type for bit-fields.

Thanks for the suggestion! I think it would be best to mirror the handling of ExtVectorElementExpr and I *think* I found all the relevant places that needed updating. I've added tests to take references/addresses with various valid/invalid cases.

clang/lib/Sema/SemaExpr.cpp

4556

Oh right! I've went back to add a ActOnMatrixSubscriptExpr, as there are a few things that can go in there now, including rejecting m[0,0].

4565

Dropped, thanks!

4633

Checking dependence is actually just as cheap as checking for C++, there's no real need to gate. But you need to check for placeholder types in the index operands before doing these type checks. The best test case here is an Objective-C property reference, something like:

Done, I've added a ActOnMatrixSubscriptExpr and added code to deal with placeholder types there.

Also, I would suggest not doing any checking on incomplete matrix expressions; just notice that it's still incomplete, build the expression, and return. You can do the checking once when you have all the operands together.

Done, I initially thought it might be good to still raise an error if the row index was invalid, but given that it's not a valid expression anyways that probably is not too helpful anyways. Doing the checks on the complete expression simplifies things a bit :)

[rjmccall]

clang/lib/Sema/SemaExpr.cpp

4633

Placeholders actually need to be handled in the Build function — they can come up in template instantiation, too.

4641

Don't check for ParenExpr specifically; there are other expressions that are handled the same way, like _Generic. You need to check for !isa<MatrixSubscriptExpr>.

4651

Overload placeholder types are used for things like &foo where foo is the name of an overloaded function. The places that resolve only non-overload placeholder types are doing so in order to leave room for overload resolution to resolve the overload later, e.g. as part of non-builtin operator handling. operator[] is like operator(): non-builtin operators are only considered when the base has class type. Since you already know that the base type is a matrix type, you know that you're using your standard rules, and your standard rules have no way to resolve overloads in the index types — correctly, since indexes can't be functions or member pointers.

tl;dr: You can (and should) resolve all placeholders here, not just non-overload placeholders.

[fhahn]

Addressed latest comments:

• Handle placeholder types in CreateBuiltinMatrixSubscriptExpr and do not limit to non-overload types there.
• Check !MatrixSubscriptExpr instead of ParenExpr.
• Only handle placeholder types for Base in ActOnMatrixSubscriptExpr. Only skip isMSPropertySubscriptExpr.

clang/lib/Sema/SemaExpr.cpp

4633

Move, thanks! I think we have to retain checking placeholder expressions for base below, as noted in the response below.

4641

Done, thanks!

4651

Thanks for the clarification. I moved the code to deal with placeholders to CreateBuiltinMatrixSubscriptExpr and removed the non-overload restriction there.

I think we still need to keep dealing with placeholders in Base below, to ensure we do not miss that the base is actually a matrix type, e.g. to support. It seems it is enough to skip` isMSPropertySubscriptExpr` there (without that restriction, some non-matrix-type codegen tests start to fail. Does that make sense?

__attribute__((objc_root_class))
@interface MatrixValue
@property double4x4 value;
@end

[rjmccall]

clang/lib/CodeGen/CGExpr.cpp

1911

This should be handled here or else a whole lot of unusual cases are going to blow up on you — compound operators and so on.

3818

You should be able to assert that these have been coerced to the right type by Sema (probably size_t or ptrdiff_t or something).

clang/lib/Sema/SemaExpr.cpp

4553

I would prefer that the checks are inlined here, since they're very likely to not trigger.

4651

Yeah, you have to resolve some placeholders before you can check whether the base is a matrix type, and you can't resolve MS properties because the property access actually merges with the subscript in some cases.

I think you may need to resolve placeholders in base even in CreateBuiltinMatrixSubscriptExpr to handle template instantiation right. The test case would be something where the matrix expression is non-dependently-typed but loaded from an ObjC property — we might need to redundantly resolve placeholders when rebuilding expressions in the instantiation.

[rjmccall]

I also want to confirm explicitly that you've decided that the right language design is for matrix components to not be addressable. You're convinced that that's an important restriction to get the code generation you want?

[fhahn]

Thanks for the latest round of comments! All expect one should be addressed. For the remaining comment, I responded inline.

Also, it would be great if you could let me know if the updated check lines in D76793 are sufficient, then I'll update the tests in this patch accordingly.

In D76791#2058460, @rjmccall wrote:

I also want to confirm explicitly that you've decided that the right language design is for matrix components to not be addressable. You're convinced that that's an important restriction to get the code generation you want?

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

[rjmccall]

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

It does make sense, although it does make it very important that code generation narrows the relevant particular load/extractvalue and load/insertvalue/store sequences, or else performance and code size will be completely unacceptable for large matrices.

The test changes in the other patch look great, thanks.

[fhahn]

In D76791#2063926, @rjmccall wrote:

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

It does make sense, although it does make it very important that code generation narrows the relevant particular load/extractvalue and load/insertvalue/store sequences, or else performance and code size will be completely unacceptable for large matrices.

Yes, dealing with large vectors is currently a weakness of LLVM unfortunately. But breaking up operations on large vectors in general is something we are planning on implementing as part of the matrix lowering. And I think even if we do not allow taking addresses, we could still just load/store a single element with the index operator, if required (maybe also based on the actual size of the matrix). And if we discover that taking addresses is really beneficial to users, it should be relatively straight forward to allow it in a backwards-compatible way I think/

The test changes in the other patch look great, thanks.

Great, I'll update them ASAP, although it takes a while. I am also submitting some inline-response that didn't got submitted when I updated the patch a few hours ago.

clang/lib/CodeGen/CGExpr.cpp

1911

Oh thanks for pointing me to those cases. Fixed and test cases added.

3818

Hm I don't think that's currently happening. Is that required? It seems a bit unfortunate if we would have to force to wider bit-widths than necessary for the given index expressions.

clang/lib/Sema/SemaExpr.cpp

4553

Done, I've removed ActOnMatrixSubscriptExpr again.

4651

Yeah, you have to resolve some placeholders before you can check whether the base is a matrix type, and you can't resolve MS properties because the property access actually merges with the subscript in some cases.

I think that should be happening in the latest version, CheckPlaceholderExpr is used on Base, RowIdx and ColumnIdx.

[rjmccall]

In D76791#2064434, @fhahn wrote:

In D76791#2063926, @rjmccall wrote:

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

It does make sense, although it does make it very important that code generation narrows the relevant particular load/extractvalue and load/insertvalue/store sequences, or else performance and code size will be completely unacceptable for large matrices.

Yes, dealing with large vectors is currently a weakness of LLVM unfortunately. But breaking up operations on large vectors in general is something we are planning on implementing as part of the matrix lowering. And I think even if we do not allow taking addresses, we could still just load/store a single element with the index operator, if required (maybe also based on the actual size of the matrix). And if we discover that taking addresses is really beneficial to users, it should be relatively straight forward to allow it in a backwards-compatible way I think/

Yeah, if you ever decide you want that IR pattern, it should be painless to make IRGen emit it. You might even consider doing it at -O0 if the backend wouldn't otherwise be narrowing then. (That doesn't need to be in this patch, of course.)

clang/lib/CodeGen/CGExpr.cpp

3818

Can you explain what cost you're worried about? I don't understand what the benefit of using a smaller IR type for the indexes is, and usually it's better to have stronger invariants coming out of Sema.

clang/lib/Sema/SemaExpr.cpp

4651

Yes, you're right.

4663

Somewhere in here would be where you'd do the index type conversion, FWIW.

I don't know if you want to allow a class with a conversion operator to an integer type, but that function you added for +/- should do it.

[fhahn]

Force index types to size_t in Sema, remove code to extend index values in Codegen. Update tests to check more targeted IR.

[fhahn]

clang/lib/CodeGen/CGExpr.cpp

3818

I now updated the code to force the index types to size_t in Sema and remove the zext generation here. If we have signed indices, we now generate sexts, which may require additional instructions down the road., which we would not require if we perform the math on the max bit width of the operands. But looking back now, I am not really sure if that would actually be safe in all cases.

We should probably just try to narrow the index types based on the number of vector elements as an IR optimization, if it becomes a problem.

clang/lib/Sema/SemaExpr.cpp

4663

I don't know if you want to allow a class with a conversion operator to an integer type, but that function you added for +/- should do it.

Yes I updated the code here to use the same conversion function and added tests for conversion operators.

[rjmccall]

LGTM, thanks.

clang/lib/CodeGen/CGExpr.cpp

3818

It's UB if it's actually negative, right? So the sext really shouldn't be a problem.

[fhahn]

Thanks for all the feedback John!

clang/lib/CodeGen/CGExpr.cpp

3818

It's UB if it's actually negative, right? So the sext really shouldn't be a problem.

Yes exactly. At the C/C++ level, the individual row/column expressions are guaranteed to be >= 0. In the IR we emit, we compute the index in a vector and we know that the computed index must be >= 0, which is slightly weaker. It might be worth checking how to encode the fact that the extension sources must be >= 0, if it becomes a problem.

[nickdesaulniers]

clang/lib/Sema/SemaExpr.cpp

4687–4688

In a release build:

llvm-project/clang/lib/Sema/SemaExpr.cpp:4737:10: warning: unused variable 'ConversionOk' [-Wunused-variable]
    bool ConversionOk = tryConvertToTy(*this, Context.getSizeType(), &ConvExpr);
         ^

[fhahn]

clang/lib/Sema/SemaExpr.cpp

4687–4688

Oh, of course, thanks for letting me know! Should be fixed in a6a42df506ca93df69725f732c396050060f026f

clang/include/clang/AST/ASTContext.h

public:
   CanQualType PseudoObjectTy, ARCUnbridgedCastTy;
   CanQualType ObjCBuiltinIdTy, ObjCBuiltinClassTy, ObjCBuiltinSelTy;
   CanQualType ObjCBuiltinBoolTy;
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) \
   CanQualType SingletonId;
 #include "clang/Basic/OpenCLImageTypes.def"
   CanQualType OCLSamplerTy, OCLEventTy, OCLClkEventTy;
   CanQualType OCLQueueTy, OCLReserveIDTy;
+  CanQualType IncompleteMatrixIdxTy;
   CanQualType OMPArraySectionTy, OMPArrayShapingTy, OMPIteratorTy;
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
   CanQualType Id##Ty;
 #include "clang/Basic/OpenCLExtensionTypes.def"
 #define SVE_TYPE(Name, Id, SingletonId) \
   CanQualType SingletonId;
 #include "clang/Basic/AArch64SVEACLETypes.def"
 

clang/include/clang/AST/BuiltinTypes.def

 
 PLACEHOLDER_TYPE(BuiltinFn, BuiltinFnTy)
 
 // The type of a cast which, in ARC, would normally require a
 // __bridge, but which might be okay depending on the immediate
 // context.
 PLACEHOLDER_TYPE(ARCUnbridgedCast, ARCUnbridgedCastTy)
 
+// A placeholder type for incomplete matrix index expressions.
+PLACEHOLDER_TYPE(IncompleteMatrixIdx, IncompleteMatrixIdxTy)
+
 // A placeholder type for OpenMP array sections.
 PLACEHOLDER_TYPE(OMPArraySection, OMPArraySectionTy)
 
 // A placeholder type for OpenMP array shaping operation.
 PLACEHOLDER_TYPE(OMPArrayShaping, OMPArrayShapingTy)
 
 // A placeholder type for OpenMP iterators.
 PLACEHOLDER_TYPE(OMPIterator, OMPIteratorTy)

clang/include/clang/AST/Expr.h

const FieldDecl *getSourceBitField() const {
     return const_cast<Expr*>(this)->getSourceBitField();
   }
 
   Decl *getReferencedDeclOfCallee();
   const Decl *getReferencedDeclOfCallee() const {
     return const_cast<Expr*>(this)->getReferencedDeclOfCallee();
   }
 
+  /// If \p Base is part of a matrix index expressions, return the access matrix
+  /// type. \p Base is part of a matrix index expression, if either Base is a
+  /// matrix (= matrix row index expr) or a matrix row index expr.
+  const ConstantMatrixType *getMatrixFromIndexExpr(bool EnableMatrix) const;
+
   /// If this expression is an l-value for an Objective C
   /// property, find the underlying property reference expression.
   const ObjCPropertyRefExpr *getObjCProperty() const;
 
   /// Check if this expression is the ObjC 'self' implicit parameter.
   bool isObjCSelfExpr() const;
 
   /// Returns whether this expression refers to a vector element.
child_range children() {
     return child_range(&SubExprs[0], &SubExprs[0]+END_EXPR);
   }
   const_child_range children() const {
     return const_child_range(&SubExprs[0], &SubExprs[0] + END_EXPR);
   }
 };
 
 /// CallExpr - Represents a function call (C99 6.5.2.2, C++ [expr.call]).
 /// CallExpr itself represents a normal function call, e.g., "f(x, 2)",
 /// while its subclasses may represent alternative syntax that (semantically)
 /// results in a function call. For example, CXXOperatorCallExpr is
 /// a subclass for overloaded operator calls that use operator syntax, e.g.,
 /// "str1 + str2" to resolve to a function call.
 class CallExpr : public Expr {
   enum { FN = 0, PREARGS_START = 1 };
 
   /// The number of arguments in the call expression.

clang/include/clang/Basic/DiagnosticSemaKinds.td

def warn_noderef_to_dereferenceable_pointer : Warning<
   "casting to dereferenceable pointer removes 'noderef' attribute">, InGroup<NoDeref>;
 
 def err_builtin_launder_invalid_arg : Error<
   "%select{non-pointer|function pointer|void pointer}0 argument to "
   "'__builtin_launder' is not allowed">;
 
 def err_builtin_matrix_disabled: Error<
   "matrix types extension is disabled. Pass -fenable-matrix to enable it">;
+def err_matrix_index_not_integer: Error<
+  "matrix %select{row|column}0 index is not an integer">;
+def err_matrix_index_outside_range: Error<
+  "matrix %select{row|column}0 index is outside the allowed range [0, %1)">;
+def err_matrix_incomplete_index: Error<
+  "single subscript expressions are not allowed for matrix values">;
 
 def err_preserve_field_info_not_field : Error<
   "__builtin_preserve_field_info argument %0 not a field access">;
 def err_preserve_field_info_not_const: Error<
   "__builtin_preserve_field_info argument %0 not a constant">;
 
 def err_bit_cast_non_trivially_copyable : Error<
   "__builtin_bit_cast %select{source|destination}0 type must be trivially copyable">;

clang/include/clang/Serialization/ASTBitCodes.h

 //===- ASTBitCodes.h - Enum values for the PCH bitcode format ---*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This header defines Bitcode enum values for Clang serialized AST files.
namespace serialization {
     ///
     /// These type IDs correspond to predefined types in the AST
     /// context, such as built-in types (int) and special place-holder
     /// types (the \<overload> and \<dependent> type markers). Such
     /// types are never actually serialized, since they will be built
     /// by the AST context when it is created.
     enum PredefinedTypeIDs {
       /// The NULL type.
       PREDEF_TYPE_NULL_ID       = 0,
 
       /// The void type.
       PREDEF_TYPE_VOID_ID       = 1,
 
       /// The 'bool' or '_Bool' type.
       PREDEF_TYPE_BOOL_ID       = 2,
 
       /// The 'char' type, when it is unsigned.
       PREDEF_TYPE_CHAR_U_ID     = 3,
 
       /// The 'unsigned char' type.
       PREDEF_TYPE_UCHAR_ID      = 4,
 
       /// The 'unsigned short' type.
       PREDEF_TYPE_USHORT_ID     = 5,
 
       /// The 'unsigned int' type.
       PREDEF_TYPE_UINT_ID       = 6,
 
       /// The 'unsigned long' type.
       PREDEF_TYPE_ULONG_ID      = 7,
 
       /// The 'unsigned long long' type.
       PREDEF_TYPE_ULONGLONG_ID  = 8,
 
       /// The 'char' type, when it is signed.
       PREDEF_TYPE_CHAR_S_ID     = 9,
 
       /// The 'signed char' type.
       PREDEF_TYPE_SCHAR_ID      = 10,
 
       /// The C++ 'wchar_t' type.
       PREDEF_TYPE_WCHAR_ID      = 11,
 
       /// The (signed) 'short' type.
       PREDEF_TYPE_SHORT_ID      = 12,
 
       /// The (signed) 'int' type.
       PREDEF_TYPE_INT_ID        = 13,
 
       /// The (signed) 'long' type.
       PREDEF_TYPE_LONG_ID       = 14,
 
       /// The (signed) 'long long' type.
       PREDEF_TYPE_LONGLONG_ID   = 15,
 
       /// The 'float' type.
       PREDEF_TYPE_FLOAT_ID      = 16,
 
       /// The 'double' type.
       PREDEF_TYPE_DOUBLE_ID     = 17,
 
       /// The 'long double' type.
       PREDEF_TYPE_LONGDOUBLE_ID = 18,
 
       /// The placeholder type for overloaded function sets.
       PREDEF_TYPE_OVERLOAD_ID   = 19,
 
       /// The placeholder type for dependent types.
       PREDEF_TYPE_DEPENDENT_ID  = 20,
 
       /// The '__uint128_t' type.
       PREDEF_TYPE_UINT128_ID    = 21,
 
       /// The '__int128_t' type.
       PREDEF_TYPE_INT128_ID     = 22,
 
       /// The type of 'nullptr'.
       PREDEF_TYPE_NULLPTR_ID    = 23,
 
       /// The C++ 'char16_t' type.
       PREDEF_TYPE_CHAR16_ID     = 24,
 
       /// The C++ 'char32_t' type.
       PREDEF_TYPE_CHAR32_ID     = 25,
 
       /// The ObjC 'id' type.
       PREDEF_TYPE_OBJC_ID       = 26,
 
       /// The ObjC 'Class' type.
       PREDEF_TYPE_OBJC_CLASS    = 27,
 
       /// The ObjC 'SEL' type.
       PREDEF_TYPE_OBJC_SEL      = 28,
 
       /// The 'unknown any' placeholder type.
       PREDEF_TYPE_UNKNOWN_ANY   = 29,
 
       /// The placeholder type for bound member functions.
       PREDEF_TYPE_BOUND_MEMBER  = 30,
 
       /// The "auto" deduction type.
       PREDEF_TYPE_AUTO_DEDUCT   = 31,
 
       /// The "auto &&" deduction type.
       PREDEF_TYPE_AUTO_RREF_DEDUCT = 32,
 
       /// The OpenCL 'half' / ARM NEON __fp16 type.
       PREDEF_TYPE_HALF_ID       = 33,
 
       /// ARC's unbridged-cast placeholder type.
       PREDEF_TYPE_ARC_UNBRIDGED_CAST = 34,
 
       /// The pseudo-object placeholder type.
       PREDEF_TYPE_PSEUDO_OBJECT = 35,
 
       /// The placeholder type for builtin functions.
       PREDEF_TYPE_BUILTIN_FN = 36,
 
       /// OpenCL event type.
       PREDEF_TYPE_EVENT_ID      = 37,
 
       /// OpenCL clk event type.
       PREDEF_TYPE_CLK_EVENT_ID  = 38,
 
       /// OpenCL sampler type.
       PREDEF_TYPE_SAMPLER_ID    = 39,
 
       /// OpenCL queue type.
       PREDEF_TYPE_QUEUE_ID      = 40,
 
       /// OpenCL reserve_id type.
       PREDEF_TYPE_RESERVE_ID_ID = 41,
 
       /// The placeholder type for OpenMP array section.
       PREDEF_TYPE_OMP_ARRAY_SECTION = 42,
 
       /// The '__float128' type
       PREDEF_TYPE_FLOAT128_ID = 43,
 
       /// The '_Float16' type
       PREDEF_TYPE_FLOAT16_ID = 44,
 
       /// The C++ 'char8_t' type.
       PREDEF_TYPE_CHAR8_ID = 45,
 
       /// \brief The 'short _Accum' type
       PREDEF_TYPE_SHORT_ACCUM_ID    = 46,
 
       /// \brief The '_Accum' type
       PREDEF_TYPE_ACCUM_ID      = 47,
 
       /// \brief The 'long _Accum' type
       PREDEF_TYPE_LONG_ACCUM_ID = 48,
 
       /// \brief The 'unsigned short _Accum' type
       PREDEF_TYPE_USHORT_ACCUM_ID   = 49,
 
       /// \brief The 'unsigned _Accum' type
       PREDEF_TYPE_UACCUM_ID     = 50,
 
       /// \brief The 'unsigned long _Accum' type
       PREDEF_TYPE_ULONG_ACCUM_ID    = 51,
 
       /// \brief The 'short _Fract' type
       PREDEF_TYPE_SHORT_FRACT_ID = 52,
 
       /// \brief The '_Fract' type
       PREDEF_TYPE_FRACT_ID = 53,
 
       /// \brief The 'long _Fract' type
enum PredefinedTypeIDs {
       PREDEF_TYPE_SAT_ULONG_FRACT_ID = 69,
 
       /// The placeholder type for OpenMP array shaping operation.
       PREDEF_TYPE_OMP_ARRAY_SHAPING = 70,
 
       /// The placeholder type for OpenMP iterator expression.
       PREDEF_TYPE_OMP_ITERATOR = 71,
 
+      /// A placeholder type for incomplete matrix index operations.
+      PREDEF_TYPE_INCOMPLETE_MATRIX_IDX = 72,
+
       /// OpenCL image types with auto numeration
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLImageTypes.def"
       /// \brief OpenCL extension types with auto numeration
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLExtensionTypes.def"
       // \brief SVE types with auto numeration

clang/lib/AST/ASTContext.cpp

void ASTContext::InitBuiltinTypes(const TargetInfo &Target,
   InitBuiltinType(BuiltinFnTy,  BuiltinType::BuiltinFn);
 
   // Placeholder type for OMP array sections.
   if (LangOpts.OpenMP) {
     InitBuiltinType(OMPArraySectionTy, BuiltinType::OMPArraySection);
     InitBuiltinType(OMPArrayShapingTy, BuiltinType::OMPArrayShaping);
     InitBuiltinType(OMPIteratorTy, BuiltinType::OMPIterator);
   }
+  if (LangOpts.MatrixTypes)
+    InitBuiltinType(IncompleteMatrixIdxTy, BuiltinType::IncompleteMatrixIdx);
 
   // C99 6.2.5p11.
   FloatComplexTy      = getComplexType(FloatTy);
   DoubleComplexTy     = getComplexType(DoubleTy);
   LongDoubleComplexTy = getComplexType(LongDoubleTy);
   Float128ComplexTy   = getComplexType(Float128Ty);
 
   // Builtin types for 'id', 'Class', and 'SEL'.

clang/lib/AST/Expr.cpp

FieldDecl *Expr::getSourceBitField() {
 
   if (UnaryOperator *UnOp = dyn_cast<UnaryOperator>(E))
     if (UnOp->isPrefix() && UnOp->isIncrementDecrementOp())
       return UnOp->getSubExpr()->getSourceBitField();
 
   return nullptr;
 }
 
+const ConstantMatrixType *
+Expr::getMatrixFromIndexExpr(bool EnableMatrix) const {
+  if (!EnableMatrix)
+    return nullptr;
+
+  if (getType()->isConstantMatrixType())
+    return getType()->getAs<ConstantMatrixType>();
+
+  auto *SubscriptE = dyn_cast<ArraySubscriptExpr>(this);
+  return SubscriptE
+             ? SubscriptE->getBase()->getType()->getAs<ConstantMatrixType>()
+             : nullptr;
+}
+
 bool Expr::refersToVectorElement() const {
   // FIXME: Why do we not just look at the ObjectKind here?
   const Expr *E = this->IgnoreParens();
 
   while (const ImplicitCastExpr *ICE = dyn_cast<ImplicitCastExpr>(E)) {
     if (ICE->getValueKind() != VK_RValue &&
         ICE->getCastKind() == CK_NoOp)
       E = ICE->getSubExpr()->IgnoreParens();

clang/lib/AST/ExprConstant.cpp

bool LValueExprEvaluator::VisitMemberExpr(const MemberExpr *E) {
   return LValueExprEvaluatorBaseTy::VisitMemberExpr(E);
 }
 
 bool LValueExprEvaluator::VisitArraySubscriptExpr(const ArraySubscriptExpr *E) {
   // FIXME: Deal with vectors as array subscript bases.
   if (E->getBase()->getType()->isVectorType())
     return Error(E);
 
+  // Skip matrixes as subscript bases.
+  if (E->getBase()->getMatrixFromIndexExpr(Info.getLangOpts().MatrixTypes))
+    return false;
+
   bool Success = true;
   if (!evaluatePointer(E->getBase(), Result)) {
     if (!Info.noteFailure())
       return false;
     Success = false;
   }
 
   APSInt Index;

clang/lib/AST/NSAPI.cpp

#include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::BoundMember:
   case BuiltinType::Dependent:
   case BuiltinType::Overload:
   case BuiltinType::UnknownAny:
   case BuiltinType::ARCUnbridgedCast:
   case BuiltinType::Half:
   case BuiltinType::PseudoObject:
   case BuiltinType::BuiltinFn:
+  case BuiltinType::IncompleteMatrixIdx:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
     break;
   }
 
   return None;
 }

clang/lib/AST/Type.cpp

#include "clang/Basic/OpenCLImageTypes.def"
   case OCLEvent:
     return "event_t";
   case OCLClkEvent:
     return "clk_event_t";
   case OCLQueue:
     return "queue_t";
   case OCLReserveID:
     return "reserve_id_t";
+  case IncompleteMatrixIdx:
+    return "<incomplete matrix index type>";
   case OMPArraySection:
     return "<OpenMP array section type>";
   case OMPArrayShaping:
     return "<OpenMP array shaping type>";
   case OMPIterator:
     return "<OpenMP iterator type>";
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
   case Id: \
#include "clang/Basic/OpenCLExtensionTypes.def"
     case BuiltinType::OCLClkEvent:
     case BuiltinType::OCLQueue:
     case BuiltinType::OCLReserveID:
 #define SVE_TYPE(Name, Id, SingletonId) \
     case BuiltinType::Id:
 #include "clang/Basic/AArch64SVEACLETypes.def"
     case BuiltinType::BuiltinFn:
     case BuiltinType::NullPtr:
+    case BuiltinType::IncompleteMatrixIdx:
     case BuiltinType::OMPArraySection:
     case BuiltinType::OMPArrayShaping:
     case BuiltinType::OMPIterator:
       return false;
     }
     llvm_unreachable("unknown builtin type");
 
   // Non-pointer types.

clang/lib/AST/TypeLoc.cpp

#include "clang/Basic/OpenCLExtensionTypes.def"
   case BuiltinType::OCLEvent:
   case BuiltinType::OCLClkEvent:
   case BuiltinType::OCLQueue:
   case BuiltinType::OCLReserveID:
 #define SVE_TYPE(Name, Id, SingletonId) \
   case BuiltinType::Id:
 #include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::BuiltinFn:
+  case BuiltinType::IncompleteMatrixIdx:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
     return TST_unspecified;
   }
 
   llvm_unreachable("Invalid BuiltinType Kind!");
 }

clang/lib/CodeGen/CGExpr.cpp

 static RValue EmitLoadOfMatrixLValue(LValue LV, SourceLocation Loc,
                                      CodeGenFunction &CGF) {
   assert(LV.getType()->isConstantMatrixType());
   Address Addr = MaybeConvertMatrixAddress(LV.getAddress(CGF), CGF);
   LV.setAddress(Addr);
   return RValue::get(CGF.EmitLoadOfScalar(LV, Loc));
 }
 
+static RValue EmitLoadOfMatrixEltLValue(LValue LV, SourceLocation Loc,
+                                        CodeGenFunction &CGF) {
+  assert(LV.getType()->isConstantMatrixType() &&
+         "matrix element LValues need to access a matrix");
+  Address Addr = MaybeConvertMatrixAddress(LV.getVectorAddress(), CGF);
+  llvm::LoadInst *Load = CGF.Builder.CreateLoad(Addr, LV.isVolatileQualified());
+  return RValue::get(
+      CGF.Builder.CreateExtractElement(Load, LV.getVectorIdx(), "matext"));
+}
+
 /// EmitLoadOfLValue - Given an expression that represents a value lvalue, this
 /// method emits the address of the lvalue, then loads the result as an rvalue,
 /// returning the rvalue.
 RValue CodeGenFunction::EmitLoadOfLValue(LValue LV, SourceLocation Loc) {
   if (LV.isObjCWeak()) {
     // load of a __weak object.
     Address AddrWeakObj = LV.getAddress(*this);
     return RValue::get(CGM.getObjCRuntime().EmitObjCWeakRead(*this,
if (LV.isSimple()) {
 
     if (LV.getType()->isConstantMatrixType())
       return EmitLoadOfMatrixLValue(LV, Loc, *this);
 
     // Everything needs a load.
     return RValue::get(EmitLoadOfScalar(LV, Loc));
   }
 
+  if (LV.isMatrixElt())
+    return EmitLoadOfMatrixEltLValue(LV, Loc, *this);
+
   if (LV.isVectorElt()) {
     llvm::LoadInst *Load = Builder.CreateLoad(LV.getVectorAddress(),
                                               LV.isVolatileQualified());
     return RValue::get(Builder.CreateExtractElement(Load, LV.getVectorIdx(),
                                                     "vecext"));
   }
 
   // If this is a reference to a subset of the elements of a vector, either
   // shuffle the input or extract/insert them as appropriate.
   if (LV.isExtVectorElt())
     return EmitLoadOfExtVectorElementLValue(LV);
 
   // Global Register variables always invoke intrinsics
   if (LV.isGlobalReg())
     return EmitLoadOfGlobalRegLValue(LV);
 
   assert(LV.isBitField() && "Unknown LValue type!");
   return EmitLoadOfBitfieldLValue(LV, Loc);
 }
 
 RValue CodeGenFunction::EmitLoadOfBitfieldLValue(LValue LV,
                                                  SourceLocation Loc) {
   const CGBitFieldInfo &Info = LV.getBitFieldInfo();
 
   // Get the output type.
   llvm::Type *ResLTy = ConvertType(LV.getType());
RValue CodeGenFunction::EmitLoadOfGlobalRegLValue(LValue LV) {
   llvm::Function *F = CGM.getIntrinsic(llvm::Intrinsic::read_register, Types);
   llvm::Value *Call = Builder.CreateCall(
       F, llvm::MetadataAsValue::get(Ty->getContext(), RegName));
   if (OrigTy->isPointerTy())
     Call = Builder.CreateIntToPtr(Call, OrigTy);
   return RValue::get(Call);
 }
 
+// Store the specified rvalue into the specified matrix element.
+static void EmitStoreThroughMatrixEltLValue(RValue Src, LValue Dst,
+                                            CodeGenFunction &CGF) {
+  Address DstAddr = MaybeConvertMatrixAddress(
+      Address(Dst.getVectorPointer(),
+              CGF.getContext().getTypeAlignInChars(Dst.getType())),
+      CGF);
+  llvm::Value *Vec = CGF.Builder.CreateLoad(DstAddr);
+  Vec = CGF.Builder.CreateInsertElement(Vec, Src.getScalarVal(),
+                                        Dst.getVectorIdx(), "matins");
+
+  CGF.Builder.CreateStore(Vec, DstAddr, Dst.isVolatileQualified());
+}
 
 /// EmitStoreThroughLValue - Store the specified rvalue into the specified
 /// lvalue, where both are guaranteed to the have the same type, and that type
 /// is 'Ty'.
 void CodeGenFunction::EmitStoreThroughLValue(RValue Src, LValue Dst,
                                              bool isInit) {
   if (!Dst.isSimple()) {
     if (Dst.isVectorElt()) {
if (!Dst.isSimple()) {
     // If this is an update of extended vector elements, insert them as
     // appropriate.
     if (Dst.isExtVectorElt())
       return EmitStoreThroughExtVectorComponentLValue(Src, Dst);
 
     if (Dst.isGlobalReg())
       return EmitStoreThroughGlobalRegLValue(Src, Dst);
 
+    if (Dst.isMatrixElt())
+      return EmitStoreThroughMatrixEltLValue(Src, Dst, *this);
+
     assert(Dst.isBitField() && "Unknown LValue type");
     return EmitStoreThroughBitfieldLValue(Src, Dst);
   }
 
   // There's special magic for assigning into an ARC-qualified l-value.
   if (Qualifiers::ObjCLifetime Lifetime = Dst.getQuals().getObjCLifetime()) {
     switch (Lifetime) {
     case Qualifiers::OCL_None:
if (E->getBase()->getType()->isVectorType() &&
     LValue LHS = EmitLValue(E->getBase());
     auto *Idx = EmitIdxAfterBase(/*Promote*/false);
     assert(LHS.isSimple() && "Can only subscript lvalue vectors here!");
     return LValue::MakeVectorElt(LHS.getAddress(*this), Idx,
                                  E->getBase()->getType(), LHS.getBaseInfo(),
                                  TBAAAccessInfo());
   }
 
+  // If the base is a matrix type, we have a matrix index expressions (with row
+  // and column indices). We are forming a matrix element lvalue with an index
+  // into the matrix as a flattened vector.
+  if (auto *MTy =
+          E->getBase()->getMatrixFromIndexExpr(getLangOpts().MatrixTypes)) {
+    auto *ColIdxExpr = cast<ArraySubscriptExpr>(E->getBase());
+    LValue Base = EmitLValue(ColIdxExpr->getBase());
+    llvm::Value *RowIdx = EmitScalarExpr(ColIdxExpr->getIdx());
+    llvm::Value *ColIdx = EmitScalarExpr(E->getIdx());
+    unsigned MaxWidth = std::max(RowIdx->getType()->getScalarSizeInBits(),
+                                 ColIdx->getType()->getScalarSizeInBits());
+    llvm::Type *IntTy = llvm::IntegerType::get(getLLVMContext(), MaxWidth);
+    RowIdx = Builder.CreateZExt(RowIdx, IntTy);
+    ColIdx = Builder.CreateZExt(ColIdx, IntTy);
+    llvm::Value *NumRows = Builder.getIntN(MaxWidth, MTy->getNumRows());
+    llvm::Value *FinalIdx =
+        Builder.CreateAdd(Builder.CreateMul(ColIdx, NumRows), RowIdx);
+    return LValue::MakeMatrixElt(Base.getAddress(*this), FinalIdx,
+                                 ColIdxExpr->getBase()->getType(),
+                                 Base.getBaseInfo(), TBAAAccessInfo());
+  }
+
   // All the other cases basically behave like simple offsetting.
 
   // Handle the extvector case we ignored above.
   if (isa<ExtVectorElementExpr>(E->getBase())) {
     LValue LV = EmitLValue(E->getBase());
     auto *Idx = EmitIdxAfterBase(/*Promote*/true);
     Address Addr = EmitExtVectorElementLValue(LV);
 
static Address emitOMPArraySectionBase(CodeGenFunction &CGF, const Expr *Base,
                                        LValueBaseInfo &BaseInfo,
                                        TBAAAccessInfo &TBAAInfo,
                                        QualType BaseTy, QualType ElTy,
                                        bool IsLowerBound) {
   LValue BaseLVal;
   if (auto *ASE = dyn_cast<OMPArraySectionExpr>(Base->IgnoreParenImpCasts())) {
     BaseLVal = CGF.EmitOMPArraySectionExpr(ASE, IsLowerBound);
     if (BaseTy->isArrayType()) {
       Address Addr = BaseLVal.getAddress(CGF);
       BaseInfo = BaseLVal.getBaseInfo();
 
       // If the array type was an incomplete type, we need to make sure
       // the decay ends up being the right type.
       llvm::Type *NewTy = CGF.ConvertType(BaseTy);
       Addr = CGF.Builder.CreateElementBitCast(Addr, NewTy);
 
       // Note that VLA pointers are always decayed, so we don't need to do

clang/lib/CodeGen/CGValue.h

 /// bitfields, this is not a simple LLVM pointer, it may be a pointer plus a
 /// bitrange.
 class LValue {
   enum {
     Simple,       // This is a normal l-value, use getAddress().
     VectorElt,    // This is a vector element l-value (V[i]), use getVector*
     BitField,     // This is a bitfield l-value, use getBitfield*.
     ExtVectorElt, // This is an extended vector subset, use getExtVectorComp
-    GlobalReg     // This is a register l-value, use getGlobalReg()
+    GlobalReg,    // This is a register l-value, use getGlobalReg()
+    MatrixElt     // This is a matrix element, use getVector*
   } LVType;
 
   llvm::Value *V;
 
   union {
     // Index into a vector subscript: V[i]
     llvm::Value *VectorIdx;
 
private:
   }
 
 public:
   bool isSimple() const { return LVType == Simple; }
   bool isVectorElt() const { return LVType == VectorElt; }
   bool isBitField() const { return LVType == BitField; }
   bool isExtVectorElt() const { return LVType == ExtVectorElt; }
   bool isGlobalReg() const { return LVType == GlobalReg; }
+  bool isMatrixElt() const { return LVType == MatrixElt; }
 
   bool isVolatileQualified() const { return Quals.hasVolatile(); }
   bool isRestrictQualified() const { return Quals.hasRestrict(); }
   unsigned getVRQualifiers() const {
     return Quals.getCVRQualifiers() & ~Qualifiers::Const;
   }
 
   QualType getType() const { return Type; }
void setAddress(Address address) {
     V = address.getPointer();
     Alignment = address.getAlignment().getQuantity();
   }
 
   // vector elt lvalue
   Address getVectorAddress() const {
     return Address(getVectorPointer(), getAlignment());
   }
-  llvm::Value *getVectorPointer() const { assert(isVectorElt()); return V; }
-  llvm::Value *getVectorIdx() const { assert(isVectorElt()); return VectorIdx; }
+  llvm::Value *getVectorPointer() const {
+    assert(isVectorElt() || isMatrixElt());
+    return V;
+  }
+  llvm::Value *getVectorIdx() const {
+    assert(isVectorElt() || isMatrixElt());
+    return VectorIdx;
+  }
 
   // extended vector elements.
   Address getExtVectorAddress() const {
     return Address(getExtVectorPointer(), getAlignment());
   }
   llvm::Value *getExtVectorPointer() const {
     assert(isExtVectorElt());
     return V;
static LValue MakeGlobalReg(Address Reg, QualType type) {
     LValue R;
     R.LVType = GlobalReg;
     R.V = Reg.getPointer();
     R.Initialize(type, type.getQualifiers(), Reg.getAlignment(),
                  LValueBaseInfo(AlignmentSource::Decl), TBAAAccessInfo());
     return R;
   }
 
+  static LValue MakeMatrixElt(Address matAddress, llvm::Value *Idx,
+                              QualType type, LValueBaseInfo BaseInfo,
+                              TBAAAccessInfo TBAAInfo) {
+    LValue R;
+    R.LVType = MatrixElt;
+    R.V = matAddress.getPointer();
+    R.VectorIdx = Idx;
+    R.Initialize(type, type.getQualifiers(), matAddress.getAlignment(),
+                 BaseInfo, TBAAInfo);
+    return R;
+  }
+
   RValue asAggregateRValue(CodeGenFunction &CGF) const {
     return RValue::getAggregate(getAddress(CGF), isVolatileQualified());
   }
 };
 
 /// An aggregate value slot.
 class AggValueSlot {
   /// The address.

clang/lib/Sema/SemaExpr.cpp

if (isa<ParenListExpr>(base)) {
     if (result.isInvalid()) return ExprError();
     base = result.get();
   }
 
   // A comma-expression as the index is deprecated in C++2a onwards.
   if (getLangOpts().CPlusPlus20 &&
       ((isa<BinaryOperator>(idx) && cast<BinaryOperator>(idx)->isCommaOp()) ||
        (isa<CXXOperatorCallExpr>(idx) &&
         cast<CXXOperatorCallExpr>(idx)->getOperator() == OO_Comma))) {
     Diag(idx->getExprLoc(), diag::warn_deprecated_comma_subscript)
       << SourceRange(base->getBeginLoc(), rbLoc);
   }
 
   // Handle any non-overload placeholder types in the base and index
   // expressions.  We can't handle overloads here because the other
   // operand might be an overloadable type, in which case the overload
   // resolution for the operator overload should get the first crack
   // at the overload.
   bool IsMSPropertySubscript = false;
-  if (base->getType()->isNonOverloadPlaceholderType()) {
+  auto BaseTy = base->getType();
+  if (BaseTy->isNonOverloadPlaceholderType() &&
+      !BaseTy->isSpecificPlaceholderType(BuiltinType::IncompleteMatrixIdx)) {
     IsMSPropertySubscript = isMSPropertySubscriptExpr(*this, base);
     if (!IsMSPropertySubscript) {
       ExprResult result = CheckPlaceholderExpr(base);
       if (result.isInvalid())
         return ExprError();
       base = result.get();
     }
   }

 
 void Sema::CheckAddressOfNoDeref(const Expr *E) {
   ExpressionEvaluationContextRecord &LastRecord = ExprEvalContexts.back();
   const Expr *StrippedExpr = E->IgnoreParenImpCasts();
 
   // For expressions like `&(*s).b`, the base is recorded and what should be
   // checked.
   const MemberExpr *Member = nullptr;
   while ((Member = dyn_cast<MemberExpr>(StrippedExpr)) && !Member->isArrow())
     StrippedExpr = Member->getBase()->IgnoreParenImpCasts();
 
   LastRecord.PossibleDerefs.erase(StrippedExpr);
 }
 
 void Sema::CheckSubscriptAccessOfNoDeref(const ArraySubscriptExpr *E) {
   QualType ResultTy = E->getType();
   ExpressionEvaluationContextRecord &LastRecord = ExprEvalContexts.back();
 
   // Bail if the element is an array since it is not memory access.
   if (isa<ArrayType>(ResultTy))
     return;
 
   if (ResultTy->hasAttr(attr::NoDeref)) {
     LastRecord.PossibleDerefs.insert(E);
     return;
   }
 
   // Check if the base type is a pointer to a member access of a struct
   // marked with noderef.
   const Expr *Base = E->getBase();
   QualType BaseTy = Base->getType();
   if (!(isa<ArrayType>(BaseTy) || isa<PointerType>(BaseTy)))
     // Not a pointer access
     return;
 
   const MemberExpr *Member = nullptr;
   while ((Member = dyn_cast<MemberExpr>(Base->IgnoreParenCasts())) &&
          Member->isArrow())
     Base = Member->getBase();
 
   if (const auto *Ptr = dyn_cast<PointerType>(Base->getType())) {
     if (Ptr->getPointeeType()->hasAttr(attr::NoDeref))
       LastRecord.PossibleDerefs.insert(E);
   }
 }
 
 ExprResult Sema::ActOnOMPArraySectionExpr(Expr *Base, SourceLocation LBLoc,
                                           Expr *LowerBound,
                                           SourceLocation ColonLoc, Expr *Length,
                                           SourceLocation RBLoc) {
   if (Base->getType()->isPlaceholderType() &&
       !Base->getType()->isSpecificPlaceholderType(
           BuiltinType::OMPArraySection)) {
     ExprResult Result = CheckPlaceholderExpr(Base);
     if (Result.isInvalid())
       return ExprError();
     Base = Result.get();
   }
   if (LowerBound && LowerBound->getType()->isNonOverloadPlaceholderType()) {
     ExprResult Result = CheckPlaceholderExpr(LowerBound);
     if (Result.isInvalid())
       return ExprError();
     Result = DefaultLvalueConversion(Result.get());
     if (Result.isInvalid())
       return ExprError();
     LowerBound = Result.get();
   }
   if (Length && Length->getType()->isNonOverloadPlaceholderType()) {
     ExprResult Result = CheckPlaceholderExpr(Length);
     if (Result.isInvalid())
       return ExprError();
     Result = DefaultLvalueConversion(Result.get());
if (getLangOpts().CPlusPlus11) {
     for (auto *Op : {LHSExp, RHSExp}) {
       Op = Op->IgnoreImplicit();
       if (Op->getType()->isArrayType() && !Op->isLValue())
         VK = VK_XValue;
     }
   }
 
   // Perform default conversions.
-  if (!LHSExp->getType()->getAs<VectorType>()) {
+  if (!LHSExp->getType()->getAs<VectorType>() &&
+      !LHSExp->getMatrixFromIndexExpr(getLangOpts().MatrixTypes)) {
     ExprResult Result = DefaultFunctionArrayLvalueConversion(LHSExp);
     if (Result.isInvalid())
       return ExprError();
     LHSExp = Result.get();
   }
   ExprResult Result = DefaultFunctionArrayLvalueConversion(RHSExp);
   if (Result.isInvalid())
     return ExprError();
Diag(LHSExp->getBeginLoc(), diag::ext_subscript_non_lvalue)
         << LHSExp->getSourceRange();
     LHSExp = ImpCastExprToType(LHSExp, Context.getArrayDecayedType(LHSTy),
                                CK_ArrayToPointerDecay).get();
     LHSTy = LHSExp->getType();
 
     BaseExpr = LHSExp;
     IndexExpr = RHSExp;
     ResultType = LHSTy->getAs<PointerType>()->getPointeeType();
+  } else if (auto *MTy =
+                 LHSExp->getMatrixFromIndexExpr(getLangOpts().MatrixTypes)) {
+    BaseExpr = LHSExp;
+    IndexExpr = RHSExp;
+
+    // Validate index.
+    bool IsRowIdx = BaseExpr->getType()->isConstantMatrixType();
+    if (!IndexExpr->getType()->isIntegerType() &&
+        !IndexExpr->isTypeDependent()) {
+      Diag(IndexExpr->getBeginLoc(), diag::err_matrix_index_not_integer)
+          << (IsRowIdx ? 0 : 1);
+      return ExprError();
+    }
+
+    unsigned Dim = IsRowIdx ? MTy->getNumRows() : MTy->getNumColumns();
+    llvm::APSInt Idx;
+    if (IndexExpr->isIntegerConstantExpr(Idx, Context) &&
+        (Idx < 0 || Idx >= Dim)) {
+      Diag(IndexExpr->getBeginLoc(), diag::err_matrix_index_outside_range)
+          << (IsRowIdx ? 0 : 1) << Dim;
+      return ExprError();
+    }
+
+    // Set the type of the outer ArraySubscriptExpr to the element type and for
+    // the inner expression to the matrix type. That ensures an error when only
+    // using a single ArraySubscriptExpr on a matrix.
+    if (IsRowIdx)
+      ResultType = Context.IncompleteMatrixIdxTy;
+    else {
+      ResultType = MTy->getElementType();
+      BaseExpr->setType(ResultType);
+    }
+
+    VK = VK_LValue;
+    OK = OK_VectorComponent;
   } else if (RHSTy->isArrayType()) {
     // Same as previous, except for 123[f().a] case
     Diag(RHSExp->getBeginLoc(), diag::ext_subscript_non_lvalue)
         << RHSExp->getSourceRange();
     RHSExp = ImpCastExprToType(RHSExp, Context.getArrayDecayedType(RHSTy),
                                CK_ArrayToPointerDecay).get();
     RHSTy = RHSExp->getType();
 
#include "clang/AST/BuiltinTypes.def"
   // The debugger mode could theoretically but currently does not try
   // to resolve unknown-typed arguments based on known parameter types.
   case BuiltinType::UnknownAny:
     return true;
 
   // These are always invalid as call arguments and should be reported.
   case BuiltinType::BoundMember:
   case BuiltinType::BuiltinFn:
+  case BuiltinType::IncompleteMatrixIdx:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
     return true;
 
   }
   llvm_unreachable("bad builtin type kind");
 }
if (DRE) {
                                 VK_RValue, SourceLocation());
       }
     }
 
     Diag(E->getBeginLoc(), diag::err_builtin_fn_use);
     return ExprError();
   }
 
+  case BuiltinType::IncompleteMatrixIdx:
+    Diag(cast<ArraySubscriptExpr>(E)->getIdx()->getBeginLoc(),
+         diag::err_matrix_incomplete_index);
+    return ExprError();
+
   // Expressions of unknown type.
   case BuiltinType::OMPArraySection:
     Diag(E->getBeginLoc(), diag::err_omp_array_section_use);
     return ExprError();
 
   // Expressions of unknown type.
   case BuiltinType::OMPArrayShaping:
     return ExprError(Diag(E->getBeginLoc(), diag::err_omp_array_shaping_use));

clang/lib/Serialization/ASTCommon.cpp

 #define SVE_TYPE(Name, Id, SingletonId) \
   case BuiltinType::Id: \
     ID = PREDEF_TYPE_##Id##_ID; \
     break;
 #include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::BuiltinFn:
     ID = PREDEF_TYPE_BUILTIN_FN;
     break;
+  case BuiltinType::IncompleteMatrixIdx:
+    ID = PREDEF_TYPE_INCOMPLETE_MATRIX_IDX;
+    break;
   case BuiltinType::OMPArraySection:
     ID = PREDEF_TYPE_OMP_ARRAY_SECTION;
     break;
   case BuiltinType::OMPArrayShaping:
     ID = PREDEF_TYPE_OMP_ARRAY_SHAPING;
     break;
   case BuiltinType::OMPIterator:
     ID = PREDEF_TYPE_OMP_ITERATOR;

clang/lib/Serialization/ASTReader.cpp

case PREDEF_TYPE_AUTO_RREF_DEDUCT:
       T = Context.getAutoRRefDeductType();
       break;
     case PREDEF_TYPE_ARC_UNBRIDGED_CAST:
       T = Context.ARCUnbridgedCastTy;
       break;
     case PREDEF_TYPE_BUILTIN_FN:
       T = Context.BuiltinFnTy;
       break;
+    case PREDEF_TYPE_INCOMPLETE_MATRIX_IDX:
+      T = Context.IncompleteMatrixIdxTy;
+      break;
     case PREDEF_TYPE_OMP_ARRAY_SECTION:
       T = Context.OMPArraySectionTy;
       break;
     case PREDEF_TYPE_OMP_ARRAY_SHAPING:
       T = Context.OMPArraySectionTy;
       break;
     case PREDEF_TYPE_OMP_ITERATOR:
       T = Context.OMPIteratorTy;

clang/test/CodeGen/matrix-type-operators.c

+// RUN: %clang_cc1 -fenable-matrix -triple x86_64-apple-darwin %s -emit-llvm -disable-llvm-passes -o - | FileCheck %s
+
+// Tests for the matrix type operators.
+
+typedef double dx5x5_t __attribute__((matrix_type(5, 5)));
+typedef float fx2x3_t __attribute__((matrix_type(2, 3)));
+
+// Check that we can use matrix index expression on different floating point
+// matrixes and indices.
+void insert_fp(dx5x5_t a, double d, fx2x3_t b, float e, int j, unsigned k) {
+  // CHECK-LABEL: define void @insert_fp(<25 x double> %a, double %d, <6 x float> %b, float %e, i32 %j, i32 %k)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %a.addr = alloca [25 x double], align 8
+  // CHECK-NEXT:    %d.addr = alloca double, align 8
+  // CHECK-NEXT:    %b.addr = alloca [6 x float], align 4
+  // CHECK-NEXT:    %e.addr = alloca float, align 4
+  // CHECK-NEXT:    %j.addr = alloca i32, align 4
+  // CHECK-NEXT:    %k.addr = alloca i32, align 4
+  // CHECK-NEXT:    %0 = bitcast [25 x double]* %a.addr to <25 x double>*
+  // CHECK-NEXT:    store <25 x double> %a, <25 x double>* %0, align 8
+  // CHECK-NEXT:    store double %d, double* %d.addr, align 8
+  // CHECK-NEXT:    %1 = bitcast [6 x float]* %b.addr to <6 x float>*
+  // CHECK-NEXT:    store <6 x float> %b, <6 x float>* %1, align 4
+  // CHECK-NEXT:    store float %e, float* %e.addr, align 4
+  // CHECK-NEXT:    store i32 %j, i32* %j.addr, align 4
+  // CHECK-NEXT:    store i32 %k, i32* %k.addr, align 4
+  // CHECK-NEXT:    %2 = load double, double* %d.addr, align 8
+  // CHECK-NEXT:    %3 = load <25 x double>, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %matins = insertelement <25 x double> %3, double %2, i64 5
+  // CHECK-NEXT:    store <25 x double> %matins, <25 x double>* %0, align 8
+  a[0ll][1u] = d;
+
+  // CHECK-NEXT:    %4 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %5 = load <6 x float>, <6 x float>* %1, align 4
+  // CHECK-NEXT:    %matins1 = insertelement <6 x float> %5, float %4, i32 1
+  // CHECK-NEXT:    store <6 x float> %matins1, <6 x float>* %1, align 4
+  b[1][0] = e;
+
+  // CHECK-NEXT:    %6 = load double, double* %d.addr, align 8
+  // CHECK-NEXT:    %7 = load <25 x double>, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %matins2 = insertelement <25 x double> %7, double %6, i32 1
+  // CHECK-NEXT:    store <25 x double> %matins2, <25 x double>* %0, align 8
+  a[1][0u] = d;
+
+  // CHECK-NEXT:    %8 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %9 = load <6 x float>, <6 x float>* %1, align 4
+  // CHECK-NEXT:    %matins3 = insertelement <6 x float> %9, float %8, i64 3
+  // CHECK-NEXT:    store <6 x float> %matins3, <6 x float>* %1, align 4
+  b[1ull][1] = e;
+
+  // CHECK-NEXT:    %10 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %11 = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:    %12 = load i32, i32* %k.addr, align 4
+  // CHECK-NEXT:    %13 = mul i32 %12, 2
+  // CHECK-NEXT:    %14 = add i32 %13, %11
+  // CHECK-NEXT:    %15 = load <6 x float>, <6 x float>* %1, align 4
+  // CHECK-NEXT:    %matins4 = insertelement <6 x float> %15, float %10, i32 %14
+  // CHECK-NEXT:    store <6 x float> %matins4, <6 x float>* %1, align 4
+  // CHECK-NEXT:    ret void
+  b[j][k] = e;
+}
+
+// Check that we can can use matrix index expressions on integer matrixes.
+typedef int ix9x3_t __attribute__((matrix_type(9, 3)));
+void insert_int(ix9x3_t a, int i) {
+  // CHECK-LABEL: define void @insert_int(<27 x i32> %a, i32 %i)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %a.addr = alloca [27 x i32], align 4
+  // CHECK-NEXT:    %i.addr = alloca i32, align 4
+  // CHECK-NEXT:    %0 = bitcast [27 x i32]* %a.addr to <27 x i32>*
+  // CHECK-NEXT:    store <27 x i32> %a, <27 x i32>* %0, align 4
+  // CHECK-NEXT:    store i32 %i, i32* %i.addr, align 4
+  // CHECK-NEXT:    %1 = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    %2 = load <27 x i32>, <27 x i32>* %0, align 4
+  // CHECK-NEXT:    %matins = insertelement <27 x i32> %2, i32 %1, i32 13
+  // CHECK-NEXT:    store <27 x i32> %matins, <27 x i32>* %0, align 4
+  // CHECK-NEXT:    ret void
+
+  a[4u][1u] = i;
+}
+
+// Check that we can can use matrix index expressions on FP and integer
+// matrixes.
+typedef int ix9x3_t __attribute__((matrix_type(9, 3)));
+void insert_int_fp(ix9x3_t *a, int i, fx2x3_t b, float e, short j, unsigned long long k) {
+  // CHECK-LABEL: define void @insert_int_fp([27 x i32]* %a, i32 %i, <6 x float> %b, float %e, i16 signext %j, i64 %k)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %a.addr = alloca [27 x i32]*, align 8
+  // CHECK-NEXT:    %i.addr = alloca i32, align 4
+  // CHECK-NEXT:    %b.addr = alloca [6 x float], align 4
+  // CHECK-NEXT:    %e.addr = alloca float, align 4
+  // CHECK-NEXT:    %j.addr = alloca i16, align 2
+  // CHECK-NEXT:    %k.addr = alloca i64, align 8
+  // CHECK-NEXT:    store [27 x i32]* %a, [27 x i32]** %a.addr, align 8
+  // CHECK-NEXT:    store i32 %i, i32* %i.addr, align 4
+  // CHECK-NEXT:    %0 = bitcast [6 x float]* %b.addr to <6 x float>*
+  // CHECK-NEXT:    store <6 x float> %b, <6 x float>* %0, align 4
+  // CHECK-NEXT:    store float %e, float* %e.addr, align 4
+  // CHECK-NEXT:    store i16 %j, i16* %j.addr, align 2
+  // CHECK-NEXT:    store i64 %k, i64* %k.addr, align 8
+  // CHECK-NEXT:    %1 = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    %2 = load [27 x i32]*, [27 x i32]** %a.addr, align 8
+  // CHECK-NEXT:    %3 = bitcast [27 x i32]* %2 to <27 x i32>*
+  // CHECK-NEXT:    %4 = load <27 x i32>, <27 x i32>* %3, align 4
+  // CHECK-NEXT:    %matins = insertelement <27 x i32> %4, i32 %1, i32 13
+  // CHECK-NEXT:    store <27 x i32> %matins, <27 x i32>* %3, align 4
+  (*a)[4u][1u] = i;
+
+  // CHECK-NEXT:    %5 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %6 = load <6 x float>, <6 x float>* %0, align 4
+  // CHECK-NEXT:    %matins1 = insertelement <6 x float> %6, float %5, i32 3
+  // CHECK-NEXT:    store <6 x float> %matins1, <6 x float>* %0, align 4
+  b[1u][1u] = e;
+
+  // CHECK-NEXT:    %7 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %8 = load i16, i16* %j.addr, align 2
+  // CHECK-NEXT:    %9 = load i64, i64* %k.addr, align 8
+  // CHECK-NEXT:    %10 = zext i16 %8 to i64
+  // CHECK-NEXT:    %11 = mul i64 %9, 2
+  // CHECK-NEXT:    %12 = add i64 %11, %10
+  // CHECK-NEXT:    %13 = load <6 x float>, <6 x float>* %0, align 4
+  // CHECK-NEXT:    %matins2 = insertelement <6 x float> %13, float %7, i64 %12
+  // CHECK-NEXT:    store <6 x float> %matins2, <6 x float>* %0, align 4
+  // CHECK-NEXT:    ret void
+  b[j][k] = e;
+}
+
+// Check that we can use overloaded matrix index expressions on matrixes with
+// matching dimensions, but different element types.
+typedef double dx3x3_t __attribute__((matrix_type(3, 3)));
+typedef float fx3x3_t __attribute__((matrix_type(3, 3)));
+void insert_matching_dimensions(dx3x3_t a, double i, fx3x3_t b, float e, long int j, char k) {
+  // CHECK-LABEL: define void @insert_matching_dimensions(<9 x double> %a, double %i, <9 x float> %b, float %e, i64 %j, i8 signext %k) #3 {
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %a.addr = alloca [9 x double], align 8
+  // CHECK-NEXT:    %i.addr = alloca double, align 8
+  // CHECK-NEXT:    %b.addr = alloca [9 x float], align 4
+  // CHECK-NEXT:    %e.addr = alloca float, align 4
+  // CHECK-NEXT:    %j.addr = alloca i64, align 8
+  // CHECK-NEXT:    %k.addr = alloca i8, align 1
+  // CHECK-NEXT:    %0 = bitcast [9 x double]* %a.addr to <9 x double>*
+  // CHECK-NEXT:    store <9 x double> %a, <9 x double>* %0, align 8
+  // CHECK-NEXT:    store double %i, double* %i.addr, align 8
+  // CHECK-NEXT:    %1 = bitcast [9 x float]* %b.addr to <9 x float>*
+  // CHECK-NEXT:    store <9 x float> %b, <9 x float>* %1, align 4
+  // CHECK-NEXT:    store float %e, float* %e.addr, align 4
+  // CHECK-NEXT:    store i64 %j, i64* %j.addr, align 8
+  // CHECK-NEXT:    store i8 %k, i8* %k.addr, align 1
+  // CHECK-NEXT:    %2 = load double, double* %i.addr, align 8
+  // CHECK-NEXT:    %3 = load <9 x double>, <9 x double>* %0, align 8
+  // CHECK-NEXT:    %matins = insertelement <9 x double> %3, double %2, i32 5
+  // CHECK-NEXT:    store <9 x double> %matins, <9 x double>* %0, align 8
+  a[2u][1u] = i;
+
+  // CHECK-NEXT:    %4 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %5 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matins1 = insertelement <9 x float> %5, float %4, i32 7
+  // CHECK-NEXT:    store <9 x float> %matins1, <9 x float>* %1, align 4
+  b[1u][2u] = e;
+
+  // CHECK-NEXT:    %6 = load double, double* %i.addr, align 8
+  // CHECK-NEXT:    %conv = fptrunc double %6 to float
+  // CHECK-NEXT:    %7 = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    %8 = load i8, i8* %k.addr, align 1
+  // CHECK-NEXT:    %9 = zext i8 %8 to i64
+  // CHECK-NEXT:    %10 = mul i64 %9, 3
+  // CHECK-NEXT:    %11 = add i64 %10, %7
+  // CHECK-NEXT:    %12 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matins2 = insertelement <9 x float> %12, float %conv, i64 %11
+  // CHECK-NEXT:    store <9 x float> %matins2, <9 x float>* %1, align 4
+  // CHECK-NEXT:    ret void
+  b[j][k] = i;
+}
+
+void extract1(dx5x5_t a, fx3x3_t b, ix9x3_t c, unsigned long j) {
+  // CHECK-LABEL: @extract1(
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %a.addr = alloca [25 x double], align 8
+  // CHECK-NEXT:    %b.addr = alloca [9 x float], align 4
+  // CHECK-NEXT:    %c.addr = alloca [27 x i32], align 4
+  // CHECK-NEXT:    %j.addr = alloca i64, align 8
+  // CHECK-NEXT:    %v1 = alloca double, align 8
+  // CHECK-NEXT:    %v2 = alloca float, align 4
+  // CHECK-NEXT:    %v3 = alloca i32, align 4
+  // CHECK-NEXT:    %v4 = alloca i32, align 4
+  // CHECK-NEXT:    %0 = bitcast [25 x double]* %a.addr to <25 x double>*
+  // CHECK-NEXT:    store <25 x double> %a, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %1 = bitcast [9 x float]* %b.addr to <9 x float>*
+  // CHECK-NEXT:    store <9 x float> %b, <9 x float>* %1, align 4
+  double v1 = a[2][3];
+
+  // CHECK-NEXT:    %2 = bitcast [27 x i32]* %c.addr to <27 x i32>*
+  // CHECK-NEXT:    store <27 x i32> %c, <27 x i32>* %2, align 4
+  // CHECK-NEXT:    store i64 %j, i64* %j.addr, align 8
+  // CHECK-NEXT:    %3 = load <25 x double>, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %matext = extractelement <25 x double> %3, i32 17
+  // CHECK-NEXT:    store double %matext, double* %v1, align 8
+  float v2 = b[2][1];
+
+  // CHECK-NEXT:    %4 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matext1 = extractelement <9 x float> %4, i32 5
+  // CHECK-NEXT:    store float %matext1, float* %v2, align 4
+  // CHECK-NEXT:    %5 = load <27 x i32>, <27 x i32>* %2, align 4
+  // CHECK-NEXT:    %matext2 = extractelement <27 x i32> %5, i32 10
+  // CHECK-NEXT:    store i32 %matext2, i32* %v3, align 4
+  int v3 = c[1][1];
+
+  // CHECK-NEXT:    %6 = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    %7 = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    %8 = mul i64 %7, 9
+  // CHECK-NEXT:    %9 = add i64 %8, %6
+  // CHECK-NEXT:    %10 = load <27 x i32>, <27 x i32>* %2, align 4
+  // CHECK-NEXT:    %matext3 = extractelement <27 x i32> %10, i64 %9
+  // CHECK-NEXT:    store i32 %matext3, i32* %v4, align 4
+  // CHECK-NEXT:    ret void
+  int v4 = c[j][j];
+}
+
+typedef double dx3x2_t __attribute__((matrix_type(3, 2)));
+double test_extract_matrix_pointer(dx5x5_t *ptr, dx3x2_t **ptr2) {
+  // CHECK-LABEL: define double @test_extract_matrix_pointer([25 x double]* %ptr, [6 x double]** %ptr2)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %ptr.addr = alloca [25 x double]*, align 8
+  // CHECK-NEXT:    %ptr2.addr = alloca [6 x double]**, align 8
+  // CHECK-NEXT:    store [25 x double]* %ptr, [25 x double]** %ptr.addr, align 8
+  // CHECK-NEXT:    store [6 x double]** %ptr2, [6 x double]*** %ptr2.addr, align 8
+  // CHECK-NEXT:    %0 = load [25 x double]*, [25 x double]** %ptr.addr, align 8
+  // CHECK-NEXT:    %arrayidx = getelementptr inbounds [25 x double], [25 x double]* %0, i64 0
+  // CHECK-NEXT:    %1 = bitcast [25 x double]* %arrayidx to <25 x double>*
+  // CHECK-NEXT:    %2 = load <25 x double>, <25 x double>* %1, align 8
+  // CHECK-NEXT:    %matext = extractelement <25 x double> %2, i32 17
+  // CHECK-NEXT:    %3 = load [6 x double]**, [6 x double]*** %ptr2.addr, align 8
+  // CHECK-NEXT:    %arrayidx1 = getelementptr inbounds [6 x double]*, [6 x double]** %3, i64 1
+  // CHECK-NEXT:    %4 = load [6 x double]*, [6 x double]** %arrayidx1, align 8
+  // CHECK-NEXT:    %arrayidx2 = getelementptr inbounds [6 x double], [6 x double]* %4, i64 2
+  // CHECK-NEXT:    %5 = bitcast [6 x double]* %arrayidx2 to <6 x double>*
+  // CHECK-NEXT:    %6 = load <6 x double>, <6 x double>* %5, align 8
+  // CHECK-NEXT:    %matext3 = extractelement <6 x double> %6, i32 3
+  // CHECK-NEXT:    %add = fadd double %matext, %matext3
+  // CHECK-NEXT:    %7 = load [6 x double]**, [6 x double]*** %ptr2.addr, align 8
+  // CHECK-NEXT:    %add.ptr = getelementptr inbounds [6 x double]*, [6 x double]** %7, i64 4
+  // CHECK-NEXT:    %8 = load [6 x double]*, [6 x double]** %add.ptr, align 8
+  // CHECK-NEXT:    %add.ptr4 = getelementptr inbounds [6 x double], [6 x double]* %8, i64 6
+  // CHECK-NEXT:    %9 = bitcast [6 x double]* %add.ptr4 to <6 x double>*
+  // CHECK-NEXT:    %10 = load <6 x double>, <6 x double>* %9, align 8
+  // CHECK-NEXT:    %matext5 = extractelement <6 x double> %10, i32 1
+  // CHECK-NEXT:    %add6 = fadd double %add, %matext5
+  // CHECK-NEXT:    ret double %add6
+
+  return (ptr[0])[2][3] + ptr2[1][2][0][1] + (*(*(ptr2 + 4) + 6))[1][0];
+}
+void insert_extract(dx5x5_t a, fx3x3_t b, unsigned long j, short k) {
+  // CHECK-LABEL: define void @insert_extract(<25 x double> %a, <9 x float> %b, i64 %j, i16 signext %k)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %a.addr = alloca [25 x double], align 8
+  // CHECK-NEXT:    %b.addr = alloca [9 x float], align 4
+  // CHECK-NEXT:    %j.addr = alloca i64, align 8
+  // CHECK-NEXT:    %k.addr = alloca i16, align 2
+  // CHECK-NEXT:    %0 = bitcast [25 x double]* %a.addr to <25 x double>*
+  // CHECK-NEXT:    store <25 x double> %a, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %1 = bitcast [9 x float]* %b.addr to <9 x float>*
+  // CHECK-NEXT:    store <9 x float> %b, <9 x float>* %1, align 4
+  // CHECK-NEXT:    store i64 %j, i64* %j.addr, align 8
+  // CHECK-NEXT:    store i16 %k, i16* %k.addr, align 2
+  // CHECK-NEXT:    %2 = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    %3 = load i16, i16* %k.addr, align 2
+  // CHECK-NEXT:    %4 = zext i16 %3 to i64
+  // CHECK-NEXT:    %5 = mul i64 %4, 3
+  // CHECK-NEXT:    %6 = add i64 %5, %2
+  // CHECK-NEXT:    %7 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matext = extractelement <9 x float> %7, i64 %6
+  // CHECK-NEXT:    %conv = fpext float %matext to double
+  // CHECK-NEXT:    %8 = load <25 x double>, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %matins = insertelement <25 x double> %8, double %conv, i32 17
+  // CHECK-NEXT:    store <25 x double> %matins, <25 x double>* %0, align 8
+  a[2][3] = b[j][k];
+
+  // CHECK-NEXT:    %9 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matext1 = extractelement <9 x float> %9, i32 3
+  // CHECK-NEXT:    %10 = load i16, i16* %k.addr, align 2
+  // CHECK-NEXT:    %11 = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    %12 = zext i16 %10 to i64
+  // CHECK-NEXT:    %13 = mul i64 %11, 3
+  // CHECK-NEXT:    %14 = add i64 %13, %12
+  // CHECK-NEXT:    %15 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matins2 = insertelement <9 x float> %15, float %matext1, i64 %14
+  // CHECK-NEXT:    store <9 x float> %matins2, <9 x float>* %1, align 4
+  b[k][j] = b[0][1];
+
+  // CHECK-NEXT:    %16 = load i16, i16* %k.addr, align 2
+  // CHECK-NEXT:    %17 = zext i16 %16 to i32
+  // CHECK-NEXT:    %18 = mul i32 %17, 3
+  // CHECK-NEXT:    %19 = add i32 %18, 0
+  // CHECK-NEXT:    %20 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matext3 = extractelement <9 x float> %20, i32 %19
+  // CHECK-NEXT:    %21 = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    %22 = mul i64 %21, 3
+  // CHECK-NEXT:    %23 = add i64 %22, 2
+  // CHECK-NEXT:    %24 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matins4 = insertelement <9 x float> %24, float %matext3, i64 %23
+  // CHECK-NEXT:    store <9 x float> %matins4, <9 x float>* %1, align 4
+  // CHECK-NEXT:    ret void
+  b[2][j] = b[0][k];
+}

clang/test/CodeGenCXX/matrix-type-operators.cpp

+// RUN: %clang_cc1 -fenable-matrix -triple x86_64-apple-darwin %s -emit-llvm -disable-llvm-passes -o - -std=c++11 | FileCheck %s
+
+typedef double dx5x5_t __attribute__((matrix_type(5, 5)));
+using fx2x3_t = float __attribute__((matrix_type(2, 3)));
+
+void insert_fp(dx5x5_t *a, double d, fx2x3_t *b, float e) {
+  (*a)[0u][1u] = d;
+  (*b)[1u][0u] = e;
+
+  // CHECK-LABEL: @_Z9insert_fpPU11matrix_typeLm5ELm5EddPU11matrix_typeLm2ELm3Eff(
+  // CHECK-NEXT: entry:
+  // CHECK-NEXT:    %a.addr = alloca [25 x double]*, align 8
+  // CHECK-NEXT:    %d.addr = alloca double, align 8
+  // CHECK-NEXT:    %b.addr = alloca [6 x float]*, align 8
+  // CHECK-NEXT:    %e.addr = alloca float, align 4
+  // CHECK-NEXT:    store [25 x double]* %a, [25 x double]** %a.addr, align 8
+  // CHECK-NEXT:    store double %d, double* %d.addr, align 8
+  // CHECK-NEXT:    store [6 x float]* %b, [6 x float]** %b.addr, align 8
+  // CHECK-NEXT:    store float %e, float* %e.addr, align 4
+  // CHECK-NEXT:    %0 = load double, double* %d.addr, align 8
+  // CHECK-NEXT:    %1 = load [25 x double]*, [25 x double]** %a.addr, align 8
+  // CHECK-NEXT:    %2 = bitcast [25 x double]* %1 to <25 x double>*
+  // CHECK-NEXT:    %3 = load <25 x double>, <25 x double>* %2, align 8
+  // CHECK-NEXT:    %matins = insertelement <25 x double> %3, double %0, i32 5
+  // CHECK-NEXT:    store <25 x double> %matins, <25 x double>* %2, align 8
+  // CHECK-NEXT:    %4 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %5 = load [6 x float]*, [6 x float]** %b.addr, align 8
+  // CHECK-NEXT:    %6 = bitcast [6 x float]* %5 to <6 x float>*
+  // CHECK-NEXT:    %7 = load <6 x float>, <6 x float>* %6, align 4
+  // CHECK-NEXT:    %matins1 = insertelement <6 x float> %7, float %4, i32 1
+  // CHECK-NEXT:    store <6 x float> %matins1, <6 x float>* %6, align 4
+  // CHECK-NEXT:    ret void
+}
+
+typedef int ix9x3_t __attribute__((matrix_type(9, 3)));
+
+void insert_int(ix9x3_t *a, int i) {
+  (*a)[4u][1u] = i;
+
+  // CHECK-LABEL: @_Z10insert_intPU11matrix_typeLm9ELm3Eii(
+  // CHECK-NEXT: entry:
+  // CHECK-NEXT:    %a.addr = alloca [27 x i32]*, align 8
+  // CHECK-NEXT:    %i.addr = alloca i32, align 4
+  // CHECK-NEXT:    store [27 x i32]* %a, [27 x i32]** %a.addr, align 8
+  // CHECK-NEXT:    store i32 %i, i32* %i.addr, align 4
+  // CHECK-NEXT:    %0 = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    %1 = load [27 x i32]*, [27 x i32]** %a.addr, align 8
+  // CHECK-NEXT:    %2 = bitcast [27 x i32]* %1 to <27 x i32>*
+  // CHECK-NEXT:    %3 = load <27 x i32>, <27 x i32>* %2, align 4
+  // CHECK-NEXT:    %matins = insertelement <27 x i32> %3, i32 %0, i32 13
+  // CHECK-NEXT:    store <27 x i32> %matins, <27 x i32>* %2, align 4
+  // CHECK-NEXT:    ret void
+}
+
+template <typename EltTy, unsigned Rows, unsigned Columns>
+struct MyMatrix {
+  using matrix_t = EltTy __attribute__((matrix_type(Rows, Columns)));
+
+  matrix_t value;
+};
+
+template <typename EltTy, unsigned Rows, unsigned Columns>
+void insert(MyMatrix<EltTy, Rows, Columns> &Mat, EltTy e) {
+  Mat.value[1u][0u] = e;
+}
+
+void test_template(unsigned *Ptr1, unsigned E1, float *Ptr2, float E2) {
+
+  // CHECK-LABEL: define void @_Z13test_templatePjjPff(i32* %Ptr1, i32 %E1, float* %Ptr2, float %E2)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %Ptr1.addr = alloca i32*, align 8
+  // CHECK-NEXT:    %E1.addr = alloca i32, align 4
+  // CHECK-NEXT:    %Ptr2.addr = alloca float*, align 8
+  // CHECK-NEXT:    %E2.addr = alloca float, align 4
+  // CHECK-NEXT:    %Mat1 = alloca %struct.MyMatrix, align 4
+  // CHECK-NEXT:    %Mat2 = alloca %struct.MyMatrix.0, align 4
+  // CHECK-NEXT:    store i32* %Ptr1, i32** %Ptr1.addr, align 8
+  // CHECK-NEXT:    store i32 %E1, i32* %E1.addr, align 4
+  // CHECK-NEXT:    store float* %Ptr2, float** %Ptr2.addr, align 8
+  // CHECK-NEXT:    store float %E2, float* %E2.addr, align 4
+  // CHECK-NEXT:    %0 = load i32*, i32** %Ptr1.addr, align 8
+  // CHECK-NEXT:    %1 = bitcast i32* %0 to [4 x i32]*
+  // CHECK-NEXT:    %2 = bitcast [4 x i32]* %1 to <4 x i32>*
+  // CHECK-NEXT:    %3 = load <4 x i32>, <4 x i32>* %2, align 4
+  // CHECK-NEXT:    %value = getelementptr inbounds %struct.MyMatrix, %struct.MyMatrix* %Mat1, i32 0, i32 0
+  // CHECK-NEXT:    %4 = bitcast [4 x i32]* %value to <4 x i32>*
+  // CHECK-NEXT:    store <4 x i32> %3, <4 x i32>* %4, align 4
+  // CHECK-NEXT:    %5 = load i32, i32* %E1.addr, align 4
+  // CHECK-NEXT:    call void @_Z6insertIjLj2ELj2EEvR8MyMatrixIT_XT0_EXT1_EES1_(%struct.MyMatrix* dereferenceable(16) %Mat1, i32 %5)
+  // CHECK-NEXT:    %6 = load float*, float** %Ptr2.addr, align 8
+  // CHECK-NEXT:    %7 = bitcast float* %6 to [24 x float]*
+  // CHECK-NEXT:    %8 = bitcast [24 x float]* %7 to <24 x float>*
+  // CHECK-NEXT:    %9 = load <24 x float>, <24 x float>* %8, align 4
+  // CHECK-NEXT:    %value1 = getelementptr inbounds %struct.MyMatrix.0, %struct.MyMatrix.0* %Mat2, i32 0, i32 0
+  // CHECK-NEXT:    %10 = bitcast [24 x float]* %value1 to <24 x float>*
+  // CHECK-NEXT:    store <24 x float> %9, <24 x float>* %10, align 4
+  // CHECK-NEXT:    %11 = load float, float* %E2.addr, align 4
+  // CHECK-NEXT:    call void @_Z6insertIfLj3ELj8EEvR8MyMatrixIT_XT0_EXT1_EES1_(%struct.MyMatrix.0* dereferenceable(96) %Mat2, float %11)
+  // CHECK-NEXT:    ret void
+
+  // CHECK-LABEL: define linkonce_odr void @_Z6insertIjLj2ELj2EEvR8MyMatrixIT_XT0_EXT1_EES1_(%struct.MyMatrix* dereferenceable(16) %Mat, i32 %e)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %Mat.addr = alloca %struct.MyMatrix*, align 8
+  // CHECK-NEXT:    %e.addr = alloca i32, align 4
+  // CHECK-NEXT:    store %struct.MyMatrix* %Mat, %struct.MyMatrix** %Mat.addr, align 8
+  // CHECK-NEXT:    store i32 %e, i32* %e.addr, align 4
+  // CHECK-NEXT:    %0 = load i32, i32* %e.addr, align 4
+  // CHECK-NEXT:    %1 = load %struct.MyMatrix*, %struct.MyMatrix** %Mat.addr, align 8
+  // CHECK-NEXT:    %value = getelementptr inbounds %struct.MyMatrix, %struct.MyMatrix* %1, i32 0, i32 0
+  // CHECK-NEXT:    %2 = bitcast [4 x i32]* %value to <4 x i32>*
+  // CHECK-NEXT:    %3 = load <4 x i32>, <4 x i32>* %2, align 4
+  // CHECK-NEXT:    %matins = insertelement <4 x i32> %3, i32 %0, i32 1
+  // CHECK-NEXT:    store <4 x i32> %matins, <4 x i32>* %2, align 4
+  // CHECK-NEXT:    ret void
+
+  // CHECK-LABEL: define linkonce_odr void @_Z6insertIfLj3ELj8EEvR8MyMatrixIT_XT0_EXT1_EES1_(%struct.MyMatrix.0* dereferenceable(96) %Mat, float %e)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %Mat.addr = alloca %struct.MyMatrix.0*, align 8
+  // CHECK-NEXT:    %e.addr = alloca float, align 4
+  // CHECK-NEXT:    store %struct.MyMatrix.0* %Mat, %struct.MyMatrix.0** %Mat.addr, align 8
+  // CHECK-NEXT:    store float %e, float* %e.addr, align 4
+  // CHECK-NEXT:    %0 = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    %1 = load %struct.MyMatrix.0*, %struct.MyMatrix.0** %Mat.addr, align 8
+  // CHECK-NEXT:    %value = getelementptr inbounds %struct.MyMatrix.0, %struct.MyMatrix.0* %1, i32 0, i32 0
+  // CHECK-NEXT:    %2 = bitcast [24 x float]* %value to <24 x float>*
+  // CHECK-NEXT:    %3 = load <24 x float>, <24 x float>* %2, align 4
+  // CHECK-NEXT:    %matins = insertelement <24 x float> %3, float %0, i32 1
+  // CHECK-NEXT:    store <24 x float> %matins, <24 x float>* %2, align 4
+  // CHECK-NEXT:    ret void
+
+  MyMatrix<unsigned, 2, 2> Mat1;
+  Mat1.value = *((decltype(Mat1)::matrix_t *)Ptr1);
+  insert(Mat1, E1);
+
+  MyMatrix<float, 3, 8> Mat2;
+  Mat2.value = *((decltype(Mat2)::matrix_t *)Ptr2);
+  insert(Mat2, E2);
+}
+
+typedef float fx3x3_t __attribute__((matrix_type(3, 3)));
+void extract1(dx5x5_t a, fx3x3_t b, ix9x3_t c) {
+  // CHECK-LABEL: @_Z8extract1U11matrix_typeLm5ELm5EdU11matrix_typeLm3ELm3EfU11matrix_typeLm9ELm3Ei(
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %a.addr = alloca [25 x double], align 8
+  // CHECK-NEXT:    %b.addr = alloca [9 x float], align 4
+  // CHECK-NEXT:    %c.addr = alloca [27 x i32], align 4
+  // CHECK-NEXT:    %v1 = alloca double, align 8
+  // CHECK-NEXT:    %v2 = alloca float, align 4
+  // CHECK-NEXT:    %v3 = alloca i32, align 4
+  // CHECK-NEXT:    %0 = bitcast [25 x double]* %a.addr to <25 x double>*
+  // CHECK-NEXT:    store <25 x double> %a, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %1 = bitcast [9 x float]* %b.addr to <9 x float>*
+  // CHECK-NEXT:    store <9 x float> %b, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %2 = bitcast [27 x i32]* %c.addr to <27 x i32>*
+  // CHECK-NEXT:    store <27 x i32> %c, <27 x i32>* %2, align 4
+  // CHECK-NEXT:    %3 = load <25 x double>, <25 x double>* %0, align 8
+  // CHECK-NEXT:    %matext = extractelement <25 x double> %3, i32 17
+  // CHECK-NEXT:    store double %matext, double* %v1, align 8
+  // CHECK-NEXT:    %4 = load <9 x float>, <9 x float>* %1, align 4
+  // CHECK-NEXT:    %matext1 = extractelement <9 x float> %4, i32 5
+  // CHECK-NEXT:    store float %matext1, float* %v2, align 4
+  // CHECK-NEXT:    %5 = load <27 x i32>, <27 x i32>* %2, align 4
+  // CHECK-NEXT:    %matext2 = extractelement <27 x i32> %5, i32 10
+  // CHECK-NEXT:    store i32 %matext2, i32* %v3, align 4
+  // CHECK-NEXT:    ret void
+
+  double v1 = a[2][3];
+  float v2 = b[2][1];
+  int v3 = c[1][1];
+}
+
+template <typename EltTy, unsigned Rows, unsigned Columns>
+EltTy extract(MyMatrix<EltTy, Rows, Columns> &Mat) {
+  return Mat.value[1u][0u];
+}
+
+void test_extract_template(unsigned *Ptr1, float *Ptr2) {
+  // CHECK-LABEL: define void @_Z21test_extract_templatePjPf(i32* %Ptr1, float* %Ptr2)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %Ptr1.addr = alloca i32*, align 8
+  // CHECK-NEXT:    %Ptr2.addr = alloca float*, align 8
+  // CHECK-NEXT:    %Mat1 = alloca %struct.MyMatrix, align 4
+  // CHECK-NEXT:    %v1 = alloca i32, align 4
+  // CHECK-NEXT:    store i32* %Ptr1, i32** %Ptr1.addr, align 8
+  // CHECK-NEXT:    store float* %Ptr2, float** %Ptr2.addr, align 8
+  // CHECK-NEXT:    %0 = load i32*, i32** %Ptr1.addr, align 8
+  // CHECK-NEXT:    %1 = bitcast i32* %0 to [4 x i32]*
+  // CHECK-NEXT:    %2 = bitcast [4 x i32]* %1 to <4 x i32>*
+  // CHECK-NEXT:    %3 = load <4 x i32>, <4 x i32>* %2, align 4
+  // CHECK-NEXT:    %value = getelementptr inbounds %struct.MyMatrix, %struct.MyMatrix* %Mat1, i32 0, i32 0
+  // CHECK-NEXT:    %4 = bitcast [4 x i32]* %value to <4 x i32>*
+  // CHECK-NEXT:    store <4 x i32> %3, <4 x i32>* %4, align 4
+  // CHECK-NEXT:    %call = call i32 @_Z7extractIjLj2ELj2EET_R8MyMatrixIS0_XT0_EXT1_EE(%struct.MyMatrix* dereferenceable(16) %Mat1)
+  // CHECK-NEXT:    store i32 %call, i32* %v1, align 4
+  // CHECK-NEXT:    ret void
+
+  // CHECK-LABEL: define linkonce_odr i32 @_Z7extractIjLj2ELj2EET_R8MyMatrixIS0_XT0_EXT1_EE(%struct.MyMatrix* dereferenceable(16) %Mat)
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    %Mat.addr = alloca %struct.MyMatrix*, align 8
+  // CHECK-NEXT:    store %struct.MyMatrix* %Mat, %struct.MyMatrix** %Mat.addr, align 8
+  // CHECK-NEXT:    %0 = load %struct.MyMatrix*, %struct.MyMatrix** %Mat.addr, align 8
+  // CHECK-NEXT:    %value = getelementptr inbounds %struct.MyMatrix, %struct.MyMatrix* %0, i32 0, i32 0
+  // CHECK-NEXT:    %1 = bitcast [4 x i32]* %value to <4 x i32>*
+  // CHECK-NEXT:    %2 = load <4 x i32>, <4 x i32>* %1, align 4
+  // CHECK-NEXT:    %matext = extractelement <4 x i32> %2, i32 1
+  // CHECK-NEXT:    ret i32 %matext
+
+  MyMatrix<unsigned, 2, 2> Mat1;
+  Mat1.value = *((decltype(Mat1)::matrix_t *)Ptr1);
+  unsigned v1 = extract(Mat1);
+}

clang/test/Sema/matrix-type-operators.c

+// RUN: %clang_cc1 %s -fenable-matrix -pedantic -verify -triple=x86_64-apple-darwin9
+
+typedef float sx5x10_t __attribute__((matrix_type(5, 10)));
+
+void insert(sx5x10_t a, float f) {
+  // Non integer indexes.
+  a[3][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+  a[f][9] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  a[f][f] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  a[0][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+
+  // Invalid element type.
+  a[3][4] = &f;
+  // expected-error@-1 {{assigning to 'float' from incompatible type 'float *'; remove &}}
+
+  // Indexes outside allowed dimensions.
+  a[-1][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[3][-1] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[3][-1u] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[-1u][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[5][2] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[4][10] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[5][10.0] = f;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+
+  a[3] = 5.0;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+}
+
+void extract(sx5x10_t a, float f) {
+  // Non integer indexes.
+  float v1 = a[3][f];
+  // expected-error@-1 {{matrix column index is not an integer}}
+  float v2 = a[f][9];
+  // expected-error@-1 {{matrix row index is not an integer}}
+  float v3 = a[f][f];
+  // expected-error@-1 {{matrix row index is not an integer}}
+
+  // Invalid element type.
+  char *v4 = a[3][4];
+  // expected-error@-1 {{initializing 'char *' with an expression of incompatible type 'float'}}
+
+  // Indexes outside allowed dimensions.
+  float v5 = a[-1][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v6 = a[3][-1];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v8 = a[-1u][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v9 = a[5][2];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v10 = a[4][10];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v11 = a[5][10.0];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+
+  float v12 = a[3];
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+}

clang/test/SemaCXX/matrix-type-operators.cpp

+// RUN: %clang_cc1 %s -fenable-matrix -pedantic -std=c++11 -verify -triple=x86_64-apple-darwin9
+
+typedef float sx5x10_t __attribute__((matrix_type(5, 10)));
+
+void insert(sx5x10_t a, float f) {
+  // Non integer indexes.
+  a[3][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+  a[f][9] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  a[f][f] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  a[0][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+
+  // Invalid element type.
+  a[3][4] = &f;
+  // expected-error@-1 {{assigning to 'float' from incompatible type 'float *'; remove &}}
+
+  // Indexes outside allowed dimensions.
+  a[-1][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[3][-1] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[3][-1u] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[-1u][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[5][2] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[4][10] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[5][10.0] = f;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+}
+
+void extract(sx5x10_t a, float f) {
+  // Non integer indexes.
+  float v1 = a[3][f];
+  // expected-error@-1 {{matrix column index is not an integer}}
+  float v2 = a[f][9];
+  // expected-error@-1 {{matrix row index is not an integer}}
+  float v3 = a[f][f];
+  // expected-error@-1 {{matrix row index is not an integer}}
+
+  // Invalid element type.
+  char *v4 = a[3][4];
+  // expected-error@-1 {{cannot initialize a variable of type 'char *' with an lvalue of type 'float'}}
+
+  // Indexes outside allowed dimensions.
+  float v5 = a[-1][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v6 = a[3][-1];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v8 = a[-1u][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v9 = a[5][2];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v10 = a[4][10];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v11 = a[5][10.0];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+}
+
+void incomplete_matrix_index_expr(sx5x10_t a, float f) {
+  float x = a[3];
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+  a[2] = f;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+}