[Matrix] Implement matrix index expressions ([][]).

[fhahn]

Summary

This patch implements matrix index expressions
(matrix[RowIdx][ColumnIdx]).

It does so by introducing a new MatrixSubscriptExpr(Base, RowIdx, ColumnIdx).
MatrixSubscriptExprs are built in 2 steps in ActOnMatrixSubscriptExpr. First,
if the base of a subscript is of matrix type, we create a incomplete
MatrixSubscriptExpr(base, idx, nullptr). Second, if the base is an incomplete
MatrixSubscriptExpr, we create a complete
MatrixSubscriptExpr(base->getBase(), base->getRowIdx(), idx)

Similar to vector elements, it is not possible to take the address of
a MatrixSubscriptExpr.
For CodeGen, a new MatrixElt type is added to LValue, which is very
similar to VectorElt. The only difference is that we may need to cast
the type of the base from an array to a vector type when accessing it.

[fhahn]

Use placeholder type for incomplete matrix index expressions, as suggested by @rjmccall

[fhahn]

Rebase on top of the latest version of D72281. Refactor code to re-use new address conversion helper.

[fhahn]

Update to support non-constant-integer-expressions as indices.

[rjmccall]

Sorry for the slow review; I'm getting crushed with review requests.

clang/lib/AST/Expr.cpp

3946

You need to IgnoreParens() here.

clang/lib/AST/ExprConstant.cpp

7768

This can also occur in the other evaluators due to indexing into an r-value.

clang/lib/Sema/SemaExpr.cpp

5452

It should have the value kind of its base. get_matrix_rvalue()[0][0] is still an r-value, so you shouldn't allow it to be assigned to. Needs tests.

I'm fine with re-using OK_VectorComponent here, but you should (1) rename it to OK_VectorOrMatrixComponent and (2) take the opportunity to audit the places that use it to make sure they do something sensible. I expect you'll need to at least update some diagnostics about e.g. disallowing taking the address of the expression.

I think you should build a new expression node instead of reusing ArraySubscriptExpr, since basically none of the code that looks for an ArraySubscriptExpr will do the right thing for your operation. That will also allow you to avoid the "is this actually enabled" check, since you'll only see this node when your feature is enabled. If you're feeling generous, you could move vector subscripting to this new node as well and leave ArraySubscriptExpr exclusively for the standard (or dependent) cases.

[fhahn]

In D76791#2048387, @rjmccall wrote:

Sorry for the slow review; I'm getting crushed with review requests.

Thanks John! I've updated the patch and introduced a new MatrixSubscriptExpr. I think it simplifies the code in a few places and makes things clearer.

[fhahn]

clang/lib/AST/Expr.cpp

3946

This code is now gone.

clang/lib/AST/ExprConstant.cpp

7768

This code is not required any longer.

clang/lib/Sema/SemaExpr.cpp

5452

It should have the value kind of its base. get_matrix_rvalue()[0][0] is still an r-value, so you shouldn't allow it to be assigned to. Needs tests.

added tests to Sema/matrix-type-operators.s and SemaCXX/matrix-type-operators.cpp.

I'm fine with re-using OK_VectorComponent here, but you should (1) rename it to OK_VectorOrMatrixComponent and (2) take the opportunity to audit the places that use it to make sure they do something sensible. I expect you'll need to at least update some diagnostics about e.g. disallowing taking the address of the expression.

I've introduced a new OK_MatrixElement, because I don't think there is enough information to distinguish between vectors/matrixes where it is used. Also added a reinterpret_cast test.

I think you should build a new expression node instead of reusing ArraySubscriptExpr, since basically none of the code that looks for an ArraySubscriptExpr will do the right thing for your operation. That will also allow you to avoid the "is this actually enabled" check, since you'll only see this node when your feature is enabled. If you're feeling generous, you could move vector subscripting to this new node as well and leave ArraySubscriptExpr exclusively for the standard (or dependent) cases.

I've added a new MatrixSubscriptExpr. That helps to simplify the code in a couple of places. ActOnArraySubscriptExpr calls ActOnMatrixSubscriptExpr if we can identify the base as matrix type or MatrixSubscriptExpr. I initially tried to move the ActOnMatrixSubscriptExpr call to the parse step, but that does not work in the presence of dependent base types I think. Until we instantiate the dependent types, there's no way to distinguish between ArraySubscriptExpr/MatrixSubscriptExpr I think and it is best to treat them as unanalyzed ArraySubscriptExpr until then.

[fhahn]

Add clarifying comment to ActOnMatrixSubscriptExpr calls in ActOnArraySubscriptExpr.

[rjmccall]

clang/include/clang/AST/Expr.h

2658

Oh, that's interesting. So you've changed this to flatten the component expressions? I think that might be inconsistent with our usual source-preservation goals unless you intend to restrict the intermediate base expression to be an immediate subscript. That is, this is okay if you're going to require the user to write matrix[i][j] and forbid (matrix[i])[j], but if you intend to allow the latter, you should preserve that structure here. You can do that while still providing this API; you just have to implement getBase() etc. by looking through parens, and you should have an accessor which returns the syntactic base expression.

What expression node do you use for the intermediate subscript expression? You should talk about this in the doc comment.

clang/include/clang/Basic/Specifiers.h

159

redundancy

clang/include/clang/Sema/Sema.h

4908

It'd be more conventional to call it BuildMatrixSubscriptExpr. You can do all the semantic checks here and make ActOnArraySubscriptExpr handle the syntactic checks. This is all assuming that you intend to impose the syntactic restriction discussed above.

clang/lib/AST/ItaniumMangle.cpp

4238

This is simple, you should just mangle them syntactically. 'ixix' <base expression> <row expression> <column expression>. Test case is

using double4x4 = double __attribute__((matrix_type(4,4)));

template <class R, class C>
auto matrix_subscript(double4x4 m, R r, C c) -> decltype(m[r][c]) {}

double test_matrix_subscript(double 4x4 m) { return m[3][2]; }

clang/lib/Sema/SemaExpr.cpp

4563

You're not handling the case of a parenthesized MatrixSubscriptExpr. If that should be an error, that's a reasonable language rule, but it should probably be a better error than whatever you'll get by default.

And you might be specifically and problematically avoiding an error because of the special treatment of IncompleteMatrixIdx below. I'd just pull that up here and emit an error.

Doing this before the C++2a comma-index check is intentional?

[fhahn]

Address John's latest comments, thanks!

clang/include/clang/AST/Expr.h

2658

Oh, that's interesting. So you've changed this to flatten the component expressions? I think that might be inconsistent with our usual source-preservation goals unless you intend to restrict the intermediate base expression to be an immediate subscript. That is, this is okay if you're going to require the user to write matrix[i][j] and forbid (matrix[i])[j], but if you intend to allow the latter, you should preserve that structure here. You can do that while still providing this API; you just have to implement getBase() etc. by looking through parens, and you should have an accessor which returns the syntactic base expression.

My intuition was that the matrix subscript operator would be a single operator with 3 arguments. Allowing things like (matrix[I])[j] has potential for confusion I think, as there is no way to create an expression just referencing a row/column on its own. (responded in more details at the SemaExpr.cpp comment).

What expression node do you use for the intermediate subscript expression? You should talk about this in the doc comment.

Intermediate subscript expressions are represented as 'incomplete' MatrixSubscriptExpr (type is MatrixIncompleteIdx, ColumnIdx is nullptr). I've updated the comment.

clang/include/clang/Basic/Specifiers.h

159

I suppose no comment is sufficient?

clang/include/clang/Sema/Sema.h

4908

Thanks, I wasn't sure about the actual distinction. I've renamed it to BuildMatrixSubscriptExpr now.

clang/lib/AST/ItaniumMangle.cpp

4238

Thanks, I've updated the code as suggested and added the test. I had to adjust the test slightly (adding a call to matrix_subscript).

This test is quite interesting, because it accidentally another issue. Matrix element subscripts are treated as Lvalues currently, but the code currently does not support taking the address of matrix elements. decltype(m[r][c]) will be & double and if you add return m[r][c]; the issue is exposed.

I am not sure how to best address this, short of computing the address of the element in memory directly. Do you have any suggestions?

I think for some vector types, we currently mis-compile here as well. For example,

using double4 = double __attribute__((ext_vector_type(4)));

template <class R>
auto matrix_subscript(const double4 m, R r) -> decltype(m[r]) { return m[r]; }

produces the IR below. Note the ret double* %ref.tmp, where %ref.tmp is an alloca.

define linkonce_odr nonnull align 8 dereferenceable(8) double* @_Z16matrix_subscriptIiEDTixfpK_fp0_EDv4_dT_(<4 x double>* byval(<4 x double>) align 16 %0, i32 %r) #0 {
entry:
  %m.addr = alloca <4 x double>, align 16
  %r.addr = alloca i32, align 4
  %ref.tmp = alloca double, align 8
  %m = load <4 x double>, <4 x double>* %0, align 16
  store <4 x double> %m, <4 x double>* %m.addr, align 16
  store i32 %r, i32* %r.addr, align 4
  %1 = load <4 x double>, <4 x double>* %m.addr, align 16
  %2 = load i32, i32* %r.addr, align 4
  %vecext = extractelement <4 x double> %1, i32 %2
  store double %vecext, double* %ref.tmp, align 8
  ret double* %ref.tmp
}

clang/lib/Sema/SemaExpr.cpp

4563

You're not handling the case of a parenthesized MatrixSubscriptExpr. If that should be an error, that's a reasonable language rule, but it should probably be a better error than whatever you'll get by default.

Yes the way I think about the matrix subscript expression is in terms of a single operator with 3 arguments (base[RowIdx][ColumnIdx]), so something like (a[i])[j] should not be allowed, similar to things like a([I]) not being allowed.

And you might be specifically and problematically avoiding an error because of the special treatment of IncompleteMatrixIdx below. I'd just pull that up here and emit an error.

I've moved the handling of IncompleteMatrixIdx here (the previous position was due to the first version of the patch).

Doing this before the C++2a comma-index check is intentional?

Not really. I've moved it after the check.

[rjmccall]

clang/include/clang/AST/Expr.h

2658

My intuition was that the matrix subscript operator would be a single operator with 3 arguments. Allowing things like (matrix[I])[j] has potential for confusion I think, as there is no way to create an expression just referencing a row/column on its own. (responded in more details at the SemaExpr.cpp comment).

I think it's a perfectly reasonable language design to disallow parentheses here; it just wasn't obvious at first that that was what you were intending to do.

Intermediate subscript expressions are represented as 'incomplete' MatrixSubscriptExpr (type is MatrixIncompleteIdx, ColumnIdx is nullptr). I've updated the comment.

Thanks. In that case, there should be a way to more directly query whether a particular expression is complete or not.

clang/include/clang/Basic/Specifiers.h

159

A comment is a good idea; I was just pointing out that you'd written "a matrix element of a matrix".

If you're intending to allow more complex matrix components in the future (e.g. row/column projections), consider going ahead and calling this OK_MatrixComponent.

clang/lib/AST/ItaniumMangle.cpp

4238

Thanks, I've updated the code as suggested and added the test. I had to adjust the test slightly (adding a call to matrix_subscript).

Er, yes, I did mean to include one of those. :)

I am not sure how to best address this, short of computing the address of the element in memory directly. Do you have any suggestions?

Well, there's two levels of this.

First, you *could* make matrix elements ordinary l-values — it's not like you have any interest in ever not storing them separately, right? Vector components use a special l-value kind specifically to prevent addressability, probably for reasons related to the swizzle projections. Although, saying that, I'm not sure you can assign to the swizzle projections, in which case I'm not really sure why vectors need a special object kind except to forbid taking the address for its own sake. The only problem I can see with allowing element l-values to be ordinary l-values is that the code coming out of IRGen after you assign to one isn't going to be quite so tidily mem2reg'able as it is with the vector-style code generation. Of course, you could peephole that in IRGen if it's important. And if you want e.g. projected column matrices to be non-addressable in the future (which I assume you do?), you can always add a special object kind for those when you add them.

If you do want to restrict taking the address of an matrix component l-value, that should be a straightforward extra restriction in the places where we already forbid taking the address of a bit-field. You'll need the same restriction in reference-binding, of course. You could make decltype not infer a reference type for your l-values if you want, although I believe decltype does infer a reference type for bit-fields.

Your example is unfortunately not a miscompile, it's just a combination of terrible language features. The returned reference is bound to a materialized temporary because vector elements aren't addressable. You wouldn't be able to do that if m weren't const because the return type would become double &, which can't bind to a temporary.

clang/lib/Sema/SemaExpr.cpp

4563

Not really. I've moved it after the check.

Okay. However, you should consider going ahead and making it an error for matrix subscripts, just in case you want that syntax to mean something else in the future. It will also let you immediately catch people trying to use m[0,0] instead of m[0][0].

4618

This change is no longer necessary.

4684

Checking dependence is actually just as cheap as checking for C++, there's no real need to gate. But you need to check for placeholder types in the index operands before doing these type checks. The best test case here is an Objective-C property reference, something like:

__attribute__((objc_root_class))
@interface IntValue
@property int value;
@end

double test(double4x4 m, IntValue *iv) {
  return m[iv.value][iv.value];
}

Also, I would suggest not doing any checking on incomplete matrix expressions; just notice that it's still incomplete, build the expression, and return. You can do the checking once when you have all the operands together.

[fhahn]

Addressed latest round of comments, thanks!

Changes include:

• OK_MatrixComponent now behaves like OK_VectorComponent with respect to taking address/reference.
• Look through non-overload placeholder expressions.
• Move matrix related syntactic checks to new ActOnMatrixSubscriptExpr.
• Add MatrixSubscriptExpr::isIncomplete & use it at a few places in assertions.
• Update CreateBuiltinMatrixType to only validate indices once, on the complete expression.
• Do not allow , in matrix subscript expressions.

[fhahn]

clang/include/clang/AST/Expr.h

2658

Thanks. In that case, there should be a way to more directly query whether a particular expression is complete or not.

Done, I've added an isIncomplete method. I also added assertions using it in a few places.

clang/include/clang/Basic/Specifiers.h

159

I've renamed it to OK_MatrixComponent and added a comment that a matrix component is a single element of a matrix.

clang/lib/AST/ItaniumMangle.cpp

4238

If you do want to restrict taking the address of an matrix component l-value, that should be a straightforward extra restriction in the places where we already forbid taking the address of a bit-field. You'll need the same restriction in reference-binding, of course. You could make decltype not infer a reference type for your l-values if you want, although I believe decltype does infer a reference type for bit-fields.

Thanks for the suggestion! I think it would be best to mirror the handling of ExtVectorElementExpr and I *think* I found all the relevant places that needed updating. I've added tests to take references/addresses with various valid/invalid cases.

clang/lib/Sema/SemaExpr.cpp

4563

Oh right! I've went back to add a ActOnMatrixSubscriptExpr, as there are a few things that can go in there now, including rejecting m[0,0].

4618

Dropped, thanks!

4684

Checking dependence is actually just as cheap as checking for C++, there's no real need to gate. But you need to check for placeholder types in the index operands before doing these type checks. The best test case here is an Objective-C property reference, something like:

Done, I've added a ActOnMatrixSubscriptExpr and added code to deal with placeholder types there.

Also, I would suggest not doing any checking on incomplete matrix expressions; just notice that it's still incomplete, build the expression, and return. You can do the checking once when you have all the operands together.

Done, I initially thought it might be good to still raise an error if the row index was invalid, but given that it's not a valid expression anyways that probably is not too helpful anyways. Doing the checks on the complete expression simplifies things a bit :)

[rjmccall]

clang/lib/Sema/SemaExpr.cpp

4684

Placeholders actually need to be handled in the Build function — they can come up in template instantiation, too.

4692

Don't check for ParenExpr specifically; there are other expressions that are handled the same way, like _Generic. You need to check for !isa<MatrixSubscriptExpr>.

4702

Overload placeholder types are used for things like &foo where foo is the name of an overloaded function. The places that resolve only non-overload placeholder types are doing so in order to leave room for overload resolution to resolve the overload later, e.g. as part of non-builtin operator handling. operator[] is like operator(): non-builtin operators are only considered when the base has class type. Since you already know that the base type is a matrix type, you know that you're using your standard rules, and your standard rules have no way to resolve overloads in the index types — correctly, since indexes can't be functions or member pointers.

tl;dr: You can (and should) resolve all placeholders here, not just non-overload placeholders.

[fhahn]

Addressed latest comments:

• Handle placeholder types in CreateBuiltinMatrixSubscriptExpr and do not limit to non-overload types there.
• Check !MatrixSubscriptExpr instead of ParenExpr.
• Only handle placeholder types for Base in ActOnMatrixSubscriptExpr. Only skip isMSPropertySubscriptExpr.

clang/lib/Sema/SemaExpr.cpp

4684

Move, thanks! I think we have to retain checking placeholder expressions for base below, as noted in the response below.

4692

Done, thanks!

4702

Thanks for the clarification. I moved the code to deal with placeholders to CreateBuiltinMatrixSubscriptExpr and removed the non-overload restriction there.

I think we still need to keep dealing with placeholders in Base below, to ensure we do not miss that the base is actually a matrix type, e.g. to support. It seems it is enough to skip` isMSPropertySubscriptExpr` there (without that restriction, some non-matrix-type codegen tests start to fail. Does that make sense?

__attribute__((objc_root_class))
@interface MatrixValue
@property double4x4 value;
@end

[rjmccall]

clang/lib/CodeGen/CGExpr.cpp

1900

This should be handled here or else a whole lot of unusual cases are going to blow up on you — compound operators and so on.

3784

You should be able to assert that these have been coerced to the right type by Sema (probably size_t or ptrdiff_t or something).

clang/lib/Sema/SemaExpr.cpp

4560

I would prefer that the checks are inlined here, since they're very likely to not trigger.

4702

Yeah, you have to resolve some placeholders before you can check whether the base is a matrix type, and you can't resolve MS properties because the property access actually merges with the subscript in some cases.

I think you may need to resolve placeholders in base even in CreateBuiltinMatrixSubscriptExpr to handle template instantiation right. The test case would be something where the matrix expression is non-dependently-typed but loaded from an ObjC property — we might need to redundantly resolve placeholders when rebuilding expressions in the instantiation.

[rjmccall]

I also want to confirm explicitly that you've decided that the right language design is for matrix components to not be addressable. You're convinced that that's an important restriction to get the code generation you want?

[fhahn]

Thanks for the latest round of comments! All expect one should be addressed. For the remaining comment, I responded inline.

Also, it would be great if you could let me know if the updated check lines in D76793 are sufficient, then I'll update the tests in this patch accordingly.

In D76791#2058460, @rjmccall wrote:

I also want to confirm explicitly that you've decided that the right language design is for matrix components to not be addressable. You're convinced that that's an important restriction to get the code generation you want?

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

[rjmccall]

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

It does make sense, although it does make it very important that code generation narrows the relevant particular load/extractvalue and load/insertvalue/store sequences, or else performance and code size will be completely unacceptable for large matrices.

The test changes in the other patch look great, thanks.

[fhahn]

In D76791#2063926, @rjmccall wrote:

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

It does make sense, although it does make it very important that code generation narrows the relevant particular load/extractvalue and load/insertvalue/store sequences, or else performance and code size will be completely unacceptable for large matrices.

Yes, dealing with large vectors is currently a weakness of LLVM unfortunately. But breaking up operations on large vectors in general is something we are planning on implementing as part of the matrix lowering. And I think even if we do not allow taking addresses, we could still just load/store a single element with the index operator, if required (maybe also based on the actual size of the matrix). And if we discover that taking addresses is really beneficial to users, it should be relatively straight forward to allow it in a backwards-compatible way I think/

The test changes in the other patch look great, thanks.

Great, I'll update them ASAP, although it takes a while. I am also submitting some inline-response that didn't got submitted when I updated the patch a few hours ago.

clang/lib/CodeGen/CGExpr.cpp

1900

Oh thanks for pointing me to those cases. Fixed and test cases added.

3784

Hm I don't think that's currently happening. Is that required? It seems a bit unfortunate if we would have to force to wider bit-widths than necessary for the given index expressions.

clang/lib/Sema/SemaExpr.cpp

4560

Done, I've removed ActOnMatrixSubscriptExpr again.

4702

Yeah, you have to resolve some placeholders before you can check whether the base is a matrix type, and you can't resolve MS properties because the property access actually merges with the subscript in some cases.

I think that should be happening in the latest version, CheckPlaceholderExpr is used on Base, RowIdx and ColumnIdx.

[rjmccall]

In D76791#2064434, @fhahn wrote:

In D76791#2063926, @rjmccall wrote:

Yes at the moment I think we want to limit element wise accesses/modifications to go through the access operator only, to guarantee we can rely on the vector forms in codegen.

Additionally I think at least initially we want to avoid handing out pointers to elements, as it may be tempting to use for pointer-based accesses to subsequent elements. I am a bit worried that allowing that would muddy the waters a bit and may lead to interpreting matrix types as similar to arrays, instead of single value types with restricted element access. Does that make sense?

It does make sense, although it does make it very important that code generation narrows the relevant particular load/extractvalue and load/insertvalue/store sequences, or else performance and code size will be completely unacceptable for large matrices.

Yes, dealing with large vectors is currently a weakness of LLVM unfortunately. But breaking up operations on large vectors in general is something we are planning on implementing as part of the matrix lowering. And I think even if we do not allow taking addresses, we could still just load/store a single element with the index operator, if required (maybe also based on the actual size of the matrix). And if we discover that taking addresses is really beneficial to users, it should be relatively straight forward to allow it in a backwards-compatible way I think/

Yeah, if you ever decide you want that IR pattern, it should be painless to make IRGen emit it. You might even consider doing it at -O0 if the backend wouldn't otherwise be narrowing then. (That doesn't need to be in this patch, of course.)

clang/lib/CodeGen/CGExpr.cpp

3784

Can you explain what cost you're worried about? I don't understand what the benefit of using a smaller IR type for the indexes is, and usually it's better to have stronger invariants coming out of Sema.

clang/lib/Sema/SemaExpr.cpp

4702

Yes, you're right.

4714

Somewhere in here would be where you'd do the index type conversion, FWIW.

I don't know if you want to allow a class with a conversion operator to an integer type, but that function you added for +/- should do it.

[fhahn]

Force index types to size_t in Sema, remove code to extend index values in Codegen. Update tests to check more targeted IR.

[fhahn]

clang/lib/CodeGen/CGExpr.cpp

3784

I now updated the code to force the index types to size_t in Sema and remove the zext generation here. If we have signed indices, we now generate sexts, which may require additional instructions down the road., which we would not require if we perform the math on the max bit width of the operands. But looking back now, I am not really sure if that would actually be safe in all cases.

We should probably just try to narrow the index types based on the number of vector elements as an IR optimization, if it becomes a problem.

clang/lib/Sema/SemaExpr.cpp

4714

I don't know if you want to allow a class with a conversion operator to an integer type, but that function you added for +/- should do it.

Yes I updated the code here to use the same conversion function and added tests for conversion operators.

[rjmccall]

LGTM, thanks.

clang/lib/CodeGen/CGExpr.cpp

3784

It's UB if it's actually negative, right? So the sext really shouldn't be a problem.

[fhahn]

Thanks for all the feedback John!

clang/lib/CodeGen/CGExpr.cpp

3784

It's UB if it's actually negative, right? So the sext really shouldn't be a problem.

Yes exactly. At the C/C++ level, the individual row/column expressions are guaranteed to be >= 0. In the IR we emit, we compute the index in a vector and we know that the computed index must be >= 0, which is slightly weaker. It might be worth checking how to encode the fact that the extension sources must be >= 0, if it becomes a problem.

[nickdesaulniers]

clang/lib/Sema/SemaExpr.cpp

4738–4739

In a release build:

llvm-project/clang/lib/Sema/SemaExpr.cpp:4737:10: warning: unused variable 'ConversionOk' [-Wunused-variable]
    bool ConversionOk = tryConvertToTy(*this, Context.getSizeType(), &ConvExpr);
         ^

[fhahn]

clang/lib/Sema/SemaExpr.cpp

4738–4739

Oh, of course, thanks for letting me know! Should be fixed in a6a42df506ca93df69725f732c396050060f026f

clang/include/clang/AST/ASTContext.h

public:
   CanQualType PseudoObjectTy, ARCUnbridgedCastTy;
   CanQualType ObjCBuiltinIdTy, ObjCBuiltinClassTy, ObjCBuiltinSelTy;
   CanQualType ObjCBuiltinBoolTy;
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) \
   CanQualType SingletonId;
 #include "clang/Basic/OpenCLImageTypes.def"
   CanQualType OCLSamplerTy, OCLEventTy, OCLClkEventTy;
   CanQualType OCLQueueTy, OCLReserveIDTy;
+  CanQualType IncompleteMatrixIdxTy;
   CanQualType OMPArraySectionTy, OMPArrayShapingTy, OMPIteratorTy;
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
   CanQualType Id##Ty;
 #include "clang/Basic/OpenCLExtensionTypes.def"
 #define SVE_TYPE(Name, Id, SingletonId) \
   CanQualType SingletonId;
 #include "clang/Basic/AArch64SVEACLETypes.def"
 

clang/include/clang/AST/BuiltinTypes.def

 
 PLACEHOLDER_TYPE(BuiltinFn, BuiltinFnTy)
 
 // The type of a cast which, in ARC, would normally require a
 // __bridge, but which might be okay depending on the immediate
 // context.
 PLACEHOLDER_TYPE(ARCUnbridgedCast, ARCUnbridgedCastTy)
 
+// A placeholder type for incomplete matrix index expressions.
+PLACEHOLDER_TYPE(IncompleteMatrixIdx, IncompleteMatrixIdxTy)
+
 // A placeholder type for OpenMP array sections.
 PLACEHOLDER_TYPE(OMPArraySection, OMPArraySectionTy)
 
 // A placeholder type for OpenMP array shaping operation.
 PLACEHOLDER_TYPE(OMPArrayShaping, OMPArrayShapingTy)
 
 // A placeholder type for OpenMP iterators.
 PLACEHOLDER_TYPE(OMPIterator, OMPIteratorTy)

clang/include/clang/AST/ComputeDependence.h

 
 class Expr;
 class FullExpr;
 class OpaqueValueExpr;
 class ParenExpr;
 class UnaryOperator;
 class UnaryExprOrTypeTraitExpr;
 class ArraySubscriptExpr;
+class MatrixSubscriptExpr;
 class CompoundLiteralExpr;
 class CastExpr;
 class BinaryOperator;
 class ConditionalOperator;
 class BinaryConditionalOperator;
 class StmtExpr;
 class ConvertVectorExpr;
 class VAArgExpr;

 // The following functions are called from constructors of `Expr`, so they
 // should not access anything beyond basic
 ExprDependence computeDependence(FullExpr *E);
 ExprDependence computeDependence(OpaqueValueExpr *E);
 ExprDependence computeDependence(ParenExpr *E);
 ExprDependence computeDependence(UnaryOperator *E);
 ExprDependence computeDependence(UnaryExprOrTypeTraitExpr *E);
 ExprDependence computeDependence(ArraySubscriptExpr *E);
+ExprDependence computeDependence(MatrixSubscriptExpr *E);
 ExprDependence computeDependence(CompoundLiteralExpr *E);
 ExprDependence computeDependence(CastExpr *E);
 ExprDependence computeDependence(BinaryOperator *E);
 ExprDependence computeDependence(ConditionalOperator *E);
 ExprDependence computeDependence(BinaryConditionalOperator *E);
 ExprDependence computeDependence(StmtExpr *E, unsigned TemplateDepth);
 ExprDependence computeDependence(ConvertVectorExpr *E);
 ExprDependence computeDependence(VAArgExpr *E);

clang/include/clang/AST/Expr.h

public:
   const ObjCPropertyRefExpr *getObjCProperty() const;
 
   /// Check if this expression is the ObjC 'self' implicit parameter.
   bool isObjCSelfExpr() const;
 
   /// Returns whether this expression refers to a vector element.
   bool refersToVectorElement() const;
 
+  /// Returns whether this expression refers to a matrix element.
+  bool refersToMatrixElement() const {
+    return getObjectKind() == OK_MatrixComponent;
+  }
+
   /// Returns whether this expression refers to a global register
   /// variable.
   bool refersToGlobalRegisterVar() const;
 
   /// Returns whether this expression has a placeholder type.
   bool hasPlaceholderType() const {
     return getType()->isPlaceholderType();
   }
class ArraySubscriptExpr : public Expr {
   bool lhsIsBase() const { return getRHS()->getType()->isIntegerType(); }
 
 public:
   ArraySubscriptExpr(Expr *lhs, Expr *rhs, QualType t, ExprValueKind VK,
                      ExprObjectKind OK, SourceLocation rbracketloc)
       : Expr(ArraySubscriptExprClass, t, VK, OK) {
     SubExprs[LHS] = lhs;
     SubExprs[RHS] = rhs;
-    ArraySubscriptExprBits.RBracketLoc = rbracketloc;
+    ArrayOrMatrixSubscriptExprBits.RBracketLoc = rbracketloc;
     setDependence(computeDependence(this));
   }
 
   /// Create an empty array subscript expression.
   explicit ArraySubscriptExpr(EmptyShell Shell)
     : Expr(ArraySubscriptExprClass, Shell) { }
 
   /// An array access can be written A[4] or 4[A] (both are equivalent).
public:
   const Expr *getIdx() const { return lhsIsBase() ? getRHS() : getLHS(); }
 
   SourceLocation getBeginLoc() const LLVM_READONLY {
     return getLHS()->getBeginLoc();
   }
   SourceLocation getEndLoc() const { return getRBracketLoc(); }
 
   SourceLocation getRBracketLoc() const {
-    return ArraySubscriptExprBits.RBracketLoc;
+    return ArrayOrMatrixSubscriptExprBits.RBracketLoc;
   }
   void setRBracketLoc(SourceLocation L) {
-    ArraySubscriptExprBits.RBracketLoc = L;
+    ArrayOrMatrixSubscriptExprBits.RBracketLoc = L;
   }
 
   SourceLocation getExprLoc() const LLVM_READONLY {
     return getBase()->getExprLoc();
   }
 
   static bool classof(const Stmt *T) {
     return T->getStmtClass() == ArraySubscriptExprClass;
   }
 
   // Iterators
   child_range children() {
     return child_range(&SubExprs[0], &SubExprs[0]+END_EXPR);
   }
   const_child_range children() const {
     return const_child_range(&SubExprs[0], &SubExprs[0] + END_EXPR);
   }
 };
 
+/// MatrixSubscriptExpr - Matrix subscript expression for the MatrixType
+/// extension.
+/// MatrixSubscriptExpr can be either incomplete (only Base and RowIdx are set
+/// so far, the type is IncompleteMatrixIdx) or complete (Base, RowIdx and
+/// ColumnIdx refer to valid expressions). Incomplete matrix expressions only
+/// exist during the initial construction of the AST.
+class MatrixSubscriptExpr : public Expr {
+  enum { BASE, ROW_IDX, COLUMN_IDX, END_EXPR };
+  Stmt *SubExprs[END_EXPR];
+
+public:
+  MatrixSubscriptExpr(Expr *Base, Expr *RowIdx, Expr *ColumnIdx, QualType T,
+                      SourceLocation RBracketLoc)
+      : Expr(MatrixSubscriptExprClass, T, Base->getValueKind(),
+             OK_MatrixComponent) {
+    SubExprs[BASE] = Base;
+    SubExprs[ROW_IDX] = RowIdx;
+    SubExprs[COLUMN_IDX] = ColumnIdx;
+    ArrayOrMatrixSubscriptExprBits.RBracketLoc = RBracketLoc;
+    setDependence(computeDependence(this));
+  }
+
+  /// Create an empty matrix subscript expression.
+  explicit MatrixSubscriptExpr(EmptyShell Shell)
+      : Expr(MatrixSubscriptExprClass, Shell) {}
+
+  bool isIncomplete() const {
+    bool IsIncomplete = hasPlaceholderType(BuiltinType::IncompleteMatrixIdx);
+    assert((SubExprs[COLUMN_IDX] || IsIncomplete) &&
+           "expressions without column index must be marked as incomplete");
+    return IsIncomplete;
+  }
+  Expr *getBase() { return cast<Expr>(SubExprs[BASE]); }
+  const Expr *getBase() const { return cast<Expr>(SubExprs[BASE]); }
+  void setBase(Expr *E) { SubExprs[BASE] = E; }
+
+  Expr *getRowIdx() { return cast<Expr>(SubExprs[ROW_IDX]); }
+  const Expr *getRowIdx() const { return cast<Expr>(SubExprs[ROW_IDX]); }
+  void setRowIdx(Expr *E) { SubExprs[ROW_IDX] = E; }
+
+  Expr *getColumnIdx() { return cast_or_null<Expr>(SubExprs[COLUMN_IDX]); }
+  const Expr *getColumnIdx() const {
+    assert(!isIncomplete() &&
+           "cannot get the column index of an incomplete expression");
+    return cast<Expr>(SubExprs[COLUMN_IDX]);
+  }
+  void setColumnIdx(Expr *E) { SubExprs[COLUMN_IDX] = E; }
+
+  SourceLocation getBeginLoc() const LLVM_READONLY {
+    return getBase()->getBeginLoc();
+  }
+
+  SourceLocation getEndLoc() const { return getRBracketLoc(); }
+
+  SourceLocation getExprLoc() const LLVM_READONLY {
+    return getBase()->getExprLoc();
+  }
+
+  SourceLocation getRBracketLoc() const {
+    return ArrayOrMatrixSubscriptExprBits.RBracketLoc;
+  }
+  void setRBracketLoc(SourceLocation L) {
+    ArrayOrMatrixSubscriptExprBits.RBracketLoc = L;
+  }
+
+  static bool classof(const Stmt *T) {
+    return T->getStmtClass() == MatrixSubscriptExprClass;
+  }
+
+  // Iterators
+  child_range children() {
+    return child_range(&SubExprs[0], &SubExprs[0] + END_EXPR);
+  }
+  const_child_range children() const {
+    return const_child_range(&SubExprs[0], &SubExprs[0] + END_EXPR);
+  }
+};
+
 /// CallExpr - Represents a function call (C99 6.5.2.2, C++ [expr.call]).
 /// CallExpr itself represents a normal function call, e.g., "f(x, 2)",
 /// while its subclasses may represent alternative syntax that (semantically)
 /// results in a function call. For example, CXXOperatorCallExpr is
 /// a subclass for overloaded operator calls that use operator syntax, e.g.,
 /// "str1 + str2" to resolve to a function call.
 class CallExpr : public Expr {
   enum { FN = 0, PREARGS_START = 1 };

clang/include/clang/AST/RecursiveASTVisitor.h

 DEF_TRAVERSE_STMT(CXXConstructExpr, {})
 DEF_TRAVERSE_STMT(CallExpr, {})
 DEF_TRAVERSE_STMT(CXXMemberCallExpr, {})
 
 // These exprs (most of them), do not need any action except iterating
 // over the children.
 DEF_TRAVERSE_STMT(AddrLabelExpr, {})
 DEF_TRAVERSE_STMT(ArraySubscriptExpr, {})
+DEF_TRAVERSE_STMT(MatrixSubscriptExpr, {})
 DEF_TRAVERSE_STMT(OMPArraySectionExpr, {})
 DEF_TRAVERSE_STMT(OMPArrayShapingExpr, {})
 DEF_TRAVERSE_STMT(OMPIteratorExpr, {})
 
 DEF_TRAVERSE_STMT(BlockExpr, {
   TRY_TO(TraverseDecl(S->getBlockDecl()));
   return true; // no child statements to loop through.
 })

clang/include/clang/AST/Stmt.h

class UnaryExprOrTypeTraitExprBitfields {
     friend class UnaryExprOrTypeTraitExpr;
 
     unsigned : NumExprBits;
 
     unsigned Kind : 3;
     unsigned IsType : 1; // true if operand is a type, false if an expression.
   };
 
-  class ArraySubscriptExprBitfields {
+  class ArrayOrMatrixSubscriptExprBitfields {
     friend class ArraySubscriptExpr;
+    friend class MatrixSubscriptExpr;
 
     unsigned : NumExprBits;
 
     SourceLocation RBracketLoc;
   };
 
   class CallExprBitfields {
     friend class CallExpr;
union {
     ConstantExprBitfields ConstantExprBits;
     PredefinedExprBitfields PredefinedExprBits;
     DeclRefExprBitfields DeclRefExprBits;
     FloatingLiteralBitfields FloatingLiteralBits;
     StringLiteralBitfields StringLiteralBits;
     CharacterLiteralBitfields CharacterLiteralBits;
     UnaryOperatorBitfields UnaryOperatorBits;
     UnaryExprOrTypeTraitExprBitfields UnaryExprOrTypeTraitExprBits;
-    ArraySubscriptExprBitfields ArraySubscriptExprBits;
+    ArrayOrMatrixSubscriptExprBitfields ArrayOrMatrixSubscriptExprBits;
     CallExprBitfields CallExprBits;
     MemberExprBitfields MemberExprBits;
     CastExprBitfields CastExprBits;
     BinaryOperatorBitfields BinaryOperatorBits;
     InitListExprBitfields InitListExprBits;
     ParenListExprBitfields ParenListExprBits;
     GenericSelectionExprBitfields GenericSelectionExprBits;
     PseudoObjectExprBitfields PseudoObjectExprBits;

clang/include/clang/Basic/DiagnosticSemaKinds.td

def warn_cxx20_compat_aggregate_init_with_ctors : Warning<
   "aggregate initialization of type %0 with user-declared constructors "
   "is incompatible with C++20">, DefaultIgnore, InGroup<CXX20Compat>;
 
 def err_reference_bind_to_bitfield : Error<
   "%select{non-const|volatile}0 reference cannot bind to "
   "bit-field%select{| %1}2">;
 def err_reference_bind_to_vector_element : Error<
   "%select{non-const|volatile}0 reference cannot bind to vector element">;
+def err_reference_bind_to_matrix_element : Error<
+  "%select{non-const|volatile}0 reference cannot bind to matrix element">;
 def err_reference_var_requires_init : Error<
   "declaration of reference variable %0 requires an initializer">;
 def err_reference_without_init : Error<
   "reference to type %0 requires an initializer">;
 def note_value_initialization_here : Note<
   "in value-initialization of type %0 here">;
 def err_reference_has_multiple_inits : Error<
   "reference cannot be initialized with multiple values">;
def ext_standalone_specifier : ExtWarn<"'%0' is not permitted on a declaration "
   "of a type">, InGroup<MissingDeclarations>;
 def err_standalone_class_nested_name_specifier : Error<
   "forward declaration of %select{class|struct|interface|union|enum}0 cannot "
   "have a nested name specifier">;
 def err_typecheck_sclass_func : Error<"illegal storage class on function">;
 def err_static_block_func : Error<
   "function declared in block scope cannot have 'static' storage class">;
 def err_typecheck_address_of : Error<"address of %select{bit-field"
-  "|vector element|property expression|register variable}0 requested">;
+  "|vector element|property expression|register variable|matrix element}0 requested">;
 def ext_typecheck_addrof_void : Extension<
   "ISO C forbids taking the address of an expression of type 'void'">;
 def err_unqualified_pointer_member_function : Error<
   "must explicitly qualify name of member function when taking its address">;
 def err_invalid_form_pointer_member_function : Error<
   "cannot create a non-constant pointer to member function">;
 def err_address_of_function_with_pass_object_size_params: Error<
   "cannot take address of function %0 because parameter %1 has "
def warn_noderef_to_dereferenceable_pointer : Warning<
   "casting to dereferenceable pointer removes 'noderef' attribute">, InGroup<NoDeref>;
 
 def err_builtin_launder_invalid_arg : Error<
   "%select{non-pointer|function pointer|void pointer}0 argument to "
   "'__builtin_launder' is not allowed">;
 
 def err_builtin_matrix_disabled: Error<
   "matrix types extension is disabled. Pass -fenable-matrix to enable it">;
+def err_matrix_index_not_integer: Error<
+  "matrix %select{row|column}0 index is not an integer">;
+def err_matrix_index_outside_range: Error<
+  "matrix %select{row|column}0 index is outside the allowed range [0, %1)">;
+def err_matrix_incomplete_index: Error<
+  "single subscript expressions are not allowed for matrix values">;
+def err_matrix_separate_incomplete_index: Error<
+  "matrix row and column subscripts cannot be separated by any expression">;
+def err_matrix_subscript_comma: Error<
+  "comma expressions are not allowed as indices in matrix subscript expressions">;
 
 def err_preserve_field_info_not_field : Error<
   "__builtin_preserve_field_info argument %0 not a field access">;
 def err_preserve_field_info_not_const: Error<
   "__builtin_preserve_field_info argument %0 not a constant">;
 def err_btf_type_id_not_const: Error<
   "__builtin_btf_type_id argument %0 not a constant">;
 

clang/include/clang/Basic/Specifiers.h

enum ExprObjectKind {
 
     /// An Objective-C property is a logical field of an Objective-C
     /// object which is read and written via Objective-C method calls.
     OK_ObjCProperty,
 
     /// An Objective-C array/dictionary subscripting which reads an
     /// object or writes at the subscripted array/dictionary element via
     /// Objective-C method calls.
-    OK_ObjCSubscript
+    OK_ObjCSubscript,
+
+    /// A matrix component is a single element of a matrix.
+    OK_MatrixComponent
   };
 
   /// The reason why a DeclRefExpr does not constitute an odr-use.
   enum NonOdrUseReason {
     /// This is an odr-use.
     NOUR_None = 0,
     /// This name appears in an unevaluated operand.
     NOUR_Unevaluated,

clang/include/clang/Basic/StmtNodes.td

 def ImaginaryLiteral : StmtNode<Expr>;
 def StringLiteral : StmtNode<Expr>;
 def CharacterLiteral : StmtNode<Expr>;
 def ParenExpr : StmtNode<Expr>;
 def UnaryOperator : StmtNode<Expr>;
 def OffsetOfExpr : StmtNode<Expr>;
 def UnaryExprOrTypeTraitExpr : StmtNode<Expr>;
 def ArraySubscriptExpr : StmtNode<Expr>;
+def MatrixSubscriptExpr : StmtNode<Expr>;
 def OMPArraySectionExpr : StmtNode<Expr>;
 def OMPIteratorExpr : StmtNode<Expr>;
 def CallExpr : StmtNode<Expr>;
 def MemberExpr : StmtNode<Expr>;
 def CastExpr : StmtNode<Expr, 1>;
 def BinaryOperator : StmtNode<Expr>;
 def CompoundAssignOperator : StmtNode<BinaryOperator>;
 def AbstractConditionalOperator : StmtNode<Expr, 1>;

clang/include/clang/Sema/Initialization.h

enum FailureKind {
     FK_NonConstLValueReferenceBindingToTemporary,
 
     /// Non-const lvalue reference binding to a bit-field.
     FK_NonConstLValueReferenceBindingToBitfield,
 
     /// Non-const lvalue reference binding to a vector element.
     FK_NonConstLValueReferenceBindingToVectorElement,
 
+    /// Non-const lvalue reference binding to a matrix element.
+    FK_NonConstLValueReferenceBindingToMatrixElement,
+
     /// Non-const lvalue reference binding to an lvalue of unrelated
     /// type.
     FK_NonConstLValueReferenceBindingToUnrelated,
 
     /// Rvalue reference binding to an lvalue.
     FK_RValueReferenceBindingToLValue,
 
     /// Reference binding drops qualifiers.

clang/include/clang/Sema/Sema.h

ExprResult ActOnSizeofParameterPackExpr(Scope *S,
                                           SourceLocation RParenLoc);
   ExprResult ActOnPostfixUnaryOp(Scope *S, SourceLocation OpLoc,
                                  tok::TokenKind Kind, Expr *Input);
 
   ExprResult ActOnArraySubscriptExpr(Scope *S, Expr *Base, SourceLocation LLoc,
                                      Expr *Idx, SourceLocation RLoc);
   ExprResult CreateBuiltinArraySubscriptExpr(Expr *Base, SourceLocation LLoc,
                                              Expr *Idx, SourceLocation RLoc);
+
+  ExprResult CreateBuiltinMatrixSubscriptExpr(Expr *Base, Expr *RowIdx,
+                                              Expr *ColumnIdx,
+                                              SourceLocation RBLoc);
+
   ExprResult ActOnOMPArraySectionExpr(Expr *Base, SourceLocation LBLoc,
                                       Expr *LowerBound, SourceLocation ColonLoc,
                                       Expr *Length, SourceLocation RBLoc);
   ExprResult ActOnOMPArrayShapingExpr(Expr *Base, SourceLocation LParenLoc,
                                       SourceLocation RParenLoc,
                                       ArrayRef<Expr *> Dims,
                                       ArrayRef<SourceRange> Brackets);
 

clang/include/clang/Serialization/ASTBitCodes.h

enum PredefinedTypeIDs {
       PREDEF_TYPE_SAT_ULONG_FRACT_ID = 69,
 
       /// The placeholder type for OpenMP array shaping operation.
       PREDEF_TYPE_OMP_ARRAY_SHAPING = 70,
 
       /// The placeholder type for OpenMP iterator expression.
       PREDEF_TYPE_OMP_ITERATOR = 71,
 
-      /// OpenCL image types with auto numeration
+      /// A placeholder type for incomplete matrix index operations.
+      PREDEF_TYPE_INCOMPLETE_MATRIX_IDX = 72,
+
 #define IMAGE_TYPE(ImgType, Id, SingletonId, Access, Suffix) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLImageTypes.def"
       /// \brief OpenCL extension types with auto numeration
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
       PREDEF_TYPE_##Id##_ID,
 #include "clang/Basic/OpenCLExtensionTypes.def"
       // \brief SVE types with auto numeration
enum StmtCode {
       EXPR_OFFSETOF,
 
       /// A SizefAlignOfExpr record.
       EXPR_SIZEOF_ALIGN_OF,
 
       /// An ArraySubscriptExpr record.
       EXPR_ARRAY_SUBSCRIPT,
 
+      /// An MatrixSubscriptExpr record.
+      EXPR_MATRIX_SUBSCRIPT,
+
       /// A CallExpr record.
       EXPR_CALL,
 
       /// A MemberExpr record.
       EXPR_MEMBER,
 
       /// A BinaryOperator record.
       EXPR_BINARY_OPERATOR,

clang/lib/AST/ASTContext.cpp

void ASTContext::InitBuiltinTypes(const TargetInfo &Target,
   InitBuiltinType(BuiltinFnTy,  BuiltinType::BuiltinFn);
 
   // Placeholder type for OMP array sections.
   if (LangOpts.OpenMP) {
     InitBuiltinType(OMPArraySectionTy, BuiltinType::OMPArraySection);
     InitBuiltinType(OMPArrayShapingTy, BuiltinType::OMPArrayShaping);
     InitBuiltinType(OMPIteratorTy, BuiltinType::OMPIterator);
   }
+  if (LangOpts.MatrixTypes)
+    InitBuiltinType(IncompleteMatrixIdxTy, BuiltinType::IncompleteMatrixIdx);
 
   // C99 6.2.5p11.
   FloatComplexTy      = getComplexType(FloatTy);
   DoubleComplexTy     = getComplexType(DoubleTy);
   LongDoubleComplexTy = getComplexType(LongDoubleTy);
   Float128ComplexTy   = getComplexType(Float128Ty);
 
   // Builtin types for 'id', 'Class', and 'SEL'.

clang/lib/AST/ComputeDependence.cpp

ExprDependence clang::computeDependence(UnaryExprOrTypeTraitExpr *E) {
   }
   return Deps;
 }
 
 ExprDependence clang::computeDependence(ArraySubscriptExpr *E) {
   return E->getLHS()->getDependence() | E->getRHS()->getDependence();
 }
 
+ExprDependence clang::computeDependence(MatrixSubscriptExpr *E) {
+  return E->getBase()->getDependence() | E->getRowIdx()->getDependence() |
+         (E->getColumnIdx() ? E->getColumnIdx()->getDependence()
+                            : ExprDependence::None);
+}
+
 ExprDependence clang::computeDependence(CompoundLiteralExpr *E) {
   return toExprDependence(E->getTypeSourceInfo()->getType()->getDependence()) |
          turnTypeToValueDependence(E->getInitializer()->getDependence());
 }
 
 ExprDependence clang::computeDependence(CastExpr *E) {
   // Cast expressions are type-dependent if the type is
   // dependent (C++ [temp.dep.expr]p3).

clang/lib/AST/Expr.cpp

bool Expr::HasSideEffects(const ASTContext &Ctx,
   case ExprWithCleanupsClass:
     if (IncludePossibleEffects)
       if (cast<ExprWithCleanups>(this)->cleanupsHaveSideEffects())
         return true;
     break;
 
   case ParenExprClass:
   case ArraySubscriptExprClass:
+  case MatrixSubscriptExprClass:
   case OMPArraySectionExprClass:
   case OMPArrayShapingExprClass:
   case OMPIteratorExprClass:
   case MemberExprClass:
   case ConditionalOperatorClass:
   case BinaryConditionalOperatorClass:
   case CompoundLiteralExprClass:
   case ExtVectorElementExprClass:

 bool Expr::refersToVectorElement() const {
   // FIXME: Why do we not just look at the ObjectKind here?
   const Expr *E = this->IgnoreParens();
 
   while (const ImplicitCastExpr *ICE = dyn_cast<ImplicitCastExpr>(E)) {
     if (ICE->getValueKind() != VK_RValue &&
         ICE->getCastKind() == CK_NoOp)
       E = ICE->getSubExpr()->IgnoreParens();
     else
       break;
   }
 
   if (const ArraySubscriptExpr *ASE = dyn_cast<ArraySubscriptExpr>(E))
     return ASE->getBase()->getType()->isVectorType();
 
   if (isa<ExtVectorElementExpr>(E))
     return true;

clang/lib/AST/ExprClassification.cpp

if (Lang.CPlusPlus11) {
       // Step over the array-to-pointer decay if present, but not over the
       // temporary materialization.
       auto *Base = cast<ArraySubscriptExpr>(E)->getBase()->IgnoreImpCasts();
       if (Base->getType()->isArrayType())
         return ClassifyInternal(Ctx, Base);
     }
     return Cl::CL_LValue;
 
+  // Subscripting matrix types behaves like member accesses.
+  case Expr::MatrixSubscriptExprClass:
+    return ClassifyInternal(Ctx, cast<MatrixSubscriptExpr>(E)->getBase());
+
     // C++ [expr.prim.general]p3: The result is an lvalue if the entity is a
     //   function or variable and a prvalue otherwise.
   case Expr::DeclRefExprClass:
     if (E->getType() == Ctx.UnknownAnyTy)
       return isa<FunctionDecl>(cast<DeclRefExpr>(E)->getDecl())
                ? Cl::CL_PRValue : Cl::CL_LValue;
     return ClassifyDecl(Ctx, cast<DeclRefExpr>(E)->getDecl());
 

clang/lib/AST/ExprConstant.cpp

 
 bool LValueExprEvaluator::VisitArraySubscriptExpr(const ArraySubscriptExpr *E) {
   // FIXME: Deal with vectors as array subscript bases.
   if (E->getBase()->getType()->isVectorType())
     return Error(E);
 
   bool Success = true;
   if (!evaluatePointer(E->getBase(), Result)) {
     if (!Info.noteFailure())
       return false;
     Success = false;
   }
 
   APSInt Index;
   if (!EvaluateInteger(E->getIdx(), Index, Info))
     return false;
 

 #define STMT(Node, Base) case Expr::Node##Class:
 #define EXPR(Node, Base)
 #include "clang/AST/StmtNodes.inc"
   case Expr::PredefinedExprClass:
   case Expr::FloatingLiteralClass:
   case Expr::ImaginaryLiteralClass:
   case Expr::StringLiteralClass:
   case Expr::ArraySubscriptExprClass:
+  case Expr::MatrixSubscriptExprClass:
   case Expr::OMPArraySectionExprClass:
   case Expr::OMPArrayShapingExprClass:
   case Expr::OMPIteratorExprClass:
   case Expr::MemberExprClass:
   case Expr::CompoundAssignOperatorClass:
   case Expr::CompoundLiteralExprClass:
   case Expr::ExtVectorElementExprClass:
   case Expr::DesignatedInitExprClass:

clang/lib/AST/ItaniumMangle.cpp

case Expr::ArraySubscriptExprClass: {
     // Array subscript is treated as a syntactically weird form of
     // binary operator.
     Out << "ix";
     mangleExpression(AE->getLHS());
     mangleExpression(AE->getRHS());
     break;
   }
 
+  case Expr::MatrixSubscriptExprClass: {
+    const MatrixSubscriptExpr *ME = cast<MatrixSubscriptExpr>(E);
+    Out << "ixix";
+    mangleExpression(ME->getBase());
+    mangleExpression(ME->getRowIdx());
+    mangleExpression(ME->getColumnIdx());
+    break;
+  }
+
   case Expr::CompoundAssignOperatorClass: // fallthrough
   case Expr::BinaryOperatorClass: {
     const BinaryOperator *BO = cast<BinaryOperator>(E);
     if (BO->getOpcode() == BO_PtrMemD)
       Out << "ds";
     else
       mangleOperatorName(BinaryOperator::getOverloadedOperator(BO->getOpcode()),
                          /*Arity=*/2);

clang/lib/AST/NSAPI.cpp

#include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::BoundMember:
   case BuiltinType::Dependent:
   case BuiltinType::Overload:
   case BuiltinType::UnknownAny:
   case BuiltinType::ARCUnbridgedCast:
   case BuiltinType::Half:
   case BuiltinType::PseudoObject:
   case BuiltinType::BuiltinFn:
+  case BuiltinType::IncompleteMatrixIdx:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
     break;
   }
 
   return None;
 }

clang/lib/AST/StmtPrinter.cpp

 
 void StmtPrinter::VisitArraySubscriptExpr(ArraySubscriptExpr *Node) {
   PrintExpr(Node->getLHS());
   OS << "[";
   PrintExpr(Node->getRHS());
   OS << "]";
 }
 
+void StmtPrinter::VisitMatrixSubscriptExpr(MatrixSubscriptExpr *Node) {
+  PrintExpr(Node->getBase());
+  OS << "[";
+  PrintExpr(Node->getRowIdx());
+  OS << "]";
+  OS << "[";
+  PrintExpr(Node->getColumnIdx());
+  OS << "]";
+}
+
 void StmtPrinter::VisitOMPArraySectionExpr(OMPArraySectionExpr *Node) {
   PrintExpr(Node->getBase());
   OS << "[";
   if (Node->getLowerBound())
     PrintExpr(Node->getLowerBound());
   if (Node->getColonLoc().isValid()) {
     OS << ":";
     if (Node->getLength())

clang/lib/AST/StmtProfile.cpp

StmtProfiler::VisitUnaryExprOrTypeTraitExpr(const UnaryExprOrTypeTraitExpr *S) {
   if (S->isArgumentType())
     VisitType(S->getArgumentType());
 }
 
 void StmtProfiler::VisitArraySubscriptExpr(const ArraySubscriptExpr *S) {
   VisitExpr(S);
 }
 
+void StmtProfiler::VisitMatrixSubscriptExpr(const MatrixSubscriptExpr *S) {
+  VisitExpr(S);
+}
+
 void StmtProfiler::VisitOMPArraySectionExpr(const OMPArraySectionExpr *S) {
   VisitExpr(S);
 }
 
 void StmtProfiler::VisitOMPArrayShapingExpr(const OMPArrayShapingExpr *S) {
   VisitExpr(S);
 }
 

clang/lib/AST/TextNodeDumper.cpp

if (E->containsErrors()) {
         OS << " objcproperty";
         break;
       case OK_ObjCSubscript:
         OS << " objcsubscript";
         break;
       case OK_VectorComponent:
         OS << " vectorcomponent";
         break;
+      case OK_MatrixComponent:
+        OS << " matrixcomponent";
+        break;
       }
     }
   }
 
   ConstStmtVisitor<TextNodeDumper>::Visit(Node);
 }
 
 void TextNodeDumper::Visit(const Type *T) {

clang/lib/AST/Type.cpp

#include "clang/Basic/OpenCLImageTypes.def"
   case OCLEvent:
     return "event_t";
   case OCLClkEvent:
     return "clk_event_t";
   case OCLQueue:
     return "queue_t";
   case OCLReserveID:
     return "reserve_id_t";
+  case IncompleteMatrixIdx:
+    return "<incomplete matrix index type>";
   case OMPArraySection:
     return "<OpenMP array section type>";
   case OMPArrayShaping:
     return "<OpenMP array shaping type>";
   case OMPIterator:
     return "<OpenMP iterator type>";
 #define EXT_OPAQUE_TYPE(ExtType, Id, Ext) \
   case Id: \
#include "clang/Basic/OpenCLExtensionTypes.def"
     case BuiltinType::OCLClkEvent:
     case BuiltinType::OCLQueue:
     case BuiltinType::OCLReserveID:
 #define SVE_TYPE(Name, Id, SingletonId) \
     case BuiltinType::Id:
 #include "clang/Basic/AArch64SVEACLETypes.def"
     case BuiltinType::BuiltinFn:
     case BuiltinType::NullPtr:
+    case BuiltinType::IncompleteMatrixIdx:
     case BuiltinType::OMPArraySection:
     case BuiltinType::OMPArrayShaping:
     case BuiltinType::OMPIterator:
       return false;
     }
     llvm_unreachable("unknown builtin type");
 
   // Non-pointer types.

clang/lib/AST/TypeLoc.cpp

#include "clang/Basic/OpenCLExtensionTypes.def"
   case BuiltinType::OCLEvent:
   case BuiltinType::OCLClkEvent:
   case BuiltinType::OCLQueue:
   case BuiltinType::OCLReserveID:
 #define SVE_TYPE(Name, Id, SingletonId) \
   case BuiltinType::Id:
 #include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::BuiltinFn:
+  case BuiltinType::IncompleteMatrixIdx:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
     return TST_unspecified;
   }
 
   llvm_unreachable("Invalid BuiltinType Kind!");
 }

clang/lib/CodeGen/CGExpr.cpp

LValue CodeGenFunction::EmitLValue(const Expr *E) {
   case Expr::ObjCIvarRefExprClass:
     return EmitObjCIvarRefLValue(cast<ObjCIvarRefExpr>(E));
   case Expr::StmtExprClass:
     return EmitStmtExprLValue(cast<StmtExpr>(E));
   case Expr::UnaryOperatorClass:
     return EmitUnaryOpLValue(cast<UnaryOperator>(E));
   case Expr::ArraySubscriptExprClass:
     return EmitArraySubscriptExpr(cast<ArraySubscriptExpr>(E));
+  case Expr::MatrixSubscriptExprClass:
+    return EmitMatrixSubscriptExpr(cast<MatrixSubscriptExpr>(E));
   case Expr::OMPArraySectionExprClass:
     return EmitOMPArraySectionExpr(cast<OMPArraySectionExpr>(E));
   case Expr::ExtVectorElementExprClass:
     return EmitExtVectorElementExpr(cast<ExtVectorElementExpr>(E));
   case Expr::MemberExprClass:
     return EmitMemberExpr(cast<MemberExpr>(E));
   case Expr::CompoundLiteralExprClass:
     return EmitCompoundLiteralLValue(cast<CompoundLiteralExpr>(E));
if (LV.isVectorElt()) {
     llvm::LoadInst *Load = Builder.CreateLoad(LV.getVectorAddress(),
                                               LV.isVolatileQualified());
     return RValue::get(Builder.CreateExtractElement(Load, LV.getVectorIdx(),
                                                     "vecext"));
   }
 
   // If this is a reference to a subset of the elements of a vector, either
   // shuffle the input or extract/insert them as appropriate.
-  if (LV.isExtVectorElt())
+  if (LV.isExtVectorElt()) {
     return EmitLoadOfExtVectorElementLValue(LV);
+  }
 
   // Global Register variables always invoke intrinsics
   if (LV.isGlobalReg())
     return EmitLoadOfGlobalRegLValue(LV);
 
+  if (LV.isMatrixElt()) {
+    llvm::LoadInst *Load =
+        Builder.CreateLoad(LV.getMatrixAddress(), LV.isVolatileQualified());
+    return RValue::get(
+        Builder.CreateExtractElement(Load, LV.getMatrixIdx(), "matrixext"));
+  }
+
   assert(LV.isBitField() && "Unknown LValue type!");
   return EmitLoadOfBitfieldLValue(LV, Loc);
 }
 
 RValue CodeGenFunction::EmitLoadOfBitfieldLValue(LValue LV,
                                                  SourceLocation Loc) {
   const CGBitFieldInfo &Info = LV.getBitFieldInfo();
 
RValue CodeGenFunction::EmitLoadOfGlobalRegLValue(LValue LV) {
   llvm::Function *F = CGM.getIntrinsic(llvm::Intrinsic::read_register, Types);
   llvm::Value *Call = Builder.CreateCall(
       F, llvm::MetadataAsValue::get(Ty->getContext(), RegName));
   if (OrigTy->isPointerTy())
     Call = Builder.CreateIntToPtr(Call, OrigTy);
   return RValue::get(Call);
 }
 
-
 /// EmitStoreThroughLValue - Store the specified rvalue into the specified
 /// lvalue, where both are guaranteed to the have the same type, and that type
 /// is 'Ty'.
 void CodeGenFunction::EmitStoreThroughLValue(RValue Src, LValue Dst,
                                              bool isInit) {
   if (!Dst.isSimple()) {
     if (Dst.isVectorElt()) {
       // Read/modify/write the vector, inserting the new element.
if (!Dst.isSimple()) {
     // If this is an update of extended vector elements, insert them as
     // appropriate.
     if (Dst.isExtVectorElt())
       return EmitStoreThroughExtVectorComponentLValue(Src, Dst);
 
     if (Dst.isGlobalReg())
       return EmitStoreThroughGlobalRegLValue(Src, Dst);
 
+    if (Dst.isMatrixElt()) {
+      llvm::Value *Vec = Builder.CreateLoad(Dst.getMatrixAddress());
+      Vec = Builder.CreateInsertElement(Vec, Src.getScalarVal(),
+                                        Dst.getMatrixIdx(), "matins");
+      Builder.CreateStore(Vec, Dst.getMatrixAddress(),
+                          Dst.isVolatileQualified());
+      return;
+    }
+
     assert(Dst.isBitField() && "Unknown LValue type");
     return EmitStoreThroughBitfieldLValue(Src, Dst);
   }
 
   // There's special magic for assigning into an ARC-qualified l-value.
   if (Qualifiers::ObjCLifetime Lifetime = Dst.getQuals().getObjCLifetime()) {
     switch (Lifetime) {
     case Qualifiers::OCL_None:
LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E,
   if (getLangOpts().ObjC &&
       getLangOpts().getGC() != LangOptions::NonGC) {
     LV.setNonGC(!E->isOBJCGCCandidate(getContext()));
     setObjCGCLValueClass(getContext(), E, LV);
   }
   return LV;
 }
 
+LValue CodeGenFunction::EmitMatrixSubscriptExpr(const MatrixSubscriptExpr *E) {
+  assert(
+      !E->isIncomplete() &&
+      "incomplete matrix subscript expressions should be rejected during Sema");
+  LValue Base = EmitLValue(E->getBase());
+  llvm::Value *RowIdx = EmitScalarExpr(E->getRowIdx());
+  llvm::Value *ColIdx = EmitScalarExpr(E->getColumnIdx());
+  llvm::Value *NumRows = Builder.getIntN(
+      RowIdx->getType()->getScalarSizeInBits(),
+      E->getBase()->getType()->getAs<ConstantMatrixType>()->getNumRows());
+  llvm::Value *FinalIdx =
+      Builder.CreateAdd(Builder.CreateMul(ColIdx, NumRows), RowIdx);
+  return LValue::MakeMatrixElt(
+      MaybeConvertMatrixAddress(Base.getAddress(*this), *this), FinalIdx,
+      E->getBase()->getType(), Base.getBaseInfo(), TBAAAccessInfo());
+}
+
 static Address emitOMPArraySectionBase(CodeGenFunction &CGF, const Expr *Base,
                                        LValueBaseInfo &BaseInfo,
                                        TBAAAccessInfo &TBAAInfo,
                                        QualType BaseTy, QualType ElTy,
                                        bool IsLowerBound) {
   LValue BaseLVal;
   if (auto *ASE = dyn_cast<OMPArraySectionExpr>(Base->IgnoreParenImpCasts())) {
     BaseLVal = CGF.EmitOMPArraySectionExpr(ASE, IsLowerBound);

clang/lib/CodeGen/CGExprScalar.cpp

llvm::Value *Args[] = {
         llvm::ConstantInt::get(CGF.CGM.Int32Ty, Min ? *Min : 0),
         llvm::ConstantInt::get(CGF.CGM.Int32Ty, SMin ? *SMin : 0),
     };
 
     return CGF.EmitBuiltinAvailable(Args);
   }
 
   Value *VisitArraySubscriptExpr(ArraySubscriptExpr *E);
+  Value *VisitMatrixSubscriptExpr(MatrixSubscriptExpr *E);
   Value *VisitShuffleVectorExpr(ShuffleVectorExpr *E);
   Value *VisitConvertVectorExpr(ConvertVectorExpr *E);
   Value *VisitMemberExpr(MemberExpr *E);
   Value *VisitExtVectorElementExpr(Expr *E) { return EmitLoadOfLValue(E); }
   Value *VisitCompoundLiteralExpr(CompoundLiteralExpr *E) {
     // Strictly speaking, we shouldn't be calling EmitLoadOfLValue, which
     // transitively calls EmitCompoundLiteralLValue, here in C++ since compound
     // literals aren't l-values in C++. We do so simply because that's the
Value *ScalarExprEmitter::VisitArraySubscriptExpr(ArraySubscriptExpr *E) {
   QualType IdxTy = E->getIdx()->getType();
 
   if (CGF.SanOpts.has(SanitizerKind::ArrayBounds))
     CGF.EmitBoundsCheck(E, E->getBase(), Idx, IdxTy, /*Accessed*/true);
 
   return Builder.CreateExtractElement(Base, Idx, "vecext");
 }
 
+Value *ScalarExprEmitter::VisitMatrixSubscriptExpr(MatrixSubscriptExpr *E) {
+  TestAndClearIgnoreResultAssign();
+
+  // Handle the vector case.  The base must be a vector, the index must be an
+  // integer value.
+  Value *RowIdx = Visit(E->getRowIdx());
+  Value *ColumnIdx = Visit(E->getColumnIdx());
+  Value *Matrix = Visit(E->getBase());
+
+  // TODO: Should we emit bounds checks with SanitizerKind::ArrayBounds?
+  llvm::MatrixBuilder<CGBuilderTy> MB(Builder);
+  return MB.CreateExtractElement(
+      Matrix, RowIdx, ColumnIdx,
+      E->getBase()->getType()->getAs<ConstantMatrixType>()->getNumRows());
+}
+
 static int getMaskElt(llvm::ShuffleVectorInst *SVI, unsigned Idx,
                       unsigned Off) {
   int MV = SVI->getMaskValue(Idx);
   if (MV == -1)
     return -1;
   return Off + MV;
 }
 

clang/lib/CodeGen/CGValue.h

 /// bitfields, this is not a simple LLVM pointer, it may be a pointer plus a
 /// bitrange.
 class LValue {
   enum {
     Simple,       // This is a normal l-value, use getAddress().
     VectorElt,    // This is a vector element l-value (V[i]), use getVector*
     BitField,     // This is a bitfield l-value, use getBitfield*.
     ExtVectorElt, // This is an extended vector subset, use getExtVectorComp
-    GlobalReg     // This is a register l-value, use getGlobalReg()
+    GlobalReg,    // This is a register l-value, use getGlobalReg()
+    MatrixElt     // This is a matrix element, use getVector*
   } LVType;
 
   llvm::Value *V;
 
   union {
     // Index into a vector subscript: V[i]
     llvm::Value *VectorIdx;
 
private:
   }
 
 public:
   bool isSimple() const { return LVType == Simple; }
   bool isVectorElt() const { return LVType == VectorElt; }
   bool isBitField() const { return LVType == BitField; }
   bool isExtVectorElt() const { return LVType == ExtVectorElt; }
   bool isGlobalReg() const { return LVType == GlobalReg; }
+  bool isMatrixElt() const { return LVType == MatrixElt; }
 
   bool isVolatileQualified() const { return Quals.hasVolatile(); }
   bool isRestrictQualified() const { return Quals.hasRestrict(); }
   unsigned getVRQualifiers() const {
     return Quals.getCVRQualifiers() & ~Qualifiers::Const;
   }
 
   QualType getType() const { return Type; }
void setAddress(Address address) {
     V = address.getPointer();
     Alignment = address.getAlignment().getQuantity();
   }
 
   // vector elt lvalue
   Address getVectorAddress() const {
     return Address(getVectorPointer(), getAlignment());
   }
-  llvm::Value *getVectorPointer() const { assert(isVectorElt()); return V; }
-  llvm::Value *getVectorIdx() const { assert(isVectorElt()); return VectorIdx; }
+  llvm::Value *getVectorPointer() const {
+    assert(isVectorElt());
+    return V;
+  }
+  llvm::Value *getVectorIdx() const {
+    assert(isVectorElt());
+    return VectorIdx;
+  }
+
+  Address getMatrixAddress() const {
+    return Address(getMatrixPointer(), getAlignment());
+  }
+  llvm::Value *getMatrixPointer() const {
+    assert(isMatrixElt());
+    return V;
+  }
+  llvm::Value *getMatrixIdx() const {
+    assert(isMatrixElt());
+    return VectorIdx;
+  }
 
   // extended vector elements.
   Address getExtVectorAddress() const {
     return Address(getExtVectorPointer(), getAlignment());
   }
   llvm::Value *getExtVectorPointer() const {
     assert(isExtVectorElt());
     return V;
static LValue MakeGlobalReg(Address Reg, QualType type) {
     LValue R;
     R.LVType = GlobalReg;
     R.V = Reg.getPointer();
     R.Initialize(type, type.getQualifiers(), Reg.getAlignment(),
                  LValueBaseInfo(AlignmentSource::Decl), TBAAAccessInfo());
     return R;
   }
 
+  static LValue MakeMatrixElt(Address matAddress, llvm::Value *Idx,
+                              QualType type, LValueBaseInfo BaseInfo,
+                              TBAAAccessInfo TBAAInfo) {
+    LValue R;
+    R.LVType = MatrixElt;
+    R.V = matAddress.getPointer();
+    R.VectorIdx = Idx;
+    R.Initialize(type, type.getQualifiers(), matAddress.getAlignment(),
+                 BaseInfo, TBAAInfo);
+    return R;
+  }
+
   RValue asAggregateRValue(CodeGenFunction &CGF) const {
     return RValue::getAggregate(getAddress(CGF), isVolatileQualified());
   }
 };
 
 /// An aggregate value slot.
 class AggValueSlot {
   /// The address.

clang/lib/CodeGen/CodeGenFunction.h

public:
   LValue EmitVAArgExprLValue(const VAArgExpr *E);
   LValue EmitDeclRefLValue(const DeclRefExpr *E);
   LValue EmitStringLiteralLValue(const StringLiteral *E);
   LValue EmitObjCEncodeExprLValue(const ObjCEncodeExpr *E);
   LValue EmitPredefinedLValue(const PredefinedExpr *E);
   LValue EmitUnaryOpLValue(const UnaryOperator *E);
   LValue EmitArraySubscriptExpr(const ArraySubscriptExpr *E,
                                 bool Accessed = false);
+  LValue EmitMatrixSubscriptExpr(const MatrixSubscriptExpr *E);
   LValue EmitOMPArraySectionExpr(const OMPArraySectionExpr *E,
                                  bool IsLowerBound = true);
   LValue EmitExtVectorElementExpr(const ExtVectorElementExpr *E);
   LValue EmitMemberExpr(const MemberExpr *E);
   LValue EmitObjCIsaExpr(const ObjCIsaExpr *E);
   LValue EmitCompoundLiteralLValue(const CompoundLiteralExpr *E);
   LValue EmitInitListLValue(const InitListExpr *E);
   LValue EmitConditionalOperatorLValue(const AbstractConditionalOperator *E);

clang/lib/Sema/SemaCast.cpp

if (const ReferenceType *DestTypeTmp = DestType->getAs<ReferenceType>()) {
     switch (SrcExpr.get()->getObjectKind()) {
     case OK_Ordinary:
       break;
     case OK_BitField:
       msg = diag::err_bad_cxx_cast_bitfield;
       return TC_NotApplicable;
       // FIXME: Use a specific diagnostic for the rest of these cases.
     case OK_VectorComponent: inappropriate = "vector element";      break;
+    case OK_MatrixComponent:
+      inappropriate = "matrix element";
+      break;
     case OK_ObjCProperty:    inappropriate = "property expression"; break;
     case OK_ObjCSubscript:   inappropriate = "container subscripting expression";
                              break;
     }
     if (inappropriate) {
       Self.Diag(OpRange.getBegin(), diag::err_bad_reinterpret_cast_reference)
           << inappropriate << DestType
           << OpRange << SrcExpr.get()->getSourceRange();

clang/lib/Sema/SemaExceptionSpec.cpp

CanThrowResult Sema::canThrow(const Stmt *S) {
   case Expr::BuiltinBitCastExprClass:
       // FIXME: Properly determine whether a variably-modified type can throw.
     if (cast<Expr>(S)->getType()->isVariablyModifiedType())
       return CT_Can;
     return canSubStmtsThrow(*this, S);
 
     // Some might be dependent for other reasons.
   case Expr::ArraySubscriptExprClass:
+  case Expr::MatrixSubscriptExprClass:
   case Expr::OMPArraySectionExprClass:
   case Expr::OMPArrayShapingExprClass:
   case Expr::OMPIteratorExprClass:
   case Expr::BinaryOperatorClass:
   case Expr::DependentCoawaitExprClass:
   case Expr::CompoundAssignOperatorClass:
   case Expr::CStyleCastExprClass:
   case Expr::CXXStaticCastExprClass:

clang/lib/Sema/SemaExpr.cpp

Sema::ActOnArraySubscriptExpr(Scope *S, Expr *base, SourceLocation lbLoc,
 
   // Since this might be a postfix expression, get rid of ParenListExprs.
   if (isa<ParenListExpr>(base)) {
     ExprResult result = MaybeConvertParenListExprToParenExpr(S, base);
     if (result.isInvalid()) return ExprError();
     base = result.get();
   }
 
+  // Check if base and idx form a MatrixSubscriptExpr.
+  //
+  // Helper to check for comma expressions, which are not allowed as indices for
+  // matrix subscript expressions.
+  auto CheckAndReportCommaError = [this, base, rbLoc](Expr *E) {
+    if (isa<BinaryOperator>(E) && cast<BinaryOperator>(E)->isCommaOp()) {
+      Diag(E->getExprLoc(), diag::err_matrix_subscript_comma)
+          << SourceRange(base->getBeginLoc(), rbLoc);
+      return true;
+    }
+    return false;
+  };
+  // The matrix subscript operator ([][])is considered a single operator.
+  // Separating the index expressions by parenthesis is not allowed.
+  if (base->getType()->isSpecificPlaceholderType(
+          BuiltinType::IncompleteMatrixIdx) &&
+      !isa<MatrixSubscriptExpr>(base)) {
+    Diag(base->getExprLoc(), diag::err_matrix_separate_incomplete_index)
+        << SourceRange(base->getBeginLoc(), rbLoc);
+    return ExprError();
+  }
+  // If the base is either a MatrixSubscriptExpr or a matrix type, try to create
+  // a new MatrixSubscriptExpr.
+  auto *matSubscriptE = dyn_cast<MatrixSubscriptExpr>(base);
+  if (matSubscriptE) {
+    if (CheckAndReportCommaError(idx))
+      return ExprError();
+
+    assert(matSubscriptE->isIncomplete() &&
+           "base has to be an incomplete matrix subscript");
+    return CreateBuiltinMatrixSubscriptExpr(
+        matSubscriptE->getBase(), matSubscriptE->getRowIdx(), idx, rbLoc);
+  }
+  Expr *matrixBase = base;
+  bool IsMSPropertySubscript = isMSPropertySubscriptExpr(*this, base);
+  if (!IsMSPropertySubscript) {
+    ExprResult result = CheckPlaceholderExpr(base);
+    if (!result.isInvalid())
+      matrixBase = result.get();
+  }
+  if (matrixBase->getType()->isMatrixType()) {
+    if (CheckAndReportCommaError(idx))
+      return ExprError();
+
+    return CreateBuiltinMatrixSubscriptExpr(matrixBase, idx, nullptr, rbLoc);
+  }
+
   // A comma-expression as the index is deprecated in C++2a onwards.
   if (getLangOpts().CPlusPlus20 &&
       ((isa<BinaryOperator>(idx) && cast<BinaryOperator>(idx)->isCommaOp()) ||
        (isa<CXXOperatorCallExpr>(idx) &&
         cast<CXXOperatorCallExpr>(idx)->getOperator() == OO_Comma))) {
     Diag(idx->getExprLoc(), diag::warn_deprecated_comma_subscript)
       << SourceRange(base->getBeginLoc(), rbLoc);
   }
 
   // Handle any non-overload placeholder types in the base and index
   // expressions.  We can't handle overloads here because the other
   // operand might be an overloadable type, in which case the overload
   // resolution for the operator overload should get the first crack
   // at the overload.
-  bool IsMSPropertySubscript = false;
   if (base->getType()->isNonOverloadPlaceholderType()) {
     IsMSPropertySubscript = isMSPropertySubscriptExpr(*this, base);
     if (!IsMSPropertySubscript) {
       ExprResult result = CheckPlaceholderExpr(base);
       if (result.isInvalid())
         return ExprError();
       base = result.get();
     }
   }
   if (idx->getType()->isNonOverloadPlaceholderType()) {
Sema::ActOnArraySubscriptExpr(Scope *S, Expr *base, SourceLocation lbLoc,
   ExprResult Res = CreateBuiltinArraySubscriptExpr(base, lbLoc, idx, rbLoc);
 
   if (!Res.isInvalid() && isa<ArraySubscriptExpr>(Res.get()))
     CheckSubscriptAccessOfNoDeref(cast<ArraySubscriptExpr>(Res.get()));
 
   return Res;
 }
 
+static bool tryConvertToTy(Sema &S, QualType ElementType, ExprResult *Scalar) {
+  InitializedEntity Entity =
+      InitializedEntity::InitializeTemporary(ElementType);
+  InitializationKind Kind = InitializationKind::CreateCopy(
+      Scalar->get()->getBeginLoc(), SourceLocation());
+  Expr *Arg = Scalar->get();
+  InitializationSequence InitSeq(S, Entity, Kind, Arg);
+  *Scalar = InitSeq.Perform(S, Entity, Kind, Arg);
+  return !Scalar->isInvalid();
+}
+
+ExprResult Sema::CreateBuiltinMatrixSubscriptExpr(Expr *Base, Expr *RowIdx,
+                                                  Expr *ColumnIdx,
+                                                  SourceLocation RBLoc) {
+  ExprResult BaseR = CheckPlaceholderExpr(Base);
+  if (BaseR.isInvalid())
+    return BaseR;
+  Base = BaseR.get();
+
+  ExprResult RowR = CheckPlaceholderExpr(RowIdx);
+  if (RowR.isInvalid())
+    return RowR;
+  RowIdx = RowR.get();
+
+  if (!ColumnIdx)
+    return new (Context) MatrixSubscriptExpr(
+        Base, RowIdx, ColumnIdx, Context.IncompleteMatrixIdxTy, RBLoc);
+
+  // Build an unanalyzed expression if any of the operands is type-dependent.
+  if (Base->isTypeDependent() || RowIdx->isTypeDependent() ||
+      ColumnIdx->isTypeDependent())
+    return new (Context) MatrixSubscriptExpr(Base, RowIdx, ColumnIdx,
+                                             Context.DependentTy, RBLoc);
+
+  ExprResult ColumnR = CheckPlaceholderExpr(ColumnIdx);
+  if (ColumnR.isInvalid())
+    return ColumnR;
+  ColumnIdx = ColumnR.get();
+
+  // Check that IndexExpr is an integer expression. If it is a constant
+  // expression, check that it is less than Dim (= the number of elements in the
+  // corresponding dimension).
+  auto IsIndexValid = [&](Expr *IndexExpr, unsigned Dim,
+                          bool IsColumnIdx) -> Expr * {
+    if (!IndexExpr->getType()->isIntegerType() &&
+        !IndexExpr->isTypeDependent()) {
+      Diag(IndexExpr->getBeginLoc(), diag::err_matrix_index_not_integer)
+          << IsColumnIdx;
+      return nullptr;
+    }
+
+    llvm::APSInt Idx;
+    if (IndexExpr->isIntegerConstantExpr(Idx, Context) &&
+        (Idx < 0 || Idx >= Dim)) {
+      Diag(IndexExpr->getBeginLoc(), diag::err_matrix_index_outside_range)
+          << IsColumnIdx << Dim;
+      return nullptr;
+    }
+
+    ExprResult ConvExpr = IndexExpr;
+    bool ConversionOk = tryConvertToTy(*this, Context.getSizeType(), &ConvExpr);
+    assert(ConversionOk &&
+           "should be able to convert any integer type to size type");
+    return ConvExpr.get();
+  };
+
+  auto *MTy = Base->getType()->getAs<ConstantMatrixType>();
+  RowIdx = IsIndexValid(RowIdx, MTy->getNumRows(), false);
+  ColumnIdx = IsIndexValid(ColumnIdx, MTy->getNumColumns(), true);
+  if (!RowIdx || !ColumnIdx)
+    return ExprError();
+
+  return new (Context) MatrixSubscriptExpr(Base, RowIdx, ColumnIdx,
+                                           MTy->getElementType(), RBLoc);
+}
+
 void Sema::CheckAddressOfNoDeref(const Expr *E) {
   ExpressionEvaluationContextRecord &LastRecord = ExprEvalContexts.back();
   const Expr *StrippedExpr = E->IgnoreParenImpCasts();
 
   // For expressions like `&(*s).b`, the base is recorded and what should be
   // checked.
   const MemberExpr *Member = nullptr;
   while ((Member = dyn_cast<MemberExpr>(StrippedExpr)) && !Member->isArrow())
Sema::CreateBuiltinArraySubscriptExpr(Expr *Base, SourceLocation LLoc,
   // C++ [expr.sub]p1: The type "T" shall be a completely-defined object
   // type. Note that Functions are not objects, and that (in C99 parlance)
   // incomplete types are not object types.
   if (ResultType->isFunctionType()) {
     Diag(BaseExpr->getBeginLoc(), diag::err_subscript_function_type)
         << ResultType << BaseExpr->getSourceRange();
     return ExprError();
   }
 
   if (ResultType->isVoidType() && !getLangOpts().CPlusPlus) {
     // GNU extension: subscripting on pointer to void
     Diag(LLoc, diag::ext_gnu_subscript_void_type)
       << BaseExpr->getSourceRange();
 
     // C forbids expressions of unqualified void type from being l-values.
     // See IsCForbiddenLValueType.
     if (!ResultType.hasQualifiers()) VK = VK_RValue;
#include "clang/AST/BuiltinTypes.def"
   // The debugger mode could theoretically but currently does not try
   // to resolve unknown-typed arguments based on known parameter types.
   case BuiltinType::UnknownAny:
     return true;
 
   // These are always invalid as call arguments and should be reported.
   case BuiltinType::BoundMember:
   case BuiltinType::BuiltinFn:
+  case BuiltinType::IncompleteMatrixIdx:
   case BuiltinType::OMPArraySection:
   case BuiltinType::OMPArrayShaping:
   case BuiltinType::OMPIterator:
     return true;
 
   }
   llvm_unreachable("bad builtin type kind");
 }
QualType Sema::CheckVectorLogicalOperands(ExprResult &LHS, ExprResult &RHS,
   //        check could be notionally dropped.
   if (!getLangOpts().CPlusPlus &&
       !(isa<ExtVectorType>(vType->getAs<VectorType>())))
     return InvalidLogicalVectorOperands(Loc, LHS, RHS);
 
   return GetSignedVectorType(LHS.get()->getType());
 }
 
-static bool tryConvertScalarToMatrixElementTy(Sema &S, QualType ElementType,
-                                              ExprResult *Scalar) {
-  InitializedEntity Entity =
-      InitializedEntity::InitializeTemporary(ElementType);
-  InitializationKind Kind = InitializationKind::CreateCopy(
-      Scalar->get()->getBeginLoc(), SourceLocation());
-  Expr *Arg = Scalar->get();
-  InitializationSequence InitSeq(S, Entity, Kind, Arg);
-  *Scalar = InitSeq.Perform(S, Entity, Kind, Arg);
-  return !Scalar->isInvalid();
-}
-
 QualType Sema::CheckMatrixElementwiseOperands(ExprResult &LHS, ExprResult &RHS,
                                               SourceLocation Loc,
                                               bool IsCompAssign) {
   if (!IsCompAssign) {
     LHS = DefaultFunctionArrayLvalueConversion(LHS.get());
     if (LHS.isInvalid())
       return QualType();
   }
QualType Sema::CheckMatrixElementwiseOperands(ExprResult &LHS, ExprResult &RHS,
   if (Context.hasSameType(LHSType, RHSType))
     return LHSType;
 
   // Type conversion may change LHS/RHS. Keep copies to the original results, in
   // case we have to return InvalidOperands.
   ExprResult OriginalLHS = LHS;
   ExprResult OriginalRHS = RHS;
   if (LHSMatType && !RHSMatType) {
-    if (tryConvertScalarToMatrixElementTy(*this, LHSMatType->getElementType(),
-                                          &RHS))
+    if (tryConvertToTy(*this, LHSMatType->getElementType(), &RHS))
       return LHSType;
     return InvalidOperands(Loc, OriginalLHS, OriginalRHS);
   }
 
   if (!LHSMatType && RHSMatType) {
-    if (tryConvertScalarToMatrixElementTy(*this, RHSMatType->getElementType(),
-                                          &LHS))
+    if (tryConvertToTy(*this, RHSMatType->getElementType(), &LHS))
       return RHSType;
     return InvalidOperands(Loc, OriginalLHS, OriginalRHS);
   }
 
   return InvalidOperands(Loc, LHS, RHS);
 }
 
 inline QualType Sema::CheckBitwiseOperands(ExprResult &LHS, ExprResult &RHS,
static ValueDecl *getPrimaryDecl(Expr *E) {
   case Stmt::CXXUuidofExprClass:
     return cast<CXXUuidofExpr>(E)->getGuidDecl();
   default:
     return nullptr;
   }
 }
 
 namespace {
-  enum {
-    AO_Bit_Field = 0,
-    AO_Vector_Element = 1,
-    AO_Property_Expansion = 2,
-    AO_Register_Variable = 3,
-    AO_No_Error = 4
-  };
+  AO_Matrix_Element = 4,
+  AO_No_Error = 5
 }
 /// Diagnose invalid operand for address of operations.
 ///
 /// \param Type The type of operand which cannot have its address taken.
 static void diagnoseAddressOfInvalidType(Sema &S, SourceLocation Loc,
                                          Expr *E, unsigned Type) {
   S.Diag(Loc, diag::err_typecheck_address_of) << Type << E->getSourceRange();
 }
if (!op->getType()->isFunctionType()) {
       }
     }
   } else if (op->getObjectKind() == OK_BitField) { // C99 6.5.3.2p1
     // The operand cannot be a bit-field
     AddressOfError = AO_Bit_Field;
   } else if (op->getObjectKind() == OK_VectorComponent) {
     // The operand cannot be an element of a vector
     AddressOfError = AO_Vector_Element;
+  } else if (op->getObjectKind() == OK_MatrixComponent) {
+    // The operand cannot be an element of a matrix.
+    AddressOfError = AO_Matrix_Element;
   } else if (dcl) { // C99 6.5.3.2p1
     // We have an lvalue with a decl. Make sure the decl is not declared
     // with the register storage-class specifier.
     if (const VarDecl *vd = dyn_cast<VarDecl>(dcl)) {
       // in C++ it is not error to take address of a register
       // variable (c++03 7.1.1P3)
       if (vd->getStorageClass() == SC_Register &&
           !getLangOpts().CPlusPlus) {
if (DRE) {
                                 VK_RValue, SourceLocation());
       }
     }
 
     Diag(E->getBeginLoc(), diag::err_builtin_fn_use);
     return ExprError();
   }
 
+  case BuiltinType::IncompleteMatrixIdx:
+    Diag(cast<MatrixSubscriptExpr>(E->IgnoreParens())
+             ->getRowIdx()
+             ->getBeginLoc(),
+         diag::err_matrix_incomplete_index);
+    return ExprError();
+
   // Expressions of unknown type.
   case BuiltinType::OMPArraySection:
     Diag(E->getBeginLoc(), diag::err_omp_array_section_use);
     return ExprError();
 
   // Expressions of unknown type.
   case BuiltinType::OMPArrayShaping:
     return ExprError(Diag(E->getBeginLoc(), diag::err_omp_array_shaping_use));

clang/lib/Sema/SemaInit.cpp

bool InitializationSequence::isAmbiguous() const {
   case FK_WideStringIntoCharArray:
   case FK_IncompatWideStringIntoWideChar:
   case FK_PlainStringIntoUTF8Char:
   case FK_UTF8StringIntoPlainChar:
   case FK_AddressOfOverloadFailed: // FIXME: Could do better
   case FK_NonConstLValueReferenceBindingToTemporary:
   case FK_NonConstLValueReferenceBindingToBitfield:
   case FK_NonConstLValueReferenceBindingToVectorElement:
+  case FK_NonConstLValueReferenceBindingToMatrixElement:
   case FK_NonConstLValueReferenceBindingToUnrelated:
   case FK_RValueReferenceBindingToLValue:
   case FK_ReferenceAddrspaceMismatchTemporary:
   case FK_ReferenceInitDropsQualifiers:
   case FK_ReferenceInitFailed:
   case FK_ConversionFailed:
   case FK_ConversionFromPropertyFailed:
   case FK_TooManyInitsForScalar:
static void TryReferenceInitialization(Sema &S,
   TryReferenceInitializationCore(S, Entity, Kind, Initializer, cv1T1, T1,
                                  T1Quals, cv2T2, T2, T2Quals, Sequence);
 }
 
 /// Determine whether an expression is a non-referenceable glvalue (one to
 /// which a reference can never bind). Attempting to bind a reference to
 /// such a glvalue will always create a temporary.
 static bool isNonReferenceableGLValue(Expr *E) {
-  return E->refersToBitField() || E->refersToVectorElement();
+  return E->refersToBitField() || E->refersToVectorElement() ||
+         E->refersToMatrixElement();
 }
 
 /// Reference initialization without resolving overloaded functions.
 static void TryReferenceInitializationCore(Sema &S,
                                            const InitializedEntity &Entity,
                                            const InitializationKind &Kind,
                                            Expr *Initializer,
                                            QualType cv1T1, QualType T1,
else {
       switch (RefRelationship) {
       case Sema::Ref_Compatible:
         if (Initializer->refersToBitField())
           FK = InitializationSequence::
               FK_NonConstLValueReferenceBindingToBitfield;
         else if (Initializer->refersToVectorElement())
           FK = InitializationSequence::
               FK_NonConstLValueReferenceBindingToVectorElement;
+        else if (Initializer->refersToMatrixElement())
+          FK = InitializationSequence::
+              FK_NonConstLValueReferenceBindingToMatrixElement;
         else
           llvm_unreachable("unexpected kind of compatible initializer");
         break;
       case Sema::Ref_Related:
         FK = InitializationSequence::FK_ReferenceInitDropsQualifiers;
         break;
       case Sema::Ref_Incompatible:
         FK = InitializationSequence::
bool InitializationSequence::Diagnose(Sema &S,
   }
 
   case FK_NonConstLValueReferenceBindingToVectorElement:
     S.Diag(Kind.getLocation(), diag::err_reference_bind_to_vector_element)
       << DestType.isVolatileQualified()
       << Args[0]->getSourceRange();
     break;
 
+  case FK_NonConstLValueReferenceBindingToMatrixElement:
+    S.Diag(Kind.getLocation(), diag::err_reference_bind_to_matrix_element)
+        << DestType.isVolatileQualified() << Args[0]->getSourceRange();
+    break;
+
   case FK_RValueReferenceBindingToLValue:
     S.Diag(Kind.getLocation(), diag::err_lvalue_to_rvalue_ref)
       << DestType.getNonReferenceType() << OnlyArg->getType()
       << Args[0]->getSourceRange();
     break;
 
   case FK_ReferenceAddrspaceMismatchTemporary:
     S.Diag(Kind.getLocation(), diag::err_reference_bind_temporary_addrspace)
case FailedSequence: {
     case FK_NonConstLValueReferenceBindingToBitfield:
       OS << "non-const lvalue reference bound to bit-field";
       break;
 
     case FK_NonConstLValueReferenceBindingToVectorElement:
       OS << "non-const lvalue reference bound to vector element";
       break;
 
+    case FK_NonConstLValueReferenceBindingToMatrixElement:
+      OS << "non-const lvalue reference bound to matrix element";
+      break;
+
     case FK_NonConstLValueReferenceBindingToUnrelated:
       OS << "non-const lvalue reference bound to unrelated type";
       break;
 
     case FK_RValueReferenceBindingToLValue:
       OS << "rvalue reference bound to an lvalue";
       break;
 

clang/lib/Sema/TreeTransform.h

ExprResult RebuildArraySubscriptExpr(Expr *LHS,
                                              SourceLocation LBracketLoc,
                                              Expr *RHS,
                                              SourceLocation RBracketLoc) {
     return getSema().ActOnArraySubscriptExpr(/*Scope=*/nullptr, LHS,
                                              LBracketLoc, RHS,
                                              RBracketLoc);
   }
 
+  /// Build a new matrix subscript expression.
+  ///
+  /// By default, performs semantic analysis to build the new expression.
+  /// Subclasses may override this routine to provide different behavior.
+  ExprResult RebuildMatrixSubscriptExpr(Expr *Base, Expr *RowIdx,
+                                        Expr *ColumnIdx,
+                                        SourceLocation RBracketLoc) {
+    return getSema().CreateBuiltinMatrixSubscriptExpr(Base, RowIdx, ColumnIdx,
+                                                      RBracketLoc);
+  }
+
   /// Build a new array section expression.
   ///
   /// By default, performs semantic analysis to build the new expression.
   /// Subclasses may override this routine to provide different behavior.
   ExprResult RebuildOMPArraySectionExpr(Expr *Base, SourceLocation LBracketLoc,
                                         Expr *LowerBound,
                                         SourceLocation ColonLoc, Expr *Length,
                                         SourceLocation RBracketLoc) {
TreeTransform<Derived>::TransformArraySubscriptExpr(ArraySubscriptExpr *E) {
 
   return getDerived().RebuildArraySubscriptExpr(
       LHS.get(),
       /*FIXME:*/ E->getLHS()->getBeginLoc(), RHS.get(), E->getRBracketLoc());
 }
 
 template <typename Derived>
 ExprResult
+TreeTransform<Derived>::TransformMatrixSubscriptExpr(MatrixSubscriptExpr *E) {
+  ExprResult Base = getDerived().TransformExpr(E->getBase());
+  if (Base.isInvalid())
+    return ExprError();
+
+  ExprResult RowIdx = getDerived().TransformExpr(E->getRowIdx());
+  if (RowIdx.isInvalid())
+    return ExprError();
+
+  ExprResult ColumnIdx = getDerived().TransformExpr(E->getColumnIdx());
+  if (ColumnIdx.isInvalid())
+    return ExprError();
+
+  if (!getDerived().AlwaysRebuild() && Base.get() == E->getBase() &&
+      RowIdx.get() == E->getRowIdx() && ColumnIdx.get() == E->getColumnIdx())
+    return E;
+
+  return getDerived().RebuildMatrixSubscriptExpr(
+      Base.get(), RowIdx.get(), ColumnIdx.get(), E->getRBracketLoc());
+}
+
+template <typename Derived>
+ExprResult
 TreeTransform<Derived>::TransformOMPArraySectionExpr(OMPArraySectionExpr *E) {
   ExprResult Base = getDerived().TransformExpr(E->getBase());
   if (Base.isInvalid())
     return ExprError();
 
   ExprResult LowerBound;
   if (E->getLowerBound()) {
     LowerBound = getDerived().TransformExpr(E->getLowerBound());

clang/lib/Serialization/ASTCommon.cpp

 #define SVE_TYPE(Name, Id, SingletonId) \
   case BuiltinType::Id: \
     ID = PREDEF_TYPE_##Id##_ID; \
     break;
 #include "clang/Basic/AArch64SVEACLETypes.def"
   case BuiltinType::BuiltinFn:
     ID = PREDEF_TYPE_BUILTIN_FN;
     break;
+  case BuiltinType::IncompleteMatrixIdx:
+    ID = PREDEF_TYPE_INCOMPLETE_MATRIX_IDX;
+    break;
   case BuiltinType::OMPArraySection:
     ID = PREDEF_TYPE_OMP_ARRAY_SECTION;
     break;
   case BuiltinType::OMPArrayShaping:
     ID = PREDEF_TYPE_OMP_ARRAY_SHAPING;
     break;
   case BuiltinType::OMPIterator:
     ID = PREDEF_TYPE_OMP_ITERATOR;

clang/lib/Serialization/ASTReader.cpp

case PREDEF_TYPE_AUTO_RREF_DEDUCT:
       T = Context.getAutoRRefDeductType();
       break;
     case PREDEF_TYPE_ARC_UNBRIDGED_CAST:
       T = Context.ARCUnbridgedCastTy;
       break;
     case PREDEF_TYPE_BUILTIN_FN:
       T = Context.BuiltinFnTy;
       break;
+    case PREDEF_TYPE_INCOMPLETE_MATRIX_IDX:
+      T = Context.IncompleteMatrixIdxTy;
+      break;
     case PREDEF_TYPE_OMP_ARRAY_SECTION:
       T = Context.OMPArraySectionTy;
       break;
     case PREDEF_TYPE_OMP_ARRAY_SHAPING:
       T = Context.OMPArraySectionTy;
       break;
     case PREDEF_TYPE_OMP_ITERATOR:
       T = Context.OMPIteratorTy;

clang/lib/Serialization/ASTReaderStmt.cpp

 
 void ASTStmtReader::VisitArraySubscriptExpr(ArraySubscriptExpr *E) {
   VisitExpr(E);
   E->setLHS(Record.readSubExpr());
   E->setRHS(Record.readSubExpr());
   E->setRBracketLoc(readSourceLocation());
 }
 
+void ASTStmtReader::VisitMatrixSubscriptExpr(MatrixSubscriptExpr *E) {
+  VisitExpr(E);
+  E->setBase(Record.readSubExpr());
+  E->setRowIdx(Record.readSubExpr());
+  E->setColumnIdx(Record.readSubExpr());
+  E->setRBracketLoc(readSourceLocation());
+}
+
 void ASTStmtReader::VisitOMPArraySectionExpr(OMPArraySectionExpr *E) {
   VisitExpr(E);
   E->setBase(Record.readSubExpr());
   E->setLowerBound(Record.readSubExpr());
   E->setLength(Record.readSubExpr());
   E->setColonLoc(readSourceLocation());
   E->setRBracketLoc(readSourceLocation());
 }
while (true) {
     case EXPR_SIZEOF_ALIGN_OF:
       S = new (Context) UnaryExprOrTypeTraitExpr(Empty);
       break;
 
     case EXPR_ARRAY_SUBSCRIPT:
       S = new (Context) ArraySubscriptExpr(Empty);
       break;
 
+    case EXPR_MATRIX_SUBSCRIPT:
+      S = new (Context) MatrixSubscriptExpr(Empty);
+      break;
+
     case EXPR_OMP_ARRAY_SECTION:
       S = new (Context) OMPArraySectionExpr(Empty);
       break;
 
     case EXPR_OMP_ARRAY_SHAPING:
       S = OMPArrayShapingExpr::CreateEmpty(
           Context, Record[ASTStmtReader::NumExprFields]);
       break;

clang/lib/Serialization/ASTWriterStmt.cpp

 void ASTStmtWriter::VisitArraySubscriptExpr(ArraySubscriptExpr *E) {
   VisitExpr(E);
   Record.AddStmt(E->getLHS());
   Record.AddStmt(E->getRHS());
   Record.AddSourceLocation(E->getRBracketLoc());
   Code = serialization::EXPR_ARRAY_SUBSCRIPT;
 }
 
+void ASTStmtWriter::VisitMatrixSubscriptExpr(MatrixSubscriptExpr *E) {
+  VisitExpr(E);
+  Record.AddStmt(E->getBase());
+  Record.AddStmt(E->getRowIdx());
+  Record.AddStmt(E->getColumnIdx());
+  Record.AddSourceLocation(E->getRBracketLoc());
+  Code = serialization::EXPR_ARRAY_SUBSCRIPT;
+}
+
 void ASTStmtWriter::VisitOMPArraySectionExpr(OMPArraySectionExpr *E) {
   VisitExpr(E);
   Record.AddStmt(E->getBase());
   Record.AddStmt(E->getLowerBound());
   Record.AddStmt(E->getLength());
   Record.AddSourceLocation(E->getColonLoc());
   Record.AddSourceLocation(E->getRBracketLoc());
   Code = serialization::EXPR_OMP_ARRAY_SECTION;

clang/lib/StaticAnalyzer/Core/ExprEngine.cpp

switch (S->getStmtClass()) {
     }
 
     case Stmt::ArraySubscriptExprClass:
       Bldr.takeNodes(Pred);
       VisitArraySubscriptExpr(cast<ArraySubscriptExpr>(S), Pred, Dst);
       Bldr.addNodes(Dst);
       break;
 
+    case Stmt::MatrixSubscriptExprClass:
+      llvm_unreachable("Support for MatrixSubscriptExpr is not implemented.");
+      break;
+
     case Stmt::GCCAsmStmtClass:
       Bldr.takeNodes(Pred);
       VisitGCCAsmStmt(cast<GCCAsmStmt>(S), Pred, Dst);
       Bldr.addNodes(Dst);
       break;
 
     case Stmt::MSAsmStmtClass:
       Bldr.takeNodes(Pred);

clang/test/CodeGen/matrix-type-operators.c

void add_matrix_scalar_long_long_int_unsigned_long_long(ullx4x2_t b, unsigned long long int vulli) {
   // CHECK:        [[SCALAR:%.*]] = load i64, i64* %vulli.addr, align 8
   // CHECK-NEXT:   [[MATRIX:%.*]] = load <8 x i64>, <8 x i64>* %0, align 8
   // CHECK-NEXT:   [[SCALAR_EMBED:%.*]] = insertelement <8 x i64> undef, i64 [[SCALAR]], i32 0
   // CHECK-NEXT:   [[SCALAR_EMBED1:%.*]] = shufflevector <8 x i64> [[SCALAR_EMBED]], <8 x i64> undef, <8 x i32> zeroinitializer
   // CHECK-NEXT:   [[RES:%.*]] = add <8 x i64> [[SCALAR_EMBED1]], [[MATRIX]]
   // CHECK-NEXT:   store <8 x i64> [[RES]], <8 x i64>* {{.*}}, align 8
   b = vulli + b;
 }
+
+// Tests for the matrix type operators.
+
+typedef double dx5x5_t __attribute__((matrix_type(5, 5)));
+typedef float fx2x3_t __attribute__((matrix_type(2, 3)));
+
+// Check that we can use matrix index expression on different floating point
+// matrixes and indices.
+void insert_double_matrix_const_idx_ll_u_double(dx5x5_t a, double d, fx2x3_t b, float e, int j, unsigned k) {
+  // CHECK-LABEL: @insert_double_matrix_const_idx_ll_u_double(
+  // CHECK:         [[D:%.*]] = load double, double* %d.addr, align 8
+  // CHECK-NEXT:    [[MAT:%.*]] = load <25 x double>, <25 x double>* {{.*}}, align 8
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <25 x double> [[MAT]], double [[D]], i64 5
+  // CHECK-NEXT:    store <25 x double> [[MATINS]], <25 x double>* {{.*}}, align 8
+  // CHECK-NEXT:    ret void
+
+  a[0ll][1u] = d;
+}
+
+void insert_double_matrix_const_idx_i_u_double(dx5x5_t a, double d) {
+  // CHECK-LABEL: @insert_double_matrix_const_idx_i_u_double(
+  // CHECK:         [[D:%.*]] = load double, double* %d.addr, align 8
+  // CHECK-NEXT:    [[MAT:%.*]] = load <25 x double>, <25 x double>* [[MAT_ADDR:%.*]], align 8
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <25 x double> [[MAT]], double [[D]], i64 21
+  // CHECK-NEXT:    store <25 x double> [[MATINS]], <25 x double>* [[MAT_ADDR]], align 8
+  // CHECK-NEXT:    ret void
+
+  a[1][4u] = d;
+}
+
+void insert_float_matrix_const_idx_ull_i_float(fx2x3_t b, float e) {
+  // CHECK-LABEL: @insert_float_matrix_const_idx_ull_i_float(
+  // CHECK:         [[E:%.*]] = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    [[MAT:%.*]] = load <6 x float>, <6 x float>* [[MAT_ADDR:%.*]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <6 x float> [[MAT]], float [[E]], i64 3
+  // CHECK-NEXT:    store <6 x float> [[MATINS]], <6 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  b[1ull][1] = e;
+}
+
+void insert_float_matrix_idx_i_u_float(fx2x3_t b, float e, int j, unsigned k) {
+  // CHECK-LABEL: @insert_float_matrix_idx_i_u_float(
+  // CHECK:         [[E:%.*]] = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    [[J:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:    [[J_EXT:%.*]] = sext i32 [[J]] to i64
+  // CHECK-NEXT:    [[K:%.*]] = load i32, i32* %k.addr, align 4
+  // CHECK-NEXT:    [[K_EXT:%.*]] = zext i32 [[K]] to i64
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[K_EXT]], 2
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[J_EXT]]
+  // CHECK-NEXT:    [[MAT:%.*]] = load <6 x float>, <6 x float>* [[MAT_ADDR:%.*]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <6 x float> [[MAT]], float [[E]], i64 [[IDX2]]
+  // CHECK-NEXT:    store <6 x float> [[MATINS]], <6 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  b[j][k] = e;
+}
+
+void insert_float_matrix_idx_s_ull_float(fx2x3_t b, float e, short j, unsigned long long k) {
+  // CHECK-LABEL: @insert_float_matrix_idx_s_ull_float(
+  // CHECK:         [[E:%.*]] = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    [[J:%.*]] = load i16, i16* %j.addr, align 2
+  // CHECK-NEXT:    [[J_EXT:%.*]] = sext i16 [[J]] to i64
+  // CHECK-NEXT:    [[K:%.*]] = load i64, i64* %k.addr, align 8
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[K]], 2
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[J_EXT]]
+  // CHECK-NEXT:    [[MAT:%.*]] = load <6 x float>, <6 x float>* [[MAT_ADDR:%.*]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <6 x float> [[MAT]], float [[E]], i64 [[IDX2]]
+  // CHECK-NEXT:    store <6 x float> [[MATINS]], <6 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  (b)[j][k] = e;
+}
+
+// Check that we can can use matrix index expressions on integer matrixes.
+typedef int ix9x3_t __attribute__((matrix_type(9, 3)));
+void insert_int_idx_expr(ix9x3_t a, int i) {
+  // CHECK-LABEL: @insert_int_idx_expr(
+  // CHECK:         [[I1:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    [[I2:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    [[I2_ADD:%.*]] = add nsw i32 4, [[I2]]
+  // CHECK-NEXT:    [[ADD_EXT:%.*]] = sext i32 [[I2_ADD]] to i64
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 18, [[ADD_EXT]]
+  // CHECK-NEXT:    [[MAT:%.*]] = load <27 x i32>, <27 x i32>* [[MAT_ADDR:%.*]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <27 x i32> [[MAT]], i32 [[I1]], i64 [[IDX2]]
+  // CHECK-NEXT:    store <27 x i32> [[MATINS]], <27 x i32>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  a[4 + i][1 + 1u] = i;
+}
+
+// Check that we can can use matrix index expressions on FP and integer
+// matrixes.
+typedef int ix9x3_t __attribute__((matrix_type(9, 3)));
+void insert_float_into_int_matrix(ix9x3_t *a, int i) {
+  // CHECK-LABEL: @insert_float_into_int_matrix(
+  // CHECK:         [[I:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    [[MAT_ADDR1:%.*]] = load [27 x i32]*, [27 x i32]** %a.addr, align 8
+  // CHECK-NEXT:    [[MAT_ADDR2:%.*]] = bitcast [27 x i32]* [[MAT_ADDR1]] to <27 x i32>*
+  // CHECK-NEXT:    [[MAT:%.*]] = load <27 x i32>, <27 x i32>* [[MAT_ADDR2]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <27 x i32> [[MAT]], i32 [[I]], i64 13
+  // CHECK-NEXT:    store <27 x i32> [[MATINS]], <27 x i32>* [[MAT_ADDR2]], align 4
+  // CHECK-NEXT:    ret void
+
+  (*a)[4][1] = i;
+}
+
+// Check that we can use overloaded matrix index expressions on matrixes with
+// matching dimensions, but different element types.
+typedef double dx3x3_t __attribute__((matrix_type(3, 3)));
+typedef float fx3x3_t __attribute__((matrix_type(3, 3)));
+void insert_matching_dimensions1(dx3x3_t a, double i) {
+  // CHECK-LABEL: @insert_matching_dimensions1(
+  // CHECK:         [[I:%.*]] = load double, double* %i.addr, align 8
+  // CHECK-NEXT:    [[MAT:%.*]] = load <9 x double>, <9 x double>* [[MAT_ADDR:%.*]], align 8
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <9 x double> [[MAT]], double [[I]], i64 5
+  // CHECK-NEXT:    store <9 x double> [[MATINS]], <9 x double>* [[MAT_ADDR]], align 8
+  // CHECK-NEXT:    ret void
+
+  a[2u][1u] = i;
+}
+
+void insert_matching_dimensions(fx3x3_t b, float e) {
+  // CHECK-LABEL: @insert_matching_dimensions(
+  // CHECK:         [[E:%.*]] = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    [[MAT:%.*]] = load <9 x float>, <9 x float>* [[MAT_ADDR:%.*]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <9 x float> [[MAT]], float [[E]], i64 7
+  // CHECK-NEXT:    store <9 x float> [[MATINS]], <9 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  b[1u][2u] = e;
+}
+
+double extract_double(dx5x5_t a) {
+  // CHECK-LABEL: @extract_double(
+  // CHECK:         [[MAT:%.*]] = load <25 x double>, <25 x double>* {{.*}}, align 8
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <25 x double> [[MAT]], i64 12
+  // CHECK-NEXT:    ret double [[MATEXT]]
+
+  return a[2][3 - 1u];
+}
+
+double extract_float(fx3x3_t b) {
+  // CHECK-LABEL: @extract_float(
+  // CHECK:         [[MAT:%.*]] = load <9 x float>, <9 x float>* {{.*}}, align 4
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <9 x float> [[MAT]], i64 5
+  // CHECK-NEXT:    [[TO_DOUBLE:%.*]] = fpext float [[MATEXT]] to double
+  // CHECK-NEXT:    ret double [[TO_DOUBLE]]
+
+  return b[2][1];
+}
+
+int extract_int(ix9x3_t c, unsigned long j) {
+  // CHECK-LABEL: @extract_int(
+  // CHECK:         [[J1:%.*]] = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    [[J2:%.*]] = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    [[MAT:%.*]] = load <27 x i32>, <27 x i32>* {{.*}}, align 4
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[J2]], 9
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[J1]]
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <27 x i32> [[MAT]], i64 [[IDX2]]
+  // CHECK-NEXT:    ret i32 [[MATEXT]]
+
+  return c[j][j];
+}
+
+typedef double dx3x2_t __attribute__((matrix_type(3, 2)));
+
+double test_extract_matrix_pointer1(dx3x2_t **ptr, unsigned j) {
+  // CHECK-LABEL: @test_extract_matrix_pointer1(
+  // CHECK:         [[J:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:    [[J_EXT:%.*]] = zext i32 [[J]] to i64
+  // CHECK-NEXT:    [[PTR:%.*]] = load [6 x double]**, [6 x double]*** %ptr.addr, align 8
+  // CHECK-NEXT:    [[PTR_IDX:%.*]] = getelementptr inbounds [6 x double]*, [6 x double]** [[PTR]], i64 1
+  // CHECK-NEXT:    [[PTR2:%.*]] = load [6 x double]*, [6 x double]** [[PTR_IDX]], align 8
+  // CHECK-NEXT:    [[PTR2_IDX:%.*]] = getelementptr inbounds [6 x double], [6 x double]* [[PTR2]], i64 2
+  // CHECK-NEXT:    [[MAT_ADDR:%.*]] = bitcast [6 x double]* [[PTR2_IDX]] to <6 x double>*
+  // CHECK-NEXT:    [[MAT:%.*]] = load <6 x double>, <6 x double>* [[MAT_ADDR]], align 8
+  // CHECK-NEXT:    [[IDX:%.*]] = add i64 3, [[J_EXT]]
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <6 x double> [[MAT]], i64 [[IDX]]
+  // CHECK-NEXT:    ret double [[MATEXT]]
+
+  return ptr[1][2][j][1];
+}
+
+double test_extract_matrix_pointer2(dx3x2_t **ptr) {
+  // CHECK-LABEL: @test_extract_matrix_pointer2(
+  // CHECK-NEXT:  entry:
+  // CHECK:         [[PTR:%.*]] = load [6 x double]**, [6 x double]*** %ptr.addr, align 8
+  // CHECK-NEXT:    [[PTR_IDX:%.*]] = getelementptr inbounds [6 x double]*, [6 x double]** [[PTR]], i64 4
+  // CHECK-NEXT:    [[PTR2:%.*]] = load [6 x double]*, [6 x double]** [[PTR_IDX]], align 8
+  // CHECK-NEXT:    [[PTR2_IDX:%.*]] = getelementptr inbounds [6 x double], [6 x double]* [[PTR2]], i64 6
+  // CHECK-NEXT:    [[MAT_ADDR:%.*]] = bitcast [6 x double]* [[PTR2_IDX]] to <6 x double>*
+  // CHECK-NEXT:    [[MAT:%.*]] = load <6 x double>, <6 x double>* [[MAT_ADDR]], align 8
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <6 x double> [[MAT]], i64 5
+  // CHECK-NEXT:    ret double [[MATEXT]]
+
+  return (*(*(ptr + 4) + 6))[2][1 * 3 - 2];
+}
+
+void insert_extract(dx5x5_t a, fx3x3_t b, unsigned long j, short k) {
+  // CHECK-LABEL: @insert_extract(
+  // CHECK:         [[K:%.*]] = load i16, i16* %k.addr, align 2
+  // CHECK-NEXT:    [[K_EXT:%.*]] = sext i16 [[K]] to i64
+  // CHECK-NEXT:    [[MAT:%.*]] = load <9 x float>, <9 x float>* [[MAT_ADDR:%.*]], align 4
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[K_EXT]], 3
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], 0
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <9 x float> [[MAT]], i64 [[IDX]]
+  // CHECK-NEXT:    [[J:%.*]] = load i64, i64* %j.addr, align 8
+  // CHECK-NEXT:    [[IDX3:%.*]] = mul i64 [[J]], 3
+  // CHECK-NEXT:    [[IDX4:%.*]] = add i64 [[IDX3]], 2
+  // CHECK-NEXT:    [[MAT2:%.*]] = load <9 x float>, <9 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <9 x float> [[MAT2]], float [[MATEXT]], i64 [[IDX4]]
+  // CHECK-NEXT:    store <9 x float> [[MATINS]], <9 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  b[2][j] = b[0][k];
+}
+
+void insert_compound_stmt(dx5x5_t a) {
+  // CHECK-LABEL: define void @insert_compound_stmt(<25 x double> %a)
+  // CHECK:        [[A:%.*]] = load <25 x double>, <25 x double>* [[A_PTR:%.*]], align 8
+  // CHECK-NEXT:   [[EXT:%.*]] = extractelement <25 x double> [[A]], i64 17
+  // CHECK-NEXT:   [[SUB:%.*]] = fsub double [[EXT]], 1.000000e+00
+  // CHECK-NEXT:   [[A2:%.*]] = load <25 x double>, <25 x double>* [[A_PTR]], align 8
+  // CHECK-NEXT:   [[INS:%.*]] = insertelement <25 x double> [[A2]], double [[SUB]], i64 17
+  // CHECK-NEXT:   store <25 x double> [[INS]], <25 x double>* [[A_PTR]], align 8
+  // CHECK-NEXT:   ret void
+
+  a[2][3] -= 1.0;
+}
+
+struct Foo {
+  fx2x3_t mat;
+};
+
+void insert_compound_stmt_field(struct Foo *a, float f, unsigned i, unsigned j) {
+  // CHECK-LABEL: define void @insert_compound_stmt_field(%struct.Foo* %a, float %f, i32 %i, i32 %j)
+  // CHECK:         [[I:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    [[I_EXT:%.*]] = zext i32 [[I]] to i64
+  // CHECK-NEXT:    [[J:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:    [[J_EXT:%.*]] = zext i32 [[J]] to i64
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[J_EXT]], 2
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[I_EXT]]
+  // CHECK-NEXT:    [[MAT_PTR:%.*]] = bitcast [6 x float]* %mat to <6 x float>*
+  // CHECK-NEXT:    [[MAT:%.*]] = load <6 x float>, <6 x float>* [[MAT_PTR]], align 4
+  // CHECK-NEXT:    [[EXT:%.*]] = extractelement <6 x float> [[MAT]], i64 [[IDX2]]
+  // CHECK-NEXT:    [[SUM:%.*]] = fadd float [[EXT]], {{.*}}
+  // CHECK-NEXT:    [[MAT2:%.*]] = load <6 x float>, <6 x float>* [[MAT_PTR]], align 4
+  // CHECK-NEXT:    [[INS:%.*]] = insertelement <6 x float> [[MAT2]], float [[SUM]], i64 [[IDX2]]
+  // CHECK-NEXT:    store <6 x float> [[INS]], <6 x float>* [[MAT_PTR]], align 4
+  // CHECK-NEXT:    ret void
+
+  a->mat[i][j] += f;
+}
+
+void matrix_as_idx(ix9x3_t a, int i, int j, dx5x5_t b) {
+  // CHECK-LABEL: define void @matrix_as_idx(<27 x i32> %a, i32 %i, i32 %j, <25 x double> %b)
+  // CHECK:       [[I1:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:  [[I1_EXT:%.*]] = sext i32 [[I1]] to i64
+  // CHECK-NEXT:  [[J1:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:  [[J1_EXT:%.*]] = sext i32 [[J1]] to i64
+  // CHECK-NEXT:  [[A:%.*]] = load <27 x i32>, <27 x i32>* %0, align 4
+  // CHECK-NEXT:  [[IDX1_1:%.*]] = mul i64 [[J1_EXT]], 9
+  // CHECK-NEXT:  [[IDX1_2:%.*]] = add i64 [[IDX1_1]], [[I1_EXT]]
+  // CHECK-NEXT:  [[MI1:%.*]] = extractelement <27 x i32> [[A]], i64 [[IDX1_2]]
+  // CHECK-NEXT:  [[MI1_EXT:%.*]] = sext i32 [[MI1]] to i64
+  // CHECK-NEXT:  [[J2:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:  [[J2_EXT:%.*]] = sext i32 [[J2]] to i64
+  // CHECK-NEXT:  [[I2:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:  [[I2_EXT:%.*]] = sext i32 [[I2]] to i64
+  // CHECK-NEXT:  [[A2:%.*]] = load <27 x i32>, <27 x i32>* {{.*}}, align 4
+  // CHECK-NEXT:  [[IDX2_1:%.*]] = mul i64 [[I2_EXT]], 9
+  // CHECK-NEXT:  [[IDX2_2:%.*]] = add i64 [[IDX2_1]], [[J2_EXT]]
+  // CHECK-NEXT:  [[MI2:%.*]] = extractelement <27 x i32> [[A2]], i64 [[IDX2_2]]
+  // CHECK-NEXT:  [[MI3:%.*]] = add nsw i32 [[MI2]], 2
+  // CHECK-NEXT:  [[MI3_EXT:%.*]] = sext i32 [[MI3]] to i64
+  // CHECK-NEXT:  [[IDX3_1:%.*]] = mul i64 [[MI3_EXT]], 5
+  // CHECK-NEXT:  [[IDX3_2:%.*]] = add i64 [[IDX3_1]], [[MI1_EXT]]
+  // CHECK-NEXT:  [[B:%.*]] = load <25 x double>, <25 x double>* [[B_PTR:%.*]], align 8
+  // CHECK-NEXT:  [[INS:%.*]] = insertelement <25 x double> [[B]], double 1.500000e+00, i64 [[IDX3_2]]
+  // CHECK-NEXT:  store <25 x double> [[INS]], <25 x double>* [[B_PTR]], align 8
+  b[a[i][j]][a[j][i] + 2] = 1.5;
+}

clang/test/CodeGenCXX/matrix-type-operators.cpp

-// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
 // RUN: %clang_cc1 -fenable-matrix -triple x86_64-apple-darwin %s -emit-llvm -disable-llvm-passes -o - -std=c++11 | FileCheck %s
 
+typedef double dx5x5_t __attribute__((matrix_type(5, 5)));
+using fx2x3_t = float __attribute__((matrix_type(2, 3)));
+
 template <typename EltTy, unsigned Rows, unsigned Columns>
 struct MyMatrix {
   using matrix_t = EltTy __attribute__((matrix_type(Rows, Columns)));
 
   matrix_t value;
 };
 
 template <typename EltTy0, unsigned R0, unsigned C0>
void test_IntWrapper_Sub(MyMatrix<double, 10, 9> &m) {
   // CHECK-NEXT:  [[SCALAR_EMBED1:%.*]] = shufflevector <90 x double> [[SCALAR_EMBED]], <90 x double> undef, <90 x i32> zeroinitializer
   // CHECK-NEXT:  [[RES:%.*]] = fsub <90 x double> [[SCALAR_EMBED1]], [[MATRIX]]
   // CHECK:       store <90 x double> [[RES]], <90 x double>* {{.*}}, align 8
 
   IntWrapper w3;
   w3.x = 'c';
   m.value = w3 - m.value;
 }
+
+template <typename EltTy, unsigned Rows, unsigned Columns>
+void insert(MyMatrix<EltTy, Rows, Columns> &Mat, EltTy e, unsigned i, unsigned j) {
+  Mat.value[i][j] = e;
+}
+
+void test_insert_template1(MyMatrix<unsigned, 2, 2> &Mat, unsigned e, unsigned i, unsigned j) {
+  // CHECK-LABEL: @_Z21test_insert_template1R8MyMatrixIjLj2ELj2EEjjj(
+  // CHECK:         [[MAT_ADDR:%.*]] = load %struct.MyMatrix.1*, %struct.MyMatrix.1** %Mat.addr, align 8
+  // CHECK-NEXT:    [[E:%.*]] = load i32, i32* %e.addr, align 4
+  // CHECK-NEXT:    [[I:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    [[J:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:    call void @_Z6insertIjLj2ELj2EEvR8MyMatrixIT_XT0_EXT1_EES1_jj(%struct.MyMatrix.1* nonnull align 4 dereferenceable(16) [[MAT_ADDR]], i32 [[E]], i32 [[I]], i32 [[J]])
+  // CHECK-NEXT:    ret void
+  //
+  // CHECK-LABEL: define linkonce_odr void @_Z6insertIjLj2ELj2EEvR8MyMatrixIT_XT0_EXT1_EES1_jj(
+  // CHECK:         [[E:%.*]] = load i32, i32* %e.addr, align 4
+  // CHECK:         [[I:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    [[I_EXT:%.*]] = zext i32 [[I]] to i64
+  // CHECK-NEXT:    [[J:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:    [[J_EXT:%.*]] = zext i32 [[J]] to i64
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[J_EXT]], 2
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[I_EXT]]
+  // CHECK-NEXT:    [[MAT_ADDR:%.*]] = bitcast [4 x i32]* {{.*}} to <4 x i32>*
+  // CHECK-NEXT:    [[MAT:%.*]] = load <4 x i32>, <4 x i32>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <4 x i32> [[MAT]], i32 [[E]], i64 [[IDX2]]
+  // CHECK-NEXT:    store <4 x i32> [[MATINS]], <4 x i32>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  insert(Mat, e, i, j);
+}
+
+void test_insert_template2(MyMatrix<float, 3, 8> &Mat, float e) {
+  // CHECK-LABEL: @_Z21test_insert_template2R8MyMatrixIfLj3ELj8EEf(
+  // CHECK:         [[MAT_ADDR:%.*]] = load %struct.MyMatrix.2*, %struct.MyMatrix.2** %Mat.addr, align 8
+  // CHECK-NEXT:    [[E:%.*]] = load float, float* %e.addr, align 4
+  // CHECK-NEXT:    call void @_Z6insertIfLj3ELj8EEvR8MyMatrixIT_XT0_EXT1_EES1_jj(%struct.MyMatrix.2* nonnull align 4 dereferenceable(96) [[MAT_ADDR]], float [[E]], i32 2, i32 5)
+  // CHECK-NEXT:    ret void
+  //
+  // CHECK-LABEL: define linkonce_odr void @_Z6insertIfLj3ELj8EEvR8MyMatrixIT_XT0_EXT1_EES1_jj(
+  // CHECK:         [[E:%.*]] = load float, float* %e.addr, align 4
+  // CHECK:         [[I:%.*]] = load i32, i32* %i.addr, align 4
+  // CHECK-NEXT:    [[I_EXT:%.*]] = zext i32 [[I]] to i64
+  // CHECK-NEXT:    [[J:%.*]] = load i32, i32* %j.addr, align 4
+  // CHECK-NEXT:    [[J_EXT:%.*]] = zext i32 [[J]] to i64
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[J_EXT]], 3
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[I_EXT]]
+  // CHECK-NEXT:    [[MAT_ADDR:%.*]] = bitcast [24 x float]* {{.*}} to <24 x float>*
+  // CHECK-NEXT:    [[MAT:%.*]] = load <24 x float>, <24 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    [[MATINS:%.*]] = insertelement <24 x float> [[MAT]], float [[E]], i64 [[IDX2]]
+  // CHECK-NEXT:    store <24 x float> [[MATINS]], <24 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:    ret void
+
+  insert(Mat, e, 2, 5);
+}
+
+template <typename EltTy, unsigned Rows, unsigned Columns>
+EltTy extract(MyMatrix<EltTy, Rows, Columns> &Mat) {
+  return Mat.value[1u][0u];
+}
+
+int test_extract_template(MyMatrix<int, 2, 2> Mat1) {
+  // CHECK-LABEL: @_Z21test_extract_template8MyMatrixIiLj2ELj2EE(
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    [[CALL:%.*]] = call i32 @_Z7extractIiLj2ELj2EET_R8MyMatrixIS0_XT0_EXT1_EE(%struct.MyMatrix.3* nonnull align 4 dereferenceable(16) [[MAT1:%.*]])
+  // CHECK-NEXT:    ret i32 [[CALL]]
+  //
+  // CHECK-LABEL: define linkonce_odr i32 @_Z7extractIiLj2ELj2EET_R8MyMatrixIS0_XT0_EXT1_EE(
+  // CHECK:         [[MAT:%.*]] = load <4 x i32>, <4 x i32>* {{.*}}, align 4
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <4 x i32> [[MAT]], i64 1
+  // CHECK-NEXT:    ret i32 [[MATEXT]]
+
+  return extract(Mat1);
+}
+
+using double4x4 = double __attribute__((matrix_type(4, 4)));
+
+template <class R, class C>
+auto matrix_subscript(double4x4 m, R r, C c) -> decltype(m[r][c]) {}
+
+double test_matrix_subscript(double4x4 m) {
+  // CHECK-LABEL: @_Z21test_matrix_subscriptU11matrix_typeLm4ELm4Ed(
+  // CHECK:         [[MAT:%.*]] = load <16 x double>, <16 x double>* {{.*}}, align 8
+  // CHECK-NEXT:    [[CALL:%.*]] = call nonnull align 8 dereferenceable(8) double* @_Z16matrix_subscriptIiiEDTixixfp_fp0_fp1_EU11matrix_typeLm4ELm4EdT_T0_(<16 x double> [[MAT]], i32 1, i32 2)
+  // CHECK-NEXT:    [[RES:%.*]] = load double, double* [[CALL]], align 8
+  // CHECK-NEXT:    ret double [[RES]]
+
+  return matrix_subscript(m, 1, 2);
+}
+
+const double &test_matrix_subscript_reference(const double4x4 m) {
+  // CHECK-LABEL: @_Z31test_matrix_subscript_referenceU11matrix_typeLm4ELm4Ed(
+  // CHECK-NEXT:  entry:
+  // CHECK-NEXT:    [[M_ADDR:%.*]] = alloca [16 x double], align 8
+  // CHECK-NEXT:    [[REF_TMP:%.*]] = alloca double, align 8
+  // CHECK-NEXT:    [[NAMELESS0:%.*]] = bitcast [16 x double]* [[M_ADDR]] to <16 x double>*
+  // CHECK-NEXT:    store <16 x double> [[M:%.*]], <16 x double>* [[NAMELESS0]], align 8
+  // CHECK-NEXT:    [[NAMELESS1:%.*]] = load <16 x double>, <16 x double>* [[NAMELESS0]], align 8
+  // CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <16 x double> [[NAMELESS1]], i64 4
+  // CHECK-NEXT:    store double [[MATEXT]], double* [[REF_TMP]], align 8
+  // CHECK-NEXT:    ret double* [[REF_TMP]]
+
+  return m[0][1];
+}
+
+struct UnsignedWrapper {
+  char x;
+  operator unsigned() {
+    return x;
+  }
+};
+
+double extract_IntWrapper_idx(double4x4 &m, IntWrapper i, UnsignedWrapper j) {
+  // CHECK-LABEL: define double @_Z22extract_IntWrapper_idxRU11matrix_typeLm4ELm4Ed10IntWrapper15UnsignedWrapper(
+  // CHECK:         [[I:%.*]] = call i32 @_ZN10IntWrappercviEv(%struct.IntWrapper* %i)
+  // CHECK-NEXT:    [[I_ADD:%.*]] = add nsw i32 [[I]], 1
+  // CHECK-NEXT:    [[I_ADD_EXT:%.*]] = sext i32 [[I_ADD]] to i64
+  // CHECK-NEXT:    [[J:%.*]] = call i32 @_ZN15UnsignedWrappercvjEv(%struct.UnsignedWrapper* %j)
+  // CHECK-NEXT:    [[J_SUB:%.*]] = sub i32 [[J]], 1
+  // CHECK-NEXT:    [[J_SUB_EXT:%.*]] = zext i32 [[J_SUB]] to i64
+  // CHECK-NEXT:    [[MAT_ADDR:%.*]] = load [16 x double]*, [16 x double]** %m.addr, align 8
+  // CHECK-NEXT:    [[MAT_ADDR2:%.*]] = bitcast [16 x double]* [[MAT_ADDR]] to <16 x double>*
+  // CHECK-NEXT:    [[MAT:%.*]] = load <16 x double>, <16 x double>* [[MAT_ADDR2]], align 8
+  // CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[J_SUB_EXT]], 4
+  // CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[I_ADD_EXT]]
+  // CHECK-NEXT:    [[MATEXT:%.*]]  = extractelement <16 x double> [[MAT]], i64 [[IDX2]]
+  // CHECK-NEXT:    ret double [[MATEXT]]
+  return m[i + 1][j - 1];
+}
+
+template <class T, unsigned R, unsigned C>
+using matrix_type = T __attribute__((matrix_type(R, C)));
+struct identmatrix_t {
+  template <class T, unsigned N>
+  operator matrix_type<T, N, N>() const {
+    matrix_type<T, N, N> result;
+    for (unsigned i = 0; i != N; ++i)
+      result[i][i] = 1;
+    return result;
+  }
+};
+
+constexpr identmatrix_t identmatrix;
+
+void test_constexpr1(matrix_type<float, 4, 4> &m) {
+  // CHECK-LABEL: define void @_Z15test_constexpr1RU11matrix_typeLm4ELm4Ef([16 x float]* nonnull align 4 dereferenceable(64) %m) #3 {
+  // CHECK:         [[MAT:%.*]] = load <16 x float>, <16 x float>* {{.*}}, align 4
+  // CHECK-NEXT:    [[IM:%.*]] = call <16 x float> @_ZNK13identmatrix_tcvU11matrix_typeXT0_EXT0_ET_IfLj4EEEv(%struct.identmatrix_t* @_ZL11identmatrix)
+  // CHECK-NEXT:    [[ADD:%.*]] = fadd <16 x float> [[MAT]], [[IM]]
+  // CHECK-NEXT:    [[MAT_ADDR:%.*]] = load [16 x float]*, [16 x float]** %m.addr, align 8
+  // CHECK-NEXT:    [[MAT_ADDR2:%.*]] = bitcast [16 x float]* [[MAT_ADDR]] to <16 x float>*
+  // CHECK-NEXT:    store <16 x float> [[ADD]], <16 x float>* [[MAT_ADDR2]], align 4
+  // CHECK-NEXT:    ret voi
+
+  // CHECK-LABEL: define linkonce_odr <16 x float> @_ZNK13identmatrix_tcvU11matrix_typeXT0_EXT0_ET_IfLj4EEEv(
+  // CHECK-LABEL: for.body:                                         ; preds = %for.cond
+  // CHECK-NEXT:   [[I:%.*]] = load i32, i32* %i, align 4
+  // CHECK-NEXT:   [[I_EXT:%.*]] = zext i32 [[I]] to i64
+  // CHECK-NEXT:   [[I2:%.*]] = load i32, i32* %i, align 4
+  // CHECK-NEXT:   [[I2_EXT:%.*]] = zext i32 [[I2]] to i64
+  // CHECK-NEXT:   [[IDX1:%.*]] = mul i64 [[I2_EXT]], 4
+  // CHECK-NEXT:   [[IDX2:%.*]] = add i64 [[IDX1]], [[I_EXT]]
+  // CHECK-NEXT:   [[MAT_ADDR:%.*]] = bitcast [16 x float]* %result to <16 x float>*
+  // CHECK-NEXT:   [[MAT:%.*]] = load <16 x float>, <16 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:   [[MATINS:%.*]] = insertelement <16 x float> [[MAT]], float 1.000000e+00, i64 [[IDX2]]
+  // CHECK-NEXT:   store <16 x float> [[MATINS]], <16 x float>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:   br label %for.inc
+  m = m + identmatrix;
+}
+
+void test_constexpr2(matrix_type<int, 5, 5> &m) {
+  // CHECK-LABEL: define void @_Z15test_constexpr2RU11matrix_typeLm5ELm5Ei([25 x i32]* nonnull align 4 dereferenceable(100) %m) #4 {
+  // CHECK:         [[IM:%.*]] = call <25 x i32> @_ZNK13identmatrix_tcvU11matrix_typeXT0_EXT0_ET_IiLj5EEEv(%struct.identmatrix_t* @_ZL11identmatrix)
+  // CHECK:         [[MAT:%.*]] = load <25 x i32>, <25 x i32>* {{.*}}, align 4
+  // CHECK-NEXT:    [[SUB:%.*]] = sub <25 x i32> [[IM]], [[MAT]]
+  // CHECK-NEXT:    [[SUB2:%.*]] = add <25 x i32> [[SUB]], <i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1, i32 1>
+  // CHECK-NEXT:    [[MAT_ADDR:%.*]] = load [25 x i32]*, [25 x i32]** %m.addr, align 8
+  // CHECK-NEXT:    [[MAT_ADDR2:%.*]] = bitcast [25 x i32]* [[MAT_ADDR]] to <25 x i32>*
+  // CHECK-NEXT:    store <25 x i32> [[SUB2]], <25 x i32>* [[MAT_ADDR2]], align 4
+  // CHECK-NEXT:    ret void
+  //
+
+  // CHECK-LABEL: define linkonce_odr <25 x i32> @_ZNK13identmatrix_tcvU11matrix_typeXT0_EXT0_ET_IiLj5EEEv(
+  // CHECK-LABEL: for.body:                                         ; preds = %for.cond
+  // CHECK-NEXT:   [[I:%.*]] = load i32, i32* %i, align 4
+  // CHECK-NEXT:   [[I_EXT:%.*]] = zext i32 [[I]] to i64
+  // CHECK-NEXT:   [[I2:%.*]] = load i32, i32* %i, align 4
+  // CHECK-NEXT:   [[I2_EXT:%.*]] = zext i32 [[I2]] to i64
+  // CHECK-NEXT:   [[IDX1:%.*]] = mul i64 [[I2_EXT]], 5
+  // CHECK-NEXT:   [[IDX2:%.*]] = add i64 [[IDX1]], [[I_EXT]]
+  // CHECK-NEXT:   [[MAT_ADDR:%.*]] = bitcast [25 x i32]* %result to <25 x i32>*
+  // CHECK-NEXT:   [[MAT:%.*]] = load <25 x i32>, <25 x i32>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:   [[MATINS:%.*]] = insertelement <25 x i32> [[MAT]], i32 1, i64 [[IDX2]]
+  // CHECK-NEXT:   store <25 x i32> [[MATINS]], <25 x i32>* [[MAT_ADDR]], align 4
+  // CHECK-NEXT:   br label %for.inc
+
+  m = identmatrix - m + 1;
+}

clang/test/CodeGenObjC/matrix-type-operators.m

+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
+// RUN: %clang_cc1 -triple x86_64-apple-darwin10 -fenable-matrix -emit-llvm -disable-llvm-optzns -o - %s | FileCheck %s
+
+__attribute__((objc_root_class))
+@interface IntValue
+@property int value;
+@end
+
+typedef double double4x4 __attribute__((matrix_type(4, 4)));
+
+// Check that we correctly deal with placeholder expressions.
+
+// CHECK-LABEL: @test_index_placeholders(
+// CHECK-NEXT:  entry:
+// CHECK:         [[IV:%.*]] = load %0*, %0** [[IV_ADDR:%.*]], align 8
+// CHECK-NEXT:    [[SEL:%.*]] = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8, !invariant.load !7
+// CHECK-NEXT:    [[IV_PTR:%.*]] = bitcast %0* [[IV]] to i8*
+// CHECK-NEXT:    [[CALL:%.*]] = call i32 bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i32 (i8*, i8*)*)(i8* [[IV_PTR]], i8* [[SEL]])
+// CHECK-NEXT:    [[CONV:%.*]] = sext i32 [[CALL]] to i64
+// CHECK-NEXT:    [[IV2:%.*]] = load %0*, %0** [[IV_ADDR]], align 8
+// CHECK-NEXT:    [[SEL2:%.*]] = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8, !invariant.load !7
+// CHECK-NEXT:    [[IV2_PTR:%.*]] = bitcast %0* [[IV2]] to i8*
+// CHECK-NEXT:    [[CALL1:%.*]] = call i32 bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i32 (i8*, i8*)*)(i8* [[IV2_PTR]], i8* [[SEL2]])
+// CHECK-NEXT:    [[CONV2:%.*]] = sext i32 [[CALL1]] to i64
+// CHECK-NEXT:    [[MAT:%.*]] = load <16 x double>, <16 x double>* {{.*}} align 8
+// CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[CONV2]], 4
+// CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[CONV]]
+// CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <16 x double> [[MAT]], i64 [[IDX2]]
+// CHECK-NEXT:    ret double [[MATEXT]]
+//
+double test_index_placeholders(double4x4 m, IntValue *iv) {
+
+  return m[iv.value][iv.value];
+}
+
+__attribute__((objc_root_class))
+@interface MatrixValue
+@property double4x4 value;
+@end
+
+// CHECK-LABEL: @test_base_and_index_placeholders(
+// CHECK:         [[IV:%.*]] = load %0*, %0** [[IV_ADDR:%.*]], align 8
+// CHECK-NEXT:    [[SEL:%.*]] = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8, !invariant.load !7
+// CHECK-NEXT:    [[IV_PTR:%.*]] = bitcast %0* [[IV]] to i8*
+// CHECK-NEXT:    [[CALL:%.*]] = call i32 bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i32 (i8*, i8*)*)(i8* [[IV_PTR]], i8* [[SEL]])
+// CHECK-NEXT:    [[CONV:%.*]] = sext i32 [[CALL]] to i64
+// CHECK-NEXT:    [[IV2:%.*]] = load %0*, %0** [[IV_ADDR]], align 8
+// CHECK-NEXT:    [[SEL2:%.*]] = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8, !invariant.load !7
+// CHECK-NEXT:    [[IV2_PTR:%.*]] = bitcast %0* [[IV2]] to i8*
+// CHECK-NEXT:    [[CALL1:%.*]] = call i32 bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to i32 (i8*, i8*)*)(i8* [[IV2_PTR]], i8* [[SEL2]])
+// CHECK-NEXT:    [[CONV2:%.*]] = sext i32 [[CALL1]] to i64
+// CHECK-NEXT:    [[M:%.*]] = load %1*, %1** %m.addr, align 8
+// CHECK-NEXT:    [[SEL3:%.*]] = load i8*, i8** @OBJC_SELECTOR_REFERENCES_, align 8, !invariant.load !7
+// CHECK-NEXT:    [[M_PTR:%.*]] = bitcast %1* [[M]] to i8*
+// CHECK-NEXT:    [[MAT:%.*]] = call <16 x double> bitcast (i8* (i8*, i8*, ...)* @objc_msgSend to <16 x double> (i8*, i8*)*)(i8* [[M_PTR]], i8* [[SEL3]])
+// CHECK-NEXT:    [[IDX1:%.*]] = mul i64 [[CONV2]], 4
+// CHECK-NEXT:    [[IDX2:%.*]] = add i64 [[IDX1]], [[CONV]]
+// CHECK-NEXT:    [[MATEXT:%.*]] = extractelement <16 x double> [[MAT]], i64 [[IDX2]]
+// CHECK-NEXT:    ret double [[MATEXT]]
+//
+double test_base_and_index_placeholders(MatrixValue *m, IntValue *iv) {
+
+  return m.value[iv.value][iv.value];
+}

clang/test/Sema/matrix-type-operators.c

void sub(sx10x10_t a, sx5x10_t b, sx10x5_t c) {
 
   a = 10 - b;
   // expected-error@-1 {{assigning to 'sx10x10_t' (aka 'float __attribute__((matrix_type(10, 10)))') from incompatible type 'sx5x10_t' (aka 'float __attribute__((matrix_type(5, 10)))')}}
 
   a = b - &c;
   // expected-error@-1 {{invalid operands to binary expression ('sx5x10_t' (aka 'float __attribute__((matrix_type(5, 10)))') and 'sx10x5_t *' (aka 'float  __attribute__((matrix_type(10, 5)))*'))}}
   // expected-error@-2 {{casting 'sx10x5_t *' (aka 'float  __attribute__((matrix_type(10, 5)))*') to incompatible type 'float'}}
 }
+
+sx5x10_t get_matrix();
+
+void insert(sx5x10_t a, float f) {
+  // Non integer indexes.
+  a[3][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+  a[f][9] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  a[f][f] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+  a[0][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+
+  a[f][f] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+
+  // Invalid element type.
+  a[3][4] = &f;
+  // expected-error@-1 {{assigning to 'float' from incompatible type 'float *'; remove &}}
+
+  // Indexes outside allowed dimensions.
+  a[-1][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[3][-1] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[3][-1u] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[-1u][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[5][2] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[4][10] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[5][0] = f;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  (a[1])[1] = f;
+  // expected-error@-1 {{matrix row and column subscripts cannot be separated by any expression}}
+
+  a[3] = 5.0;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+
+  (a[3]) = 5.0;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+
+  get_matrix()[0][0] = f;
+  // expected-error@-1 {{expression is not assignable}}
+  get_matrix()[5][1] = f;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  get_matrix()[3] = 5.0;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+
+  (get_matrix()[5])[10.0] = f;
+  // expected-error@-1 {{matrix row and column subscripts cannot be separated by any expression}}
+  (get_matrix()[3]) = 5.0;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+
+  a([0])[0] = f;
+  // expected-error@-1 {{expected expression}}
+  a[0]([0]) = f;
+  // expected-error@-1 {{expected expression}}
+}
+
+void extract(sx5x10_t a, float f) {
+  // Non integer indexes.
+  float v1 = a[3][f];
+  // expected-error@-1 {{matrix column index is not an integer}}
+  float v2 = a[f][9];
+  // expected-error@-1 {{matrix row index is not an integer}}
+  float v3 = a[f][f];
+  // expected-error@-1 {{matrix row index is not an integer}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+
+  // Invalid element type.
+  char *v4 = a[3][4];
+  // expected-error@-1 {{initializing 'char *' with an expression of incompatible type 'float'}}
+
+  // Indexes outside allowed dimensions.
+  float v5 = a[-1][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v6 = a[3][-1];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v8 = a[-1u][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v9 = a[5][2];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v10 = a[4][10];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v11 = a[5][9];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+
+  float v12 = a[3];
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+}
+
+float *address_of_element(sx5x10_t *a) {
+  return &(*a)[0][1];
+  // expected-error@-1 {{address of matrix element requested}}
+}

clang/test/SemaCXX/matrix-type-operators.cpp

void test_DoubleWrapper(MyMatrix<double, 10, 9> &m, StructWithC &c) {
   m.value = m.value - c;
   // expected-error@-1 {{no viable conversion from 'StructWithC' to 'double'}}
   // expected-error@-2 {{invalid operands to binary expression ('MyMatrix<double, 10, 9>::matrix_t' (aka 'double __attribute__((matrix_type(10, 9)))') and 'StructWithC')}}
 
   m.value = c - m.value;
   // expected-error@-1 {{no viable conversion from 'StructWithC' to 'double'}}
   // expected-error@-2 {{invalid operands to binary expression ('StructWithC' and 'MyMatrix<double, 10, 9>::matrix_t' (aka 'double __attribute__((matrix_type(10, 9)))'))}}
 }
+
+sx5x10_t get_matrix();
+
+void insert(sx5x10_t a, float f) {
+  // Non integer indexes.
+  a[3][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+  a[f][9] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  a[f][f] = 0;
+  // expected-error@-1 {{matrix row index is not an integer}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+  a[0][f] = 0;
+  // expected-error@-1 {{matrix column index is not an integer}}
+
+  // Invalid element type.
+  a[3][4] = &f;
+  // expected-error@-1 {{assigning to 'float' from incompatible type 'float *'; remove &}}
+
+  // Indexes outside allowed dimensions.
+  a[-1][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[3][-1] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[3][-1u] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[-1u][3] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[5][2] = 10.0;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  a[4][10] = 10.0;
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  a[5][10.0] = f;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+
+  get_matrix()[0][0] = f;
+  // expected-error@-1 {{expression is not assignable}}
+  get_matrix()[5][10.0] = f;
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+  get_matrix()[3] = 5.0;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+
+  float &x = reinterpret_cast<float &>(a[3][3]);
+  // expected-error@-1 {{reinterpret_cast of a matrix element to 'float &' needs its address, which is not allowed}}
+
+  a[4, 5] = 5.0;
+  // expected-error@-1 {{comma expressions are not allowed as indices in matrix subscript expressions}}
+  // expected-warning@-2 {{expression result unused}}
+
+  a[4, 5, 4] = 5.0;
+  // expected-error@-1 {{comma expressions are not allowed as indices in matrix subscript expressions}}
+  // expected-warning@-2 {{expression result unused}}
+  // expected-warning@-3 {{expression result unused}}
+}
+
+void extract(sx5x10_t a, float f) {
+  // Non integer indexes.
+  float v1 = a[3][f];
+  // expected-error@-1 {{matrix column index is not an integer}}
+  float v2 = a[f][9];
+  // expected-error@-1 {{matrix row index is not an integer}}
+  float v3 = a[f][f];
+  // expected-error@-1 {{matrix row index is not an integer}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+
+  // Invalid element type.
+  char *v4 = a[3][4];
+  // expected-error@-1 {{cannot initialize a variable of type 'char *' with an lvalue of type 'float'}}
+
+  // Indexes outside allowed dimensions.
+  float v5 = a[-1][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v6 = a[3][-1];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v8 = a[-1u][3];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v9 = a[5][2];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  float v10 = a[4][10];
+  // expected-error@-1 {{matrix column index is outside the allowed range [0, 10)}}
+  float v11 = a[5][10.0];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+
+  float v12 = get_matrix()[0][0];
+  float v13 = get_matrix()[5][10.0];
+  // expected-error@-1 {{matrix row index is outside the allowed range [0, 5)}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+}
+
+const float &const_subscript_reference(sx5x10_t m) {
+  return m[2][2];
+  // expected-warning@-1 {{returning reference to local temporary object}}
+}
+
+const float &const_subscript_reference(const sx5x10_t &m) {
+  return m[2][2];
+  // expected-warning@-1 {{returning reference to local temporary object}}
+}
+
+float &nonconst_subscript_reference(sx5x10_t m) {
+  return m[2][2];
+  // expected-error@-1 {{non-const reference cannot bind to matrix element}}
+}
+
+void incomplete_matrix_index_expr(sx5x10_t a, float f) {
+  float x = a[3];
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+  a[2] = f;
+  // expected-error@-1 {{single subscript expressions are not allowed for matrix values}}
+}

clang/test/SemaObjC/matrix-type-operators.m

+// RUN: %clang_cc1 -fsyntax-only -verify -fenable-matrix %s
+
+struct Foo {};
+__attribute__((objc_root_class))
+@interface FooValue
+@property struct Foo value;
+@end
+
+typedef double double4x4 __attribute__((matrix_type(4, 4)));
+
+// Check that we generate proper error messages for invalid placeholder types.
+//
+double test_index_placeholders(double4x4 m, FooValue *iv) {
+  return m[iv.value][iv.value];
+  // expected-error@-1 {{matrix row index is not an integer}}
+  // expected-error@-2 {{matrix column index is not an integer}}
+}
+
+double test_base_and_index_placeholders(FooValue *m, FooValue *iv) {
+  return m.value[iv.value][iv.value];
+  // expected-error@-1 {{subscripted value is not an array, pointer, or vector}}
+}

clang/tools/libclang/CXCursor.cpp

case Stmt::CXXNoexceptExprClass:
     K = CXCursor_UnaryExpr;
     break;
 
   case Stmt::MSPropertySubscriptExprClass:
   case Stmt::ArraySubscriptExprClass:
     K = CXCursor_ArraySubscriptExpr;
     break;
 
+  case Stmt::MatrixSubscriptExprClass:
+    // TODO: add support for MatrixSubscriptExpr.
+    K = CXCursor_UnexposedExpr;
+    break;
+
   case Stmt::OMPArraySectionExprClass:
     K = CXCursor_OMPArraySectionExpr;
     break;
 
   case Stmt::OMPArrayShapingExprClass:
     K = CXCursor_OMPArrayShapingExpr;
     break;
 

llvm/include/llvm/IR/MatrixBuilder.h

Value *ScalarVector =
         B.CreateVectorSplat(cast<VectorType>(LHS->getType())->getNumElements(),
                             RHS, "scalar.splat");
     if (RHS->getType()->isFloatingPointTy())
       return B.CreateFMul(LHS, ScalarVector);
 
     return B.CreateMul(LHS, ScalarVector);
   }
 
-  /// Extracts the element at (\p Row, \p Column) from \p Matrix.
-  Value *CreateExtractMatrix(Value *Matrix, Value *Row, Value *Column,
+  /// Extracts the element at (\p RowIdx, \p ColumnIdx) from \p Matrix.
+  Value *CreateExtractElement(Value *Matrix, Value *RowIdx, Value *ColumnIdx,
                               unsigned NumRows, Twine const &Name = "") {
 
+    unsigned MaxWidth = std::max(RowIdx->getType()->getScalarSizeInBits(),
+                                 ColumnIdx->getType()->getScalarSizeInBits());
+    Type *IntTy = IntegerType::get(RowIdx->getType()->getContext(), MaxWidth);
+    RowIdx = B.CreateZExt(RowIdx, IntTy);
+    ColumnIdx = B.CreateZExt(ColumnIdx, IntTy);
+    Value *NumRowsV = B.getIntN(MaxWidth, NumRows);
     return B.CreateExtractElement(
-        Matrix,
-        B.CreateAdd(
-            B.CreateMul(Column, ConstantInt::get(Column->getType(), NumRows)),
-            Row));
+        Matrix, B.CreateAdd(B.CreateMul(ColumnIdx, NumRowsV), RowIdx),
+        "matext");
   }
 };
 
 } // end namespace llvm
 
 #endif // LLVM_IR_MATRIXBUILDER_H