[SVE] Fixed-length vector MVT ranges

[huntergr]

Summary

• New range functions in MachineValueType.h to only iterate over the fixed-length int/fp vector types
• Stopped backends which don't support scalable vector types from iterating over scalable types.

[huntergr]

Split out from D53137.

I added new range functions for fp and int fixedlen types, but not all fixedlen vector valuetypes; the fixedlen MVTs are not contiguous.

If we want to provide that, I could reorder the MVT list to group all fixedlen types together (and the same for all scalable types), but we then lose direct lists of all fp/int types. The concat or filter iterators from STLExtras could be used to fix those if needed. Any preferences?

[sdesmalen]

In D66339#1632857, @huntergr wrote:

Split out from D53137.

Thanks!

I added new range functions for fp and int fixedlen types, but not all fixedlen vector valuetypes; the fixedlen MVTs are not contiguous.

If we want to provide that, I could reorder the MVT list to group all fixedlen types together (and the same for all scalable types), but we then lose direct lists of all fp/int types. The concat or filter iterators from STLExtras could be used to fix those if needed. Any preferences?

I like your suggestion of using concat_iterator(_range), but I think grouping all fixed-width types together is sufficient for now. If there is ever a good reason to iterate through (all combined scalable and fixed-width fp vector types)or (all combined scalable and fixed-width integer vector types), we can always introduce the extra concat_iterators then.

[greened]

In D66339#1632975, @sdesmalen wrote:

I like your suggestion of using concat_iterator(_range), but I think grouping all fixed-width types together is sufficient for now. If there is ever a good reason to iterate through (all combined scalable and fixed-width fp vector types)or (all combined scalable and fixed-width integer vector types), we can always introduce the extra concat_iterators then.

I'm not sure what you're suggesting here. Are you asking Graham to keep all the fixed-width types contiguous (i.e. int and fp)? This patch does not do that. I don't have a strong preference either way. Given that the existing state is non-contiguous I see no compelling reason to change it.

[greened]

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

312 ↗

(On Diff #215558)

Should we replace this with a concat/filter iterator on the vector value types in the future? Not suggesting this needs to be done now. Obviously this would apply to the many similar patterns below.

[sdesmalen]

In D66339#1633368, @greened wrote:

In D66339#1632975, @sdesmalen wrote:

I like your suggestion of using concat_iterator(_range), but I think grouping all fixed-width types together is sufficient for now. If there is ever a good reason to iterate through (all combined scalable and fixed-width fp vector types)or (all combined scalable and fixed-width integer vector types), we can always introduce the extra concat_iterators then.

I'm not sure what you're suggesting here. Are you asking Graham to keep all the fixed-width types contiguous (i.e. int and fp)? This patch does not do that. I don't have a strong preference either way. Given that the existing state is non-contiguous I see no compelling reason to change it.

I was suggesting to change the order of the MVTs to keep all the fixed-width types contiguous (int and fp), as a lighter weight alternative to introducing the filter/concat iterators, since this would probably suffice for most use-cases. We can always introduce the concat iterators later if needed.

[greened]

In D66339#1634903, @sdesmalen wrote:

I was suggesting to change the order of the MVTs to keep all the fixed-width types contiguous (int and fp), as a lighter weight alternative to introducing the filter/concat iterators, since this would probably suffice for most use-cases. We can always introduce the concat iterators later if needed.

Got it. That sounds fine to me.

[huntergr]

• Reordered MVTs to group fixed-length types together (and the same for scalable).
• Replaced existing range accessors to remove explicit isScalableVector checks on MVTs in various backends.

[greened]

llvm/lib/Target/ARM/ARMISelLowering.cpp

899 ↗

(On Diff #216415)

This is confusing to me. It doesn't do what the comment says (and maybe that comment should have been updated earlier) but it also seems like this could do this:

setLoadExtAction(ISD::EXTLOAD, v4i8, v4i16, Legal);

That seems wrong. What am I missing?

I know this patch didn't introduce this.

[greened]

Since I don't see anything strange that was introduced by this patch, LGTM.

llvm/trunk/include/llvm/CodeGen/ValueTypes.td

 def v2i64  : ValueType<128, 57>;   //  2 x i64 vector value
 def v4i64  : ValueType<256, 58>;   //  4 x i64 vector value
 def v8i64  : ValueType<512, 59>;   //  8 x i64 vector value
 def v16i64 : ValueType<1024,60>;   // 16 x i64 vector value
 def v32i64 : ValueType<2048,61>;   // 32 x i64 vector value
 
 def v1i128 : ValueType<128, 62>;   //  1 x i128 vector value
 
-def nxv1i1  : ValueType<1,   63>;  // n x  1 x i1  vector value
-def nxv2i1  : ValueType<2,   64>;  // n x  2 x i1  vector value
-def nxv4i1  : ValueType<4,   65>;  // n x  4 x i1  vector value
-def nxv8i1  : ValueType<8,   66>;  // n x  8 x i1  vector value
-def nxv16i1 : ValueType<16,  67>;  // n x 16 x i1  vector value
-def nxv32i1 : ValueType<32,  68>;  // n x 32 x i1  vector value
-
-def nxv1i8  : ValueType<8,   69>;  // n x  1 x i8  vector value
-def nxv2i8  : ValueType<16,  70>;  // n x  2 x i8  vector value
-def nxv4i8  : ValueType<32,  71>;  // n x  4 x i8  vector value
-def nxv8i8  : ValueType<64,  72>;  // n x  8 x i8  vector value
-def nxv16i8 : ValueType<128, 73>;  // n x 16 x i8  vector value
-def nxv32i8 : ValueType<256, 74>;  // n x 32 x i8  vector value
-
-def nxv1i16 : ValueType<16,  75>;  // n x  1 x i16 vector value
-def nxv2i16 : ValueType<32,  76>;  // n x  2 x i16 vector value
-def nxv4i16 : ValueType<64,  77>;  // n x  4 x i16 vector value
-def nxv8i16 : ValueType<128, 78>;  // n x  8 x i16 vector value
-def nxv16i16: ValueType<256, 79>;  // n x 16 x i16 vector value
-def nxv32i16: ValueType<512, 80>;  // n x 32 x i16 vector value
-
-def nxv1i32 : ValueType<32,  81>;  // n x  1 x i32 vector value
-def nxv2i32 : ValueType<64,  82>;  // n x  2 x i32 vector value
-def nxv4i32 : ValueType<128, 83>;  // n x  4 x i32 vector value
-def nxv8i32 : ValueType<256, 84>;  // n x  8 x i32 vector value
-def nxv16i32: ValueType<512, 85>;  // n x 16 x i32 vector value
-def nxv32i32: ValueType<1024,86>;  // n x 32 x i32 vector value
-
-def nxv1i64 : ValueType<64,  87>;  // n x  1 x i64 vector value
-def nxv2i64 : ValueType<128, 88>;  // n x  2 x i64 vector value
-def nxv4i64 : ValueType<256, 89>;  // n x  4 x i64 vector value
-def nxv8i64 : ValueType<512, 90>;  // n x  8 x i64 vector value
-def nxv16i64: ValueType<1024,91>;  // n x 16 x i64 vector value
-def nxv32i64: ValueType<2048,92>;  // n x 32 x i64 vector value
-
-def v2f16    : ValueType<32 , 93>;    //    2 x f16 vector value
-def v3f16    : ValueType<48 , 94>;    //    3 x f16 vector value
-def v4f16    : ValueType<64 , 95>;    //    4 x f16 vector value
-def v8f16    : ValueType<128, 96>;    //    8 x f16 vector value
-def v16f16   : ValueType<256, 97>;    //    8 x f16 vector value
-def v32f16   : ValueType<512, 98>;    //    8 x f16 vector value
-def v1f32    : ValueType<32 , 99>;    //    1 x f32 vector value
-def v2f32    : ValueType<64 , 100>;   //    2 x f32 vector value
-def v3f32    : ValueType<96 , 101>;   //    3 x f32 vector value
-def v4f32    : ValueType<128, 102>;   //    4 x f32 vector value
-def v5f32    : ValueType<160, 103>;   //    5 x f32 vector value
-def v8f32    : ValueType<256, 104>;   //    8 x f32 vector value
-def v16f32   : ValueType<512,  105>;  //   16 x f32 vector value
-def v32f32   : ValueType<1024, 106>;  //   32 x f32 vector value
-def v64f32   : ValueType<2048, 107>;  //   64 x f32 vector value
-def v128f32  : ValueType<4096, 108>;  //  128 x f32 vector value
-def v256f32  : ValueType<8182, 109>;  //  256 x f32 vector value
-def v512f32  : ValueType<16384, 110>; //  512 x f32 vector value
-def v1024f32 : ValueType<32768, 111>; // 1024 x f32 vector value
-def v2048f32 : ValueType<65536, 112>; // 2048 x f32 vector value
-def v1f64    : ValueType<64, 113>;    //    1 x f64 vector value
-def v2f64    : ValueType<128, 114>;   //    2 x f64 vector value
-def v4f64    : ValueType<256, 115>;   //    4 x f64 vector value
-def v8f64    : ValueType<512, 116>;   //    8 x f64 vector value
+def v2f16    : ValueType<32 , 63>;    //    2 x f16 vector value
+def v3f16    : ValueType<48 , 64>;    //    3 x f16 vector value
+def v4f16    : ValueType<64 , 65>;    //    4 x f16 vector value
+def v8f16    : ValueType<128, 66>;    //    8 x f16 vector value
+def v16f16   : ValueType<256, 67>;    //    8 x f16 vector value
+def v32f16   : ValueType<512, 68>;    //    8 x f16 vector value
+def v1f32    : ValueType<32 , 69>;    //    1 x f32 vector value
+def v2f32    : ValueType<64 , 70>;    //    2 x f32 vector value
+def v3f32    : ValueType<96 , 71>;    //    3 x f32 vector value
+def v4f32    : ValueType<128, 72>;    //    4 x f32 vector value
+def v5f32    : ValueType<160, 73>;    //    5 x f32 vector value
+def v8f32    : ValueType<256, 74>;    //    8 x f32 vector value
+def v16f32   : ValueType<512,  75>;   //   16 x f32 vector value
+def v32f32   : ValueType<1024, 76>;   //   32 x f32 vector value
+def v64f32   : ValueType<2048, 77>;   //   64 x f32 vector value
+def v128f32  : ValueType<4096, 78>;   //  128 x f32 vector value
+def v256f32  : ValueType<8182, 79>;   //  256 x f32 vector value
+def v512f32  : ValueType<16384, 80>;  //  512 x f32 vector value
+def v1024f32 : ValueType<32768, 81>;  // 1024 x f32 vector value
+def v2048f32 : ValueType<65536, 82>;  // 2048 x f32 vector value
+def v1f64    : ValueType<64, 83>;     //    1 x f64 vector value
+def v2f64    : ValueType<128, 84>;    //    2 x f64 vector value
+def v4f64    : ValueType<256, 85>;    //    4 x f64 vector value
+def v8f64    : ValueType<512, 86>;    //    8 x f64 vector value
+
+def nxv1i1  : ValueType<1,   87>;  // n x  1 x i1  vector value
+def nxv2i1  : ValueType<2,   88>;  // n x  2 x i1  vector value
+def nxv4i1  : ValueType<4,   89>;  // n x  4 x i1  vector value
+def nxv8i1  : ValueType<8,   90>;  // n x  8 x i1  vector value
+def nxv16i1 : ValueType<16,  91>;  // n x 16 x i1  vector value
+def nxv32i1 : ValueType<32,  92>;  // n x 32 x i1  vector value
+
+def nxv1i8  : ValueType<8,   93>;  // n x  1 x i8  vector value
+def nxv2i8  : ValueType<16,  94>;  // n x  2 x i8  vector value
+def nxv4i8  : ValueType<32,  95>;  // n x  4 x i8  vector value
+def nxv8i8  : ValueType<64,  96>;  // n x  8 x i8  vector value
+def nxv16i8 : ValueType<128, 97>;  // n x 16 x i8  vector value
+def nxv32i8 : ValueType<256, 98>;  // n x 32 x i8  vector value
+
+def nxv1i16 : ValueType<16,  99>;  // n x  1 x i16 vector value
+def nxv2i16 : ValueType<32,  100>; // n x  2 x i16 vector value
+def nxv4i16 : ValueType<64,  101>; // n x  4 x i16 vector value
+def nxv8i16 : ValueType<128, 102>; // n x  8 x i16 vector value
+def nxv16i16: ValueType<256, 103>; // n x 16 x i16 vector value
+def nxv32i16: ValueType<512, 104>; // n x 32 x i16 vector value
+
+def nxv1i32 : ValueType<32,  105>; // n x  1 x i32 vector value
+def nxv2i32 : ValueType<64,  106>; // n x  2 x i32 vector value
+def nxv4i32 : ValueType<128, 107>; // n x  4 x i32 vector value
+def nxv8i32 : ValueType<256, 108>; // n x  8 x i32 vector value
+def nxv16i32: ValueType<512, 109>; // n x 16 x i32 vector value
+def nxv32i32: ValueType<1024,110>; // n x 32 x i32 vector value
+
+def nxv1i64 : ValueType<64,  111>; // n x  1 x i64 vector value
+def nxv2i64 : ValueType<128, 112>; // n x  2 x i64 vector value
+def nxv4i64 : ValueType<256, 113>; // n x  4 x i64 vector value
+def nxv8i64 : ValueType<512, 114>; // n x  8 x i64 vector value
+def nxv16i64: ValueType<1024,115>; // n x 16 x i64 vector value
+def nxv32i64: ValueType<2048,116>; // n x 32 x i64 vector value
 
 def nxv2f16  : ValueType<32 , 117>; // n x  2 x f16 vector value
 def nxv4f16  : ValueType<64 , 118>; // n x  4 x f16 vector value
 def nxv8f16  : ValueType<128, 119>; // n x  8 x f16 vector value
 def nxv1f32  : ValueType<32 , 120>; // n x  1 x f32 vector value
 def nxv2f32  : ValueType<64 , 121>; // n x  2 x f32 vector value
 def nxv4f32  : ValueType<128, 122>; // n x  4 x f32 vector value
 def nxv8f32  : ValueType<256, 123>; // n x  8 x f32 vector value

llvm/trunk/include/llvm/Support/MachineValueType.h

enum SimpleValueType : uint8_t {
       v2i64          =  57,   //  2 x i64
       v4i64          =  58,   //  4 x i64
       v8i64          =  59,   //  8 x i64
       v16i64         =  60,   // 16 x i64
       v32i64         =  61,   // 32 x i64
 
       v1i128         =  62,   //  1 x i128
 
-      // Scalable integer types
-      nxv1i1         =  63,   // n x  1 x i1
-      nxv2i1         =  64,   // n x  2 x i1
-      nxv4i1         =  65,   // n x  4 x i1
-      nxv8i1         =  66,   // n x  8 x i1
-      nxv16i1        =  67,   // n x 16 x i1
-      nxv32i1        =  68,   // n x 32 x i1
-
-      nxv1i8         =  69,   // n x  1 x i8
-      nxv2i8         =  70,   // n x  2 x i8
-      nxv4i8         =  71,   // n x  4 x i8
-      nxv8i8         =  72,   // n x  8 x i8
-      nxv16i8        =  73,   // n x 16 x i8
-      nxv32i8        =  74,   // n x 32 x i8
-
-      nxv1i16        =  75,   // n x  1 x i16
-      nxv2i16        =  76,   // n x  2 x i16
-      nxv4i16        =  77,   // n x  4 x i16
-      nxv8i16        =  78,   // n x  8 x i16
-      nxv16i16       =  79,   // n x 16 x i16
-      nxv32i16       =  80,   // n x 32 x i16
-
-      nxv1i32        =  81,   // n x  1 x i32
-      nxv2i32        =  82,   // n x  2 x i32
-      nxv4i32        =  83,   // n x  4 x i32
-      nxv8i32        =  84,   // n x  8 x i32
-      nxv16i32       =  85,   // n x 16 x i32
-      nxv32i32       =  86,   // n x 32 x i32
-
-      nxv1i64        =  87,   // n x  1 x i64
-      nxv2i64        =  88,   // n x  2 x i64
-      nxv4i64        =  89,   // n x  4 x i64
-      nxv8i64        =  90,   // n x  8 x i64
-      nxv16i64       =  91,   // n x 16 x i64
-      nxv32i64       =  92,   // n x 32 x i64
-
-      FIRST_INTEGER_VECTOR_VALUETYPE = v1i1,
-      LAST_INTEGER_VECTOR_VALUETYPE = nxv32i64,
-
-      FIRST_INTEGER_SCALABLE_VALUETYPE = nxv1i1,
-      LAST_INTEGER_SCALABLE_VALUETYPE = nxv32i64,
-
-      v2f16          =  93,   //    2 x f16
-      v3f16          =  94,   //    3 x f16
-      v4f16          =  95,   //    4 x f16
-      v8f16          =  96,   //    8 x f16
-      v16f16         =  97,   //   16 x f16
-      v32f16         =  98,   //   32 x f16
-      v1f32          =  99,   //    1 x f32
-      v2f32          =  100,  //    2 x f32
-      v3f32          =  101,  //    3 x f32
-      v4f32          =  102,  //    4 x f32
-      v5f32          =  103,  //    5 x f32
-      v8f32          =  104,  //    8 x f32
-      v16f32         =  105,  //   16 x f32
-      v32f32         =  106,  //   32 x f32
-      v64f32         =  107,  //   64 x f32
-      v128f32        =  108,  //  128 x f32
-      v256f32        =  109,  //  256 x f32
-      v512f32        =  110,  //  512 x f32
-      v1024f32       =  111,  // 1024 x f32
-      v2048f32       =  112,  // 2048 x f32
-      v1f64          =  113,  //    1 x f64
-      v2f64          =  114,  //    2 x f64
-      v4f64          =  115,  //    4 x f64
-      v8f64          =  116,  //    8 x f64
+      FIRST_INTEGER_FIXEDLEN_VECTOR_VALUETYPE = v1i1,
+      LAST_INTEGER_FIXEDLEN_VECTOR_VALUETYPE = v1i128,
+
+      v2f16          =  63,   //    2 x f16
+      v3f16          =  64,   //    3 x f16
+      v4f16          =  65,   //    4 x f16
+      v8f16          =  66,   //    8 x f16
+      v16f16         =  67,   //   16 x f16
+      v32f16         =  68,   //   32 x f16
+      v1f32          =  69,   //    1 x f32
+      v2f32          =  70,   //    2 x f32
+      v3f32          =  71,   //    3 x f32
+      v4f32          =  72,   //    4 x f32
+      v5f32          =  73,   //    5 x f32
+      v8f32          =  74,   //    8 x f32
+      v16f32         =  75,   //   16 x f32
+      v32f32         =  76,   //   32 x f32
+      v64f32         =  77,   //   64 x f32
+      v128f32        =  78,   //  128 x f32
+      v256f32        =  79,   //  256 x f32
+      v512f32        =  80,   //  512 x f32
+      v1024f32       =  81,   // 1024 x f32
+      v2048f32       =  82,   // 2048 x f32
+      v1f64          =  83,   //    1 x f64
+      v2f64          =  84,   //    2 x f64
+      v4f64          =  85,   //    4 x f64
+      v8f64          =  86,   //    8 x f64
+
+      FIRST_FP_FIXEDLEN_VECTOR_VALUETYPE = v2f16,
+      LAST_FP_FIXEDLEN_VECTOR_VALUETYPE = v8f64,
+
+      FIRST_FIXEDLEN_VECTOR_VALUETYPE = v1i1,
+      LAST_FIXEDLEN_VECTOR_VALUETYPE = v8f64,
+
+      nxv1i1         =  87,   // n x  1 x i1
+      nxv2i1         =  88,   // n x  2 x i1
+      nxv4i1         =  89,   // n x  4 x i1
+      nxv8i1         =  90,   // n x  8 x i1
+      nxv16i1        =  91,   // n x 16 x i1
+      nxv32i1        =  92,   // n x 32 x i1
+
+      nxv1i8         =  93,   // n x  1 x i8
+      nxv2i8         =  94,   // n x  2 x i8
+      nxv4i8         =  95,   // n x  4 x i8
+      nxv8i8         =  96,   // n x  8 x i8
+      nxv16i8        =  97,   // n x 16 x i8
+      nxv32i8        =  98,   // n x 32 x i8
+
+      nxv1i16        =  99,   // n x  1 x i16
+      nxv2i16        =  100,  // n x  2 x i16
+      nxv4i16        =  101,  // n x  4 x i16
+      nxv8i16        =  102,  // n x  8 x i16
+      nxv16i16       =  103,  // n x 16 x i16
+      nxv32i16       =  104,  // n x 32 x i16
+
+      nxv1i32        =  105,  // n x  1 x i32
+      nxv2i32        =  106,  // n x  2 x i32
+      nxv4i32        =  107,  // n x  4 x i32
+      nxv8i32        =  108,  // n x  8 x i32
+      nxv16i32       =  109,  // n x 16 x i32
+      nxv32i32       =  110,  // n x 32 x i32
+
+      nxv1i64        =  111,  // n x  1 x i64
+      nxv2i64        =  112,  // n x  2 x i64
+      nxv4i64        =  113,  // n x  4 x i64
+      nxv8i64        =  114,  // n x  8 x i64
+      nxv16i64       =  115,  // n x 16 x i64
+      nxv32i64       =  116,  // n x 32 x i64
+
+      FIRST_INTEGER_SCALABLE_VECTOR_VALUETYPE = nxv1i1,
+      LAST_INTEGER_SCALABLE_VECTOR_VALUETYPE = nxv32i64,
 
       nxv2f16        =  117,  // n x  2 x f16
       nxv4f16        =  118,  // n x  4 x f16
       nxv8f16        =  119,  // n x  8 x f16
       nxv1f32        =  120,  // n x  1 x f32
       nxv2f32        =  121,  // n x  2 x f32
       nxv4f32        =  122,  // n x  4 x f32
       nxv8f32        =  123,  // n x  8 x f32
       nxv16f32       =  124,  // n x 16 x f32
       nxv1f64        =  125,  // n x  1 x f64
       nxv2f64        =  126,  // n x  2 x f64
       nxv4f64        =  127,  // n x  4 x f64
       nxv8f64        =  128,  // n x  8 x f64
 
-      FIRST_FP_VECTOR_VALUETYPE = v2f16,
-      LAST_FP_VECTOR_VALUETYPE = nxv8f64,
+      FIRST_FP_SCALABLE_VECTOR_VALUETYPE = nxv2f16,
+      LAST_FP_SCALABLE_VECTOR_VALUETYPE = nxv8f64,
 
-      FIRST_FP_SCALABLE_VALUETYPE = nxv2f16,
-      LAST_FP_SCALABLE_VALUETYPE = nxv8f64,
+      FIRST_SCALABLE_VECTOR_VALUETYPE = nxv1i1,
+      LAST_SCALABLE_VECTOR_VALUETYPE = nxv8f64,
 
       FIRST_VECTOR_VALUETYPE = v1i1,
       LAST_VECTOR_VALUETYPE  = nxv8f64,
 
       x86mmx         =  129,   // This is an X86 MMX value
 
       Glue           =  130,   // This glues nodes together during pre-RA sched
 
bool isValid() const {
       return (SimpleTy >= MVT::FIRST_VALUETYPE &&
               SimpleTy < MVT::LAST_VALUETYPE);
     }
 
     /// Return true if this is a FP or a vector FP type.
     bool isFloatingPoint() const {
       return ((SimpleTy >= MVT::FIRST_FP_VALUETYPE &&
                SimpleTy <= MVT::LAST_FP_VALUETYPE) ||
-              (SimpleTy >= MVT::FIRST_FP_VECTOR_VALUETYPE &&
-               SimpleTy <= MVT::LAST_FP_VECTOR_VALUETYPE));
+              (SimpleTy >= MVT::FIRST_FP_FIXEDLEN_VECTOR_VALUETYPE &&
+               SimpleTy <= MVT::LAST_FP_FIXEDLEN_VECTOR_VALUETYPE) ||
+              (SimpleTy >= MVT::FIRST_FP_SCALABLE_VECTOR_VALUETYPE &&
+               SimpleTy <= MVT::LAST_FP_SCALABLE_VECTOR_VALUETYPE));
     }
 
     /// Return true if this is an integer or a vector integer type.
     bool isInteger() const {
       return ((SimpleTy >= MVT::FIRST_INTEGER_VALUETYPE &&
                SimpleTy <= MVT::LAST_INTEGER_VALUETYPE) ||
-              (SimpleTy >= MVT::FIRST_INTEGER_VECTOR_VALUETYPE &&
-               SimpleTy <= MVT::LAST_INTEGER_VECTOR_VALUETYPE));
+              (SimpleTy >= MVT::FIRST_INTEGER_FIXEDLEN_VECTOR_VALUETYPE &&
+               SimpleTy <= MVT::LAST_INTEGER_FIXEDLEN_VECTOR_VALUETYPE) ||
+              (SimpleTy >= MVT::FIRST_INTEGER_SCALABLE_VECTOR_VALUETYPE &&
+               SimpleTy <= MVT::LAST_INTEGER_SCALABLE_VECTOR_VALUETYPE));
     }
 
     /// Return true if this is an integer, not including vectors.
     bool isScalarInteger() const {
       return (SimpleTy >= MVT::FIRST_INTEGER_VALUETYPE &&
               SimpleTy <= MVT::LAST_INTEGER_VALUETYPE);
     }
 
     /// Return true if this is a vector value type.
     bool isVector() const {
       return (SimpleTy >= MVT::FIRST_VECTOR_VALUETYPE &&
               SimpleTy <= MVT::LAST_VECTOR_VALUETYPE);
     }
 
     /// Return true if this is a vector value type where the
     /// runtime length is machine dependent
     bool isScalableVector() const {
-      return ((SimpleTy >= MVT::FIRST_INTEGER_SCALABLE_VALUETYPE &&
-               SimpleTy <= MVT::LAST_INTEGER_SCALABLE_VALUETYPE) ||
-              (SimpleTy >= MVT::FIRST_FP_SCALABLE_VALUETYPE &&
-               SimpleTy <= MVT::LAST_FP_SCALABLE_VALUETYPE));
+      return (SimpleTy >= MVT::FIRST_SCALABLE_VECTOR_VALUETYPE &&
+              SimpleTy <= MVT::LAST_SCALABLE_VECTOR_VALUETYPE);
+    }
+
+    bool isFixedLengthVector() const {
+      return (SimpleTy >= MVT::FIRST_FIXEDLEN_VECTOR_VALUETYPE &&
+              SimpleTy <= MVT::LAST_FIXEDLEN_VECTOR_VALUETYPE);
     }
 
     /// Return true if this is a 16-bit vector type.
     bool is16BitVector() const {
       return (SimpleTy == MVT::v2i8  || SimpleTy == MVT::v1i16 ||
               SimpleTy == MVT::v16i1);
     }
 
static mvt_range fp_valuetypes() {
                        (MVT::SimpleValueType)(MVT::LAST_FP_VALUETYPE + 1));
     }
 
     static mvt_range vector_valuetypes() {
       return mvt_range(MVT::FIRST_VECTOR_VALUETYPE,
                        (MVT::SimpleValueType)(MVT::LAST_VECTOR_VALUETYPE + 1));
     }
 
-    static mvt_range integer_vector_valuetypes() {
+    static mvt_range fixedlen_vector_valuetypes() {
+      return mvt_range(
+               MVT::FIRST_FIXEDLEN_VECTOR_VALUETYPE,
+               (MVT::SimpleValueType)(MVT::LAST_FIXEDLEN_VECTOR_VALUETYPE + 1));
+    }
+
+    static mvt_range scalable_vector_valuetypes() {
+      return mvt_range(
+               MVT::FIRST_SCALABLE_VECTOR_VALUETYPE,
+               (MVT::SimpleValueType)(MVT::LAST_SCALABLE_VECTOR_VALUETYPE + 1));
+    }
+
+    static mvt_range integer_fixedlen_vector_valuetypes() {
       return mvt_range(
-          MVT::FIRST_INTEGER_VECTOR_VALUETYPE,
-          (MVT::SimpleValueType)(MVT::LAST_INTEGER_VECTOR_VALUETYPE + 1));
+       MVT::FIRST_INTEGER_FIXEDLEN_VECTOR_VALUETYPE,
+       (MVT::SimpleValueType)(MVT::LAST_INTEGER_FIXEDLEN_VECTOR_VALUETYPE + 1));
     }
 
-    static mvt_range fp_vector_valuetypes() {
+    static mvt_range fp_fixedlen_vector_valuetypes() {
       return mvt_range(
-          MVT::FIRST_FP_VECTOR_VALUETYPE,
-          (MVT::SimpleValueType)(MVT::LAST_FP_VECTOR_VALUETYPE + 1));
+          MVT::FIRST_FP_FIXEDLEN_VECTOR_VALUETYPE,
+          (MVT::SimpleValueType)(MVT::LAST_FP_FIXEDLEN_VECTOR_VALUETYPE + 1));
     }
 
     static mvt_range integer_scalable_vector_valuetypes() {
-      return mvt_range(MVT::FIRST_INTEGER_SCALABLE_VALUETYPE,
-              (MVT::SimpleValueType)(MVT::LAST_INTEGER_SCALABLE_VALUETYPE + 1));
+      return mvt_range(
+       MVT::FIRST_INTEGER_SCALABLE_VECTOR_VALUETYPE,
+       (MVT::SimpleValueType)(MVT::LAST_INTEGER_SCALABLE_VECTOR_VALUETYPE + 1));
     }
 
     static mvt_range fp_scalable_vector_valuetypes() {
-      return mvt_range(MVT::FIRST_FP_SCALABLE_VALUETYPE,
-                   (MVT::SimpleValueType)(MVT::LAST_FP_SCALABLE_VALUETYPE + 1));
+      return mvt_range(
+            MVT::FIRST_FP_SCALABLE_VECTOR_VALUETYPE,
+            (MVT::SimpleValueType)(MVT::LAST_FP_SCALABLE_VECTOR_VALUETYPE + 1));
     }
     /// @}
   };
 
 } // end namespace llvm
 
 #endif // LLVM_CODEGEN_MACHINEVALUETYPE_H

llvm/trunk/lib/CodeGen/TargetLoweringBase.cpp

for (unsigned i = MVT::FIRST_VECTOR_VALUETYPE;
     MVT EltVT = VT.getVectorElementType();
     unsigned NElts = VT.getVectorNumElements();
     bool IsLegalWiderType = false;
     LegalizeTypeAction PreferredAction = getPreferredVectorAction(VT);
     switch (PreferredAction) {
     case TypePromoteInteger:
       // Try to promote the elements of integer vectors. If no legal
       // promotion was found, fall through to the widen-vector method.
-      for (unsigned nVT = i + 1; nVT <= MVT::LAST_INTEGER_VECTOR_VALUETYPE; ++nVT) {
+      for (unsigned nVT = i + 1;
+           nVT <= MVT::LAST_INTEGER_FIXEDLEN_VECTOR_VALUETYPE; ++nVT) {
         MVT SVT = (MVT::SimpleValueType) nVT;
         // Promote vectors of integers to vectors with the same number
         // of elements, with a wider element type.
         if (SVT.getScalarSizeInBits() > EltVT.getSizeInBits() &&
             SVT.getVectorNumElements() == NElts && isTypeLegal(SVT)) {
           TransformToType[i] = SVT;
           RegisterTypeForVT[i] = SVT;
           NumRegistersForVT[i] = 1;

llvm/trunk/lib/Target/AArch64/AArch64ISelLowering.cpp

AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM,
   setOperationAction(ISD::ADDC, MVT::i64, Custom);
   setOperationAction(ISD::ADDE, MVT::i64, Custom);
   setOperationAction(ISD::SUBC, MVT::i64, Custom);
   setOperationAction(ISD::SUBE, MVT::i64, Custom);
 
   // AArch64 lacks both left-rotate and popcount instructions.
   setOperationAction(ISD::ROTL, MVT::i32, Expand);
   setOperationAction(ISD::ROTL, MVT::i64, Expand);
-  for (MVT VT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     setOperationAction(ISD::ROTL, VT, Expand);
     setOperationAction(ISD::ROTR, VT, Expand);
   }
 
   // AArch64 doesn't have {U|S}MUL_LOHI.
   setOperationAction(ISD::UMUL_LOHI, MVT::i64, Expand);
   setOperationAction(ISD::SMUL_LOHI, MVT::i64, Expand);
 
   setOperationAction(ISD::CTPOP, MVT::i32, Custom);
   setOperationAction(ISD::CTPOP, MVT::i64, Custom);
 
   setOperationAction(ISD::SDIVREM, MVT::i32, Expand);
   setOperationAction(ISD::SDIVREM, MVT::i64, Expand);
-  for (MVT VT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     setOperationAction(ISD::SDIVREM, VT, Expand);
     setOperationAction(ISD::UDIVREM, VT, Expand);
   }
   setOperationAction(ISD::SREM, MVT::i32, Expand);
   setOperationAction(ISD::SREM, MVT::i64, Expand);
   setOperationAction(ISD::UDIVREM, MVT::i32, Expand);
   setOperationAction(ISD::UDIVREM, MVT::i64, Expand);
   setOperationAction(ISD::UREM, MVT::i32, Expand);
for (MVT VT : { MVT::v4f16, MVT::v2f32,
       setOperationAction(ISD::VECREDUCE_FMAX, VT, Custom);
       setOperationAction(ISD::VECREDUCE_FMIN, VT, Custom);
     }
 
     setOperationAction(ISD::ANY_EXTEND, MVT::v4i32, Legal);
     setTruncStoreAction(MVT::v2i32, MVT::v2i16, Expand);
     // Likewise, narrowing and extending vector loads/stores aren't handled
     // directly.
-    for (MVT VT : MVT::vector_valuetypes()) {
+    for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
       setOperationAction(ISD::SIGN_EXTEND_INREG, VT, Expand);
 
       if (VT == MVT::v16i8 || VT == MVT::v8i16 || VT == MVT::v4i32) {
         setOperationAction(ISD::MULHS, VT, Legal);
         setOperationAction(ISD::MULHU, VT, Legal);
       } else {
         setOperationAction(ISD::MULHS, VT, Expand);
         setOperationAction(ISD::MULHU, VT, Expand);
       }
       setOperationAction(ISD::SMUL_LOHI, VT, Expand);
       setOperationAction(ISD::UMUL_LOHI, VT, Expand);
 
       setOperationAction(ISD::BSWAP, VT, Expand);
       setOperationAction(ISD::CTTZ, VT, Expand);
 
-      for (MVT InnerVT : MVT::vector_valuetypes()) {
+      for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) {
         setTruncStoreAction(VT, InnerVT, Expand);
         setLoadExtAction(ISD::SEXTLOAD, VT, InnerVT, Expand);
         setLoadExtAction(ISD::ZEXTLOAD, VT, InnerVT, Expand);
         setLoadExtAction(ISD::EXTLOAD, VT, InnerVT, Expand);
       }
     }
 
     // AArch64 has implementations of a lot of rounding-like FP operations.

llvm/trunk/lib/Target/AMDGPU/AMDGPUISelLowering.cpp

for (MVT VT : MVT::integer_valuetypes()) {
     setLoadExtAction(ISD::ZEXTLOAD, VT, MVT::i32, Expand);
 
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::i1, Promote);
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::i8, Legal);
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::i16, Legal);
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::i32, Expand);
   }
 
-  for (MVT VT : MVT::integer_vector_valuetypes()) {
+  for (MVT VT : MVT::integer_fixedlen_vector_valuetypes()) {
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::v2i8, Expand);
     setLoadExtAction(ISD::SEXTLOAD, VT, MVT::v2i8, Expand);
     setLoadExtAction(ISD::ZEXTLOAD, VT, MVT::v2i8, Expand);
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::v4i8, Expand);
     setLoadExtAction(ISD::SEXTLOAD, VT, MVT::v4i8, Expand);
     setLoadExtAction(ISD::ZEXTLOAD, VT, MVT::v4i8, Expand);
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::v2i16, Expand);
     setLoadExtAction(ISD::SEXTLOAD, VT, MVT::v2i16, Expand);

llvm/trunk/lib/Target/ARM/ARMISelLowering.cpp

if (Subtarget->hasFullFP16()) {
     setOperationAction(ISD::BITCAST, MVT::i16, Custom);
     setOperationAction(ISD::BITCAST, MVT::i32, Custom);
     setOperationAction(ISD::BITCAST, MVT::f16, Custom);
 
     setOperationAction(ISD::FMINNUM, MVT::f16, Legal);
     setOperationAction(ISD::FMAXNUM, MVT::f16, Legal);
   }
 
-  for (MVT VT : MVT::vector_valuetypes()) {
-    for (MVT InnerVT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
+    for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) {
       setTruncStoreAction(VT, InnerVT, Expand);
       addAllExtLoads(VT, InnerVT, Expand);
     }
 
     setOperationAction(ISD::MULHS, VT, Expand);
     setOperationAction(ISD::SMUL_LOHI, VT, Expand);
     setOperationAction(ISD::MULHU, VT, Expand);
     setOperationAction(ISD::UMUL_LOHI, VT, Expand);
if (Subtarget->hasNEON()) {
     setTargetDAGCombine(ISD::FP_TO_SINT);
     setTargetDAGCombine(ISD::FP_TO_UINT);
     setTargetDAGCombine(ISD::FDIV);
     setTargetDAGCombine(ISD::LOAD);
 
     // It is legal to extload from v4i8 to v4i16 or v4i32.
     for (MVT Ty : {MVT::v8i8, MVT::v4i8, MVT::v2i8, MVT::v4i16, MVT::v2i16,
                    MVT::v2i32}) {
-      for (MVT VT : MVT::integer_vector_valuetypes()) {
+      for (MVT VT : MVT::integer_fixedlen_vector_valuetypes()) {
         setLoadExtAction(ISD::EXTLOAD, VT, Ty, Legal);
         setLoadExtAction(ISD::ZEXTLOAD, VT, Ty, Legal);
         setLoadExtAction(ISD::SEXTLOAD, VT, Ty, Legal);
       }
     }
   }
 
   if (Subtarget->hasNEON() || Subtarget->hasMVEIntegerOps()) {
if (Subtarget->isThumb1Only()) {
     setOperationAction(ISD::SRL_PARTS, MVT::i32, Expand);
   }
 
   if (!Subtarget->isThumb1Only() && Subtarget->hasV6T2Ops())
     setOperationAction(ISD::BITREVERSE, MVT::i32, Legal);
 
   // ARM does not have ROTL.
   setOperationAction(ISD::ROTL, MVT::i32, Expand);
-  for (MVT VT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     setOperationAction(ISD::ROTL, VT, Expand);
     setOperationAction(ISD::ROTR, VT, Expand);
   }
   setOperationAction(ISD::CTTZ,  MVT::i32, Custom);
   setOperationAction(ISD::CTPOP, MVT::i32, Expand);
   if (!Subtarget->hasV5TOps() || Subtarget->isThumb1Only()) {
     setOperationAction(ISD::CTLZ, MVT::i32, Expand);
     setOperationAction(ISD::CTLZ_ZERO_UNDEF, MVT::i32, LibCall);

llvm/trunk/lib/Target/Hexagon/HexagonISelLowering.cpp

static const unsigned VectExpOps[] = {
     ISD::BR_CC,   ISD::SELECT_CC,             ISD::ConstantPool,
     // Vector:
     ISD::BUILD_VECTOR,          ISD::SCALAR_TO_VECTOR,
     ISD::EXTRACT_VECTOR_ELT,    ISD::INSERT_VECTOR_ELT,
     ISD::EXTRACT_SUBVECTOR,     ISD::INSERT_SUBVECTOR,
     ISD::CONCAT_VECTORS,        ISD::VECTOR_SHUFFLE
   };
 
-  for (MVT VT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     for (unsigned VectExpOp : VectExpOps)
       setOperationAction(VectExpOp, VT, Expand);
 
     // Expand all extending loads and truncating stores:
-    for (MVT TargetVT : MVT::vector_valuetypes()) {
+    for (MVT TargetVT : MVT::fixedlen_vector_valuetypes()) {
       if (TargetVT == VT)
         continue;
       setLoadExtAction(ISD::EXTLOAD, TargetVT, VT, Expand);
       setLoadExtAction(ISD::ZEXTLOAD, TargetVT, VT, Expand);
       setLoadExtAction(ISD::SEXTLOAD, TargetVT, VT, Expand);
       setTruncStoreAction(VT, TargetVT, Expand);
     }
 

 
 bool HexagonTargetLowering::isShuffleMaskLegal(ArrayRef<int> Mask,
                                                EVT VT) const {
   return true;
 }
 
 TargetLoweringBase::LegalizeTypeAction
 HexagonTargetLowering::getPreferredVectorAction(MVT VT) const {
-  if (VT.getVectorNumElements() == 1)
+  if (VT.getVectorNumElements() == 1 || VT.isScalableVector())
     return TargetLoweringBase::TypeScalarizeVector;
 
   // Always widen vectors of i1.
   MVT ElemTy = VT.getVectorElementType();
   if (ElemTy == MVT::i1)
     return TargetLoweringBase::TypeWidenVector;
 
   if (Subtarget.useHVXOps()) {

llvm/trunk/lib/Target/Hexagon/HexagonSubtarget.h

public:
   }
 
   ArrayRef<MVT> getHVXElementTypes() const {
     static MVT Types[] = { MVT::i8, MVT::i16, MVT::i32 };
     return makeArrayRef(Types);
   }
 
   bool isHVXVectorType(MVT VecTy, bool IncludeBool = false) const {
-    if (!VecTy.isVector() || !useHVXOps())
+    if (!VecTy.isVector() || !useHVXOps() || VecTy.isScalableVector())
       return false;
     MVT ElemTy = VecTy.getVectorElementType();
     if (!IncludeBool && ElemTy == MVT::i1)
       return false;
 
     unsigned HwLen = getVectorLength();
     unsigned NumElems = VecTy.getVectorNumElements();
     ArrayRef<MVT> ElemTypes = getHVXElementTypes();

llvm/trunk/lib/Target/Hexagon/HexagonTargetTransformInfo.cpp

 static const unsigned FloatFactor = 4;
 
 bool HexagonTTIImpl::useHVX() const {
   return ST.useHVXOps() && HexagonAutoHVX;
 }
 
 bool HexagonTTIImpl::isTypeForHVX(Type *VecTy) const {
   assert(VecTy->isVectorTy());
+  if (cast<VectorType>(VecTy)->isScalable())
+    return false;
   // Avoid types like <2 x i32*>.
   if (!cast<VectorType>(VecTy)->getElementType()->isIntegerTy())
     return false;
   EVT VecVT = EVT::getEVT(VecTy);
   if (!VecVT.isSimple() || VecVT.getSizeInBits() <= 64)
     return false;
   if (ST.isHVXVectorType(VecVT.getSimpleVT()))
     return true;

llvm/trunk/lib/Target/Mips/MipsISelLowering.cpp

MipsTargetLowering::MipsTargetLowering(const MipsTargetMachine &TM,
   // MIPS doesn't have extending float->double load/store.  Set LoadExtAction
   // for f32, f16
   for (MVT VT : MVT::fp_valuetypes()) {
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::f32, Expand);
     setLoadExtAction(ISD::EXTLOAD, VT, MVT::f16, Expand);
   }
 
   // Set LoadExtAction for f16 vectors to Expand
-  for (MVT VT : MVT::fp_vector_valuetypes()) {
+  for (MVT VT : MVT::fp_fixedlen_vector_valuetypes()) {
     MVT F16VT = MVT::getVectorVT(MVT::f16, VT.getVectorNumElements());
     if (F16VT.isValid())
       setLoadExtAction(ISD::EXTLOAD, VT, F16VT, Expand);
   }
 
   setTruncStoreAction(MVT::f32, MVT::f16, Expand);
   setTruncStoreAction(MVT::f64, MVT::f16, Expand);
 

llvm/trunk/lib/Target/Mips/MipsSEISelLowering.cpp

MipsSETargetLowering::MipsSETargetLowering(const MipsTargetMachine &TM,
   // Set up the register classes
   addRegisterClass(MVT::i32, &Mips::GPR32RegClass);
 
   if (Subtarget.isGP64bit())
     addRegisterClass(MVT::i64, &Mips::GPR64RegClass);
 
   if (Subtarget.hasDSP() || Subtarget.hasMSA()) {
     // Expand all truncating stores and extending loads.
-    for (MVT VT0 : MVT::vector_valuetypes()) {
-      for (MVT VT1 : MVT::vector_valuetypes()) {
+    for (MVT VT0 : MVT::fixedlen_vector_valuetypes()) {
+      for (MVT VT1 : MVT::fixedlen_vector_valuetypes()) {
         setTruncStoreAction(VT0, VT1, Expand);
         setLoadExtAction(ISD::SEXTLOAD, VT0, VT1, Expand);
         setLoadExtAction(ISD::ZEXTLOAD, VT0, VT1, Expand);
         setLoadExtAction(ISD::EXTLOAD, VT0, VT1, Expand);
       }
     }
   }
 

llvm/trunk/lib/Target/NVPTX/NVPTXISelLowering.cpp

NVPTXTargetLowering::NVPTXTargetLowering(const NVPTXTargetMachine &TM,
   setOperationAction(ISD::ConstantFP, MVT::f64, Legal);
   setOperationAction(ISD::ConstantFP, MVT::f32, Legal);
   setOperationAction(ISD::ConstantFP, MVT::f16, Legal);
 
   // TRAP can be lowered to PTX trap
   setOperationAction(ISD::TRAP, MVT::Other, Legal);
 
   // Register custom handling for vector loads/stores
-  for (MVT VT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     if (IsPTXVectorType(VT)) {
       setOperationAction(ISD::LOAD, VT, Custom);
       setOperationAction(ISD::STORE, VT, Custom);
       setOperationAction(ISD::INTRINSIC_W_CHAIN, VT, Custom);
     }
   }
 
   // Custom handling for i8 intrinsics

llvm/trunk/lib/Target/PowerPC/PPCISelLowering.cpp

if (Subtarget.use64BitRegs()) {
     setOperationAction(ISD::SHL_PARTS, MVT::i32, Custom);
     setOperationAction(ISD::SRA_PARTS, MVT::i32, Custom);
     setOperationAction(ISD::SRL_PARTS, MVT::i32, Custom);
   }
 
   if (Subtarget.hasAltivec()) {
     // First set operation action for all vector types to expand. Then we
     // will selectively turn on ones that can be effectively codegen'd.
-    for (MVT VT : MVT::vector_valuetypes()) {
+    for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
       // add/sub are legal for all supported vector VT's.
       setOperationAction(ISD::ADD, VT, Legal);
       setOperationAction(ISD::SUB, VT, Legal);
 
       // For v2i64, these are only valid with P8Vector. This is corrected after
       // the loop.
       if (VT.getSizeInBits() <= 128 && VT.getScalarSizeInBits() <= 64) {
         setOperationAction(ISD::SMAX, VT, Legal);
for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
       setOperationAction(ISD::SDIVREM, VT, Expand);
       setOperationAction(ISD::SCALAR_TO_VECTOR, VT, Expand);
       setOperationAction(ISD::FPOW, VT, Expand);
       setOperationAction(ISD::BSWAP, VT, Expand);
       setOperationAction(ISD::SIGN_EXTEND_INREG, VT, Expand);
       setOperationAction(ISD::ROTL, VT, Expand);
       setOperationAction(ISD::ROTR, VT, Expand);
 
-      for (MVT InnerVT : MVT::vector_valuetypes()) {
+      for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) {
         setTruncStoreAction(VT, InnerVT, Expand);
         setLoadExtAction(ISD::SEXTLOAD, VT, InnerVT, Expand);
         setLoadExtAction(ISD::ZEXTLOAD, VT, InnerVT, Expand);
         setLoadExtAction(ISD::EXTLOAD, VT, InnerVT, Expand);
       }
     }
     if (!Subtarget.hasP8Vector()) {
       setOperationAction(ISD::SMAX, MVT::v2i64, Expand);

llvm/trunk/lib/Target/Sparc/SparcISelLowering.cpp

if (Subtarget->is64Bit()) {
     addRegisterClass(MVT::v2i32, &SP::IntPairRegClass);
 
     // ...but almost all operations must be expanded, so set that as
     // the default.
     for (unsigned Op = 0; Op < ISD::BUILTIN_OP_END; ++Op) {
       setOperationAction(Op, MVT::v2i32, Expand);
     }
     // Truncating/extending stores/loads are also not supported.
-    for (MVT VT : MVT::integer_vector_valuetypes()) {
+    for (MVT VT : MVT::integer_fixedlen_vector_valuetypes()) {
       setLoadExtAction(ISD::SEXTLOAD, VT, MVT::v2i32, Expand);
       setLoadExtAction(ISD::ZEXTLOAD, VT, MVT::v2i32, Expand);
       setLoadExtAction(ISD::EXTLOAD, VT, MVT::v2i32, Expand);
 
       setLoadExtAction(ISD::SEXTLOAD, MVT::v2i32, VT, Expand);
       setLoadExtAction(ISD::ZEXTLOAD, MVT::v2i32, VT, Expand);
       setLoadExtAction(ISD::EXTLOAD, MVT::v2i32, VT, Expand);
 

llvm/trunk/lib/Target/SystemZ/SystemZISelLowering.cpp

SystemZTargetLowering::SystemZTargetLowering(const TargetMachine &TM,
   // Use custom expanders so that we can force the function to use
   // a frame pointer.
   setOperationAction(ISD::STACKSAVE,    MVT::Other, Custom);
   setOperationAction(ISD::STACKRESTORE, MVT::Other, Custom);
 
   // Handle prefetches with PFD or PFDRL.
   setOperationAction(ISD::PREFETCH, MVT::Other, Custom);
 
-  for (MVT VT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     // Assume by default that all vector operations need to be expanded.
     for (unsigned Opcode = 0; Opcode < ISD::BUILTIN_OP_END; ++Opcode)
       if (getOperationAction(Opcode, VT) == Legal)
         setOperationAction(Opcode, VT, Expand);
 
     // Likewise all truncating stores and extending loads.
-    for (MVT InnerVT : MVT::vector_valuetypes()) {
+    for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) {
       setTruncStoreAction(VT, InnerVT, Expand);
       setLoadExtAction(ISD::SEXTLOAD, VT, InnerVT, Expand);
       setLoadExtAction(ISD::ZEXTLOAD, VT, InnerVT, Expand);
       setLoadExtAction(ISD::EXTLOAD, VT, InnerVT, Expand);
     }
 
     if (isTypeLegal(VT)) {
       // These operations are legal for anything that can be stored in a
if (isTypeLegal(VT)) {
       // Likewise, except that we need to replace the nodes with something
       // more specific.
       setOperationAction(ISD::BUILD_VECTOR, VT, Custom);
       setOperationAction(ISD::VECTOR_SHUFFLE, VT, Custom);
     }
   }
 
   // Handle integer vector types.
-  for (MVT VT : MVT::integer_vector_valuetypes()) {
+  for (MVT VT : MVT::integer_fixedlen_vector_valuetypes()) {
     if (isTypeLegal(VT)) {
       // These operations have direct equivalents.
       setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Legal);
       setOperationAction(ISD::INSERT_VECTOR_ELT, VT, Legal);
       setOperationAction(ISD::ADD, VT, Legal);
       setOperationAction(ISD::SUB, VT, Legal);
       if (VT != MVT::v2i64)
         setOperationAction(ISD::MUL, VT, Legal);

llvm/trunk/lib/Target/WebAssembly/WebAssemblyISelLowering.cpp

WebAssemblyTargetLowering::WebAssemblyTargetLowering(
   // sign-extend from.
   setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i1, Expand);
   if (!Subtarget->hasSignExt()) {
     // Sign extends are legal only when extending a vector extract
     auto Action = Subtarget->hasSIMD128() ? Custom : Expand;
     for (auto T : {MVT::i8, MVT::i16, MVT::i32})
       setOperationAction(ISD::SIGN_EXTEND_INREG, T, Action);
   }
-  for (auto T : MVT::integer_vector_valuetypes())
+  for (auto T : MVT::integer_fixedlen_vector_valuetypes())
     setOperationAction(ISD::SIGN_EXTEND_INREG, T, Expand);
 
   // Dynamic stack allocation: use the default expansion.
   setOperationAction(ISD::STACKSAVE, MVT::Other, Expand);
   setOperationAction(ISD::STACKRESTORE, MVT::Other, Expand);
   setOperationAction(ISD::DYNAMIC_STACKALLOC, MVTPtr, Expand);
 
   setOperationAction(ISD::FrameIndex, MVT::i32, Custom);
WebAssemblyTargetLowering::WebAssemblyTargetLowering(
   setLoadExtAction(ISD::EXTLOAD, MVT::f64, MVT::f32, Expand);
   setTruncStoreAction(MVT::f64, MVT::f32, Expand);
   for (auto T : MVT::integer_valuetypes())
     for (auto Ext : {ISD::EXTLOAD, ISD::ZEXTLOAD, ISD::SEXTLOAD})
       setLoadExtAction(Ext, T, MVT::i1, Promote);
   if (Subtarget->hasSIMD128()) {
     for (auto T : {MVT::v16i8, MVT::v8i16, MVT::v4i32, MVT::v2i64, MVT::v4f32,
                    MVT::v2f64}) {
-      for (auto MemT : MVT::vector_valuetypes()) {
+      for (auto MemT : MVT::fixedlen_vector_valuetypes()) {
         if (MVT(T) != MemT) {
           setTruncStoreAction(T, MemT, Expand);
           for (auto Ext : {ISD::EXTLOAD, ISD::ZEXTLOAD, ISD::SEXTLOAD})
             setLoadExtAction(Ext, T, MemT, Expand);
         }
       }
     }
   }

llvm/trunk/lib/Target/X86/X86ISelLowering.cpp

for (auto VT : { MVT::v4f32, MVT::v8f32, MVT::v16f32,
     setOperationAction(ISD::FLOG10,    VT, Expand);
     setOperationAction(ISD::FEXP,      VT, Expand);
     setOperationAction(ISD::FEXP2,     VT, Expand);
   }
 
   // First set operation action for all vector types to either promote
   // (for widening) or expand (for scalarization). Then we will selectively
   // turn on ones that can be effectively codegen'd.
-  for (MVT VT : MVT::vector_valuetypes()) {
+  for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     setOperationAction(ISD::SDIV, VT, Expand);
     setOperationAction(ISD::UDIV, VT, Expand);
     setOperationAction(ISD::SREM, VT, Expand);
     setOperationAction(ISD::UREM, VT, Expand);
     setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT,Expand);
     setOperationAction(ISD::INSERT_VECTOR_ELT, VT, Expand);
     setOperationAction(ISD::EXTRACT_SUBVECTOR, VT,Expand);
     setOperationAction(ISD::INSERT_SUBVECTOR, VT,Expand);
for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
     setOperationAction(ISD::UINT_TO_FP, VT, Expand);
     setOperationAction(ISD::SINT_TO_FP, VT, Expand);
     setOperationAction(ISD::SIGN_EXTEND_INREG, VT,Expand);
     setOperationAction(ISD::TRUNCATE, VT, Expand);
     setOperationAction(ISD::SIGN_EXTEND, VT, Expand);
     setOperationAction(ISD::ZERO_EXTEND, VT, Expand);
     setOperationAction(ISD::ANY_EXTEND, VT, Expand);
     setOperationAction(ISD::SELECT_CC, VT, Expand);
-    for (MVT InnerVT : MVT::vector_valuetypes()) {
+    for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) {
       setTruncStoreAction(InnerVT, VT, Expand);
 
       setLoadExtAction(ISD::SEXTLOAD, InnerVT, VT, Expand);
       setLoadExtAction(ISD::ZEXTLOAD, InnerVT, VT, Expand);
 
       // N.b. ISD::EXTLOAD legality is basically ignored except for i1-like
       // types, we have to deal with them whether we ask for Expansion or not.
       // Setting Expand causes its own optimisation problems though, so leave
for (auto VT : { MVT::v16i8, MVT::v8i16, MVT::v4i32 }) {
       setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Custom);
     }
 
     // We support custom legalizing of sext and anyext loads for specific
     // memory vector types which we can load as a scalar (or sequence of
     // scalars) and extend in-register to a legal 128-bit vector type. For sext
     // loads these must work with a single scalar load.
     if (!ExperimentalVectorWideningLegalization) {
-      for (MVT VT : MVT::integer_vector_valuetypes()) {
+      for (MVT VT : MVT::integer_fixedlen_vector_valuetypes()) {
         setLoadExtAction(ISD::EXTLOAD, VT, MVT::v2i8, Custom);
         setLoadExtAction(ISD::EXTLOAD, VT, MVT::v2i16, Custom);
         setLoadExtAction(ISD::EXTLOAD, VT, MVT::v2i32, Custom);
         setLoadExtAction(ISD::EXTLOAD, VT, MVT::v4i8, Custom);
         setLoadExtAction(ISD::EXTLOAD, VT, MVT::v4i16, Custom);
         setLoadExtAction(ISD::EXTLOAD, VT, MVT::v8i8, Custom);
       }
     }
if (!Subtarget.useSoftFloat() && Subtarget.hasSSE41()) {
     for (auto VT : { MVT::v8i16, MVT::v4i32, MVT::v2i64 }) {
       setOperationAction(ISD::SIGN_EXTEND_VECTOR_INREG, VT, Legal);
       setOperationAction(ISD::ZERO_EXTEND_VECTOR_INREG, VT, Legal);
     }
 
     if (!ExperimentalVectorWideningLegalization) {
       // Avoid narrow result types when widening. The legal types are listed
       // in the next loop.
-      for (MVT VT : MVT::integer_vector_valuetypes()) {
+      for (MVT VT : MVT::integer_fixedlen_vector_valuetypes()) {
         setLoadExtAction(ISD::SEXTLOAD, VT, MVT::v2i8, Custom);
         setLoadExtAction(ISD::SEXTLOAD, VT, MVT::v2i16, Custom);
         setLoadExtAction(ISD::SEXTLOAD, VT, MVT::v2i32, Custom);
       }
     }
 
     // SSE41 also has vector sign/zero extending loads, PMOV[SZ]X
     for (auto LoadExtOp : { ISD::SEXTLOAD, ISD::ZEXTLOAD }) {

llvm/trunk/utils/TableGen/CodeGenDAGPatterns.cpp

for (MVT Ov : Ovs) {
     switch (Ov.SimpleTy) {
       case MVT::iPTRAny:
         Out.insert(MVT::iPTR);
         return;
       case MVT::iAny:
         for (MVT T : MVT::integer_valuetypes())
           if (Legal.count(T))
             Out.insert(T);
-        for (MVT T : MVT::integer_vector_valuetypes())
+        for (MVT T : MVT::integer_fixedlen_vector_valuetypes())
+          if (Legal.count(T))
+            Out.insert(T);
+        for (MVT T : MVT::integer_scalable_vector_valuetypes())
           if (Legal.count(T))
             Out.insert(T);
         return;
       case MVT::fAny:
         for (MVT T : MVT::fp_valuetypes())
           if (Legal.count(T))
             Out.insert(T);
-        for (MVT T : MVT::fp_vector_valuetypes())
+        for (MVT T : MVT::fp_fixedlen_vector_valuetypes())
+          if (Legal.count(T))
+            Out.insert(T);
+        for (MVT T : MVT::fp_scalable_vector_valuetypes())
           if (Legal.count(T))
             Out.insert(T);
         return;
       case MVT::vAny:
         for (MVT T : MVT::vector_valuetypes())
           if (Legal.count(T))
             Out.insert(T);
         return;