[AArch64][GlobalISel] Add G_EXT and select ext using it

[paquette]

Summary

Add selection support for ext via a new opcode, G_EXT and a post-legalizer combine which matches it.

Add an applyEXT function, because the AArch64ext patterns require a register for the immediate. So, we have to create a G_CONSTANT to get these without writing new patterns or modifying the existing ones.

Tests are the same as arm64-ext.ll.

For reference, here are the patterns we get with G_EXT:

(AArch64ext:{ *:[v1i64] } V64:{ *:[v1i64] }:$Rn, V64:{ *:[v1i64] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v1i64] } V64:{ *:[v1i64] }:$Rn, V64:{ *:[v1i64] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v1f64] } V64:{ *:[v1f64] }:$Rn, V64:{ *:[v1f64] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v1f64] } V64:{ *:[v1f64] }:$Rn, V64:{ *:[v1f64] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v2i32] } V64:{ *:[v2i32] }:$Rn, V64:{ *:[v2i32] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v2i32] } V64:{ *:[v2i32] }:$Rn, V64:{ *:[v2i32] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v2f32] } V64:{ *:[v2f32] }:$Rn, V64:{ *:[v2f32] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v2f32] } V64:{ *:[v2f32] }:$Rn, V64:{ *:[v2f32] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v2i64] } V128:{ *:[v2i64] }:$Rn, V128:{ *:[v2i64] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v2i64] } V128:{ *:[v2i64] }:$Rn, V128:{ *:[v2i64] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v2f64] } V128:{ *:[v2f64] }:$Rn, V128:{ *:[v2f64] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v2f64] } V128:{ *:[v2f64] }:$Rn, V128:{ *:[v2f64] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v4i16] } V64:{ *:[v4i16] }:$Rn, V64:{ *:[v4i16] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v4i16] } V64:{ *:[v4i16] }:$Rn, V64:{ *:[v4i16] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v4f16] } V64:{ *:[v4f16] }:$Rn, V64:{ *:[v4f16] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v4f16] } V64:{ *:[v4f16] }:$Rn, V64:{ *:[v4f16] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v4bf16] } V64:{ *:[v4bf16] }:$Rn, V64:{ *:[v4bf16] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v4bf16] } V64:{ *:[v4bf16] }:$Rn, V64:{ *:[v4bf16] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v4i32] } V128:{ *:[v4i32] }:$Rn, V128:{ *:[v4i32] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v4i32] } V128:{ *:[v4i32] }:$Rn, V128:{ *:[v4i32] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v4f32] } V128:{ *:[v4f32] }:$Rn, V128:{ *:[v4f32] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v4f32] } V128:{ *:[v4f32] }:$Rn, V128:{ *:[v4f32] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v8i8] } V64:{ *:[v8i8] }:$Rn, V64:{ *:[v8i8] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v8i8] } V64:{ *:[v8i8] }:$Rn, V64:{ *:[v8i8] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v8i8] } V64:{ *:[v8i8] }:$Rn, V64:{ *:[v8i8] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv8i8:{ *:[v8i8] } V64:{ *:[v8i8] }:$Rn, V64:{ *:[v8i8] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v8i16] } V128:{ *:[v8i16] }:$Rn, V128:{ *:[v8i16] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v8i16] } V128:{ *:[v8i16] }:$Rn, V128:{ *:[v8i16] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v8f16] } V128:{ *:[v8f16] }:$Rn, V128:{ *:[v8f16] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v8f16] } V128:{ *:[v8f16] }:$Rn, V128:{ *:[v8f16] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v8bf16] } V128:{ *:[v8bf16] }:$Rn, V128:{ *:[v8bf16] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v8bf16] } V128:{ *:[v8bf16] }:$Rn, V128:{ *:[v8bf16] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v16i8] } V128:{ *:[v16i8] }:$Rn, V128:{ *:[v16i8] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v16i8] } V128:{ *:[v16i8] }:$Rn, V128:{ *:[v16i8] }:$Rm, (imm:{ *:[i32] }):$imm)
(AArch64ext:{ *:[v16i8] } V128:{ *:[v16i8] }:$Rn, V128:{ *:[v16i8] }:$Rm, (imm:{ *:[i32] }):$imm)  =>  (EXTv16i8:{ *:[v16i8] } V128:{ *:[v16i8] }:$Rn, V128:{ *:[v16i8] }:$Rm, (imm:{ *:[i32] }):$imm)

[aemerson]

llvm/lib/Target/AArch64/GISel/AArch64PostLegalizerCombiner.cpp

355

You can just do:
auto Cst = IRBuilder.buildConstant(LLT::scalar(32), MatchInfo.SrcOps[2].getImm());

[aemerson]

llvm/lib/Target/AArch64/GISel/AArch64PostLegalizerCombiner.cpp

320

Tiny nit: if you're returning optional instead of bool unlike the DAG version, I think its better to name this getExtMask rather than isExtMask.

[paquette]

Rename function and address nit.

Also update a couple testcases I apparently forgot to run.

For the zip test, disable the ext combine. The ext combine has a higher priority than the zip combine, and G_EXT was being produced for a couple of the negative zip tests. Since those tests are supposed that we will not produce a G_ZIP in those situations, running the ext combine (and thus not exercising the code) is undesirable.

For the splat test, it was messed up because it was already regbankselected, and the ext combine introduces G_CONSTANTs.

[aemerson]

LGTM.

llvm/lib/Target/AArch64/AArch64Combine.td

 
 def trn : GICombineRule<
   (defs root:$root, shuffle_matchdata:$matchinfo),
   (match (wip_match_opcode G_SHUFFLE_VECTOR):$root,
          [{ return matchTRN(*${root}, MRI, ${matchinfo}); }]),
   (apply [{ applyShuffleVectorPseudo(*${root}, ${matchinfo}); }])
 >;
 
+def ext: GICombineRule <
+  (defs root:$root, shuffle_matchdata:$matchinfo),
+  (match (wip_match_opcode G_SHUFFLE_VECTOR):$root,
+         [{ return matchEXT(*${root}, MRI, ${matchinfo}); }]),
+  (apply [{ applyEXT(*${root}, ${matchinfo}); }])
+>;
+
 // Combines which replace a G_SHUFFLE_VECTOR with a target-specific pseudo
 // instruction.
-def shuffle_vector_pseudos : GICombineGroup<[dup, rev, zip, uzp, trn]>;
+def shuffle_vector_pseudos : GICombineGroup<[dup, rev, ext, zip, uzp, trn]>;
 
 def AArch64PostLegalizerCombinerHelper
     : GICombinerHelper<"AArch64GenPostLegalizerCombinerHelper",
                        [erase_undef_store, combines_for_extload,
                         shuffle_vector_pseudos]> {
   let DisableRuleOption = "aarch64postlegalizercombiner-disable-rule";
 }

llvm/lib/Target/AArch64/AArch64InstrGISel.td

 // Represents a trn2 instruction. Produced post-legalization from
 // G_SHUFFLE_VECTORs with appropriate masks.
 def G_TRN2 : AArch64GenericInstruction {
   let OutOperandList = (outs type0:$dst);
   let InOperandList = (ins type0:$v1, type0:$v2);
   let hasSideEffects = 0;
 }
 
+// Represents an ext instruction. Produced post-legalization from
+// G_SHUFFLE_VECTORs with appropriate masks.
+def G_EXT: AArch64GenericInstruction {
+  let OutOperandList = (outs type0:$dst);
+  let InOperandList = (ins type0:$v1, type0:$v2, untyped_imm_0:$imm);
+}
+
 def : GINodeEquiv<G_REV16, AArch64rev16>;
 def : GINodeEquiv<G_REV32, AArch64rev32>;
 def : GINodeEquiv<G_REV64, AArch64rev64>;
 def : GINodeEquiv<G_UZP1, AArch64uzp1>;
 def : GINodeEquiv<G_UZP2, AArch64uzp2>;
 def : GINodeEquiv<G_ZIP1, AArch64zip1>;
 def : GINodeEquiv<G_ZIP2, AArch64zip2>;
 def : GINodeEquiv<G_DUP, AArch64dup>;
 def : GINodeEquiv<G_TRN1, AArch64trn1>;
 def : GINodeEquiv<G_TRN2, AArch64trn2>;
+def : GINodeEquiv<G_EXT, AArch64ext>;

llvm/lib/Target/AArch64/GISel/AArch64PostLegalizerCombiner.cpp

for (unsigned i = 0; i < NumElts; i += 2) {
     if ((M[i] >= 0 && static_cast<unsigned>(M[i]) != i + WhichResult) ||
         (M[i + 1] >= 0 &&
          static_cast<unsigned>(M[i + 1]) != i + NumElts + WhichResult))
       return false;
   }
   return true;
 }
 
+/// Check if a G_EXT instruction can handle a shuffle mask \p M when the vector
+/// sources of the shuffle are different.
+static Optional<std::pair<bool, uint64_t>> getExtMask(ArrayRef<int> M,
+                                                      unsigned NumElts) {
+  // Look for the first non-undef element.
+  auto FirstRealElt = find_if(M, [](int Elt) { return Elt >= 0; });
+  if (FirstRealElt == M.end())
+    return None;
+
+  // Use APInt to handle overflow when calculating expected element.
+  unsigned MaskBits = APInt(32, NumElts * 2).logBase2();
+  APInt ExpectedElt = APInt(MaskBits, *FirstRealElt + 1);
+
+  // The following shuffle indices must be the successive elements after the
+  // first real element.
+  if (any_of(
+          make_range(std::next(FirstRealElt), M.end()),
+          [&ExpectedElt](int Elt) { return Elt != ExpectedElt++ && Elt >= 0; }))
+    return None;
+
+  // The index of an EXT is the first element if it is not UNDEF.
+  // Watch out for the beginning UNDEFs. The EXT index should be the expected
+  // value of the first element.  E.g.
+  // <-1, -1, 3, ...> is treated as <1, 2, 3, ...>.
+  // <-1, -1, 0, 1, ...> is treated as <2*NumElts-2, 2*NumElts-1, 0, 1, ...>.
+  // ExpectedElt is the last mask index plus 1.
+  uint64_t Imm = ExpectedElt.getZExtValue();
+  bool ReverseExt = false;
+
+  // There are two difference cases requiring to reverse input vectors.
+  // For example, for vector <4 x i32> we have the following cases,
+  // Case 1: shufflevector(<4 x i32>,<4 x i32>,<-1, -1, -1, 0>)
+  // Case 2: shufflevector(<4 x i32>,<4 x i32>,<-1, -1, 7, 0>)
+  // For both cases, we finally use mask <5, 6, 7, 0>, which requires
+  // to reverse two input vectors.
+  if (Imm < NumElts)
+    ReverseExt = true;
+  else
+    Imm -= NumElts;
+  return std::make_pair(ReverseExt, Imm);
+}
+
 /// Determines if \p M is a shuffle vector mask for a UZP of \p NumElts.
 /// Whether or not G_UZP1 or G_UZP2 should be used is stored in \p WhichResult.
 static bool isUZPMask(ArrayRef<int> M, unsigned NumElts,
                       unsigned &WhichResult) {
   WhichResult = (M[0] == 0 ? 0 : 1);
   for (unsigned i = 0; i != NumElts; ++i) {
     // Skip undef indices.
     if (M[i] < 0)
if (!mi_match(InsMI->getOperand(3).getReg(), MRI, m_ICst(Index)) || Index)
     return false;
 
   Register Dst = MI.getOperand(0).getReg();
   MatchInfo =
       ShuffleVectorPseudo(AArch64::G_DUP, Dst, {InsMI->getOperand(2).getReg()});
   return true;
 }
 
+static bool matchEXT(MachineInstr &MI, MachineRegisterInfo &MRI,
+                     ShuffleVectorPseudo &MatchInfo) {
+  assert(MI.getOpcode() == TargetOpcode::G_SHUFFLE_VECTOR);
+  Register Dst = MI.getOperand(0).getReg();
+  auto ExtInfo = getExtMask(MI.getOperand(3).getShuffleMask(),
+                            MRI.getType(Dst).getNumElements());
+  if (!ExtInfo)
+    return false;
+  bool ReverseExt;
+  uint64_t Imm;
+  std::tie(ReverseExt, Imm) = *ExtInfo;
+  Register V1 = MI.getOperand(1).getReg();
+  Register V2 = MI.getOperand(2).getReg();
+  if (ReverseExt)
+    std::swap(V1, V2);
+  uint64_t ExtFactor = MRI.getType(V1).getScalarSizeInBits() / 8;
+  Imm *= ExtFactor;
+  MatchInfo = ShuffleVectorPseudo(AArch64::G_EXT, Dst, {V1, V2, Imm});
+  return true;
+}
+
 /// Replace a G_SHUFFLE_VECTOR instruction with a pseudo.
 /// \p Opc is the opcode to use. \p MI is the G_SHUFFLE_VECTOR.
 static bool applyShuffleVectorPseudo(MachineInstr &MI,
                                      ShuffleVectorPseudo &MatchInfo) {
   MachineIRBuilder MIRBuilder(MI);
   MIRBuilder.buildInstr(MatchInfo.Opc, {MatchInfo.Dst}, MatchInfo.SrcOps);
   MI.eraseFromParent();
   return true;
 }
 
+/// Replace a G_SHUFFLE_VECTOR instruction with G_EXT.
+/// Special-cased because the constant operand must be emitted as a G_CONSTANT
+/// for the imported tablegen patterns to work.
+static bool applyEXT(MachineInstr &MI, ShuffleVectorPseudo &MatchInfo) {
+  MachineIRBuilder MIRBuilder(MI);
+  // Tablegen patterns expect an i32 G_CONSTANT as the final op.
+  auto Cst =
+      MIRBuilder.buildConstant(LLT::scalar(32), MatchInfo.SrcOps[2].getImm());
+  MIRBuilder.buildInstr(MatchInfo.Opc, {MatchInfo.Dst},
+                        {MatchInfo.SrcOps[0], MatchInfo.SrcOps[1], Cst});
+  MI.eraseFromParent();
+  return true;
+}
+
 #define AARCH64POSTLEGALIZERCOMBINERHELPER_GENCOMBINERHELPER_DEPS
 #include "AArch64GenPostLegalizeGICombiner.inc"
 #undef AARCH64POSTLEGALIZERCOMBINERHELPER_GENCOMBINERHELPER_DEPS
 
 namespace {
 #define AARCH64POSTLEGALIZERCOMBINERHELPER_GENCOMBINERHELPER_H
 #include "AArch64GenPostLegalizeGICombiner.inc"
 #undef AARCH64POSTLEGALIZERCOMBINERHELPER_GENCOMBINERHELPER_H

llvm/test/CodeGen/AArch64/GlobalISel/postlegalizer-combiner-ext.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple aarch64 -run-pass=aarch64-postlegalizer-combiner -verify-machineinstrs %s -o - | FileCheck %s
+#
+# Check that we can combine a G_SHUFFLE_VECTOR into a G_EXT.
+
+...
+---
+name:            v8s8_cst3
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+    ; CHECK-LABEL: name: v8s8_cst3
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:_(<8 x s8>) = COPY $d0
+    ; CHECK: %v2:_(<8 x s8>) = COPY $d1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 3
+    ; CHECK: %shuf:_(<8 x s8>) = G_EXT %v1, %v2, [[C]](s32)
+    ; CHECK: $d0 = COPY %shuf(<8 x s8>)
+    ; CHECK: RET_ReallyLR implicit $d0
+    %v1:_(<8 x s8>) = COPY $d0
+    %v2:_(<8 x s8>) = COPY $d1
+    %shuf:_(<8 x s8>) = G_SHUFFLE_VECTOR %v1(<8 x s8>), %v2, shufflemask(3, 4, 5, 6, 7, 8, 9, 10)
+    $d0 = COPY %shuf(<8 x s8>)
+    RET_ReallyLR implicit $d0
+...
+---
+name:            v8s8_cst5
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+    ; CHECK-LABEL: name: v8s8_cst5
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:_(<8 x s8>) = COPY $d0
+    ; CHECK: %v2:_(<8 x s8>) = COPY $d1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 5
+    ; CHECK: %shuf:_(<8 x s8>) = G_EXT %v2, %v1, [[C]](s32)
+    ; CHECK: $d0 = COPY %shuf(<8 x s8>)
+    ; CHECK: RET_ReallyLR implicit $d0
+    %v1:_(<8 x s8>) = COPY $d0
+    %v2:_(<8 x s8>) = COPY $d1
+    %shuf:_(<8 x s8>) = G_SHUFFLE_VECTOR %v1(<8 x s8>), %v2, shufflemask(13, 14, 15, 0, 1, 2, 3, 4)
+    $d0 = COPY %shuf(<8 x s8>)
+    RET_ReallyLR implicit $d0
+...
+---
+name:            v16s8_cst3
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; CHECK-LABEL: name: v16s8_cst3
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:_(<16 x s8>) = COPY $q0
+    ; CHECK: %v2:_(<16 x s8>) = COPY $q1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 3
+    ; CHECK: %shuf:_(<16 x s8>) = G_EXT %v1, %v2, [[C]](s32)
+    ; CHECK: $q0 = COPY %shuf(<16 x s8>)
+    ; CHECK: RET_ReallyLR implicit $q0
+    %v1:_(<16 x s8>) = COPY $q0
+    %v2:_(<16 x s8>) = COPY $q1
+    %shuf:_(<16 x s8>) = G_SHUFFLE_VECTOR %v1(<16 x s8>), %v2, shufflemask(3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
+    $q0 = COPY %shuf(<16 x s8>)
+    RET_ReallyLR implicit $q0
+...
+---
+name:            v16s8_cst7
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; CHECK-LABEL: name: v16s8_cst7
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:_(<16 x s8>) = COPY $q0
+    ; CHECK: %v2:_(<16 x s8>) = COPY $q1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; CHECK: %shuf:_(<16 x s8>) = G_EXT %v2, %v1, [[C]](s32)
+    ; CHECK: $q0 = COPY %shuf(<16 x s8>)
+    ; CHECK: RET_ReallyLR implicit $q0
+    %v1:_(<16 x s8>) = COPY $q0
+    %v2:_(<16 x s8>) = COPY $q1
+    %shuf:_(<16 x s8>) = G_SHUFFLE_VECTOR %v1(<16 x s8>), %v2, shufflemask(23, 24, 25, 26, 27, 28, 29, 30, 31, 0, 1, 2, 3, 4, 5, 6)
+    $q0 = COPY %shuf(<16 x s8>)
+    RET_ReallyLR implicit $q0
+...
+---
+name:            v4s16_cst6
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+    ; CHECK-LABEL: name: v4s16_cst6
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:_(<4 x s16>) = COPY $d0
+    ; CHECK: %v2:_(<4 x s16>) = COPY $d1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 6
+    ; CHECK: %shuf:_(<4 x s16>) = G_EXT %v1, %v2, [[C]](s32)
+    ; CHECK: $d0 = COPY %shuf(<4 x s16>)
+    ; CHECK: RET_ReallyLR implicit $d0
+    %v1:_(<4 x s16>) = COPY $d0
+    %v2:_(<4 x s16>) = COPY $d1
+    %shuf:_(<4 x s16>) = G_SHUFFLE_VECTOR %v1(<4 x s16>), %v2, shufflemask(3, 4, 5, 6)
+    $d0 = COPY %shuf(<4 x s16>)
+    RET_ReallyLR implicit $d0
+...
+---
+name:            v4s32_cst12
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; CHECK-LABEL: name: v4s32_cst12
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:_(<4 x s32>) = COPY $q0
+    ; CHECK: %v2:_(<4 x s32>) = COPY $q1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 12
+    ; CHECK: %shuf:_(<4 x s32>) = G_EXT %v1, %v2, [[C]](s32)
+    ; CHECK: $q0 = COPY %shuf(<4 x s32>)
+    ; CHECK: RET_ReallyLR implicit $q0
+    %v1:_(<4 x s32>) = COPY $q0
+    %v2:_(<4 x s32>) = COPY $q1
+    %shuf:_(<4 x s32>) = G_SHUFFLE_VECTOR %v1(<4 x s32>), %v2, shufflemask(3, 4, 5, 6)
+    $q0 = COPY %shuf(<4 x s32>)
+    RET_ReallyLR implicit $q0
+...
+---
+name:            undef_elts_should_match_1
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+    ; Undef shuffle indices should not prevent matching G_EXT.
+    ; We should get a constant 3 here.
+    ;
+    ; CHECK-LABEL: name: undef_elts_should_match_1
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:_(<8 x s8>) = COPY $d0
+    ; CHECK: %v2:_(<8 x s8>) = COPY $d1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 3
+    ; CHECK: %shuf:_(<8 x s8>) = G_EXT %v1, %v2, [[C]](s32)
+    ; CHECK: $d0 = COPY %shuf(<8 x s8>)
+    ; CHECK: RET_ReallyLR implicit $d0
+    %v1:_(<8 x s8>) = COPY $d0
+    %v2:_(<8 x s8>) = COPY $d1
+    %shuf:_(<8 x s8>) = G_SHUFFLE_VECTOR %v1(<8 x s8>), %v2, shufflemask(3, -1, -1, 6, 7, 8, 9, 10)
+    $d0 = COPY %shuf(<8 x s8>)
+    RET_ReallyLR implicit $d0
+...
+---
+name:            undef_elts_should_match_2
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+    ; Undef shuffle indices should not prevent matching G_EXT.
+    ; We should get a constant 6 here.
+    ;
+    ; CHECK-LABEL: name: undef_elts_should_match_2
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:_(<8 x s8>) = COPY $d0
+    ; CHECK: %v2:_(<8 x s8>) = COPY $d1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 6
+    ; CHECK: %shuf:_(<8 x s8>) = G_EXT %v2, %v1, [[C]](s32)
+    ; CHECK: $d0 = COPY %shuf(<8 x s8>)
+    ; CHECK: RET_ReallyLR implicit $d0
+    %v1:_(<8 x s8>) = COPY $d0
+    %v2:_(<8 x s8>) = COPY $d1
+    %shuf:_(<8 x s8>) = G_SHUFFLE_VECTOR %v1(<8 x s8>), %v2, shufflemask(-1, -1, -1, -1, 2, 3, 4, 5)
+    $d0 = COPY %shuf(<8 x s8>)
+    RET_ReallyLR implicit $d0
+...
+---
+name:            undef_elts_should_match_3
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; Undef shuffle indices should not prevent matching G_EXT.
+    ; We should get a constant 7 here.
+    ; CHECK-LABEL: name: undef_elts_should_match_3
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:_(<16 x s8>) = COPY $q0
+    ; CHECK: %v2:_(<16 x s8>) = COPY $q1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; CHECK: %shuf:_(<16 x s8>) = G_EXT %v2, %v1, [[C]](s32)
+    ; CHECK: $q0 = COPY %shuf(<16 x s8>)
+    ; CHECK: RET_ReallyLR implicit $q0
+    %v1:_(<16 x s8>) = COPY $q0
+    %v2:_(<16 x s8>) = COPY $q1
+    %shuf:_(<16 x s8>) = G_SHUFFLE_VECTOR %v1(<16 x s8>), %v2, shufflemask(23, 24, 25, 26, -1, -1, 29, 30, 31, 0, 1, 2, 3, 4, -1, 6)
+    $q0 = COPY %shuf(<16 x s8>)
+    RET_ReallyLR implicit $q0
+...
+---
+name:            undef_elts_should_match_4
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; Undef shuffle indices should not prevent matching G_EXT.
+    ; We should get a constant 10 here.
+    ; CHECK-LABEL: name: undef_elts_should_match_4
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:_(<8 x s16>) = COPY $q0
+    ; CHECK: %v2:_(<8 x s16>) = COPY $q1
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 10
+    ; CHECK: %shuf:_(<8 x s16>) = G_EXT %v2, %v1, [[C]](s32)
+    ; CHECK: $q0 = COPY %shuf(<8 x s16>)
+    ; CHECK: RET_ReallyLR implicit $q0
+    %v1:_(<8 x s16>) = COPY $q0
+    %v2:_(<8 x s16>) = COPY $q1
+    %shuf:_(<8 x s16>) = G_SHUFFLE_VECTOR %v1(<8 x s16>), %v2, shufflemask(-1, -1, -1, -1, 1, 2, 3, 4)
+    $q0 = COPY %shuf(<8 x s16>)
+    RET_ReallyLR implicit $q0
+...
+---
+name:            all_undef
+alignment:       4
+legalized:       true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+    ; We expect at least one defined element in the shuffle mask.
+    ;
+    ; CHECK-LABEL: name: all_undef
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:_(<8 x s16>) = COPY $q0
+    ; CHECK: %shuf:_(<8 x s16>) = G_REV64 %v1
+    ; CHECK: $q0 = COPY %shuf(<8 x s16>)
+    ; CHECK: RET_ReallyLR implicit $q0
+    %v1:_(<8 x s16>) = COPY $q0
+    %v2:_(<8 x s16>) = COPY $q1
+    %shuf:_(<8 x s16>) = G_SHUFFLE_VECTOR %v1(<8 x s16>), %v2, shufflemask(-1, -1, -1, -1, -1, -1, -1, -1)
+    $q0 = COPY %shuf(<8 x s16>)
+    RET_ReallyLR implicit $q0
+...

llvm/test/CodeGen/AArch64/GlobalISel/postlegalizer-combiner-shuffle-splat.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 # RUN: llc -mtriple=aarch64 -run-pass=aarch64-postlegalizer-combiner -verify-machineinstrs %s -o - | FileCheck %s
 
 ---
 name:            splat_4xi32
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $w0
 
     ; CHECK-LABEL: name: splat_4xi32
     ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr(s32) = COPY $w0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<4 x s32>) = G_DUP [[COPY]](s32)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<4 x s32>) = G_DUP [[COPY]](s32)
     ; CHECK: $q0 = COPY [[DUP]](<4 x s32>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:gpr(s32) = COPY $w0
-    %2:fpr(<4 x s32>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
-    %4:fpr(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(0, 0, 0, 0)
+    %0:_(s32) = COPY $w0
+    %2:_(<4 x s32>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
+    %4:_(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(0, 0, 0, 0)
     $q0 = COPY %4(<4 x s32>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            splat_2xi64
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $x0
 
     ; CHECK-LABEL: name: splat_2xi64
     ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr(s64) = COPY $x0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<2 x s64>) = G_DUP [[COPY]](s64)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $x0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<2 x s64>) = G_DUP [[COPY]](s64)
     ; CHECK: $q0 = COPY [[DUP]](<2 x s64>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:gpr(s64) = COPY $x0
-    %2:fpr(<2 x s64>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
-    %4:fpr(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(0, 0)
+    %0:_(s64) = COPY $x0
+    %2:_(<2 x s64>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
+    %4:_(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(0, 0)
     $q0 = COPY %4(<2 x s64>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            splat_2xi32
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $w0
 
     ; CHECK-LABEL: name: splat_2xi32
     ; CHECK: liveins: $w0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr(s32) = COPY $w0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<2 x s32>) = G_DUP [[COPY]](s32)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $w0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<2 x s32>) = G_DUP [[COPY]](s32)
     ; CHECK: $d0 = COPY [[DUP]](<2 x s32>)
     ; CHECK: RET_ReallyLR implicit $d0
-    %0:gpr(s32) = COPY $w0
-    %2:fpr(<2 x s32>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<2 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
-    %4:fpr(<2 x s32>) = G_SHUFFLE_VECTOR %1(<2 x s32>), %2, shufflemask(0, 0)
+    %0:_(s32) = COPY $w0
+    %2:_(<2 x s32>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<2 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
+    %4:_(<2 x s32>) = G_SHUFFLE_VECTOR %1(<2 x s32>), %2, shufflemask(0, 0)
     $d0 = COPY %4(<2 x s32>)
     RET_ReallyLR implicit $d0
 
 ...
 ---
 name:            splat_4xf32
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $s0
 
     ; CHECK-LABEL: name: splat_4xf32
     ; CHECK: liveins: $s0
-    ; CHECK: [[COPY:%[0-9]+]]:fpr(s32) = COPY $s0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<4 x s32>) = G_DUP [[COPY]](s32)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $s0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<4 x s32>) = G_DUP [[COPY]](s32)
     ; CHECK: $q0 = COPY [[DUP]](<4 x s32>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:fpr(s32) = COPY $s0
-    %2:fpr(<4 x s32>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
-    %4:fpr(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(0, 0, 0, 0)
+    %0:_(s32) = COPY $s0
+    %2:_(<4 x s32>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
+    %4:_(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(0, 0, 0, 0)
     $q0 = COPY %4(<4 x s32>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            splat_2xf64
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $d0
 
     ; CHECK-LABEL: name: splat_2xf64
     ; CHECK: liveins: $d0
-    ; CHECK: [[COPY:%[0-9]+]]:fpr(s64) = COPY $d0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<2 x s64>) = G_DUP [[COPY]](s64)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $d0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<2 x s64>) = G_DUP [[COPY]](s64)
     ; CHECK: $q0 = COPY [[DUP]](<2 x s64>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:fpr(s64) = COPY $d0
-    %2:fpr(<2 x s64>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
-    %4:fpr(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(0, 0)
+    %0:_(s64) = COPY $d0
+    %2:_(<2 x s64>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
+    %4:_(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(0, 0)
     $q0 = COPY %4(<2 x s64>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            splat_2xf32
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $s0
 
     ; CHECK-LABEL: name: splat_2xf32
     ; CHECK: liveins: $s0
-    ; CHECK: [[COPY:%[0-9]+]]:fpr(s32) = COPY $s0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<2 x s32>) = G_DUP [[COPY]](s32)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $s0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<2 x s32>) = G_DUP [[COPY]](s32)
     ; CHECK: $d0 = COPY [[DUP]](<2 x s32>)
     ; CHECK: RET_ReallyLR implicit $d0
-    %0:fpr(s32) = COPY $s0
-    %2:fpr(<2 x s32>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<2 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
-    %4:fpr(<2 x s32>) = G_SHUFFLE_VECTOR %1(<2 x s32>), %2, shufflemask(0, 0)
+    %0:_(s32) = COPY $s0
+    %2:_(<2 x s32>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<2 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
+    %4:_(<2 x s32>) = G_SHUFFLE_VECTOR %1(<2 x s32>), %2, shufflemask(0, 0)
     $d0 = COPY %4(<2 x s32>)
     RET_ReallyLR implicit $d0
 
 ...
 ---
 name:            splat_2xf64_copies
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $d0
 
     ; This test is exactly the same as splat_2xf64, except it adds two copies.
     ; These copies shouldn't get in the way of matching the dup pattern.
     ; CHECK-LABEL: name: splat_2xf64_copies
     ; CHECK: liveins: $d0
-    ; CHECK: [[COPY:%[0-9]+]]:fpr(s64) = COPY $d0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<2 x s64>) = G_DUP [[COPY]](s64)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $d0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<2 x s64>) = G_DUP [[COPY]](s64)
     ; CHECK: $q0 = COPY [[DUP]](<2 x s64>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:fpr(s64) = COPY $d0
-    %2:fpr(<2 x s64>) = G_IMPLICIT_DEF
-    %6:fpr(<2 x s64>) = COPY %2
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<2 x s64>) = G_INSERT_VECTOR_ELT %6, %0(s64), %3(s32)
-    %7:fpr(<2 x s64>) = COPY %1
-    %4:fpr(<2 x s64>) = G_SHUFFLE_VECTOR %7(<2 x s64>), %2, shufflemask(0, 0)
+    %0:_(s64) = COPY $d0
+    %2:_(<2 x s64>) = G_IMPLICIT_DEF
+    %6:_(<2 x s64>) = COPY %2
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<2 x s64>) = G_INSERT_VECTOR_ELT %6, %0(s64), %3(s32)
+    %7:_(<2 x s64>) = COPY %1
+    %4:_(<2 x s64>) = G_SHUFFLE_VECTOR %7(<2 x s64>), %2, shufflemask(0, 0)
     $q0 = COPY %4(<2 x s64>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            not_all_zeros
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $x0
     ; Make sure that we don't do the optimization when it's not all zeroes.
     ; CHECK-LABEL: name: not_all_zeros
     ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr(s64) = COPY $x0
-    ; CHECK: [[DEF:%[0-9]+]]:fpr(<2 x s64>) = G_IMPLICIT_DEF
-    ; CHECK: [[C:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 0
-    ; CHECK: [[IVEC:%[0-9]+]]:fpr(<2 x s64>) = G_INSERT_VECTOR_ELT [[DEF]], [[COPY]](s64), [[C]](s32)
-    ; CHECK: [[SHUF:%[0-9]+]]:fpr(<2 x s64>) = G_SHUFFLE_VECTOR [[IVEC]](<2 x s64>), [[DEF]], shufflemask(0, 1)
-    ; CHECK: $q0 = COPY [[SHUF]](<2 x s64>)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $x0
+    ; CHECK: [[DEF:%[0-9]+]]:_(<2 x s64>) = G_IMPLICIT_DEF
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; CHECK: [[IVEC:%[0-9]+]]:_(<2 x s64>) = G_INSERT_VECTOR_ELT [[DEF]], [[COPY]](s64), [[C]](s32)
+    ; CHECK: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; CHECK: [[EXT:%[0-9]+]]:_(<2 x s64>) = G_EXT [[IVEC]], [[DEF]], [[C1]](s32)
+    ; CHECK: $q0 = COPY [[EXT]](<2 x s64>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:gpr(s64) = COPY $x0
-    %2:fpr(<2 x s64>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
-    %4:fpr(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(0, 1)
+    %0:_(s64) = COPY $x0
+    %2:_(<2 x s64>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
+    %4:_(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(0, 1)
     $q0 = COPY %4(<2 x s64>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            all_undef
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $x0
     ; If all the elements are undefined, we consider it a splat. In this case,
     ; we can choose 0 as our index.
     ;
     ; We should get a G_DUP here.
     ;
     ; CHECK-LABEL: name: all_undef
     ; CHECK: liveins: $x0
-    ; CHECK: [[COPY:%[0-9]+]]:gpr(s64) = COPY $x0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<2 x s64>) = G_DUP [[COPY]](s64)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s64) = COPY $x0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<2 x s64>) = G_DUP [[COPY]](s64)
     ; CHECK: $q0 = COPY [[DUP]](<2 x s64>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:gpr(s64) = COPY $x0
-    %2:fpr(<2 x s64>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
-    %4:fpr(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(-1, -1)
+    %0:_(s64) = COPY $x0
+    %2:_(<2 x s64>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<2 x s64>) = G_INSERT_VECTOR_ELT %2, %0(s64), %3(s32)
+    %4:_(<2 x s64>) = G_SHUFFLE_VECTOR %1(<2 x s64>), %2, shufflemask(-1, -1)
     $q0 = COPY %4(<2 x s64>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            one_undef
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $s0
     ; Make sure we can skip past undef values.
     ;
     ; We should get a G_DUP here.
     ;
     ; CHECK-LABEL: name: one_undef
     ; CHECK: liveins: $s0
-    ; CHECK: [[COPY:%[0-9]+]]:fpr(s32) = COPY $s0
-    ; CHECK: [[DUP:%[0-9]+]]:fpr(<4 x s32>) = G_DUP [[COPY]](s32)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $s0
+    ; CHECK: [[DUP:%[0-9]+]]:_(<4 x s32>) = G_DUP [[COPY]](s32)
     ; CHECK: $q0 = COPY [[DUP]](<4 x s32>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:fpr(s32) = COPY $s0
-    %2:fpr(<4 x s32>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
-    %4:fpr(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(0, -1, 0, 0)
+    %0:_(s32) = COPY $s0
+    %2:_(<4 x s32>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
+    %4:_(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(0, -1, 0, 0)
     $q0 = COPY %4(<4 x s32>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            not_all_zeros_with_undefs
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $s0
     ; Check a non-splat mask with an undef value. We shouldn't get a G_DUP here.
     ;
     ; CHECK-LABEL: name: not_all_zeros_with_undefs
     ; CHECK: liveins: $s0
-    ; CHECK: [[COPY:%[0-9]+]]:fpr(s32) = COPY $s0
-    ; CHECK: [[DEF:%[0-9]+]]:fpr(<4 x s32>) = G_IMPLICIT_DEF
-    ; CHECK: [[C:%[0-9]+]]:gpr(s32) = G_CONSTANT i32 0
-    ; CHECK: [[IVEC:%[0-9]+]]:fpr(<4 x s32>) = G_INSERT_VECTOR_ELT [[DEF]], [[COPY]](s32), [[C]](s32)
-    ; CHECK: [[SHUF:%[0-9]+]]:fpr(<4 x s32>) = G_SHUFFLE_VECTOR [[IVEC]](<4 x s32>), [[DEF]], shufflemask(undef, 0, 0, 3)
+    ; CHECK: [[COPY:%[0-9]+]]:_(s32) = COPY $s0
+    ; CHECK: [[DEF:%[0-9]+]]:_(<4 x s32>) = G_IMPLICIT_DEF
+    ; CHECK: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; CHECK: [[IVEC:%[0-9]+]]:_(<4 x s32>) = G_INSERT_VECTOR_ELT [[DEF]], [[COPY]](s32), [[C]](s32)
+    ; CHECK: [[SHUF:%[0-9]+]]:_(<4 x s32>) = G_SHUFFLE_VECTOR [[IVEC]](<4 x s32>), [[DEF]], shufflemask(undef, 0, 0, 3)
     ; CHECK: $q0 = COPY [[SHUF]](<4 x s32>)
     ; CHECK: RET_ReallyLR implicit $q0
-    %0:fpr(s32) = COPY $s0
-    %2:fpr(<4 x s32>) = G_IMPLICIT_DEF
-    %3:gpr(s32) = G_CONSTANT i32 0
-    %1:fpr(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
-    %4:fpr(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(-1, 0, 0, 3)
+    %0:_(s32) = COPY $s0
+    %2:_(<4 x s32>) = G_IMPLICIT_DEF
+    %3:_(s32) = G_CONSTANT i32 0
+    %1:_(<4 x s32>) = G_INSERT_VECTOR_ELT %2, %0(s32), %3(s32)
+    %4:_(<4 x s32>) = G_SHUFFLE_VECTOR %1(<4 x s32>), %2, shufflemask(-1, 0, 0, 3)
     $q0 = COPY %4(<4 x s32>)
     RET_ReallyLR implicit $q0
 
 ...
 ---
 name:            splat_4xi16
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $h0
     ; CHECK-LABEL: name: splat_4xi16
     ; CHECK: liveins: $h0
-    ; CHECK: %copy:fpr(s16) = COPY $h0
-    ; CHECK: %splat:fpr(<4 x s16>) = G_DUP %copy(s16)
+    ; CHECK: %copy:_(s16) = COPY $h0
+    ; CHECK: %splat:_(<4 x s16>) = G_DUP %copy(s16)
     ; CHECK: $d0 = COPY %splat(<4 x s16>)
     ; CHECK: RET_ReallyLR implicit $d0
-    %copy:fpr(s16) = COPY $h0
-    %undef:fpr(<4 x s16>) = G_IMPLICIT_DEF
-    %cst:gpr(s32) = G_CONSTANT i32 0
-    %ins:fpr(<4 x s16>) = G_INSERT_VECTOR_ELT %undef, %copy(s16), %cst(s32)
-    %splat:fpr(<4 x s16>) = G_SHUFFLE_VECTOR %ins(<4 x s16>), %undef, shufflemask(0, 0, 0, 0)
+    %copy:_(s16) = COPY $h0
+    %undef:_(<4 x s16>) = G_IMPLICIT_DEF
+    %cst:_(s32) = G_CONSTANT i32 0
+    %ins:_(<4 x s16>) = G_INSERT_VECTOR_ELT %undef, %copy(s16), %cst(s32)
+    %splat:_(<4 x s16>) = G_SHUFFLE_VECTOR %ins(<4 x s16>), %undef, shufflemask(0, 0, 0, 0)
     $d0 = COPY %splat(<4 x s16>)
     RET_ReallyLR implicit $d0
 
 ...
 ---
 name:            splat_8xi8
 alignment:       4
 legalized:       true
-regBankSelected: true
 tracksRegLiveness: true
 body:             |
   bb.1.entry:
     liveins: $w0
     ; CHECK-LABEL: name: splat_8xi8
     ; CHECK: liveins: $w0
-    ; CHECK: %copy:gpr(s32) = COPY $w0
-    ; CHECK: %splat:fpr(<8 x s8>) = G_DUP %copy(s32)
+    ; CHECK: %copy:_(s32) = COPY $w0
+    ; CHECK: %splat:_(<8 x s8>) = G_DUP %copy(s32)
     ; CHECK: $d0 = COPY %splat(<8 x s8>)
     ; CHECK: RET_ReallyLR implicit $d0
-    %copy:gpr(s32) = COPY $w0
-    %undef:fpr(<8 x s8>) = G_IMPLICIT_DEF
-    %cst:gpr(s32) = G_CONSTANT i32 0
-    %ins:fpr(<8 x s8>) = G_INSERT_VECTOR_ELT %undef, %copy(s32), %cst(s32)
-    %splat:fpr(<8 x s8>) = G_SHUFFLE_VECTOR %ins(<8 x s8>), %undef, shufflemask(0, 0, 0, 0, 0, 0, 0, 0)
+    %copy:_(s32) = COPY $w0
+    %undef:_(<8 x s8>) = G_IMPLICIT_DEF
+    %cst:_(s32) = G_CONSTANT i32 0
+    %ins:_(<8 x s8>) = G_INSERT_VECTOR_ELT %undef, %copy(s32), %cst(s32)
+    %splat:_(<8 x s8>) = G_SHUFFLE_VECTOR %ins(<8 x s8>), %undef, shufflemask(0, 0, 0, 0, 0, 0, 0, 0)
     $d0 = COPY %splat(<8 x s8>)
     RET_ReallyLR implicit $d0

llvm/test/CodeGen/AArch64/GlobalISel/postlegalizer-combiner-zip.mir

 # NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
 #
 # Check that we can recognize a shuffle mask for a zip instruction, and produce
 # G_ZIP1 or G_ZIP2 where appropriate.
 #
-# RUN: llc -mtriple aarch64 -run-pass=aarch64-postlegalizer-combiner -verify-machineinstrs %s -o - | FileCheck %s
+# RUN: llc -mtriple aarch64 -run-pass=aarch64-postlegalizer-combiner --aarch64postlegalizercombinerhelper-disable-rule=ext -verify-machineinstrs %s -o - | FileCheck %s
 
 ...
 ---
 name:            zip1_v2s32
 alignment:       4
 legalized:       true
 tracksRegLiveness: true
 body:             |

llvm/test/CodeGen/AArch64/GlobalISel/select-ext.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=aarch64 -run-pass=instruction-select -verify-machineinstrs %s -o - | FileCheck %s
+#
+# Test G_EXT selection using AArch64ext patterns.
+
+...
+---
+name:            v8s8_EXTv8i8
+alignment:       4
+legalized:       true
+regBankSelected: true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+
+    ; CHECK-LABEL: name: v8s8_EXTv8i8
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:fpr64 = COPY $d0
+    ; CHECK: %v2:fpr64 = COPY $d1
+    ; CHECK: %shuf:fpr64 = EXTv8i8 %v1, %v2, 3
+    %v1:fpr(<8 x s8>) = COPY $d0
+    %v2:fpr(<8 x s8>) = COPY $d1
+    %3:gpr(s32) = G_CONSTANT i32 3
+    %shuf:fpr(<8 x s8>) = G_EXT %v1, %v2, %3(s32)
+
+...
+---
+name:            v16s8_EXTv16i8
+alignment:       4
+legalized:       true
+regBankSelected: true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+
+    ; CHECK-LABEL: name: v16s8_EXTv16i8
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:fpr128 = COPY $q0
+    ; CHECK: %v2:fpr128 = COPY $q1
+    ; CHECK: %shuf:fpr128 = EXTv16i8 %v1, %v2, 3
+    %v1:fpr(<16 x s8>) = COPY $q0
+    %v2:fpr(<16 x s8>) = COPY $q1
+    %3:gpr(s32) = G_CONSTANT i32 3
+    %shuf:fpr(<16 x s8>) = G_EXT %v1, %v2, %3(s32)
+
+...
+---
+name:            v4s16_EXTv8i8
+alignment:       4
+legalized:       true
+regBankSelected: true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+
+    ; CHECK-LABEL: name: v4s16_EXTv8i8
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:fpr64 = COPY $d0
+    ; CHECK: %v2:fpr64 = COPY $d1
+    ; CHECK: %shuf:fpr64 = EXTv8i8 %v1, %v2, 6
+    %v1:fpr(<4 x s16>) = COPY $d0
+    %v2:fpr(<4 x s16>) = COPY $d1
+    %3:gpr(s32) = G_CONSTANT i32 6
+    %shuf:fpr(<4 x s16>) = G_EXT %v1, %v2, %3(s32)
+
+...
+---
+name:            v8s16_EXTv16i8
+alignment:       4
+legalized:       true
+regBankSelected: true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+
+    ; CHECK-LABEL: name: v8s16_EXTv16i8
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:fpr128 = COPY $q0
+    ; CHECK: %v2:fpr128 = COPY $q1
+    ; CHECK: %shuf:fpr128 = EXTv16i8 %v2, %v1, 10
+    %v1:fpr(<8 x s16>) = COPY $q0
+    %v2:fpr(<8 x s16>) = COPY $q1
+    %3:gpr(s32) = G_CONSTANT i32 10
+    %shuf:fpr(<8 x s16>) = G_EXT %v2, %v1, %3(s32)
+...
+
+...
+---
+name:            v4s32_EXTv16i8
+alignment:       4
+legalized:       true
+regBankSelected: true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+
+    ; CHECK-LABEL: name: v4s32_EXTv16i8
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:fpr128 = COPY $q0
+    ; CHECK: %v2:fpr128 = COPY $q1
+    ; CHECK: %shuf:fpr128 = EXTv16i8 %v1, %v2, 12
+    %v1:fpr(<4 x s32>) = COPY $q0
+    %v2:fpr(<4 x s32>) = COPY $q1
+    %3:gpr(s32) = G_CONSTANT i32 12
+    %shuf:fpr(<4 x s32>) = G_EXT %v1, %v2, %3(s32)
+
+...
+---
+name:            v2s32_EXTv8i8
+alignment:       4
+legalized:       true
+regBankSelected: true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $d0, $d1
+
+    ; CHECK-LABEL: name: v2s32_EXTv8i8
+    ; CHECK: liveins: $d0, $d1
+    ; CHECK: %v1:fpr64 = COPY $d0
+    ; CHECK: %v2:fpr64 = COPY $d1
+    ; CHECK: %shuf:fpr64 = EXTv8i8 %v1, %v2, 2
+    %v1:fpr(<2 x s32>) = COPY $d0
+    %v2:fpr(<2 x s32>) = COPY $d1
+    %3:gpr(s32) = G_CONSTANT i32 2
+    %shuf:fpr(<2 x s32>) = G_EXT %v1, %v2, %3(s32)
+
+...
+---
+name:            v2s64_EXTv16i8
+alignment:       4
+legalized:       true
+regBankSelected: true
+tracksRegLiveness: true
+body:             |
+  bb.0:
+    liveins: $q0, $q1
+
+    ; CHECK-LABEL: name: v2s64_EXTv16i8
+    ; CHECK: liveins: $q0, $q1
+    ; CHECK: %v1:fpr128 = COPY $q0
+    ; CHECK: %v2:fpr128 = COPY $q1
+    ; CHECK: %shuf:fpr128 = EXTv16i8 %v1, %v2, 2
+    %v1:fpr(<2 x s64>) = COPY $q0
+    %v2:fpr(<2 x s64>) = COPY $q1
+    %3:gpr(s32) = G_CONSTANT i32 2
+    %shuf:fpr(<2 x s64>) = G_EXT %v1, %v2, %3(s32)
+...