GlobalISel: Lower funnel shifts

[arsenm]

Summary

128-bit cases are broken, tests need completion and fixes, and divide-by-constant optimizations are missing (these would also apply after the legalizer, not during as would be required)

[foad]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4167–4168

This won't work when Z is 0 if shifting by BW is undefined.

[arsenm]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4167–4168

That's handled below but not included in the high level comment

[foad]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4196–4197

It would be slicker to implement (X << (BW - (Z % BW))) as (X << 1 << ((BW - 1) - (Z % BW))).

[arsenm]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4196–4197

I copied this from the DAG version. Maybe that should be updated first? (or maybe it happens to turn into this anyway from other combines)

[arsenm]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4196–4197

I don't actually see how this is better? It doesn't eliminate the urem, and adds an extra instruction?

[foad]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4196–4197

Adding one instruction here (the << 1) avoids the need for the compare and select at the end to handle the shift-by-zero case.

[RKSimon]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4190

@arsenm In D80489 for SelectionDAG we changed the pow2 InvShAmt case to ~ShAmt & Mask to help targets that can only use the bottom bits of the shift amount.

[aemerson]

Is this still being worked on?

[arsenm]

In D76500#2301319, @aemerson wrote:

Is this still being worked on?

Yes, I have an updated version I haven't posted yet. The problem I ended up blocked on was the fact that we don't have the divide-by-constant optimizations, combined with 128-bit division legalization not working

[foad]

llvm/include/llvm/CodeGen/GlobalISel/Utils.h

249 ↗

(On Diff #295041)

Comment about nullptr seems wrong. Match doesn't take any pointer arguments.

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4172

Should be C && C->getValue().urem(BW) != 0.

4276–4277

Please add some kind of comment to the effect that these expansions are not quite right, cos they might try to shift by BW.

llvm/lib/CodeGen/GlobalISel/Utils.cpp

750 ↗

(On Diff #295041)

Could return a bit earlier with:

if (Def->getOpcode() == TargetOpcode::G_IMPLICIT_DEF)
  return AllowUndefs;

770 ↗

(On Diff #295041)

You need this return true inside the body of the if and a return false at the end of the function. Alternatively, return early as soon as we know that Def is not G_BUILD_VECTOR*.

[aemerson]

In D76500#2301324, @arsenm wrote:

In D76500#2301319, @aemerson wrote:

Is this still being worked on?

Yes, I have an updated version I haven't posted yet. The problem I ended up blocked on was the fact that we don't have the divide-by-constant optimizations, combined with 128-bit division legalization not working

Anything we can help on? For the divide by constant opts, I guess this is: X / C -> ashr X, log2(C)? Anything else?

[arsenm]

In D76500#2371797, @aemerson wrote:

In D76500#2301324, @arsenm wrote:

In D76500#2301319, @aemerson wrote:

Is this still being worked on?

Yes, I have an updated version I haven't posted yet. The problem I ended up blocked on was the fact that we don't have the divide-by-constant optimizations, combined with 128-bit division legalization not working

Anything we can help on? For the divide by constant opts, I guess this is: X / C -> ashr X, log2(C)? Anything else?

It's a urem here, I think this was the important one:

// fold (urem x, pow2) -> (and x, pow2-1)

// fold (urem x, (shl pow2, y)) -> (and x, (add (shl pow2, y), -1))

[foad]

Reverse ping! We are now seeing funnel shifts in the wild, because InstCombine has started generating them. Can this patch go in as-is? Are there correctness problems or is it just a matter of optimizing the udiv/urem into shifts?

[arsenm]

Fix i128 case and rebase. matchUnaryPredicate was broken so it was emitting urem in cases where it wasn't needed. We're still emitting worse code in the end from urem expansion though, but I think everything works

[foad]

llvm/include/llvm/CodeGen/GlobalISel/Utils.h

328–330 ↗

(On Diff #332044)

Nit: "AllowUndefs"

Comment about nullptr seems wrong. Match doesn't take any pointer arguments.

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4172

I still think this check for !C is either wrong or redundant, depending on what the behaviour of matchUnaryPredicate is supposed to be.

4196–4197

Using ~Z here only works if BitWidth is a power of two. Otherwise you need something like BitWidth - 1 - (Z % BitWidth).

llvm/lib/CodeGen/GlobalISel/Utils.cpp

878 ↗

(On Diff #332044)

At the top level, matchUnaryPredicate only calls Match for G_CONSTANTs (not for G_IMPLICIT_DEF despite what the comment in Utils.h says).

Inside a G_BUILD_VECTOR(_TRUNC) you call Match on anything. Did you mean to have a check for G_CONSTANT here too, like the SelectionDAG version does?

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fshl.mir

49

Not related to this patch, but I think it would work out better for AMDGPU to implement 32-bit G_FSHL as a 64-bit left shift and then extract the high part of the result.

[arsenm]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4172

This is is correct because in the variable case, you have to assume it's a non-zero mod

4196–4197

Isn't that what the isNonZeroModBitWidthOrUndef check is for? I just directly copied this from the DAG, so this should match the behavior

[arsenm]

Address comments

[foad]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4196–4197

DAG only tries to use a shift in the other direction if isPowerOf2_32(BW) (TargetLowering.cpp line 6456).

[foad]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4172

Huh? In the variable case you can't assume it's a non-zero mod, because it might not be. This function is really isKnownToBeNonZeroModBitWidthOrUndef, and the safe default is to return false.

[arsenm]

Make matchUnaryPredicate behave like the DAG version, and only handle constant or undef

[foad]

LGTM, thanks!

[arsenm]

b24436ac96bdf3f2c545fc85dc8af239d618c9c4

[foad]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4196–4197

I realise I approved the patch but I still think this needs fixing. The ~Z thing only works if BW is a power of two.

[arsenm]

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

4196–4197

D99541

llvm/include/llvm/CodeGen/GlobalISel/LegalizerHelper.h

public:
   LegalizeResult narrowScalarExt(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
   LegalizeResult narrowScalarSelect(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
   LegalizeResult narrowScalarCTLZ(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
   LegalizeResult narrowScalarCTTZ(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
   LegalizeResult narrowScalarCTPOP(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
 
   LegalizeResult lowerBitcast(MachineInstr &MI);
   LegalizeResult lowerBitCount(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
+  LegalizeResult lowerFunnelShift(MachineInstr &MI);
 
   LegalizeResult lowerU64ToF32BitOps(MachineInstr &MI);
   LegalizeResult lowerUITOFP(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
   LegalizeResult lowerSITOFP(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
   LegalizeResult lowerFPTOUI(MachineInstr &MI, unsigned TypeIdx, LLT Ty);
   LegalizeResult lowerFPTOSI(MachineInstr &MI);
 
   LegalizeResult lowerFPTRUNC_F64_TO_F16(MachineInstr &MI);

llvm/include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h

   }
 
   MachineInstrBuilder buildSMulH(const DstOp &Dst, const SrcOp &Src0,
                                  const SrcOp &Src1,
                                  Optional<unsigned> Flags = None) {
     return buildInstr(TargetOpcode::G_SMULH, {Dst}, {Src0, Src1}, Flags);
   }
 
+  /// Build and insert \p Res = G_UREM \p Op0, \p Op1
+  MachineInstrBuilder buildURem(const DstOp &Dst, const SrcOp &Src0,
+                                const SrcOp &Src1,
+                                Optional<unsigned> Flags = None) {
+    return buildInstr(TargetOpcode::G_UREM, {Dst}, {Src0, Src1}, Flags);
+  }
+
   MachineInstrBuilder buildFMul(const DstOp &Dst, const SrcOp &Src0,
                                 const SrcOp &Src1,
                                 Optional<unsigned> Flags = None) {
     return buildInstr(TargetOpcode::G_FMUL, {Dst}, {Src0, Src1}, Flags);
   }
 
   MachineInstrBuilder buildFMinNum(const DstOp &Dst, const SrcOp &Src0,
                                    const SrcOp &Src1,

llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp

case G_INSERT:
     return lowerInsert(MI);
   case G_BSWAP:
     return lowerBswap(MI);
   case G_BITREVERSE:
     return lowerBitreverse(MI);
   case G_READ_REGISTER:
   case G_WRITE_REGISTER:
     return lowerReadWriteRegister(MI);
+  case G_FSHL:
+  case G_FSHR:
+    return lowerFunnelShift(MI);
   }
 }
 
 LegalizerHelper::LegalizeResult LegalizerHelper::fewerElementsVectorImplicitDef(
     MachineInstr &MI, unsigned TypeIdx, LLT NarrowTy) {
   SmallVector<Register, 2> DstRegs;
 
   unsigned NarrowSize = NarrowTy.getSizeInBits();
case TargetOpcode::G_CTPOP: {
     B.buildLShr(MI.getOperand(0).getReg(), ResTmp, C_SizeM8);
 
     MI.eraseFromParent();
     return Legalized;
   }
   }
 }
 
+LegalizerHelper::LegalizeResult
+LegalizerHelper::lowerFunnelShift(MachineInstr &MI) {
+  // G_FSHL: (X << (Z % BW)) | (Y >> (BW - (Z % BW)))
+  // G_FSHR: (X << (BW - (Z % BW))) | (Y >> (Z % BW))
+  Register Dst = MI.getOperand(0).getReg();
+  Register X = MI.getOperand(1).getReg();
+  Register Y = MI.getOperand(2).getReg();
+  Register Z = MI.getOperand(3).getReg();
+  LLT Ty = MRI.getType(Dst);
+  LLT ShTy = MRI.getType(Z);
+
+  const unsigned EltSizeInBits = Ty.getScalarSizeInBits();
+  const bool IsFSHL = MI.getOpcode() == TargetOpcode::G_FSHL;
+
+  auto BitWidthC = MIRBuilder.buildConstant(ShTy, EltSizeInBits);
+  auto Zero = MIRBuilder.buildConstant(ShTy, 0);
+
+  Register ShAmt;
+  if (isPowerOf2_32(EltSizeInBits)) {
+    auto Mask = MIRBuilder.buildConstant(ShTy, EltSizeInBits - 1);
+    ShAmt = MIRBuilder.buildAnd(ShTy, Z, Mask).getReg(0);
+  } else {
+    ShAmt = MIRBuilder.buildURem(ShTy, Z, BitWidthC).getReg(0);
+  }
+
+  Register InvShAmt = MIRBuilder.buildSub(ShTy, BitWidthC, ShAmt).getReg(0);
+  auto ShX = MIRBuilder.buildShl(Ty, X, IsFSHL ? ShAmt : InvShAmt);
+  auto ShY = MIRBuilder.buildLShr(Ty, Y, IsFSHL ? InvShAmt : ShAmt);
+  auto Or = MIRBuilder.buildOr(Ty, ShX, ShY);
+
+  auto CCTy = Ty.changeElementSize(1);
+  // If (Z % BW == 0), then the opposite direction shift is shift-by-bitwidth,
+  // and that is undefined. We must compare and select to avoid UB.
+
+  // For fshl, 0-shift returns the 1st arg (X).
+  // For fshr, 0-shift returns the 2nd arg (Y).
+  auto IsZeroShift = MIRBuilder.buildICmp(CmpInst::ICMP_EQ, CCTy, ShAmt, Zero);
+
+  MIRBuilder.buildSelect(Dst, IsZeroShift, IsFSHL ? X : Y, Or);
+  MI.eraseFromParent();
+  return Legalized;
+}
+
 // Expand s32 = G_UITOFP s64 using bit operations to an IEEE float
 // representation.
 LegalizerHelper::LegalizeResult
 LegalizerHelper::lowerU64ToF32BitOps(MachineInstr &MI) {
   Register Dst = MI.getOperand(0).getReg();
   Register Src = MI.getOperand(1).getReg();
   const LLT S64 = LLT::scalar(64);
   const LLT S32 = LLT::scalar(32);
LegalizerHelper::lowerUITOFP(MachineInstr &MI, unsigned TypeIdx, LLT Ty) {
   Register Dst = MI.getOperand(0).getReg();
   Register Src = MI.getOperand(1).getReg();
   LLT DstTy = MRI.getType(Dst);
   LLT SrcTy = MRI.getType(Src);
 
   if (SrcTy == LLT::scalar(1)) {
     auto True = MIRBuilder.buildFConstant(DstTy, 1.0);
     auto False = MIRBuilder.buildFConstant(DstTy, 0.0);
     MIRBuilder.buildSelect(Dst, Src, True, False);
     MI.eraseFromParent();
     return Legalized;
   }
 
   if (SrcTy != LLT::scalar(64))
     return UnableToLegalize;
 
   if (DstTy == LLT::scalar(32)) {
     // TODO: SelectionDAG has several alternative expansions to port which may

llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp

if (ST.hasVOP3PInsts()) {
     SextInReg.lowerFor({{S32}, {S64}});
   }
 
   SextInReg
     .scalarize(0)
     .clampScalar(0, S32, S64)
     .lower();
 
+  // TODO: Only Try to form v2s16 with legal packed instructions.
   getActionDefinitionsBuilder(G_FSHR)
     .legalFor({{S32, S32}})
+    .lowerFor({{V2S16, V2S16}})
+    .fewerElementsIf(elementTypeIs(0, S16), changeTo(0, V2S16))
     .scalarize(0)
     .lower();
 
   getActionDefinitionsBuilder(G_READCYCLECOUNTER)
     .legalFor({S64});
 
   getActionDefinitionsBuilder({
       // TODO: Verify V_BFI_B32 is generated from expanded bit ops

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fshl.mir

+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=tahiti -O0 -run-pass=legalizer -global-isel-abort=0 %s -o - | FileCheck -check-prefix=SI %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=fiji -O0 -run-pass=legalizer  -global-isel-abort=0 %s -o - | FileCheck -check-prefix=VI %s
+# RUN: llc -mtriple=amdgcn-mesa-mesa3d -mcpu=gfx900 -O0 -run-pass=legalizer  -global-isel-abort=0 %s -o - | FileCheck -check-prefix=GFX9  %s
+
+---
+name: test_fshl_s32_s32
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1, $vgpr2
+
+    ; SI-LABEL: name: test_fshl_s32_s32
+    ; SI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; SI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; SI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 31
+    ; SI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY2]], [[C2]]
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C]], [[AND]]
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY]], [[AND]](s32)
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[COPY1]], [[SUB]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[SHL]], [[LSHR]]
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s32), [[C1]]
+    ; SI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[COPY]], [[OR]]
+    ; SI: $vgpr0 = COPY [[SELECT]](s32)
+    ; VI-LABEL: name: test_fshl_s32_s32
+    ; VI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; VI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; VI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; VI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
+    ; VI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 31
+    ; VI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY2]], [[C2]]
+    ; VI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C]], [[AND]]
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY]], [[AND]](s32)
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[COPY1]], [[SUB]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[SHL]], [[LSHR]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s32), [[C1]]
+    ; VI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[COPY]], [[OR]]
+    ; VI: $vgpr0 = COPY [[SELECT]](s32)
+    ; GFX9-LABEL: name: test_fshl_s32_s32
+    ; GFX9: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; GFX9: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; GFX9: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
+    ; GFX9: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 31
+    ; GFX9: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY2]], [[C2]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY]], [[AND]](s32)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[COPY1]], [[SUB]](s32)
+    ; GFX9: [[OR:%[0-9]+]]:_(s32) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s32), [[C1]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[COPY]], [[OR]]
+    ; GFX9: $vgpr0 = COPY [[SELECT]](s32)
+    %0:_(s32) = COPY $vgpr0
+    %1:_(s32) = COPY $vgpr1
+    %2:_(s32) = COPY $vgpr2
+    %3:_(s32) = G_FSHL %0, %1, %2
+    $vgpr0 = COPY %3
+...
+
+---
+name: test_fshl_v2s32_v2s32
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1, $vgpr2_vgpr3, $vgpr4_vgpr5
+
+    ; SI-LABEL: name: test_fshl_v2s32_v2s32
+    ; SI: [[COPY:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr0_vgpr1
+    ; SI: [[COPY1:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr2_vgpr3
+    ; SI: [[COPY2:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr4_vgpr5
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 31
+    ; SI: [[BUILD_VECTOR:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[C2]](s32), [[C2]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(<2 x s32>) = G_AND [[COPY2]], [[BUILD_VECTOR]]
+    ; SI: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C]], [[UV]]
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[C]], [[UV1]]
+    ; SI: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY]](<2 x s32>)
+    ; SI: [[UV4:%[0-9]+]]:_(s32), [[UV5:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[UV2]], [[UV4]](s32)
+    ; SI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[UV3]], [[UV5]](s32)
+    ; SI: [[BUILD_VECTOR1:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[SHL]](s32), [[SHL1]](s32)
+    ; SI: [[UV6:%[0-9]+]]:_(s32), [[UV7:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY1]](<2 x s32>)
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[UV6]], [[SUB]](s32)
+    ; SI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[UV7]], [[SUB1]](s32)
+    ; SI: [[BUILD_VECTOR2:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[LSHR]](s32), [[LSHR1]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(<2 x s32>) = G_OR [[BUILD_VECTOR1]], [[BUILD_VECTOR2]]
+    ; SI: [[UV8:%[0-9]+]]:_(s32), [[UV9:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UV8]](s32), [[C1]]
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UV9]](s32), [[C1]]
+    ; SI: [[UV10:%[0-9]+]]:_(s32), [[UV11:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY]](<2 x s32>)
+    ; SI: [[UV12:%[0-9]+]]:_(s32), [[UV13:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[OR]](<2 x s32>)
+    ; SI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[UV10]], [[UV12]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP1]](s1), [[UV11]], [[UV13]]
+    ; SI: [[BUILD_VECTOR3:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[SELECT]](s32), [[SELECT1]](s32)
+    ; SI: $vgpr0_vgpr1 = COPY [[BUILD_VECTOR3]](<2 x s32>)
+    ; VI-LABEL: name: test_fshl_v2s32_v2s32
+    ; VI: [[COPY:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr0_vgpr1
+    ; VI: [[COPY1:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr2_vgpr3
+    ; VI: [[COPY2:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr4_vgpr5
+    ; VI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
+    ; VI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 31
+    ; VI: [[BUILD_VECTOR:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[C2]](s32), [[C2]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(<2 x s32>) = G_AND [[COPY2]], [[BUILD_VECTOR]]
+    ; VI: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; VI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C]], [[UV]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[C]], [[UV1]]
+    ; VI: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY]](<2 x s32>)
+    ; VI: [[UV4:%[0-9]+]]:_(s32), [[UV5:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[UV2]], [[UV4]](s32)
+    ; VI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[UV3]], [[UV5]](s32)
+    ; VI: [[BUILD_VECTOR1:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[SHL]](s32), [[SHL1]](s32)
+    ; VI: [[UV6:%[0-9]+]]:_(s32), [[UV7:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY1]](<2 x s32>)
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[UV6]], [[SUB]](s32)
+    ; VI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[UV7]], [[SUB1]](s32)
+    ; VI: [[BUILD_VECTOR2:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[LSHR]](s32), [[LSHR1]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(<2 x s32>) = G_OR [[BUILD_VECTOR1]], [[BUILD_VECTOR2]]
+    ; VI: [[UV8:%[0-9]+]]:_(s32), [[UV9:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UV8]](s32), [[C1]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UV9]](s32), [[C1]]
+    ; VI: [[UV10:%[0-9]+]]:_(s32), [[UV11:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY]](<2 x s32>)
+    ; VI: [[UV12:%[0-9]+]]:_(s32), [[UV13:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[OR]](<2 x s32>)
+    ; VI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[UV10]], [[UV12]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP1]](s1), [[UV11]], [[UV13]]
+    ; VI: [[BUILD_VECTOR3:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[SELECT]](s32), [[SELECT1]](s32)
+    ; VI: $vgpr0_vgpr1 = COPY [[BUILD_VECTOR3]](<2 x s32>)
+    ; GFX9-LABEL: name: test_fshl_v2s32_v2s32
+    ; GFX9: [[COPY:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr0_vgpr1
+    ; GFX9: [[COPY1:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr2_vgpr3
+    ; GFX9: [[COPY2:%[0-9]+]]:_(<2 x s32>) = COPY $vgpr4_vgpr5
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 32
+    ; GFX9: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 31
+    ; GFX9: [[BUILD_VECTOR:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[C2]](s32), [[C2]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(<2 x s32>) = G_AND [[COPY2]], [[BUILD_VECTOR]]
+    ; GFX9: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; GFX9: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C]], [[UV]]
+    ; GFX9: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[C]], [[UV1]]
+    ; GFX9: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY]](<2 x s32>)
+    ; GFX9: [[UV4:%[0-9]+]]:_(s32), [[UV5:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; GFX9: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[UV2]], [[UV4]](s32)
+    ; GFX9: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[UV3]], [[UV5]](s32)
+    ; GFX9: [[BUILD_VECTOR1:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[SHL]](s32), [[SHL1]](s32)
+    ; GFX9: [[UV6:%[0-9]+]]:_(s32), [[UV7:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY1]](<2 x s32>)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[UV6]], [[SUB]](s32)
+    ; GFX9: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[UV7]], [[SUB1]](s32)
+    ; GFX9: [[BUILD_VECTOR2:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[LSHR]](s32), [[LSHR1]](s32)
+    ; GFX9: [[OR:%[0-9]+]]:_(<2 x s32>) = G_OR [[BUILD_VECTOR1]], [[BUILD_VECTOR2]]
+    ; GFX9: [[UV8:%[0-9]+]]:_(s32), [[UV9:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](<2 x s32>)
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UV8]](s32), [[C1]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UV9]](s32), [[C1]]
+    ; GFX9: [[UV10:%[0-9]+]]:_(s32), [[UV11:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[COPY]](<2 x s32>)
+    ; GFX9: [[UV12:%[0-9]+]]:_(s32), [[UV13:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[OR]](<2 x s32>)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[UV10]], [[UV12]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP1]](s1), [[UV11]], [[UV13]]
+    ; GFX9: [[BUILD_VECTOR3:%[0-9]+]]:_(<2 x s32>) = G_BUILD_VECTOR [[SELECT]](s32), [[SELECT1]](s32)
+    ; GFX9: $vgpr0_vgpr1 = COPY [[BUILD_VECTOR3]](<2 x s32>)
+    %0:_(<2 x s32>) = COPY $vgpr0_vgpr1
+    %1:_(<2 x s32>) = COPY $vgpr2_vgpr3
+    %2:_(<2 x s32>) = COPY $vgpr4_vgpr5
+    %3:_(<2 x s32>) = G_FSHL %0, %1, %2
+    $vgpr0_vgpr1 = COPY %3
+...
+
+---
+name: test_fshl_s16_s16
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1, $vgpr2
+
+    ; SI-LABEL: name: test_fshl_s16_s16
+    ; SI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; SI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; SI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY2]](s32)
+    ; SI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; SI: [[AND:%[0-9]+]]:_(s16) = G_AND [[TRUNC1]], [[C]]
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; SI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[AND]](s16)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C1]], [[ANYEXT]]
+    ; SI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[AND]](s16)
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY3]], [[ZEXT]](s32)
+    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[SHL]](s32)
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY4]], [[C2]]
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY5]], [[C2]]
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND2]], [[AND1]](s32)
+    ; SI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s16) = G_OR [[TRUNC2]], [[TRUNC3]]
+    ; SI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[AND]](s16)
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[ZEXT1]](s32), [[C3]]
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[OR]]
+    ; SI: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; SI: $vgpr0 = COPY [[ANYEXT1]](s32)
+    ; VI-LABEL: name: test_fshl_s16_s16
+    ; VI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; VI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; VI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY2]](s32)
+    ; VI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; VI: [[AND:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C2]]
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[AND]]
+    ; VI: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC]], [[AND]](s16)
+    ; VI: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC1]], [[SUB]](s16)
+    ; VI: [[OR:%[0-9]+]]:_(s16) = G_OR [[SHL]], [[LSHR]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s16), [[C1]]
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[OR]]
+    ; VI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; VI: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; GFX9-LABEL: name: test_fshl_s16_s16
+    ; GFX9: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; GFX9: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; GFX9: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY2]](s32)
+    ; GFX9: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; GFX9: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; GFX9: [[AND:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C2]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC]], [[AND]](s16)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC1]], [[SUB]](s16)
+    ; GFX9: [[OR:%[0-9]+]]:_(s16) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s16), [[C1]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[OR]]
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: $vgpr0 = COPY [[ANYEXT]](s32)
+    %0:_(s32) = COPY $vgpr0
+    %1:_(s32) = COPY $vgpr1
+    %2:_(s32) = COPY $vgpr2
+    %3:_(s16) = G_TRUNC %0
+    %4:_(s16) = G_TRUNC %1
+    %5:_(s16) = G_TRUNC %2
+    %6:_(s16) = G_FSHL %3, %4, %5
+    %7:_(s32) = G_ANYEXT %6
+    $vgpr0 = COPY %7
+...
+
+---
+name: test_fshl_v2s16_v2s16
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1, $vgpr2
+
+    ; SI-LABEL: name: test_fshl_v2s16_v2s16
+    ; SI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; SI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; SI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[C]], [[C1]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; SI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[COPY2]], [[BITCAST]]
+    ; SI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C1]](s32)
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[BITCAST1]](s32)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY4]], [[COPY5]]
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[C1]], [[COPY6]]
+    ; SI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[COPY]](<2 x s16>)
+    ; SI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C1]](s32)
+    ; SI: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C1]](s32)
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[BITCAST3]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[BITCAST2]](s32)
+    ; SI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND1]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[LSHR2]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[LSHR1]](s32)
+    ; SI: [[SHL2:%[0-9]+]]:_(s32) = G_SHL [[COPY10]], [[AND2]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL1]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY11]], [[C2]]
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[SHL2]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY12]], [[C2]]
+    ; SI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[AND4]], [[C1]](s32)
+    ; SI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[AND3]], [[SHL3]]
+    ; SI: [[BITCAST4:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; SI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[COPY1]](<2 x s16>)
+    ; SI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C1]](s32)
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[BITCAST5]](s32)
+    ; SI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; SI: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[AND6]], [[AND5]](s32)
+    ; SI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[SUB1]](s32)
+    ; SI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY15]], [[C2]]
+    ; SI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[LSHR3]](s32)
+    ; SI: [[AND8:%[0-9]+]]:_(s32) = G_AND [[COPY16]], [[C2]]
+    ; SI: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[AND8]], [[AND7]](s32)
+    ; SI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[LSHR4]](s32)
+    ; SI: [[AND9:%[0-9]+]]:_(s32) = G_AND [[COPY17]], [[C2]]
+    ; SI: [[COPY18:%[0-9]+]]:_(s32) = COPY [[LSHR5]](s32)
+    ; SI: [[AND10:%[0-9]+]]:_(s32) = G_AND [[COPY18]], [[C2]]
+    ; SI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[AND10]], [[C1]](s32)
+    ; SI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[AND9]], [[SHL4]]
+    ; SI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; SI: [[OR3:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST4]], [[BITCAST6]]
+    ; SI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C1]](s32)
+    ; SI: [[COPY19:%[0-9]+]]:_(s32) = COPY [[BITCAST7]](s32)
+    ; SI: [[AND11:%[0-9]+]]:_(s32) = G_AND [[COPY19]], [[C2]]
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[COPY20:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND11]](s32), [[COPY20]]
+    ; SI: [[COPY21:%[0-9]+]]:_(s32) = COPY [[LSHR6]](s32)
+    ; SI: [[AND12:%[0-9]+]]:_(s32) = G_AND [[COPY21]], [[C2]]
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND12]](s32), [[C3]]
+    ; SI: [[BITCAST8:%[0-9]+]]:_(s32) = G_BITCAST [[COPY]](<2 x s16>)
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST8]](s32)
+    ; SI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST8]], [[C1]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; SI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[OR3]](<2 x s16>)
+    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; SI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C1]](s32)
+    ; SI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR8]](s32)
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC2]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC1]], [[TRUNC3]]
+    ; SI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; SI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; SI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C1]](s32)
+    ; SI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL5]]
+    ; SI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; SI: $vgpr0 = COPY [[BITCAST10]](<2 x s16>)
+    ; VI-LABEL: name: test_fshl_v2s16_v2s16
+    ; VI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; VI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; VI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; VI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[C2]], [[C3]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; VI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[COPY2]], [[BITCAST]]
+    ; VI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C3]](s32)
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR]](s32)
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC1]]
+    ; VI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[COPY]](<2 x s16>)
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; VI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C3]](s32)
+    ; VI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR1]](s32)
+    ; VI: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST3]](s32)
+    ; VI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C3]](s32)
+    ; VI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; VI: [[SHL1:%[0-9]+]]:_(s16) = G_SHL [[TRUNC2]], [[TRUNC4]](s16)
+    ; VI: [[SHL2:%[0-9]+]]:_(s16) = G_SHL [[TRUNC3]], [[TRUNC5]](s16)
+    ; VI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SHL1]](s16)
+    ; VI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SHL2]](s16)
+    ; VI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C3]](s32)
+    ; VI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL3]]
+    ; VI: [[BITCAST4:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; VI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[COPY1]](<2 x s16>)
+    ; VI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST5]](s32)
+    ; VI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C3]](s32)
+    ; VI: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; VI: [[LSHR4:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC6]], [[SUB]](s16)
+    ; VI: [[LSHR5:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC7]], [[SUB1]](s16)
+    ; VI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR4]](s16)
+    ; VI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR5]](s16)
+    ; VI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C3]](s32)
+    ; VI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL4]]
+    ; VI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; VI: [[OR3:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST4]], [[BITCAST6]]
+    ; VI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST7]](s32)
+    ; VI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C3]](s32)
+    ; VI: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR6]](s32)
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC8]](s16), [[C1]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC9]](s16), [[C1]]
+    ; VI: [[BITCAST8:%[0-9]+]]:_(s32) = G_BITCAST [[COPY]](<2 x s16>)
+    ; VI: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST8]](s32)
+    ; VI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST8]], [[C3]](s32)
+    ; VI: [[TRUNC11:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; VI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[OR3]](<2 x s16>)
+    ; VI: [[TRUNC12:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; VI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C3]](s32)
+    ; VI: [[TRUNC13:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR8]](s32)
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC10]], [[TRUNC12]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC11]], [[TRUNC13]]
+    ; VI: [[ZEXT4:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; VI: [[ZEXT5:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; VI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[ZEXT5]], [[C3]](s32)
+    ; VI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT4]], [[SHL5]]
+    ; VI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; VI: $vgpr0 = COPY [[BITCAST10]](<2 x s16>)
+    ; GFX9-LABEL: name: test_fshl_v2s16_v2s16
+    ; GFX9: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; GFX9: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; GFX9: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY3]](s32), [[COPY4]](s32)
+    ; GFX9: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC1:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY5]](s32), [[C2]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[COPY2]], [[BUILD_VECTOR_TRUNC1]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(<2 x s16>) = G_SHL [[COPY]], [[AND]](<2 x s16>)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[COPY1]], [[SUB]](<2 x s16>)
+    ; GFX9: [[OR:%[0-9]+]]:_(<2 x s16>) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[BITCAST:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST]](s32)
+    ; GFX9: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST]], [[C]](s32)
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR1]](s32)
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC]](s16), [[C1]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC1]](s16), [[C1]]
+    ; GFX9: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[COPY]](<2 x s16>)
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; GFX9: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C]](s32)
+    ; GFX9: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; GFX9: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[OR]](<2 x s16>)
+    ; GFX9: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; GFX9: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C]](s32)
+    ; GFX9: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC2]], [[TRUNC4]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC3]], [[TRUNC5]]
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT1]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC2:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT]](s32), [[ANYEXT1]](s32)
+    ; GFX9: $vgpr0 = COPY [[BUILD_VECTOR_TRUNC2]](<2 x s16>)
+    %0:_(<2 x s16>) = COPY $vgpr0
+    %1:_(<2 x s16>) = COPY $vgpr1
+    %2:_(<2 x s16>) = COPY $vgpr2
+    %3:_(<2 x s16>) = G_FSHL %0, %1, %2
+    $vgpr0 = COPY %3
+...
+
+---
+name: test_fshl_s64_s64
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1, $vgpr2_vgpr3, $vgpr4_vgpr5
+
+    ; SI-LABEL: name: test_fshl_s64_s64
+    ; SI: [[COPY:%[0-9]+]]:_(s64) = COPY $vgpr0_vgpr1
+    ; SI: [[COPY1:%[0-9]+]]:_(s64) = COPY $vgpr2_vgpr3
+    ; SI: [[COPY2:%[0-9]+]]:_(s64) = COPY $vgpr4_vgpr5
+    ; SI: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 64
+    ; SI: [[C1:%[0-9]+]]:_(s64) = G_CONSTANT i64 0
+    ; SI: [[C2:%[0-9]+]]:_(s64) = G_CONSTANT i64 63
+    ; SI: [[AND:%[0-9]+]]:_(s64) = G_AND [[COPY2]], [[C2]]
+    ; SI: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
+    ; SI: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](s64)
+    ; SI: [[USUBO:%[0-9]+]]:_(s32), [[USUBO1:%[0-9]+]]:_(s1) = G_USUBO [[UV]], [[UV2]]
+    ; SI: [[USUBE:%[0-9]+]]:_(s32), [[USUBE1:%[0-9]+]]:_(s1) = G_USUBE [[UV1]], [[UV3]], [[USUBO1]]
+    ; SI: [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[AND]](s64)
+    ; SI: [[SHL:%[0-9]+]]:_(s64) = G_SHL [[COPY]], [[TRUNC]](s32)
+    ; SI: [[LSHR:%[0-9]+]]:_(s64) = G_LSHR [[COPY1]], [[USUBO]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s64) = G_OR [[SHL]], [[LSHR]]
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s64), [[C1]]
+    ; SI: [[SELECT:%[0-9]+]]:_(s64) = G_SELECT [[ICMP]](s1), [[COPY]], [[OR]]
+    ; SI: $vgpr0_vgpr1 = COPY [[SELECT]](s64)
+    ; VI-LABEL: name: test_fshl_s64_s64
+    ; VI: [[COPY:%[0-9]+]]:_(s64) = COPY $vgpr0_vgpr1
+    ; VI: [[COPY1:%[0-9]+]]:_(s64) = COPY $vgpr2_vgpr3
+    ; VI: [[COPY2:%[0-9]+]]:_(s64) = COPY $vgpr4_vgpr5
+    ; VI: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 64
+    ; VI: [[C1:%[0-9]+]]:_(s64) = G_CONSTANT i64 0
+    ; VI: [[C2:%[0-9]+]]:_(s64) = G_CONSTANT i64 63
+    ; VI: [[AND:%[0-9]+]]:_(s64) = G_AND [[COPY2]], [[C2]]
+    ; VI: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
+    ; VI: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](s64)
+    ; VI: [[USUBO:%[0-9]+]]:_(s32), [[USUBO1:%[0-9]+]]:_(s1) = G_USUBO [[UV]], [[UV2]]
+    ; VI: [[USUBE:%[0-9]+]]:_(s32), [[USUBE1:%[0-9]+]]:_(s1) = G_USUBE [[UV1]], [[UV3]], [[USUBO1]]
+    ; VI: [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[AND]](s64)
+    ; VI: [[SHL:%[0-9]+]]:_(s64) = G_SHL [[COPY]], [[TRUNC]](s32)
+    ; VI: [[LSHR:%[0-9]+]]:_(s64) = G_LSHR [[COPY1]], [[USUBO]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s64) = G_OR [[SHL]], [[LSHR]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s64), [[C1]]
+    ; VI: [[SELECT:%[0-9]+]]:_(s64) = G_SELECT [[ICMP]](s1), [[COPY]], [[OR]]
+    ; VI: $vgpr0_vgpr1 = COPY [[SELECT]](s64)
+    ; GFX9-LABEL: name: test_fshl_s64_s64
+    ; GFX9: [[COPY:%[0-9]+]]:_(s64) = COPY $vgpr0_vgpr1
+    ; GFX9: [[COPY1:%[0-9]+]]:_(s64) = COPY $vgpr2_vgpr3
+    ; GFX9: [[COPY2:%[0-9]+]]:_(s64) = COPY $vgpr4_vgpr5
+    ; GFX9: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 64
+    ; GFX9: [[C1:%[0-9]+]]:_(s64) = G_CONSTANT i64 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s64) = G_CONSTANT i64 63
+    ; GFX9: [[AND:%[0-9]+]]:_(s64) = G_AND [[COPY2]], [[C2]]
+    ; GFX9: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
+    ; GFX9: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](s64)
+    ; GFX9: [[USUBO:%[0-9]+]]:_(s32), [[USUBO1:%[0-9]+]]:_(s1) = G_USUBO [[UV]], [[UV2]]
+    ; GFX9: [[USUBE:%[0-9]+]]:_(s32), [[USUBE1:%[0-9]+]]:_(s1) = G_USUBE [[UV1]], [[UV3]], [[USUBO1]]
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[AND]](s64)
+    ; GFX9: [[SHL:%[0-9]+]]:_(s64) = G_SHL [[COPY]], [[TRUNC]](s32)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s64) = G_LSHR [[COPY1]], [[USUBO]](s32)
+    ; GFX9: [[OR:%[0-9]+]]:_(s64) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s64), [[C1]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s64) = G_SELECT [[ICMP]](s1), [[COPY]], [[OR]]
+    ; GFX9: $vgpr0_vgpr1 = COPY [[SELECT]](s64)
+    %0:_(s64) = COPY $vgpr0_vgpr1
+    %1:_(s64) = COPY $vgpr2_vgpr3
+    %2:_(s64) = COPY $vgpr4_vgpr5
+    %3:_(s64) = G_FSHL %0, %1, %2
+    $vgpr0_vgpr1 = COPY %3
+...
+
+---
+name: test_fshl_s8_s8
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1, $vgpr2
+
+    ; SI-LABEL: name: test_fshl_s8_s8
+    ; SI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; SI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; SI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 8
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[COPY4]]
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 255
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C3]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND1]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C3]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C3]]
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND3]], [[AND2]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C3]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C3]]
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND4]](s32), [[AND5]]
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[OR]](s32)
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC1]]
+    ; SI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; SI: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; VI-LABEL: name: test_fshl_s8_s8
+    ; VI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; VI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; VI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; VI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; VI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[COPY4]]
+    ; VI: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 8
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[AND]](s32)
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C2]], [[TRUNC]]
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 255
+    ; VI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; VI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY5]], [[C3]]
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; VI: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC1]], [[AND1]](s32)
+    ; VI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SUB]](s16)
+    ; VI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[ANYEXT]], [[C3]]
+    ; VI: [[C4:%[0-9]+]]:_(s16) = G_CONSTANT i16 255
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; VI: [[AND3:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C4]]
+    ; VI: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[AND3]], [[AND2]](s32)
+    ; VI: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SHL]](s16)
+    ; VI: [[ANYEXT2:%[0-9]+]]:_(s32) = G_ANYEXT [[LSHR]](s16)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[ANYEXT1]], [[ANYEXT2]]
+    ; VI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; VI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY6]], [[C3]]
+    ; VI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; VI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C3]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND4]](s32), [[AND5]]
+    ; VI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; VI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[OR]](s32)
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC3]], [[TRUNC4]]
+    ; VI: [[ANYEXT3:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; VI: $vgpr0 = COPY [[ANYEXT3]](s32)
+    ; GFX9-LABEL: name: test_fshl_s8_s8
+    ; GFX9: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; GFX9: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; GFX9: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[COPY4]]
+    ; GFX9: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 8
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[AND]](s32)
+    ; GFX9: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C2]], [[TRUNC]]
+    ; GFX9: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 255
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; GFX9: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY5]], [[C3]]
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; GFX9: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC1]], [[AND1]](s32)
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SUB]](s16)
+    ; GFX9: [[AND2:%[0-9]+]]:_(s32) = G_AND [[ANYEXT]], [[C3]]
+    ; GFX9: [[C4:%[0-9]+]]:_(s16) = G_CONSTANT i16 255
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; GFX9: [[AND3:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C4]]
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[AND3]], [[AND2]](s32)
+    ; GFX9: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SHL]](s16)
+    ; GFX9: [[ANYEXT2:%[0-9]+]]:_(s32) = G_ANYEXT [[LSHR]](s16)
+    ; GFX9: [[OR:%[0-9]+]]:_(s32) = G_OR [[ANYEXT1]], [[ANYEXT2]]
+    ; GFX9: [[COPY6:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; GFX9: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY6]], [[C3]]
+    ; GFX9: [[COPY7:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C3]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND4]](s32), [[AND5]]
+    ; GFX9: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; GFX9: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[OR]](s32)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC3]], [[TRUNC4]]
+    ; GFX9: [[ANYEXT3:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: $vgpr0 = COPY [[ANYEXT3]](s32)
+    %0:_(s32) = COPY $vgpr0
+    %1:_(s32) = COPY $vgpr1
+    %2:_(s32) = COPY $vgpr2
+    %3:_(s8) = G_TRUNC %0
+    %4:_(s8) = G_TRUNC %1
+    %5:_(s8) = G_TRUNC %2
+    %6:_(s8) = G_FSHL %3, %4, %5
+    %7:_(s32) = G_ANYEXT %6
+    $vgpr0 = COPY %7
+...
+
+---
+name: test_fshl_s24_s24
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1, $vgpr2
+
+    ; SI-LABEL: name: test_fshl_s24_s24
+    ; SI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; SI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; SI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 24
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 16777215
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[C2]]
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY4]], [[C2]]
+    ; SI: [[UITOFP:%[0-9]+]]:_(s32) = G_UITOFP [[AND1]](s32)
+    ; SI: [[AMDGPU_RCP_IFLAG:%[0-9]+]]:_(s32) = G_AMDGPU_RCP_IFLAG [[UITOFP]](s32)
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_FCONSTANT float 0x41F0000000000000
+    ; SI: [[FMUL:%[0-9]+]]:_(s32) = G_FMUL [[AMDGPU_RCP_IFLAG]], [[C3]]
+    ; SI: [[FPTOUI:%[0-9]+]]:_(s32) = G_FPTOUI [[FMUL]](s32)
+    ; SI: [[MUL:%[0-9]+]]:_(s32) = G_MUL [[FPTOUI]], [[AND1]]
+    ; SI: [[UMULH:%[0-9]+]]:_(s32) = G_UMULH [[FPTOUI]], [[AND1]]
+    ; SI: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C4]], [[MUL]]
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UMULH]](s32), [[C4]]
+    ; SI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[SUB]], [[MUL]]
+    ; SI: [[UMULH1:%[0-9]+]]:_(s32) = G_UMULH [[SELECT]], [[FPTOUI]]
+    ; SI: [[ADD:%[0-9]+]]:_(s32) = G_ADD [[FPTOUI]], [[UMULH1]]
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[FPTOUI]], [[UMULH1]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[ADD]], [[SUB1]]
+    ; SI: [[UMULH2:%[0-9]+]]:_(s32) = G_UMULH [[SELECT1]], [[AND]]
+    ; SI: [[MUL1:%[0-9]+]]:_(s32) = G_MUL [[UMULH2]], [[AND1]]
+    ; SI: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[AND]], [[MUL1]]
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[SUB2]](s32), [[AND1]]
+    ; SI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[AND]](s32), [[MUL1]]
+    ; SI: [[AND2:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP2]]
+    ; SI: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[SUB2]], [[AND1]]
+    ; SI: [[ADD1:%[0-9]+]]:_(s32) = G_ADD [[SUB2]], [[AND1]]
+    ; SI: [[SELECT2:%[0-9]+]]:_(s32) = G_SELECT [[AND2]](s1), [[SUB3]], [[SUB2]]
+    ; SI: [[SELECT3:%[0-9]+]]:_(s32) = G_SELECT [[ICMP2]](s1), [[SELECT2]], [[ADD1]]
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; SI: [[SUB4:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND3]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SUB4]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C2]]
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND5]], [[AND4]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; SI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; SI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND6]](s32), [[AND7]]
+    ; SI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; SI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[OR]](s32)
+    ; SI: [[SELECT4:%[0-9]+]]:_(s32) = G_SELECT [[ICMP3]](s1), [[COPY15]], [[COPY16]]
+    ; SI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[SELECT4]](s32)
+    ; SI: $vgpr0 = COPY [[COPY17]](s32)
+    ; VI-LABEL: name: test_fshl_s24_s24
+    ; VI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; VI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; VI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; VI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 24
+    ; VI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 16777215
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[C2]]
+    ; VI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; VI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY4]], [[C2]]
+    ; VI: [[UITOFP:%[0-9]+]]:_(s32) = G_UITOFP [[AND1]](s32)
+    ; VI: [[AMDGPU_RCP_IFLAG:%[0-9]+]]:_(s32) = G_AMDGPU_RCP_IFLAG [[UITOFP]](s32)
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_FCONSTANT float 0x41F0000000000000
+    ; VI: [[FMUL:%[0-9]+]]:_(s32) = G_FMUL [[AMDGPU_RCP_IFLAG]], [[C3]]
+    ; VI: [[FPTOUI:%[0-9]+]]:_(s32) = G_FPTOUI [[FMUL]](s32)
+    ; VI: [[MUL:%[0-9]+]]:_(s32) = G_MUL [[FPTOUI]], [[AND1]]
+    ; VI: [[UMULH:%[0-9]+]]:_(s32) = G_UMULH [[FPTOUI]], [[AND1]]
+    ; VI: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C4]], [[MUL]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UMULH]](s32), [[C4]]
+    ; VI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[SUB]], [[MUL]]
+    ; VI: [[UMULH1:%[0-9]+]]:_(s32) = G_UMULH [[SELECT]], [[FPTOUI]]
+    ; VI: [[ADD:%[0-9]+]]:_(s32) = G_ADD [[FPTOUI]], [[UMULH1]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[FPTOUI]], [[UMULH1]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[ADD]], [[SUB1]]
+    ; VI: [[UMULH2:%[0-9]+]]:_(s32) = G_UMULH [[SELECT1]], [[AND]]
+    ; VI: [[MUL1:%[0-9]+]]:_(s32) = G_MUL [[UMULH2]], [[AND1]]
+    ; VI: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[AND]], [[MUL1]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[SUB2]](s32), [[AND1]]
+    ; VI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[AND]](s32), [[MUL1]]
+    ; VI: [[AND2:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP2]]
+    ; VI: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[SUB2]], [[AND1]]
+    ; VI: [[ADD1:%[0-9]+]]:_(s32) = G_ADD [[SUB2]], [[AND1]]
+    ; VI: [[SELECT2:%[0-9]+]]:_(s32) = G_SELECT [[AND2]](s1), [[SUB3]], [[SUB2]]
+    ; VI: [[SELECT3:%[0-9]+]]:_(s32) = G_SELECT [[ICMP2]](s1), [[SELECT2]], [[ADD1]]
+    ; VI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; VI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; VI: [[SUB4:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; VI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; VI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; VI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND3]](s32)
+    ; VI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SUB4]](s32)
+    ; VI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; VI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; VI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C2]]
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND5]], [[AND4]](s32)
+    ; VI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; VI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; VI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; VI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; VI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; VI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; VI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND6]](s32), [[AND7]]
+    ; VI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; VI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[OR]](s32)
+    ; VI: [[SELECT4:%[0-9]+]]:_(s32) = G_SELECT [[ICMP3]](s1), [[COPY15]], [[COPY16]]
+    ; VI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[SELECT4]](s32)
+    ; VI: $vgpr0 = COPY [[COPY17]](s32)
+    ; GFX9-LABEL: name: test_fshl_s24_s24
+    ; GFX9: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
+    ; GFX9: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
+    ; GFX9: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 24
+    ; GFX9: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 16777215
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[C2]]
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY4]], [[C2]]
+    ; GFX9: [[UITOFP:%[0-9]+]]:_(s32) = G_UITOFP [[AND1]](s32)
+    ; GFX9: [[AMDGPU_RCP_IFLAG:%[0-9]+]]:_(s32) = G_AMDGPU_RCP_IFLAG [[UITOFP]](s32)
+    ; GFX9: [[C3:%[0-9]+]]:_(s32) = G_FCONSTANT float 0x41F0000000000000
+    ; GFX9: [[FMUL:%[0-9]+]]:_(s32) = G_FMUL [[AMDGPU_RCP_IFLAG]], [[C3]]
+    ; GFX9: [[FPTOUI:%[0-9]+]]:_(s32) = G_FPTOUI [[FMUL]](s32)
+    ; GFX9: [[MUL:%[0-9]+]]:_(s32) = G_MUL [[FPTOUI]], [[AND1]]
+    ; GFX9: [[UMULH:%[0-9]+]]:_(s32) = G_UMULH [[FPTOUI]], [[AND1]]
+    ; GFX9: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C4]], [[MUL]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UMULH]](s32), [[C4]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[SUB]], [[MUL]]
+    ; GFX9: [[UMULH1:%[0-9]+]]:_(s32) = G_UMULH [[SELECT]], [[FPTOUI]]
+    ; GFX9: [[ADD:%[0-9]+]]:_(s32) = G_ADD [[FPTOUI]], [[UMULH1]]
+    ; GFX9: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[FPTOUI]], [[UMULH1]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[ADD]], [[SUB1]]
+    ; GFX9: [[UMULH2:%[0-9]+]]:_(s32) = G_UMULH [[SELECT1]], [[AND]]
+    ; GFX9: [[MUL1:%[0-9]+]]:_(s32) = G_MUL [[UMULH2]], [[AND1]]
+    ; GFX9: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[AND]], [[MUL1]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[SUB2]](s32), [[AND1]]
+    ; GFX9: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[AND]](s32), [[MUL1]]
+    ; GFX9: [[AND2:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP2]]
+    ; GFX9: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[SUB2]], [[AND1]]
+    ; GFX9: [[ADD1:%[0-9]+]]:_(s32) = G_ADD [[SUB2]], [[AND1]]
+    ; GFX9: [[SELECT2:%[0-9]+]]:_(s32) = G_SELECT [[AND2]](s1), [[SUB3]], [[SUB2]]
+    ; GFX9: [[SELECT3:%[0-9]+]]:_(s32) = G_SELECT [[ICMP2]](s1), [[SELECT2]], [[ADD1]]
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY6:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; GFX9: [[SUB4:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; GFX9: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; GFX9: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; GFX9: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; GFX9: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND3]](s32)
+    ; GFX9: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SUB4]](s32)
+    ; GFX9: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; GFX9: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; GFX9: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C2]]
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND5]], [[AND4]](s32)
+    ; GFX9: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; GFX9: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; GFX9: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; GFX9: [[COPY13:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; GFX9: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; GFX9: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; GFX9: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; GFX9: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND6]](s32), [[AND7]]
+    ; GFX9: [[COPY15:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; GFX9: [[COPY16:%[0-9]+]]:_(s32) = COPY [[OR]](s32)
+    ; GFX9: [[SELECT4:%[0-9]+]]:_(s32) = G_SELECT [[ICMP3]](s1), [[COPY15]], [[COPY16]]
+    ; GFX9: [[COPY17:%[0-9]+]]:_(s32) = COPY [[SELECT4]](s32)
+    ; GFX9: $vgpr0 = COPY [[COPY17]](s32)
+    %0:_(s32) = COPY $vgpr0
+    %1:_(s32) = COPY $vgpr1
+    %2:_(s32) = COPY $vgpr2
+    %3:_(s24) = G_TRUNC %0
+    %4:_(s24) = G_TRUNC %1
+    %5:_(s24) = G_TRUNC %2
+    %6:_(s24) = G_FSHL %3, %4, %5
+    %7:_(s32) = G_ANYEXT %6
+    $vgpr0 = COPY %7
+...
+
+---
+name: test_fshl_v3s16_v3s16
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4, $vgpr5
+
+    ; SI-LABEL: name: test_fshl_v3s16_v3s16
+    ; SI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; SI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; SI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; SI: [[COPY3:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr3
+    ; SI: [[COPY4:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr4
+    ; SI: [[COPY5:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr5
+    ; SI: [[DEF:%[0-9]+]]:_(<2 x s16>) = G_IMPLICIT_DEF
+    ; SI: [[CONCAT_VECTORS:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY]](<2 x s16>), [[COPY1]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; SI: [[UV:%[0-9]+]]:_(<3 x s16>), [[UV1:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS]](<6 x s16>)
+    ; SI: [[CONCAT_VECTORS1:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY2]](<2 x s16>), [[COPY3]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; SI: [[UV2:%[0-9]+]]:_(<3 x s16>), [[UV3:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS1]](<6 x s16>)
+    ; SI: [[CONCAT_VECTORS2:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY4]](<2 x s16>), [[COPY5]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; SI: [[UV4:%[0-9]+]]:_(<3 x s16>), [[UV5:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS2]](<6 x s16>)
+    ; SI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; SI: [[BUILD_VECTOR:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C]](s16), [[C]](s16), [[C]](s16)
+    ; SI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; SI: [[BUILD_VECTOR1:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C1]](s16), [[C1]](s16), [[C1]](s16)
+    ; SI: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; SI: [[BUILD_VECTOR2:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C2]](s16), [[C2]](s16), [[C2]](s16)
+    ; SI: [[AND:%[0-9]+]]:_(<3 x s16>) = G_AND [[UV4]], [[BUILD_VECTOR2]]
+    ; SI: [[SUB:%[0-9]+]]:_(<3 x s16>) = G_SUB [[BUILD_VECTOR]], [[AND]]
+    ; SI: [[SHL:%[0-9]+]]:_(<3 x s16>) = G_SHL [[UV]], [[AND]](<3 x s16>)
+    ; SI: [[LSHR:%[0-9]+]]:_(<3 x s16>) = G_LSHR [[UV2]], [[SUB]](<3 x s16>)
+    ; SI: [[OR:%[0-9]+]]:_(<3 x s16>) = G_OR [[SHL]], [[LSHR]]
+    ; SI: [[ICMP:%[0-9]+]]:_(<3 x s1>) = G_ICMP intpred(eq), [[AND]](<3 x s16>), [[BUILD_VECTOR1]]
+    ; SI: [[SELECT:%[0-9]+]]:_(<3 x s16>) = G_SELECT [[ICMP]](<3 x s1>), [[UV]], [[OR]]
+    ; SI: [[DEF1:%[0-9]+]]:_(<4 x s16>) = G_IMPLICIT_DEF
+    ; SI: [[EXTRACT:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[DEF1]](<4 x s16>), 0
+    ; SI: [[CONCAT_VECTORS3:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[SELECT]](<3 x s16>), [[EXTRACT]](<3 x s16>)
+    ; SI: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>), [[UV8:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS3]](<6 x s16>)
+    ; SI: $vgpr0 = COPY [[UV6]](<2 x s16>)
+    ; SI: $vgpr1 = COPY [[UV7]](<2 x s16>)
+    ; SI: $vgpr2 = COPY [[UV8]](<2 x s16>)
+    ; VI-LABEL: name: test_fshl_v3s16_v3s16
+    ; VI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; VI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; VI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; VI: [[COPY3:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr3
+    ; VI: [[COPY4:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr4
+    ; VI: [[COPY5:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr5
+    ; VI: [[DEF:%[0-9]+]]:_(<2 x s16>) = G_IMPLICIT_DEF
+    ; VI: [[CONCAT_VECTORS:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY]](<2 x s16>), [[COPY1]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; VI: [[UV:%[0-9]+]]:_(<3 x s16>), [[UV1:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS]](<6 x s16>)
+    ; VI: [[CONCAT_VECTORS1:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY2]](<2 x s16>), [[COPY3]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; VI: [[UV2:%[0-9]+]]:_(<3 x s16>), [[UV3:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS1]](<6 x s16>)
+    ; VI: [[CONCAT_VECTORS2:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY4]](<2 x s16>), [[COPY5]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; VI: [[UV4:%[0-9]+]]:_(<3 x s16>), [[UV5:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS2]](<6 x s16>)
+    ; VI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[BUILD_VECTOR:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C]](s16), [[C]](s16), [[C]](s16)
+    ; VI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[BUILD_VECTOR1:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C1]](s16), [[C1]](s16), [[C1]](s16)
+    ; VI: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; VI: [[BUILD_VECTOR2:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C2]](s16), [[C2]](s16), [[C2]](s16)
+    ; VI: [[AND:%[0-9]+]]:_(<3 x s16>) = G_AND [[UV4]], [[BUILD_VECTOR2]]
+    ; VI: [[SUB:%[0-9]+]]:_(<3 x s16>) = G_SUB [[BUILD_VECTOR]], [[AND]]
+    ; VI: [[SHL:%[0-9]+]]:_(<3 x s16>) = G_SHL [[UV]], [[AND]](<3 x s16>)
+    ; VI: [[LSHR:%[0-9]+]]:_(<3 x s16>) = G_LSHR [[UV2]], [[SUB]](<3 x s16>)
+    ; VI: [[OR:%[0-9]+]]:_(<3 x s16>) = G_OR [[SHL]], [[LSHR]]
+    ; VI: [[ICMP:%[0-9]+]]:_(<3 x s1>) = G_ICMP intpred(eq), [[AND]](<3 x s16>), [[BUILD_VECTOR1]]
+    ; VI: [[SELECT:%[0-9]+]]:_(<3 x s16>) = G_SELECT [[ICMP]](<3 x s1>), [[UV]], [[OR]]
+    ; VI: [[DEF1:%[0-9]+]]:_(<4 x s16>) = G_IMPLICIT_DEF
+    ; VI: [[EXTRACT:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[DEF1]](<4 x s16>), 0
+    ; VI: [[CONCAT_VECTORS3:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[SELECT]](<3 x s16>), [[EXTRACT]](<3 x s16>)
+    ; VI: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>), [[UV8:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS3]](<6 x s16>)
+    ; VI: $vgpr0 = COPY [[UV6]](<2 x s16>)
+    ; VI: $vgpr1 = COPY [[UV7]](<2 x s16>)
+    ; VI: $vgpr2 = COPY [[UV8]](<2 x s16>)
+    ; GFX9-LABEL: name: test_fshl_v3s16_v3s16
+    ; GFX9: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; GFX9: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; GFX9: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; GFX9: [[COPY3:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr3
+    ; GFX9: [[COPY4:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr4
+    ; GFX9: [[COPY5:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr5
+    ; GFX9: [[DEF:%[0-9]+]]:_(<2 x s16>) = G_IMPLICIT_DEF
+    ; GFX9: [[CONCAT_VECTORS:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY]](<2 x s16>), [[COPY1]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; GFX9: [[UV:%[0-9]+]]:_(<3 x s16>), [[UV1:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS]](<6 x s16>)
+    ; GFX9: [[CONCAT_VECTORS1:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY2]](<2 x s16>), [[COPY3]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; GFX9: [[UV2:%[0-9]+]]:_(<3 x s16>), [[UV3:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS1]](<6 x s16>)
+    ; GFX9: [[CONCAT_VECTORS2:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY4]](<2 x s16>), [[COPY5]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; GFX9: [[UV4:%[0-9]+]]:_(<3 x s16>), [[UV5:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS2]](<6 x s16>)
+    ; GFX9: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; GFX9: [[BUILD_VECTOR:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C]](s16), [[C]](s16), [[C]](s16)
+    ; GFX9: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[BUILD_VECTOR1:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C1]](s16), [[C1]](s16), [[C1]](s16)
+    ; GFX9: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; GFX9: [[BUILD_VECTOR2:%[0-9]+]]:_(<3 x s16>) = G_BUILD_VECTOR [[C2]](s16), [[C2]](s16), [[C2]](s16)
+    ; GFX9: [[AND:%[0-9]+]]:_(<3 x s16>) = G_AND [[UV4]], [[BUILD_VECTOR2]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(<3 x s16>) = G_SUB [[BUILD_VECTOR]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(<3 x s16>) = G_SHL [[UV]], [[AND]](<3 x s16>)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(<3 x s16>) = G_LSHR [[UV2]], [[SUB]](<3 x s16>)
+    ; GFX9: [[OR:%[0-9]+]]:_(<3 x s16>) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(<3 x s1>) = G_ICMP intpred(eq), [[AND]](<3 x s16>), [[BUILD_VECTOR1]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(<3 x s16>) = G_SELECT [[ICMP]](<3 x s1>), [[UV]], [[OR]]
+    ; GFX9: [[DEF1:%[0-9]+]]:_(<4 x s16>) = G_IMPLICIT_DEF
+    ; GFX9: [[EXTRACT:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[DEF1]](<4 x s16>), 0
+    ; GFX9: [[CONCAT_VECTORS3:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[SELECT]](<3 x s16>), [[EXTRACT]](<3 x s16>)
+    ; GFX9: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>), [[UV8:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS3]](<6 x s16>)
+    ; GFX9: $vgpr0 = COPY [[UV6]](<2 x s16>)
+    ; GFX9: $vgpr1 = COPY [[UV7]](<2 x s16>)
+    ; GFX9: $vgpr2 = COPY [[UV8]](<2 x s16>)
+    %0:_(<2 x s16>) = COPY $vgpr0
+    %1:_(<2 x s16>) = COPY $vgpr1
+    %2:_(<2 x s16>) = COPY $vgpr2
+    %3:_(<2 x s16>) = COPY $vgpr3
+    %4:_(<2 x s16>) = COPY $vgpr4
+    %5:_(<2 x s16>) = COPY $vgpr5
+    %6:_(<2 x s16>) = G_IMPLICIT_DEF
+    %7:_(<6 x s16>) = G_CONCAT_VECTORS %0(<2 x s16>), %1(<2 x s16>), %6(<2 x s16>)
+    %8:_(<3 x s16>), %9:_(<3 x s16>) = G_UNMERGE_VALUES %7(<6 x s16>)
+    %10:_(<6 x s16>) = G_CONCAT_VECTORS %2(<2 x s16>), %3(<2 x s16>), %6(<2 x s16>)
+    %11:_(<3 x s16>), %12:_(<3 x s16>) = G_UNMERGE_VALUES %10(<6 x s16>)
+    %13:_(<6 x s16>) = G_CONCAT_VECTORS %4(<2 x s16>), %5(<2 x s16>), %6(<2 x s16>)
+    %14:_(<3 x s16>), %15:_(<3 x s16>) = G_UNMERGE_VALUES %13(<6 x s16>)
+    %16:_(<3 x s16>) = G_FSHL %8, %11, %14(<3 x s16>)
+    %17:_(<3 x s16>) = G_IMPLICIT_DEF
+    %18:_(<6 x s16>) = G_CONCAT_VECTORS %16(<3 x s16>), %17(<3 x s16>)
+    %19:_(<2 x s16>), %20:_(<2 x s16>), %21:_(<2 x s16>) = G_UNMERGE_VALUES %18(<6 x s16>)
+    $vgpr0 = COPY %19(<2 x s16>)
+    $vgpr1 = COPY %20(<2 x s16>)
+    $vgpr2 = COPY %21(<2 x s16>)
+...
+
+---
+name: test_fshl_v4s16_v4s16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1, $vgpr2_vgpr3, $vgpr4_vgpr5
+
+    ; SI-LABEL: name: test_fshl_v4s16_v4s16
+    ; SI: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    ; SI: [[COPY1:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    ; SI: [[COPY2:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY4]], [[C1]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; SI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[C]], [[C1]](s32)
+    ; SI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[COPY5]], [[SHL1]]
+    ; SI: [[BITCAST1:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; SI: [[CONCAT_VECTORS:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST]](<2 x s16>), [[BITCAST1]](<2 x s16>)
+    ; SI: [[AND:%[0-9]+]]:_(<4 x s16>) = G_AND [[COPY2]], [[CONCAT_VECTORS]]
+    ; SI: [[UV:%[0-9]+]]:_(<2 x s16>), [[UV1:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; SI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV]](<2 x s16>)
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C1]](s32)
+    ; SI: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[UV1]](<2 x s16>)
+    ; SI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C1]](s32)
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[BITCAST2]](s32)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY6]], [[COPY7]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[COPY8]], [[COPY9]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[BITCAST3]](s32)
+    ; SI: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[COPY10]], [[COPY11]]
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR1]](s32)
+    ; SI: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[C1]], [[COPY12]]
+    ; SI: [[UV2:%[0-9]+]]:_(<2 x s16>), [[UV3:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; SI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV2]](<2 x s16>)
+    ; SI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C1]](s32)
+    ; SI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[UV3]](<2 x s16>)
+    ; SI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C1]](s32)
+    ; SI: [[UV4:%[0-9]+]]:_(<2 x s16>), [[UV5:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; SI: [[BITCAST6:%[0-9]+]]:_(s32) = G_BITCAST [[UV4]](<2 x s16>)
+    ; SI: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST6]], [[C1]](s32)
+    ; SI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[UV5]](<2 x s16>)
+    ; SI: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C1]](s32)
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[BITCAST6]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[BITCAST4]](s32)
+    ; SI: [[SHL2:%[0-9]+]]:_(s32) = G_SHL [[COPY14]], [[AND1]](s32)
+    ; SI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[LSHR4]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY15]], [[C2]]
+    ; SI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[LSHR2]](s32)
+    ; SI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[COPY16]], [[AND2]](s32)
+    ; SI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[BITCAST7]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY17]], [[C2]]
+    ; SI: [[COPY18:%[0-9]+]]:_(s32) = COPY [[BITCAST5]](s32)
+    ; SI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[COPY18]], [[AND3]](s32)
+    ; SI: [[COPY19:%[0-9]+]]:_(s32) = COPY [[LSHR5]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY19]], [[C2]]
+    ; SI: [[COPY20:%[0-9]+]]:_(s32) = COPY [[LSHR3]](s32)
+    ; SI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[COPY20]], [[AND4]](s32)
+    ; SI: [[COPY21:%[0-9]+]]:_(s32) = COPY [[SHL2]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY21]], [[C2]]
+    ; SI: [[COPY22:%[0-9]+]]:_(s32) = COPY [[SHL3]](s32)
+    ; SI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY22]], [[C2]]
+    ; SI: [[SHL6:%[0-9]+]]:_(s32) = G_SHL [[AND6]], [[C1]](s32)
+    ; SI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[AND5]], [[SHL6]]
+    ; SI: [[BITCAST8:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; SI: [[COPY23:%[0-9]+]]:_(s32) = COPY [[SHL4]](s32)
+    ; SI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY23]], [[C2]]
+    ; SI: [[COPY24:%[0-9]+]]:_(s32) = COPY [[SHL5]](s32)
+    ; SI: [[AND8:%[0-9]+]]:_(s32) = G_AND [[COPY24]], [[C2]]
+    ; SI: [[SHL7:%[0-9]+]]:_(s32) = G_SHL [[AND8]], [[C1]](s32)
+    ; SI: [[OR3:%[0-9]+]]:_(s32) = G_OR [[AND7]], [[SHL7]]
+    ; SI: [[BITCAST9:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR3]](s32)
+    ; SI: [[CONCAT_VECTORS1:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST8]](<2 x s16>), [[BITCAST9]](<2 x s16>)
+    ; SI: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY1]](<4 x s16>)
+    ; SI: [[BITCAST10:%[0-9]+]]:_(s32) = G_BITCAST [[UV6]](<2 x s16>)
+    ; SI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST10]], [[C1]](s32)
+    ; SI: [[BITCAST11:%[0-9]+]]:_(s32) = G_BITCAST [[UV7]](<2 x s16>)
+    ; SI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST11]], [[C1]](s32)
+    ; SI: [[COPY25:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND9:%[0-9]+]]:_(s32) = G_AND [[COPY25]], [[C2]]
+    ; SI: [[COPY26:%[0-9]+]]:_(s32) = COPY [[BITCAST10]](s32)
+    ; SI: [[AND10:%[0-9]+]]:_(s32) = G_AND [[COPY26]], [[C2]]
+    ; SI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[AND10]], [[AND9]](s32)
+    ; SI: [[COPY27:%[0-9]+]]:_(s32) = COPY [[SUB1]](s32)
+    ; SI: [[AND11:%[0-9]+]]:_(s32) = G_AND [[COPY27]], [[C2]]
+    ; SI: [[COPY28:%[0-9]+]]:_(s32) = COPY [[LSHR6]](s32)
+    ; SI: [[AND12:%[0-9]+]]:_(s32) = G_AND [[COPY28]], [[C2]]
+    ; SI: [[LSHR9:%[0-9]+]]:_(s32) = G_LSHR [[AND12]], [[AND11]](s32)
+    ; SI: [[COPY29:%[0-9]+]]:_(s32) = COPY [[SUB2]](s32)
+    ; SI: [[AND13:%[0-9]+]]:_(s32) = G_AND [[COPY29]], [[C2]]
+    ; SI: [[COPY30:%[0-9]+]]:_(s32) = COPY [[BITCAST11]](s32)
+    ; SI: [[AND14:%[0-9]+]]:_(s32) = G_AND [[COPY30]], [[C2]]
+    ; SI: [[LSHR10:%[0-9]+]]:_(s32) = G_LSHR [[AND14]], [[AND13]](s32)
+    ; SI: [[COPY31:%[0-9]+]]:_(s32) = COPY [[SUB3]](s32)
+    ; SI: [[AND15:%[0-9]+]]:_(s32) = G_AND [[COPY31]], [[C2]]
+    ; SI: [[COPY32:%[0-9]+]]:_(s32) = COPY [[LSHR7]](s32)
+    ; SI: [[AND16:%[0-9]+]]:_(s32) = G_AND [[COPY32]], [[C2]]
+    ; SI: [[LSHR11:%[0-9]+]]:_(s32) = G_LSHR [[AND16]], [[AND15]](s32)
+    ; SI: [[COPY33:%[0-9]+]]:_(s32) = COPY [[LSHR8]](s32)
+    ; SI: [[AND17:%[0-9]+]]:_(s32) = G_AND [[COPY33]], [[C2]]
+    ; SI: [[COPY34:%[0-9]+]]:_(s32) = COPY [[LSHR9]](s32)
+    ; SI: [[AND18:%[0-9]+]]:_(s32) = G_AND [[COPY34]], [[C2]]
+    ; SI: [[SHL8:%[0-9]+]]:_(s32) = G_SHL [[AND18]], [[C1]](s32)
+    ; SI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[AND17]], [[SHL8]]
+    ; SI: [[BITCAST12:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; SI: [[COPY35:%[0-9]+]]:_(s32) = COPY [[LSHR10]](s32)
+    ; SI: [[AND19:%[0-9]+]]:_(s32) = G_AND [[COPY35]], [[C2]]
+    ; SI: [[COPY36:%[0-9]+]]:_(s32) = COPY [[LSHR11]](s32)
+    ; SI: [[AND20:%[0-9]+]]:_(s32) = G_AND [[COPY36]], [[C2]]
+    ; SI: [[SHL9:%[0-9]+]]:_(s32) = G_SHL [[AND20]], [[C1]](s32)
+    ; SI: [[OR5:%[0-9]+]]:_(s32) = G_OR [[AND19]], [[SHL9]]
+    ; SI: [[BITCAST13:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR5]](s32)
+    ; SI: [[CONCAT_VECTORS2:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST12]](<2 x s16>), [[BITCAST13]](<2 x s16>)
+    ; SI: [[OR6:%[0-9]+]]:_(<4 x s16>) = G_OR [[CONCAT_VECTORS1]], [[CONCAT_VECTORS2]]
+    ; SI: [[UV8:%[0-9]+]]:_(<2 x s16>), [[UV9:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; SI: [[BITCAST14:%[0-9]+]]:_(s32) = G_BITCAST [[UV8]](<2 x s16>)
+    ; SI: [[LSHR12:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST14]], [[C1]](s32)
+    ; SI: [[BITCAST15:%[0-9]+]]:_(s32) = G_BITCAST [[UV9]](<2 x s16>)
+    ; SI: [[LSHR13:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST15]], [[C1]](s32)
+    ; SI: [[COPY37:%[0-9]+]]:_(s32) = COPY [[BITCAST14]](s32)
+    ; SI: [[AND21:%[0-9]+]]:_(s32) = G_AND [[COPY37]], [[C2]]
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[COPY38:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND21]](s32), [[COPY38]]
+    ; SI: [[COPY39:%[0-9]+]]:_(s32) = COPY [[LSHR12]](s32)
+    ; SI: [[AND22:%[0-9]+]]:_(s32) = G_AND [[COPY39]], [[C2]]
+    ; SI: [[COPY40:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND22]](s32), [[COPY40]]
+    ; SI: [[COPY41:%[0-9]+]]:_(s32) = COPY [[BITCAST15]](s32)
+    ; SI: [[AND23:%[0-9]+]]:_(s32) = G_AND [[COPY41]], [[C2]]
+    ; SI: [[COPY42:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND23]](s32), [[COPY42]]
+    ; SI: [[COPY43:%[0-9]+]]:_(s32) = COPY [[LSHR13]](s32)
+    ; SI: [[AND24:%[0-9]+]]:_(s32) = G_AND [[COPY43]], [[C2]]
+    ; SI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND24]](s32), [[C3]]
+    ; SI: [[UV10:%[0-9]+]]:_(<2 x s16>), [[UV11:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; SI: [[BITCAST16:%[0-9]+]]:_(s32) = G_BITCAST [[UV10]](<2 x s16>)
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST16]](s32)
+    ; SI: [[LSHR14:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST16]], [[C1]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR14]](s32)
+    ; SI: [[BITCAST17:%[0-9]+]]:_(s32) = G_BITCAST [[UV11]](<2 x s16>)
+    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST17]](s32)
+    ; SI: [[LSHR15:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST17]], [[C1]](s32)
+    ; SI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR15]](s32)
+    ; SI: [[UV12:%[0-9]+]]:_(<2 x s16>), [[UV13:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[OR6]](<4 x s16>)
+    ; SI: [[BITCAST18:%[0-9]+]]:_(s32) = G_BITCAST [[UV12]](<2 x s16>)
+    ; SI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST18]](s32)
+    ; SI: [[LSHR16:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST18]], [[C1]](s32)
+    ; SI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR16]](s32)
+    ; SI: [[BITCAST19:%[0-9]+]]:_(s32) = G_BITCAST [[UV13]](<2 x s16>)
+    ; SI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST19]](s32)
+    ; SI: [[LSHR17:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST19]], [[C1]](s32)
+    ; SI: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR17]](s32)
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC4]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC1]], [[TRUNC5]]
+    ; SI: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC2]], [[TRUNC6]]
+    ; SI: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[TRUNC3]], [[TRUNC7]]
+    ; SI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; SI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; SI: [[SHL10:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C1]](s32)
+    ; SI: [[OR7:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL10]]
+    ; SI: [[BITCAST20:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR7]](s32)
+    ; SI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT2]](s16)
+    ; SI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT3]](s16)
+    ; SI: [[SHL11:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C1]](s32)
+    ; SI: [[OR8:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL11]]
+    ; SI: [[BITCAST21:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR8]](s32)
+    ; SI: [[CONCAT_VECTORS3:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST20]](<2 x s16>), [[BITCAST21]](<2 x s16>)
+    ; SI: $vgpr0_vgpr1 = COPY [[CONCAT_VECTORS3]](<4 x s16>)
+    ; VI-LABEL: name: test_fshl_v4s16_v4s16
+    ; VI: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    ; VI: [[COPY1:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    ; VI: [[COPY2:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    ; VI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY4]], [[C3]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; VI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; VI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[C2]], [[C3]](s32)
+    ; VI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[COPY5]], [[SHL1]]
+    ; VI: [[BITCAST1:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; VI: [[CONCAT_VECTORS:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST]](<2 x s16>), [[BITCAST1]](<2 x s16>)
+    ; VI: [[AND:%[0-9]+]]:_(<4 x s16>) = G_AND [[COPY2]], [[CONCAT_VECTORS]]
+    ; VI: [[UV:%[0-9]+]]:_(<2 x s16>), [[UV1:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; VI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV]](<2 x s16>)
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C3]](s32)
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR]](s32)
+    ; VI: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[UV1]](<2 x s16>)
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST3]](s32)
+    ; VI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C3]](s32)
+    ; VI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR1]](s32)
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC1]]
+    ; VI: [[SUB2:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC2]]
+    ; VI: [[SUB3:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC3]]
+    ; VI: [[UV2:%[0-9]+]]:_(<2 x s16>), [[UV3:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; VI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV2]](<2 x s16>)
+    ; VI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST4]](s32)
+    ; VI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C3]](s32)
+    ; VI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; VI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[UV3]](<2 x s16>)
+    ; VI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST5]](s32)
+    ; VI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C3]](s32)
+    ; VI: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; VI: [[UV4:%[0-9]+]]:_(<2 x s16>), [[UV5:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; VI: [[BITCAST6:%[0-9]+]]:_(s32) = G_BITCAST [[UV4]](<2 x s16>)
+    ; VI: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST6]](s32)
+    ; VI: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST6]], [[C3]](s32)
+    ; VI: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR4]](s32)
+    ; VI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[UV5]](<2 x s16>)
+    ; VI: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST7]](s32)
+    ; VI: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C3]](s32)
+    ; VI: [[TRUNC11:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR5]](s32)
+    ; VI: [[SHL2:%[0-9]+]]:_(s16) = G_SHL [[TRUNC4]], [[TRUNC8]](s16)
+    ; VI: [[SHL3:%[0-9]+]]:_(s16) = G_SHL [[TRUNC5]], [[TRUNC9]](s16)
+    ; VI: [[SHL4:%[0-9]+]]:_(s16) = G_SHL [[TRUNC6]], [[TRUNC10]](s16)
+    ; VI: [[SHL5:%[0-9]+]]:_(s16) = G_SHL [[TRUNC7]], [[TRUNC11]](s16)
+    ; VI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SHL2]](s16)
+    ; VI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SHL3]](s16)
+    ; VI: [[SHL6:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C3]](s32)
+    ; VI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL6]]
+    ; VI: [[BITCAST8:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; VI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[SHL4]](s16)
+    ; VI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[SHL5]](s16)
+    ; VI: [[SHL7:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C3]](s32)
+    ; VI: [[OR3:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL7]]
+    ; VI: [[BITCAST9:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR3]](s32)
+    ; VI: [[CONCAT_VECTORS1:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST8]](<2 x s16>), [[BITCAST9]](<2 x s16>)
+    ; VI: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY1]](<4 x s16>)
+    ; VI: [[BITCAST10:%[0-9]+]]:_(s32) = G_BITCAST [[UV6]](<2 x s16>)
+    ; VI: [[TRUNC12:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST10]](s32)
+    ; VI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST10]], [[C3]](s32)
+    ; VI: [[TRUNC13:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR6]](s32)
+    ; VI: [[BITCAST11:%[0-9]+]]:_(s32) = G_BITCAST [[UV7]](<2 x s16>)
+    ; VI: [[TRUNC14:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST11]](s32)
+    ; VI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST11]], [[C3]](s32)
+    ; VI: [[TRUNC15:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; VI: [[LSHR8:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC12]], [[SUB]](s16)
+    ; VI: [[LSHR9:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC13]], [[SUB1]](s16)
+    ; VI: [[LSHR10:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC14]], [[SUB2]](s16)
+    ; VI: [[LSHR11:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC15]], [[SUB3]](s16)
+    ; VI: [[ZEXT4:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR8]](s16)
+    ; VI: [[ZEXT5:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR9]](s16)
+    ; VI: [[SHL8:%[0-9]+]]:_(s32) = G_SHL [[ZEXT5]], [[C3]](s32)
+    ; VI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT4]], [[SHL8]]
+    ; VI: [[BITCAST12:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; VI: [[ZEXT6:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR10]](s16)
+    ; VI: [[ZEXT7:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR11]](s16)
+    ; VI: [[SHL9:%[0-9]+]]:_(s32) = G_SHL [[ZEXT7]], [[C3]](s32)
+    ; VI: [[OR5:%[0-9]+]]:_(s32) = G_OR [[ZEXT6]], [[SHL9]]
+    ; VI: [[BITCAST13:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR5]](s32)
+    ; VI: [[CONCAT_VECTORS2:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST12]](<2 x s16>), [[BITCAST13]](<2 x s16>)
+    ; VI: [[OR6:%[0-9]+]]:_(<4 x s16>) = G_OR [[CONCAT_VECTORS1]], [[CONCAT_VECTORS2]]
+    ; VI: [[UV8:%[0-9]+]]:_(<2 x s16>), [[UV9:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; VI: [[BITCAST14:%[0-9]+]]:_(s32) = G_BITCAST [[UV8]](<2 x s16>)
+    ; VI: [[TRUNC16:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST14]](s32)
+    ; VI: [[LSHR12:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST14]], [[C3]](s32)
+    ; VI: [[TRUNC17:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR12]](s32)
+    ; VI: [[BITCAST15:%[0-9]+]]:_(s32) = G_BITCAST [[UV9]](<2 x s16>)
+    ; VI: [[TRUNC18:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST15]](s32)
+    ; VI: [[LSHR13:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST15]], [[C3]](s32)
+    ; VI: [[TRUNC19:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR13]](s32)
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC16]](s16), [[C1]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC17]](s16), [[C1]]
+    ; VI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC18]](s16), [[C1]]
+    ; VI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC19]](s16), [[C1]]
+    ; VI: [[UV10:%[0-9]+]]:_(<2 x s16>), [[UV11:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; VI: [[BITCAST16:%[0-9]+]]:_(s32) = G_BITCAST [[UV10]](<2 x s16>)
+    ; VI: [[TRUNC20:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST16]](s32)
+    ; VI: [[LSHR14:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST16]], [[C3]](s32)
+    ; VI: [[TRUNC21:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR14]](s32)
+    ; VI: [[BITCAST17:%[0-9]+]]:_(s32) = G_BITCAST [[UV11]](<2 x s16>)
+    ; VI: [[TRUNC22:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST17]](s32)
+    ; VI: [[LSHR15:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST17]], [[C3]](s32)
+    ; VI: [[TRUNC23:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR15]](s32)
+    ; VI: [[UV12:%[0-9]+]]:_(<2 x s16>), [[UV13:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[OR6]](<4 x s16>)
+    ; VI: [[BITCAST18:%[0-9]+]]:_(s32) = G_BITCAST [[UV12]](<2 x s16>)
+    ; VI: [[TRUNC24:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST18]](s32)
+    ; VI: [[LSHR16:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST18]], [[C3]](s32)
+    ; VI: [[TRUNC25:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR16]](s32)
+    ; VI: [[BITCAST19:%[0-9]+]]:_(s32) = G_BITCAST [[UV13]](<2 x s16>)
+    ; VI: [[TRUNC26:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST19]](s32)
+    ; VI: [[LSHR17:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST19]], [[C3]](s32)
+    ; VI: [[TRUNC27:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR17]](s32)
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC20]], [[TRUNC24]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC21]], [[TRUNC25]]
+    ; VI: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC22]], [[TRUNC26]]
+    ; VI: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[TRUNC23]], [[TRUNC27]]
+    ; VI: [[ZEXT8:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; VI: [[ZEXT9:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; VI: [[SHL10:%[0-9]+]]:_(s32) = G_SHL [[ZEXT9]], [[C3]](s32)
+    ; VI: [[OR7:%[0-9]+]]:_(s32) = G_OR [[ZEXT8]], [[SHL10]]
+    ; VI: [[BITCAST20:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR7]](s32)
+    ; VI: [[ZEXT10:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT2]](s16)
+    ; VI: [[ZEXT11:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT3]](s16)
+    ; VI: [[SHL11:%[0-9]+]]:_(s32) = G_SHL [[ZEXT11]], [[C3]](s32)
+    ; VI: [[OR8:%[0-9]+]]:_(s32) = G_OR [[ZEXT10]], [[SHL11]]
+    ; VI: [[BITCAST21:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR8]](s32)
+    ; VI: [[CONCAT_VECTORS3:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST20]](<2 x s16>), [[BITCAST21]](<2 x s16>)
+    ; VI: $vgpr0_vgpr1 = COPY [[CONCAT_VECTORS3]](<4 x s16>)
+    ; GFX9-LABEL: name: test_fshl_v4s16_v4s16
+    ; GFX9: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    ; GFX9: [[COPY1:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    ; GFX9: [[COPY2:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    ; GFX9: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY3]](s32), [[COPY4]](s32)
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC1:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY5]](s32), [[C1]](s32)
+    ; GFX9: [[CONCAT_VECTORS:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BUILD_VECTOR_TRUNC]](<2 x s16>), [[BUILD_VECTOR_TRUNC1]](<2 x s16>)
+    ; GFX9: [[AND:%[0-9]+]]:_(<4 x s16>) = G_AND [[COPY2]], [[CONCAT_VECTORS]]
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; GFX9: [[COPY6:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[COPY7:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC2:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY6]](s32), [[COPY7]](s32)
+    ; GFX9: [[COPY8:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC3:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY8]](s32), [[C2]](s32)
+    ; GFX9: [[UV:%[0-9]+]]:_(<2 x s16>), [[UV1:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; GFX9: [[SUB:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC2]], [[UV]]
+    ; GFX9: [[SUB1:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC3]], [[UV1]]
+    ; GFX9: [[UV2:%[0-9]+]]:_(<2 x s16>), [[UV3:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; GFX9: [[UV4:%[0-9]+]]:_(<2 x s16>), [[UV5:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; GFX9: [[SHL:%[0-9]+]]:_(<2 x s16>) = G_SHL [[UV2]], [[UV4]](<2 x s16>)
+    ; GFX9: [[SHL1:%[0-9]+]]:_(<2 x s16>) = G_SHL [[UV3]], [[UV5]](<2 x s16>)
+    ; GFX9: [[CONCAT_VECTORS1:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[SHL]](<2 x s16>), [[SHL1]](<2 x s16>)
+    ; GFX9: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY1]](<4 x s16>)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[UV6]], [[SUB]](<2 x s16>)
+    ; GFX9: [[LSHR1:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[UV7]], [[SUB1]](<2 x s16>)
+    ; GFX9: [[CONCAT_VECTORS2:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[LSHR]](<2 x s16>), [[LSHR1]](<2 x s16>)
+    ; GFX9: [[OR:%[0-9]+]]:_(<4 x s16>) = G_OR [[CONCAT_VECTORS1]], [[CONCAT_VECTORS2]]
+    ; GFX9: [[UV8:%[0-9]+]]:_(<2 x s16>), [[UV9:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[AND]](<4 x s16>)
+    ; GFX9: [[BITCAST:%[0-9]+]]:_(s32) = G_BITCAST [[UV8]](<2 x s16>)
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST]](s32)
+    ; GFX9: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST]], [[C2]](s32)
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; GFX9: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[UV9]](<2 x s16>)
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; GFX9: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C2]](s32)
+    ; GFX9: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC]](s16), [[C]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC1]](s16), [[C]]
+    ; GFX9: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC2]](s16), [[C]]
+    ; GFX9: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC3]](s16), [[C]]
+    ; GFX9: [[UV10:%[0-9]+]]:_(<2 x s16>), [[UV11:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; GFX9: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV10]](<2 x s16>)
+    ; GFX9: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; GFX9: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C2]](s32)
+    ; GFX9: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR4]](s32)
+    ; GFX9: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[UV11]](<2 x s16>)
+    ; GFX9: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST3]](s32)
+    ; GFX9: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C2]](s32)
+    ; GFX9: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR5]](s32)
+    ; GFX9: [[UV12:%[0-9]+]]:_(<2 x s16>), [[UV13:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[OR]](<4 x s16>)
+    ; GFX9: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV12]](<2 x s16>)
+    ; GFX9: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST4]](s32)
+    ; GFX9: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C2]](s32)
+    ; GFX9: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR6]](s32)
+    ; GFX9: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[UV13]](<2 x s16>)
+    ; GFX9: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST5]](s32)
+    ; GFX9: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C2]](s32)
+    ; GFX9: [[TRUNC11:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC4]], [[TRUNC8]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC5]], [[TRUNC9]]
+    ; GFX9: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC6]], [[TRUNC10]]
+    ; GFX9: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[TRUNC7]], [[TRUNC11]]
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT1]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC4:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT]](s32), [[ANYEXT1]](s32)
+    ; GFX9: [[ANYEXT2:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT2]](s16)
+    ; GFX9: [[ANYEXT3:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT3]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC5:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT2]](s32), [[ANYEXT3]](s32)
+    ; GFX9: [[CONCAT_VECTORS3:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BUILD_VECTOR_TRUNC4]](<2 x s16>), [[BUILD_VECTOR_TRUNC5]](<2 x s16>)
+    ; GFX9: $vgpr0_vgpr1 = COPY [[CONCAT_VECTORS3]](<4 x s16>)
+    %0:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    %1:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    %2:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    %3:_(<4 x s16>) = G_FSHL %0, %1, %2
+    $vgpr0_vgpr1 = COPY %3
+...

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-fshr.mir

 body: |
   bb.0:
     liveins: $vgpr0, $vgpr1, $vgpr2
 
     ; SI-LABEL: name: test_fshr_s16_s16
     ; SI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; SI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; SI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
-    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
-    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
-    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY2]](s32)
-    ; SI: [[FSHR:%[0-9]+]]:_(s16) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s16)
-    ; SI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s16)
-    ; SI: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY2]](s32)
+    ; SI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; SI: [[AND:%[0-9]+]]:_(s16) = G_AND [[TRUNC1]], [[C]]
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; SI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[AND]](s16)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C1]], [[ANYEXT]]
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[C2]]
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY4]], [[AND1]](s32)
+    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[SHL]](s32)
+    ; SI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[AND]](s16)
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY5]], [[C2]]
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND2]], [[ZEXT]](s32)
+    ; SI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s16) = G_OR [[TRUNC2]], [[TRUNC3]]
+    ; SI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[AND]](s16)
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[ZEXT1]](s32), [[C3]]
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[OR]]
+    ; SI: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; SI: $vgpr0 = COPY [[ANYEXT1]](s32)
     ; VI-LABEL: name: test_fshr_s16_s16
     ; VI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; VI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; VI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
     ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
     ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
     ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY2]](s32)
-    ; VI: [[FSHR:%[0-9]+]]:_(s16) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s16)
-    ; VI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s16)
+    ; VI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; VI: [[AND:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C2]]
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[AND]]
+    ; VI: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC]], [[SUB]](s16)
+    ; VI: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC1]], [[AND]](s16)
+    ; VI: [[OR:%[0-9]+]]:_(s16) = G_OR [[SHL]], [[LSHR]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s16), [[C1]]
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC1]], [[OR]]
+    ; VI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
     ; VI: $vgpr0 = COPY [[ANYEXT]](s32)
     ; GFX9-LABEL: name: test_fshr_s16_s16
     ; GFX9: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; GFX9: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; GFX9: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
     ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
     ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
     ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY2]](s32)
-    ; GFX9: [[FSHR:%[0-9]+]]:_(s16) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s16)
-    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s16)
+    ; GFX9: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; GFX9: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 15
+    ; GFX9: [[AND:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C2]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC]], [[SUB]](s16)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC1]], [[AND]](s16)
+    ; GFX9: [[OR:%[0-9]+]]:_(s16) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s16), [[C1]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC1]], [[OR]]
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
     ; GFX9: $vgpr0 = COPY [[ANYEXT]](s32)
     %0:_(s32) = COPY $vgpr0
     %1:_(s32) = COPY $vgpr1
     %2:_(s32) = COPY $vgpr2
     %3:_(s16) = G_TRUNC %0
     %4:_(s16) = G_TRUNC %1
     %5:_(s16) = G_TRUNC %2
     %6:_(s16) = G_FSHR %3, %4, %5
     %7:_(s32) = G_ANYEXT %6
     $vgpr0 = COPY %7
 ...
 
 ---
 name: test_fshr_v2s16_v2s16
 body: |
   bb.0:
     liveins: $vgpr0, $vgpr1, $vgpr2
 
     ; SI-LABEL: name: test_fshr_v2s16_v2s16
     ; SI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
     ; SI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
     ; SI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
-    ; SI: [[UV:%[0-9]+]]:_(s16), [[UV1:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY]](<2 x s16>)
-    ; SI: [[UV2:%[0-9]+]]:_(s16), [[UV3:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY1]](<2 x s16>)
-    ; SI: [[UV4:%[0-9]+]]:_(s16), [[UV5:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY2]](<2 x s16>)
-    ; SI: [[FSHR:%[0-9]+]]:_(s16) = G_FSHR [[UV]], [[UV2]], [[UV4]](s16)
-    ; SI: [[FSHR1:%[0-9]+]]:_(s16) = G_FSHR [[UV1]], [[UV3]], [[UV5]](s16)
-    ; SI: [[BUILD_VECTOR:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR [[FSHR]](s16), [[FSHR1]](s16)
-    ; SI: $vgpr0 = COPY [[BUILD_VECTOR]](<2 x s16>)
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[C]], [[C1]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; SI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[COPY2]], [[BITCAST]]
+    ; SI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C1]](s32)
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[BITCAST1]](s32)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY4]], [[COPY5]]
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[C1]], [[COPY6]]
+    ; SI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[COPY]](<2 x s16>)
+    ; SI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C1]](s32)
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[BITCAST2]](s32)
+    ; SI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND1]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SUB1]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[LSHR1]](s32)
+    ; SI: [[SHL2:%[0-9]+]]:_(s32) = G_SHL [[COPY10]], [[AND2]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL1]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY11]], [[C2]]
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[SHL2]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY12]], [[C2]]
+    ; SI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[AND4]], [[C1]](s32)
+    ; SI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[AND3]], [[SHL3]]
+    ; SI: [[BITCAST3:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; SI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[COPY1]](<2 x s16>)
+    ; SI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C1]](s32)
+    ; SI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C1]](s32)
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[BITCAST5]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[BITCAST4]](s32)
+    ; SI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; SI: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[AND6]], [[AND5]](s32)
+    ; SI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[LSHR3]](s32)
+    ; SI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY15]], [[C2]]
+    ; SI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[LSHR2]](s32)
+    ; SI: [[AND8:%[0-9]+]]:_(s32) = G_AND [[COPY16]], [[C2]]
+    ; SI: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[AND8]], [[AND7]](s32)
+    ; SI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[LSHR4]](s32)
+    ; SI: [[AND9:%[0-9]+]]:_(s32) = G_AND [[COPY17]], [[C2]]
+    ; SI: [[COPY18:%[0-9]+]]:_(s32) = COPY [[LSHR5]](s32)
+    ; SI: [[AND10:%[0-9]+]]:_(s32) = G_AND [[COPY18]], [[C2]]
+    ; SI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[AND10]], [[C1]](s32)
+    ; SI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[AND9]], [[SHL4]]
+    ; SI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; SI: [[OR3:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST3]], [[BITCAST6]]
+    ; SI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C1]](s32)
+    ; SI: [[COPY19:%[0-9]+]]:_(s32) = COPY [[BITCAST7]](s32)
+    ; SI: [[AND11:%[0-9]+]]:_(s32) = G_AND [[COPY19]], [[C2]]
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[COPY20:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND11]](s32), [[COPY20]]
+    ; SI: [[COPY21:%[0-9]+]]:_(s32) = COPY [[LSHR6]](s32)
+    ; SI: [[AND12:%[0-9]+]]:_(s32) = G_AND [[COPY21]], [[C2]]
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND12]](s32), [[C3]]
+    ; SI: [[BITCAST8:%[0-9]+]]:_(s32) = G_BITCAST [[COPY1]](<2 x s16>)
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST8]](s32)
+    ; SI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST8]], [[C1]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; SI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[OR3]](<2 x s16>)
+    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; SI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C1]](s32)
+    ; SI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR8]](s32)
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC2]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC1]], [[TRUNC3]]
+    ; SI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; SI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; SI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C1]](s32)
+    ; SI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL5]]
+    ; SI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; SI: $vgpr0 = COPY [[BITCAST10]](<2 x s16>)
     ; VI-LABEL: name: test_fshr_v2s16_v2s16
     ; VI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
     ; VI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
     ; VI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
-    ; VI: [[UV:%[0-9]+]]:_(s16), [[UV1:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY]](<2 x s16>)
-    ; VI: [[UV2:%[0-9]+]]:_(s16), [[UV3:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY1]](<2 x s16>)
-    ; VI: [[UV4:%[0-9]+]]:_(s16), [[UV5:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY2]](<2 x s16>)
-    ; VI: [[FSHR:%[0-9]+]]:_(s16) = G_FSHR [[UV]], [[UV2]], [[UV4]](s16)
-    ; VI: [[FSHR1:%[0-9]+]]:_(s16) = G_FSHR [[UV1]], [[UV3]], [[UV5]](s16)
-    ; VI: [[BUILD_VECTOR:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR [[FSHR]](s16), [[FSHR1]](s16)
-    ; VI: $vgpr0 = COPY [[BUILD_VECTOR]](<2 x s16>)
+    ; VI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[C2]], [[C3]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; VI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[COPY2]], [[BITCAST]]
+    ; VI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C3]](s32)
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR]](s32)
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC1]]
+    ; VI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[COPY]](<2 x s16>)
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; VI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C3]](s32)
+    ; VI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR1]](s32)
+    ; VI: [[SHL1:%[0-9]+]]:_(s16) = G_SHL [[TRUNC2]], [[SUB]](s16)
+    ; VI: [[SHL2:%[0-9]+]]:_(s16) = G_SHL [[TRUNC3]], [[SUB1]](s16)
+    ; VI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SHL1]](s16)
+    ; VI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SHL2]](s16)
+    ; VI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C3]](s32)
+    ; VI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL3]]
+    ; VI: [[BITCAST3:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; VI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[COPY1]](<2 x s16>)
+    ; VI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST4]](s32)
+    ; VI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C3]](s32)
+    ; VI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; VI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST5]](s32)
+    ; VI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C3]](s32)
+    ; VI: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; VI: [[LSHR4:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC4]], [[TRUNC6]](s16)
+    ; VI: [[LSHR5:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC5]], [[TRUNC7]](s16)
+    ; VI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR4]](s16)
+    ; VI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR5]](s16)
+    ; VI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C3]](s32)
+    ; VI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL4]]
+    ; VI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; VI: [[OR3:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST3]], [[BITCAST6]]
+    ; VI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST7]](s32)
+    ; VI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C3]](s32)
+    ; VI: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR6]](s32)
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC8]](s16), [[C1]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC9]](s16), [[C1]]
+    ; VI: [[BITCAST8:%[0-9]+]]:_(s32) = G_BITCAST [[COPY1]](<2 x s16>)
+    ; VI: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST8]](s32)
+    ; VI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST8]], [[C3]](s32)
+    ; VI: [[TRUNC11:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; VI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[OR3]](<2 x s16>)
+    ; VI: [[TRUNC12:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; VI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C3]](s32)
+    ; VI: [[TRUNC13:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR8]](s32)
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC10]], [[TRUNC12]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC11]], [[TRUNC13]]
+    ; VI: [[ZEXT4:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; VI: [[ZEXT5:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; VI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[ZEXT5]], [[C3]](s32)
+    ; VI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT4]], [[SHL5]]
+    ; VI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; VI: $vgpr0 = COPY [[BITCAST10]](<2 x s16>)
     ; GFX9-LABEL: name: test_fshr_v2s16_v2s16
     ; GFX9: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
     ; GFX9: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
     ; GFX9: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
-    ; GFX9: [[UV:%[0-9]+]]:_(s16), [[UV1:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY]](<2 x s16>)
-    ; GFX9: [[UV2:%[0-9]+]]:_(s16), [[UV3:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY1]](<2 x s16>)
-    ; GFX9: [[UV4:%[0-9]+]]:_(s16), [[UV5:%[0-9]+]]:_(s16) = G_UNMERGE_VALUES [[COPY2]](<2 x s16>)
-    ; GFX9: [[FSHR:%[0-9]+]]:_(s16) = G_FSHR [[UV]], [[UV2]], [[UV4]](s16)
-    ; GFX9: [[FSHR1:%[0-9]+]]:_(s16) = G_FSHR [[UV1]], [[UV3]], [[UV5]](s16)
-    ; GFX9: [[BUILD_VECTOR:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR [[FSHR]](s16), [[FSHR1]](s16)
-    ; GFX9: $vgpr0 = COPY [[BUILD_VECTOR]](<2 x s16>)
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY3]](s32), [[COPY4]](s32)
+    ; GFX9: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC1:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY5]](s32), [[C2]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[COPY2]], [[BUILD_VECTOR_TRUNC1]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(<2 x s16>) = G_SHL [[COPY]], [[SUB]](<2 x s16>)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[COPY1]], [[AND]](<2 x s16>)
+    ; GFX9: [[OR:%[0-9]+]]:_(<2 x s16>) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[BITCAST:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST]](s32)
+    ; GFX9: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST]], [[C]](s32)
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR1]](s32)
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC]](s16), [[C1]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC1]](s16), [[C1]]
+    ; GFX9: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[COPY1]](<2 x s16>)
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; GFX9: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C]](s32)
+    ; GFX9: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; GFX9: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[OR]](<2 x s16>)
+    ; GFX9: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; GFX9: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C]](s32)
+    ; GFX9: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC2]], [[TRUNC4]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC3]], [[TRUNC5]]
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT1]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC2:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT]](s32), [[ANYEXT1]](s32)
+    ; GFX9: $vgpr0 = COPY [[BUILD_VECTOR_TRUNC2]](<2 x s16>)
     %0:_(<2 x s16>) = COPY $vgpr0
     %1:_(<2 x s16>) = COPY $vgpr1
     %2:_(<2 x s16>) = COPY $vgpr2
     %3:_(<2 x s16>) = G_FSHR %0, %1, %2
     $vgpr0 = COPY %3
 ...
 
 ---
 name: test_fshr_s64_s64
 body: |
   bb.0:
     liveins: $vgpr0_vgpr1, $vgpr2_vgpr3, $vgpr4_vgpr5
 
     ; SI-LABEL: name: test_fshr_s64_s64
     ; SI: [[COPY:%[0-9]+]]:_(s64) = COPY $vgpr0_vgpr1
     ; SI: [[COPY1:%[0-9]+]]:_(s64) = COPY $vgpr2_vgpr3
     ; SI: [[COPY2:%[0-9]+]]:_(s64) = COPY $vgpr4_vgpr5
-    ; SI: [[FSHR:%[0-9]+]]:_(s64) = G_FSHR [[COPY]], [[COPY1]], [[COPY2]](s64)
-    ; SI: $vgpr0_vgpr1 = COPY [[FSHR]](s64)
+    ; SI: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 64
+    ; SI: [[C1:%[0-9]+]]:_(s64) = G_CONSTANT i64 0
+    ; SI: [[C2:%[0-9]+]]:_(s64) = G_CONSTANT i64 63
+    ; SI: [[AND:%[0-9]+]]:_(s64) = G_AND [[COPY2]], [[C2]]
+    ; SI: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
+    ; SI: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](s64)
+    ; SI: [[USUBO:%[0-9]+]]:_(s32), [[USUBO1:%[0-9]+]]:_(s1) = G_USUBO [[UV]], [[UV2]]
+    ; SI: [[USUBE:%[0-9]+]]:_(s32), [[USUBE1:%[0-9]+]]:_(s1) = G_USUBE [[UV1]], [[UV3]], [[USUBO1]]
+    ; SI: [[SHL:%[0-9]+]]:_(s64) = G_SHL [[COPY]], [[USUBO]](s32)
+    ; SI: [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[AND]](s64)
+    ; SI: [[LSHR:%[0-9]+]]:_(s64) = G_LSHR [[COPY1]], [[TRUNC]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s64) = G_OR [[SHL]], [[LSHR]]
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s64), [[C1]]
+    ; SI: [[SELECT:%[0-9]+]]:_(s64) = G_SELECT [[ICMP]](s1), [[COPY1]], [[OR]]
+    ; SI: $vgpr0_vgpr1 = COPY [[SELECT]](s64)
     ; VI-LABEL: name: test_fshr_s64_s64
     ; VI: [[COPY:%[0-9]+]]:_(s64) = COPY $vgpr0_vgpr1
     ; VI: [[COPY1:%[0-9]+]]:_(s64) = COPY $vgpr2_vgpr3
     ; VI: [[COPY2:%[0-9]+]]:_(s64) = COPY $vgpr4_vgpr5
-    ; VI: [[FSHR:%[0-9]+]]:_(s64) = G_FSHR [[COPY]], [[COPY1]], [[COPY2]](s64)
-    ; VI: $vgpr0_vgpr1 = COPY [[FSHR]](s64)
+    ; VI: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 64
+    ; VI: [[C1:%[0-9]+]]:_(s64) = G_CONSTANT i64 0
+    ; VI: [[C2:%[0-9]+]]:_(s64) = G_CONSTANT i64 63
+    ; VI: [[AND:%[0-9]+]]:_(s64) = G_AND [[COPY2]], [[C2]]
+    ; VI: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
+    ; VI: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](s64)
+    ; VI: [[USUBO:%[0-9]+]]:_(s32), [[USUBO1:%[0-9]+]]:_(s1) = G_USUBO [[UV]], [[UV2]]
+    ; VI: [[USUBE:%[0-9]+]]:_(s32), [[USUBE1:%[0-9]+]]:_(s1) = G_USUBE [[UV1]], [[UV3]], [[USUBO1]]
+    ; VI: [[SHL:%[0-9]+]]:_(s64) = G_SHL [[COPY]], [[USUBO]](s32)
+    ; VI: [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[AND]](s64)
+    ; VI: [[LSHR:%[0-9]+]]:_(s64) = G_LSHR [[COPY1]], [[TRUNC]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s64) = G_OR [[SHL]], [[LSHR]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s64), [[C1]]
+    ; VI: [[SELECT:%[0-9]+]]:_(s64) = G_SELECT [[ICMP]](s1), [[COPY1]], [[OR]]
+    ; VI: $vgpr0_vgpr1 = COPY [[SELECT]](s64)
     ; GFX9-LABEL: name: test_fshr_s64_s64
     ; GFX9: [[COPY:%[0-9]+]]:_(s64) = COPY $vgpr0_vgpr1
     ; GFX9: [[COPY1:%[0-9]+]]:_(s64) = COPY $vgpr2_vgpr3
     ; GFX9: [[COPY2:%[0-9]+]]:_(s64) = COPY $vgpr4_vgpr5
-    ; GFX9: [[FSHR:%[0-9]+]]:_(s64) = G_FSHR [[COPY]], [[COPY1]], [[COPY2]](s64)
-    ; GFX9: $vgpr0_vgpr1 = COPY [[FSHR]](s64)
+    ; GFX9: [[C:%[0-9]+]]:_(s64) = G_CONSTANT i64 64
+    ; GFX9: [[C1:%[0-9]+]]:_(s64) = G_CONSTANT i64 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s64) = G_CONSTANT i64 63
+    ; GFX9: [[AND:%[0-9]+]]:_(s64) = G_AND [[COPY2]], [[C2]]
+    ; GFX9: [[UV:%[0-9]+]]:_(s32), [[UV1:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[C]](s64)
+    ; GFX9: [[UV2:%[0-9]+]]:_(s32), [[UV3:%[0-9]+]]:_(s32) = G_UNMERGE_VALUES [[AND]](s64)
+    ; GFX9: [[USUBO:%[0-9]+]]:_(s32), [[USUBO1:%[0-9]+]]:_(s1) = G_USUBO [[UV]], [[UV2]]
+    ; GFX9: [[USUBE:%[0-9]+]]:_(s32), [[USUBE1:%[0-9]+]]:_(s1) = G_USUBE [[UV1]], [[UV3]], [[USUBO1]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(s64) = G_SHL [[COPY]], [[USUBO]](s32)
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s32) = G_TRUNC [[AND]](s64)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s64) = G_LSHR [[COPY1]], [[TRUNC]](s32)
+    ; GFX9: [[OR:%[0-9]+]]:_(s64) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND]](s64), [[C1]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s64) = G_SELECT [[ICMP]](s1), [[COPY1]], [[OR]]
+    ; GFX9: $vgpr0_vgpr1 = COPY [[SELECT]](s64)
     %0:_(s64) = COPY $vgpr0_vgpr1
     %1:_(s64) = COPY $vgpr2_vgpr3
     %2:_(s64) = COPY $vgpr4_vgpr5
     %3:_(s64) = G_FSHR %0, %1, %2
     $vgpr0_vgpr1 = COPY %3
 ...
 
 ---
 name: test_fshr_s8_s8
 body: |
   bb.0:
     liveins: $vgpr0, $vgpr1, $vgpr2
 
     ; SI-LABEL: name: test_fshr_s8_s8
     ; SI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; SI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; SI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
-    ; SI: [[TRUNC:%[0-9]+]]:_(s8) = G_TRUNC [[COPY]](s32)
-    ; SI: [[TRUNC1:%[0-9]+]]:_(s8) = G_TRUNC [[COPY1]](s32)
-    ; SI: [[TRUNC2:%[0-9]+]]:_(s8) = G_TRUNC [[COPY2]](s32)
-    ; SI: [[FSHR:%[0-9]+]]:_(s8) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s8)
-    ; SI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s8)
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 8
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[COPY4]]
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 255
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C3]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND1]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C3]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C3]]
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND3]], [[AND2]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C3]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C3]]
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND4]](s32), [[AND5]]
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[OR]](s32)
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC1]]
+    ; SI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
     ; SI: $vgpr0 = COPY [[ANYEXT]](s32)
     ; VI-LABEL: name: test_fshr_s8_s8
     ; VI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; VI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; VI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
-    ; VI: [[TRUNC:%[0-9]+]]:_(s8) = G_TRUNC [[COPY]](s32)
-    ; VI: [[TRUNC1:%[0-9]+]]:_(s8) = G_TRUNC [[COPY1]](s32)
-    ; VI: [[TRUNC2:%[0-9]+]]:_(s8) = G_TRUNC [[COPY2]](s32)
-    ; VI: [[FSHR:%[0-9]+]]:_(s8) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s8)
-    ; VI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s8)
-    ; VI: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; VI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; VI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[COPY4]]
+    ; VI: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 8
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[AND]](s32)
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C2]], [[TRUNC]]
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 255
+    ; VI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SUB]](s16)
+    ; VI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[ANYEXT]], [[C3]]
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; VI: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC1]], [[AND1]](s32)
+    ; VI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; VI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY5]], [[C3]]
+    ; VI: [[C4:%[0-9]+]]:_(s16) = G_CONSTANT i16 255
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; VI: [[AND3:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C4]]
+    ; VI: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[AND3]], [[AND2]](s32)
+    ; VI: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SHL]](s16)
+    ; VI: [[ANYEXT2:%[0-9]+]]:_(s32) = G_ANYEXT [[LSHR]](s16)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[ANYEXT1]], [[ANYEXT2]]
+    ; VI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; VI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY6]], [[C3]]
+    ; VI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; VI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C3]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND4]](s32), [[AND5]]
+    ; VI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; VI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[OR]](s32)
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC3]], [[TRUNC4]]
+    ; VI: [[ANYEXT3:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; VI: $vgpr0 = COPY [[ANYEXT3]](s32)
     ; GFX9-LABEL: name: test_fshr_s8_s8
     ; GFX9: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; GFX9: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; GFX9: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
-    ; GFX9: [[TRUNC:%[0-9]+]]:_(s8) = G_TRUNC [[COPY]](s32)
-    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s8) = G_TRUNC [[COPY1]](s32)
-    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s8) = G_TRUNC [[COPY2]](s32)
-    ; GFX9: [[FSHR:%[0-9]+]]:_(s8) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s8)
-    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s8)
-    ; GFX9: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 7
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[COPY4]]
+    ; GFX9: [[C2:%[0-9]+]]:_(s16) = G_CONSTANT i16 8
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[AND]](s32)
+    ; GFX9: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C2]], [[TRUNC]]
+    ; GFX9: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 255
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SUB]](s16)
+    ; GFX9: [[AND1:%[0-9]+]]:_(s32) = G_AND [[ANYEXT]], [[C3]]
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[COPY]](s32)
+    ; GFX9: [[SHL:%[0-9]+]]:_(s16) = G_SHL [[TRUNC1]], [[AND1]](s32)
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; GFX9: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY5]], [[C3]]
+    ; GFX9: [[C4:%[0-9]+]]:_(s16) = G_CONSTANT i16 255
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; GFX9: [[AND3:%[0-9]+]]:_(s16) = G_AND [[TRUNC2]], [[C4]]
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s16) = G_LSHR [[AND3]], [[AND2]](s32)
+    ; GFX9: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SHL]](s16)
+    ; GFX9: [[ANYEXT2:%[0-9]+]]:_(s32) = G_ANYEXT [[LSHR]](s16)
+    ; GFX9: [[OR:%[0-9]+]]:_(s32) = G_OR [[ANYEXT1]], [[ANYEXT2]]
+    ; GFX9: [[COPY6:%[0-9]+]]:_(s32) = COPY [[AND]](s32)
+    ; GFX9: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY6]], [[C3]]
+    ; GFX9: [[COPY7:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C3]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND4]](s32), [[AND5]]
+    ; GFX9: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[COPY1]](s32)
+    ; GFX9: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[OR]](s32)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC3]], [[TRUNC4]]
+    ; GFX9: [[ANYEXT3:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: $vgpr0 = COPY [[ANYEXT3]](s32)
     %0:_(s32) = COPY $vgpr0
     %1:_(s32) = COPY $vgpr1
     %2:_(s32) = COPY $vgpr2
     %3:_(s8) = G_TRUNC %0
     %4:_(s8) = G_TRUNC %1
     %5:_(s8) = G_TRUNC %2
     %6:_(s8) = G_FSHR %3, %4, %5
     %7:_(s32) = G_ANYEXT %6
     $vgpr0 = COPY %7
 ...
 
 ---
 name: test_fshr_s24_s24
 body: |
   bb.0:
     liveins: $vgpr0, $vgpr1, $vgpr2
 
     ; SI-LABEL: name: test_fshr_s24_s24
     ; SI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; SI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; SI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
-    ; SI: [[TRUNC:%[0-9]+]]:_(s24) = G_TRUNC [[COPY]](s32)
-    ; SI: [[TRUNC1:%[0-9]+]]:_(s24) = G_TRUNC [[COPY1]](s32)
-    ; SI: [[TRUNC2:%[0-9]+]]:_(s24) = G_TRUNC [[COPY2]](s32)
-    ; SI: [[FSHR:%[0-9]+]]:_(s24) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s24)
-    ; SI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s24)
-    ; SI: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 24
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 16777215
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[C2]]
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY4]], [[C2]]
+    ; SI: [[UITOFP:%[0-9]+]]:_(s32) = G_UITOFP [[AND1]](s32)
+    ; SI: [[AMDGPU_RCP_IFLAG:%[0-9]+]]:_(s32) = G_AMDGPU_RCP_IFLAG [[UITOFP]](s32)
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_FCONSTANT float 0x41F0000000000000
+    ; SI: [[FMUL:%[0-9]+]]:_(s32) = G_FMUL [[AMDGPU_RCP_IFLAG]], [[C3]]
+    ; SI: [[FPTOUI:%[0-9]+]]:_(s32) = G_FPTOUI [[FMUL]](s32)
+    ; SI: [[MUL:%[0-9]+]]:_(s32) = G_MUL [[FPTOUI]], [[AND1]]
+    ; SI: [[UMULH:%[0-9]+]]:_(s32) = G_UMULH [[FPTOUI]], [[AND1]]
+    ; SI: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C4]], [[MUL]]
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UMULH]](s32), [[C4]]
+    ; SI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[SUB]], [[MUL]]
+    ; SI: [[UMULH1:%[0-9]+]]:_(s32) = G_UMULH [[SELECT]], [[FPTOUI]]
+    ; SI: [[ADD:%[0-9]+]]:_(s32) = G_ADD [[FPTOUI]], [[UMULH1]]
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[FPTOUI]], [[UMULH1]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[ADD]], [[SUB1]]
+    ; SI: [[UMULH2:%[0-9]+]]:_(s32) = G_UMULH [[SELECT1]], [[AND]]
+    ; SI: [[MUL1:%[0-9]+]]:_(s32) = G_MUL [[UMULH2]], [[AND1]]
+    ; SI: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[AND]], [[MUL1]]
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[SUB2]](s32), [[AND1]]
+    ; SI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[AND]](s32), [[MUL1]]
+    ; SI: [[AND2:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP2]]
+    ; SI: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[SUB2]], [[AND1]]
+    ; SI: [[ADD1:%[0-9]+]]:_(s32) = G_ADD [[SUB2]], [[AND1]]
+    ; SI: [[SELECT2:%[0-9]+]]:_(s32) = G_SELECT [[AND2]](s1), [[SUB3]], [[SUB2]]
+    ; SI: [[SELECT3:%[0-9]+]]:_(s32) = G_SELECT [[ICMP2]](s1), [[SELECT2]], [[ADD1]]
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; SI: [[SUB4:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SUB4]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND3]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C2]]
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND5]], [[AND4]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; SI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; SI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND6]](s32), [[AND7]]
+    ; SI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; SI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[OR]](s32)
+    ; SI: [[SELECT4:%[0-9]+]]:_(s32) = G_SELECT [[ICMP3]](s1), [[COPY15]], [[COPY16]]
+    ; SI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[SELECT4]](s32)
+    ; SI: $vgpr0 = COPY [[COPY17]](s32)
     ; VI-LABEL: name: test_fshr_s24_s24
     ; VI: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; VI: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; VI: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
-    ; VI: [[TRUNC:%[0-9]+]]:_(s24) = G_TRUNC [[COPY]](s32)
-    ; VI: [[TRUNC1:%[0-9]+]]:_(s24) = G_TRUNC [[COPY1]](s32)
-    ; VI: [[TRUNC2:%[0-9]+]]:_(s24) = G_TRUNC [[COPY2]](s32)
-    ; VI: [[FSHR:%[0-9]+]]:_(s24) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s24)
-    ; VI: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s24)
-    ; VI: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; VI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 24
+    ; VI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 16777215
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[C2]]
+    ; VI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; VI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY4]], [[C2]]
+    ; VI: [[UITOFP:%[0-9]+]]:_(s32) = G_UITOFP [[AND1]](s32)
+    ; VI: [[AMDGPU_RCP_IFLAG:%[0-9]+]]:_(s32) = G_AMDGPU_RCP_IFLAG [[UITOFP]](s32)
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_FCONSTANT float 0x41F0000000000000
+    ; VI: [[FMUL:%[0-9]+]]:_(s32) = G_FMUL [[AMDGPU_RCP_IFLAG]], [[C3]]
+    ; VI: [[FPTOUI:%[0-9]+]]:_(s32) = G_FPTOUI [[FMUL]](s32)
+    ; VI: [[MUL:%[0-9]+]]:_(s32) = G_MUL [[FPTOUI]], [[AND1]]
+    ; VI: [[UMULH:%[0-9]+]]:_(s32) = G_UMULH [[FPTOUI]], [[AND1]]
+    ; VI: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C4]], [[MUL]]
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UMULH]](s32), [[C4]]
+    ; VI: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[SUB]], [[MUL]]
+    ; VI: [[UMULH1:%[0-9]+]]:_(s32) = G_UMULH [[SELECT]], [[FPTOUI]]
+    ; VI: [[ADD:%[0-9]+]]:_(s32) = G_ADD [[FPTOUI]], [[UMULH1]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[FPTOUI]], [[UMULH1]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[ADD]], [[SUB1]]
+    ; VI: [[UMULH2:%[0-9]+]]:_(s32) = G_UMULH [[SELECT1]], [[AND]]
+    ; VI: [[MUL1:%[0-9]+]]:_(s32) = G_MUL [[UMULH2]], [[AND1]]
+    ; VI: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[AND]], [[MUL1]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[SUB2]](s32), [[AND1]]
+    ; VI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[AND]](s32), [[MUL1]]
+    ; VI: [[AND2:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP2]]
+    ; VI: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[SUB2]], [[AND1]]
+    ; VI: [[ADD1:%[0-9]+]]:_(s32) = G_ADD [[SUB2]], [[AND1]]
+    ; VI: [[SELECT2:%[0-9]+]]:_(s32) = G_SELECT [[AND2]](s1), [[SUB3]], [[SUB2]]
+    ; VI: [[SELECT3:%[0-9]+]]:_(s32) = G_SELECT [[ICMP2]](s1), [[SELECT2]], [[ADD1]]
+    ; VI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; VI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; VI: [[SUB4:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; VI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SUB4]](s32)
+    ; VI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; VI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND3]](s32)
+    ; VI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; VI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; VI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; VI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C2]]
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND5]], [[AND4]](s32)
+    ; VI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; VI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; VI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; VI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; VI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; VI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; VI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND6]](s32), [[AND7]]
+    ; VI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; VI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[OR]](s32)
+    ; VI: [[SELECT4:%[0-9]+]]:_(s32) = G_SELECT [[ICMP3]](s1), [[COPY15]], [[COPY16]]
+    ; VI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[SELECT4]](s32)
+    ; VI: $vgpr0 = COPY [[COPY17]](s32)
     ; GFX9-LABEL: name: test_fshr_s24_s24
     ; GFX9: [[COPY:%[0-9]+]]:_(s32) = COPY $vgpr0
     ; GFX9: [[COPY1:%[0-9]+]]:_(s32) = COPY $vgpr1
     ; GFX9: [[COPY2:%[0-9]+]]:_(s32) = COPY $vgpr2
-    ; GFX9: [[TRUNC:%[0-9]+]]:_(s24) = G_TRUNC [[COPY]](s32)
-    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s24) = G_TRUNC [[COPY1]](s32)
-    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s24) = G_TRUNC [[COPY2]](s32)
-    ; GFX9: [[FSHR:%[0-9]+]]:_(s24) = G_FSHR [[TRUNC]], [[TRUNC1]], [[TRUNC2]](s24)
-    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[FSHR]](s24)
-    ; GFX9: $vgpr0 = COPY [[ANYEXT]](s32)
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 24
+    ; GFX9: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 16777215
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[COPY2]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY3]], [[C2]]
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY4]], [[C2]]
+    ; GFX9: [[UITOFP:%[0-9]+]]:_(s32) = G_UITOFP [[AND1]](s32)
+    ; GFX9: [[AMDGPU_RCP_IFLAG:%[0-9]+]]:_(s32) = G_AMDGPU_RCP_IFLAG [[UITOFP]](s32)
+    ; GFX9: [[C3:%[0-9]+]]:_(s32) = G_FCONSTANT float 0x41F0000000000000
+    ; GFX9: [[FMUL:%[0-9]+]]:_(s32) = G_FMUL [[AMDGPU_RCP_IFLAG]], [[C3]]
+    ; GFX9: [[FPTOUI:%[0-9]+]]:_(s32) = G_FPTOUI [[FMUL]](s32)
+    ; GFX9: [[MUL:%[0-9]+]]:_(s32) = G_MUL [[FPTOUI]], [[AND1]]
+    ; GFX9: [[UMULH:%[0-9]+]]:_(s32) = G_UMULH [[FPTOUI]], [[AND1]]
+    ; GFX9: [[C4:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; GFX9: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[C4]], [[MUL]]
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[UMULH]](s32), [[C4]]
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[SUB]], [[MUL]]
+    ; GFX9: [[UMULH1:%[0-9]+]]:_(s32) = G_UMULH [[SELECT]], [[FPTOUI]]
+    ; GFX9: [[ADD:%[0-9]+]]:_(s32) = G_ADD [[FPTOUI]], [[UMULH1]]
+    ; GFX9: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[FPTOUI]], [[UMULH1]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s32) = G_SELECT [[ICMP]](s1), [[ADD]], [[SUB1]]
+    ; GFX9: [[UMULH2:%[0-9]+]]:_(s32) = G_UMULH [[SELECT1]], [[AND]]
+    ; GFX9: [[MUL1:%[0-9]+]]:_(s32) = G_MUL [[UMULH2]], [[AND1]]
+    ; GFX9: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[AND]], [[MUL1]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[SUB2]](s32), [[AND1]]
+    ; GFX9: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(uge), [[AND]](s32), [[MUL1]]
+    ; GFX9: [[AND2:%[0-9]+]]:_(s1) = G_AND [[ICMP1]], [[ICMP2]]
+    ; GFX9: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[SUB2]], [[AND1]]
+    ; GFX9: [[ADD1:%[0-9]+]]:_(s32) = G_ADD [[SUB2]], [[AND1]]
+    ; GFX9: [[SELECT2:%[0-9]+]]:_(s32) = G_SELECT [[AND2]](s1), [[SUB3]], [[SUB2]]
+    ; GFX9: [[SELECT3:%[0-9]+]]:_(s32) = G_SELECT [[ICMP2]](s1), [[SELECT2]], [[ADD1]]
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY6:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; GFX9: [[SUB4:%[0-9]+]]:_(s32) = G_SUB [[COPY5]], [[COPY6]]
+    ; GFX9: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SUB4]](s32)
+    ; GFX9: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; GFX9: [[COPY8:%[0-9]+]]:_(s32) = COPY [[COPY]](s32)
+    ; GFX9: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND3]](s32)
+    ; GFX9: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; GFX9: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; GFX9: [[COPY10:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; GFX9: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY10]], [[C2]]
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[AND5]], [[AND4]](s32)
+    ; GFX9: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL]](s32)
+    ; GFX9: [[COPY12:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; GFX9: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY11]], [[COPY12]]
+    ; GFX9: [[COPY13:%[0-9]+]]:_(s32) = COPY [[SELECT3]](s32)
+    ; GFX9: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; GFX9: [[COPY14:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; GFX9: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; GFX9: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND6]](s32), [[AND7]]
+    ; GFX9: [[COPY15:%[0-9]+]]:_(s32) = COPY [[COPY1]](s32)
+    ; GFX9: [[COPY16:%[0-9]+]]:_(s32) = COPY [[OR]](s32)
+    ; GFX9: [[SELECT4:%[0-9]+]]:_(s32) = G_SELECT [[ICMP3]](s1), [[COPY15]], [[COPY16]]
+    ; GFX9: [[COPY17:%[0-9]+]]:_(s32) = COPY [[SELECT4]](s32)
+    ; GFX9: $vgpr0 = COPY [[COPY17]](s32)
     %0:_(s32) = COPY $vgpr0
     %1:_(s32) = COPY $vgpr1
     %2:_(s32) = COPY $vgpr2
     %3:_(s24) = G_TRUNC %0
     %4:_(s24) = G_TRUNC %1
     %5:_(s24) = G_TRUNC %2
     %6:_(s24) = G_FSHR %3, %4, %5
     %7:_(s32) = G_ANYEXT %6
     $vgpr0 = COPY %7
 ...
 
+---
+name: test_fshr_v3s16_v3s16
+body: |
+  bb.0:
+    liveins: $vgpr0, $vgpr1, $vgpr2, $vgpr3, $vgpr4, $vgpr5
+
+    ; SI-LABEL: name: test_fshr_v3s16_v3s16
+    ; SI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; SI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; SI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; SI: [[COPY3:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr3
+    ; SI: [[COPY4:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr4
+    ; SI: [[COPY5:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr5
+    ; SI: [[DEF:%[0-9]+]]:_(<2 x s16>) = G_IMPLICIT_DEF
+    ; SI: [[CONCAT_VECTORS:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY]](<2 x s16>), [[COPY1]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; SI: [[UV:%[0-9]+]]:_(<3 x s16>), [[UV1:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS]](<6 x s16>)
+    ; SI: [[CONCAT_VECTORS1:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY2]](<2 x s16>), [[COPY3]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; SI: [[UV2:%[0-9]+]]:_(<3 x s16>), [[UV3:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS1]](<6 x s16>)
+    ; SI: [[CONCAT_VECTORS2:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY4]](<2 x s16>), [[COPY5]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; SI: [[UV4:%[0-9]+]]:_(<3 x s16>), [[UV5:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS2]](<6 x s16>)
+    ; SI: [[DEF1:%[0-9]+]]:_(<4 x s16>) = G_IMPLICIT_DEF
+    ; SI: [[INSERT:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV]](<3 x s16>), 0
+    ; SI: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT]](<4 x s16>)
+    ; SI: [[BITCAST:%[0-9]+]]:_(s32) = G_BITCAST [[UV6]](<2 x s16>)
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST]], [[C]](s32)
+    ; SI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[UV7]](<2 x s16>)
+    ; SI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C]](s32)
+    ; SI: [[DEF2:%[0-9]+]]:_(s16) = G_IMPLICIT_DEF
+    ; SI: [[INSERT1:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV2]](<3 x s16>), 0
+    ; SI: [[UV8:%[0-9]+]]:_(<2 x s16>), [[UV9:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT1]](<4 x s16>)
+    ; SI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV8]](<2 x s16>)
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; SI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; SI: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[UV9]](<2 x s16>)
+    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST3]](s32)
+    ; SI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C]](s32)
+    ; SI: [[INSERT2:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV4]](<3 x s16>), 0
+    ; SI: [[UV10:%[0-9]+]]:_(<2 x s16>), [[UV11:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT2]](<4 x s16>)
+    ; SI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV10]](<2 x s16>)
+    ; SI: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C]](s32)
+    ; SI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[UV11]](<2 x s16>)
+    ; SI: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C]](s32)
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[BITCAST4]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY6]], [[C1]]
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[LSHR4]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C1]]
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[AND1]], [[C]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[AND]], [[SHL]]
+    ; SI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[BITCAST5]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY8]], [[C1]]
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; SI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[COPY9]], [[C]](s32)
+    ; SI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[AND2]], [[SHL1]]
+    ; SI: [[BITCAST7:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[SHL2:%[0-9]+]]:_(s32) = G_SHL [[C3]], [[C]](s32)
+    ; SI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[COPY10]], [[SHL2]]
+    ; SI: [[BITCAST8:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(<2 x s16>) = G_AND [[BITCAST6]], [[BITCAST8]]
+    ; SI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[AND3]](<2 x s16>)
+    ; SI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[BITCAST9]](s32)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY11]], [[COPY12]]
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[LSHR6]](s32)
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[C]], [[COPY13]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C1]]
+    ; SI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[BITCAST]](s32)
+    ; SI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[COPY15]], [[AND4]](s32)
+    ; SI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[SUB1]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY16]], [[C1]]
+    ; SI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[COPY17]], [[AND5]](s32)
+    ; SI: [[COPY18:%[0-9]+]]:_(s32) = COPY [[SHL3]](s32)
+    ; SI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY18]], [[C1]]
+    ; SI: [[COPY19:%[0-9]+]]:_(s32) = COPY [[SHL4]](s32)
+    ; SI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY19]], [[C1]]
+    ; SI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[AND7]], [[C]](s32)
+    ; SI: [[OR3:%[0-9]+]]:_(s32) = G_OR [[AND6]], [[SHL5]]
+    ; SI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR3]](s32)
+    ; SI: [[BITCAST11:%[0-9]+]]:_(s32) = G_BITCAST [[AND3]](<2 x s16>)
+    ; SI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST11]], [[C]](s32)
+    ; SI: [[COPY20:%[0-9]+]]:_(s32) = COPY [[BITCAST11]](s32)
+    ; SI: [[AND8:%[0-9]+]]:_(s32) = G_AND [[COPY20]], [[C1]]
+    ; SI: [[COPY21:%[0-9]+]]:_(s32) = COPY [[BITCAST2]](s32)
+    ; SI: [[AND9:%[0-9]+]]:_(s32) = G_AND [[COPY21]], [[C1]]
+    ; SI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[AND9]], [[AND8]](s32)
+    ; SI: [[COPY22:%[0-9]+]]:_(s32) = COPY [[LSHR7]](s32)
+    ; SI: [[AND10:%[0-9]+]]:_(s32) = G_AND [[COPY22]], [[C1]]
+    ; SI: [[COPY23:%[0-9]+]]:_(s32) = COPY [[LSHR2]](s32)
+    ; SI: [[AND11:%[0-9]+]]:_(s32) = G_AND [[COPY23]], [[C1]]
+    ; SI: [[LSHR9:%[0-9]+]]:_(s32) = G_LSHR [[AND11]], [[AND10]](s32)
+    ; SI: [[COPY24:%[0-9]+]]:_(s32) = COPY [[LSHR8]](s32)
+    ; SI: [[AND12:%[0-9]+]]:_(s32) = G_AND [[COPY24]], [[C1]]
+    ; SI: [[COPY25:%[0-9]+]]:_(s32) = COPY [[LSHR9]](s32)
+    ; SI: [[AND13:%[0-9]+]]:_(s32) = G_AND [[COPY25]], [[C1]]
+    ; SI: [[SHL6:%[0-9]+]]:_(s32) = G_SHL [[AND13]], [[C]](s32)
+    ; SI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[AND12]], [[SHL6]]
+    ; SI: [[BITCAST12:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; SI: [[OR5:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST10]], [[BITCAST12]]
+    ; SI: [[BITCAST13:%[0-9]+]]:_(s32) = G_BITCAST [[AND3]](<2 x s16>)
+    ; SI: [[LSHR10:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST13]], [[C]](s32)
+    ; SI: [[COPY26:%[0-9]+]]:_(s32) = COPY [[BITCAST13]](s32)
+    ; SI: [[AND14:%[0-9]+]]:_(s32) = G_AND [[COPY26]], [[C1]]
+    ; SI: [[COPY27:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND14]](s32), [[COPY27]]
+    ; SI: [[COPY28:%[0-9]+]]:_(s32) = COPY [[LSHR10]](s32)
+    ; SI: [[AND15:%[0-9]+]]:_(s32) = G_AND [[COPY28]], [[C1]]
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND15]](s32), [[C2]]
+    ; SI: [[BITCAST14:%[0-9]+]]:_(s32) = G_BITCAST [[OR5]](<2 x s16>)
+    ; SI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST14]](s32)
+    ; SI: [[LSHR11:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST14]], [[C]](s32)
+    ; SI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR11]](s32)
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC3]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC1]], [[TRUNC4]]
+    ; SI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; SI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; SI: [[SHL7:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C]](s32)
+    ; SI: [[OR6:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL7]]
+    ; SI: [[BITCAST15:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR6]](s32)
+    ; SI: [[COPY29:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[COPY30:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[SHL8:%[0-9]+]]:_(s32) = G_SHL [[COPY30]], [[C]](s32)
+    ; SI: [[OR7:%[0-9]+]]:_(s32) = G_OR [[COPY29]], [[SHL8]]
+    ; SI: [[BITCAST16:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR7]](s32)
+    ; SI: [[AND16:%[0-9]+]]:_(<2 x s16>) = G_AND [[BITCAST7]], [[BITCAST16]]
+    ; SI: [[BITCAST17:%[0-9]+]]:_(s32) = G_BITCAST [[AND16]](<2 x s16>)
+    ; SI: [[LSHR12:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST17]], [[C]](s32)
+    ; SI: [[COPY31:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY32:%[0-9]+]]:_(s32) = COPY [[BITCAST17]](s32)
+    ; SI: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[COPY31]], [[COPY32]]
+    ; SI: [[COPY33:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY34:%[0-9]+]]:_(s32) = COPY [[LSHR12]](s32)
+    ; SI: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[COPY33]], [[COPY34]]
+    ; SI: [[COPY35:%[0-9]+]]:_(s32) = COPY [[SUB2]](s32)
+    ; SI: [[AND17:%[0-9]+]]:_(s32) = G_AND [[COPY35]], [[C1]]
+    ; SI: [[COPY36:%[0-9]+]]:_(s32) = COPY [[BITCAST1]](s32)
+    ; SI: [[SHL9:%[0-9]+]]:_(s32) = G_SHL [[COPY36]], [[AND17]](s32)
+    ; SI: [[COPY37:%[0-9]+]]:_(s32) = COPY [[SUB3]](s32)
+    ; SI: [[AND18:%[0-9]+]]:_(s32) = G_AND [[COPY37]], [[C1]]
+    ; SI: [[DEF3:%[0-9]+]]:_(s32) = G_IMPLICIT_DEF
+    ; SI: [[SHL10:%[0-9]+]]:_(s32) = G_SHL [[DEF3]], [[AND18]](s32)
+    ; SI: [[COPY38:%[0-9]+]]:_(s32) = COPY [[SHL9]](s32)
+    ; SI: [[AND19:%[0-9]+]]:_(s32) = G_AND [[COPY38]], [[C1]]
+    ; SI: [[COPY39:%[0-9]+]]:_(s32) = COPY [[SHL10]](s32)
+    ; SI: [[AND20:%[0-9]+]]:_(s32) = G_AND [[COPY39]], [[C1]]
+    ; SI: [[SHL11:%[0-9]+]]:_(s32) = G_SHL [[AND20]], [[C]](s32)
+    ; SI: [[OR8:%[0-9]+]]:_(s32) = G_OR [[AND19]], [[SHL11]]
+    ; SI: [[BITCAST18:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR8]](s32)
+    ; SI: [[BITCAST19:%[0-9]+]]:_(s32) = G_BITCAST [[AND16]](<2 x s16>)
+    ; SI: [[LSHR13:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST19]], [[C]](s32)
+    ; SI: [[COPY40:%[0-9]+]]:_(s32) = COPY [[BITCAST19]](s32)
+    ; SI: [[AND21:%[0-9]+]]:_(s32) = G_AND [[COPY40]], [[C1]]
+    ; SI: [[COPY41:%[0-9]+]]:_(s32) = COPY [[BITCAST3]](s32)
+    ; SI: [[AND22:%[0-9]+]]:_(s32) = G_AND [[COPY41]], [[C1]]
+    ; SI: [[LSHR14:%[0-9]+]]:_(s32) = G_LSHR [[AND22]], [[AND21]](s32)
+    ; SI: [[COPY42:%[0-9]+]]:_(s32) = COPY [[LSHR13]](s32)
+    ; SI: [[AND23:%[0-9]+]]:_(s32) = G_AND [[COPY42]], [[C1]]
+    ; SI: [[COPY43:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; SI: [[LSHR15:%[0-9]+]]:_(s32) = G_LSHR [[COPY43]], [[AND23]](s32)
+    ; SI: [[COPY44:%[0-9]+]]:_(s32) = COPY [[LSHR14]](s32)
+    ; SI: [[AND24:%[0-9]+]]:_(s32) = G_AND [[COPY44]], [[C1]]
+    ; SI: [[COPY45:%[0-9]+]]:_(s32) = COPY [[LSHR15]](s32)
+    ; SI: [[AND25:%[0-9]+]]:_(s32) = G_AND [[COPY45]], [[C1]]
+    ; SI: [[SHL12:%[0-9]+]]:_(s32) = G_SHL [[AND25]], [[C]](s32)
+    ; SI: [[OR9:%[0-9]+]]:_(s32) = G_OR [[AND24]], [[SHL12]]
+    ; SI: [[BITCAST20:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR9]](s32)
+    ; SI: [[OR10:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST18]], [[BITCAST20]]
+    ; SI: [[BITCAST21:%[0-9]+]]:_(s32) = G_BITCAST [[AND16]](<2 x s16>)
+    ; SI: [[LSHR16:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST21]], [[C]](s32)
+    ; SI: [[COPY46:%[0-9]+]]:_(s32) = COPY [[BITCAST21]](s32)
+    ; SI: [[AND26:%[0-9]+]]:_(s32) = G_AND [[COPY46]], [[C1]]
+    ; SI: [[COPY47:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; SI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND26]](s32), [[COPY47]]
+    ; SI: [[COPY48:%[0-9]+]]:_(s32) = COPY [[LSHR16]](s32)
+    ; SI: [[AND27:%[0-9]+]]:_(s32) = G_AND [[COPY48]], [[C1]]
+    ; SI: [[COPY49:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; SI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND27]](s32), [[COPY49]]
+    ; SI: [[BITCAST22:%[0-9]+]]:_(s32) = G_BITCAST [[OR10]](<2 x s16>)
+    ; SI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST22]](s32)
+    ; SI: [[LSHR17:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST22]], [[C]](s32)
+    ; SI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR17]](s32)
+    ; SI: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC2]], [[TRUNC5]]
+    ; SI: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[DEF2]], [[TRUNC6]]
+    ; SI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT2]](s16)
+    ; SI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT3]](s16)
+    ; SI: [[SHL13:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C]](s32)
+    ; SI: [[OR11:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL13]]
+    ; SI: [[BITCAST23:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR11]](s32)
+    ; SI: [[CONCAT_VECTORS3:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[BITCAST15]](<2 x s16>), [[BITCAST23]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; SI: [[EXTRACT:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[CONCAT_VECTORS3]](<6 x s16>), 0
+    ; SI: [[EXTRACT1:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[DEF1]](<4 x s16>), 0
+    ; SI: [[CONCAT_VECTORS4:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[EXTRACT]](<3 x s16>), [[EXTRACT1]](<3 x s16>)
+    ; SI: [[UV12:%[0-9]+]]:_(<2 x s16>), [[UV13:%[0-9]+]]:_(<2 x s16>), [[UV14:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS4]](<6 x s16>)
+    ; SI: $vgpr0 = COPY [[UV12]](<2 x s16>)
+    ; SI: $vgpr1 = COPY [[UV13]](<2 x s16>)
+    ; SI: $vgpr2 = COPY [[UV14]](<2 x s16>)
+    ; VI-LABEL: name: test_fshr_v3s16_v3s16
+    ; VI: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; VI: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; VI: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; VI: [[COPY3:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr3
+    ; VI: [[COPY4:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr4
+    ; VI: [[COPY5:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr5
+    ; VI: [[DEF:%[0-9]+]]:_(<2 x s16>) = G_IMPLICIT_DEF
+    ; VI: [[CONCAT_VECTORS:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY]](<2 x s16>), [[COPY1]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; VI: [[UV:%[0-9]+]]:_(<3 x s16>), [[UV1:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS]](<6 x s16>)
+    ; VI: [[CONCAT_VECTORS1:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY2]](<2 x s16>), [[COPY3]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; VI: [[UV2:%[0-9]+]]:_(<3 x s16>), [[UV3:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS1]](<6 x s16>)
+    ; VI: [[CONCAT_VECTORS2:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY4]](<2 x s16>), [[COPY5]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; VI: [[UV4:%[0-9]+]]:_(<3 x s16>), [[UV5:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS2]](<6 x s16>)
+    ; VI: [[DEF1:%[0-9]+]]:_(<4 x s16>) = G_IMPLICIT_DEF
+    ; VI: [[INSERT:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV]](<3 x s16>), 0
+    ; VI: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT]](<4 x s16>)
+    ; VI: [[BITCAST:%[0-9]+]]:_(s32) = G_BITCAST [[UV6]](<2 x s16>)
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST]](s32)
+    ; VI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST]], [[C]](s32)
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR]](s32)
+    ; VI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[UV7]](<2 x s16>)
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; VI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C]](s32)
+    ; VI: [[DEF2:%[0-9]+]]:_(s16) = G_IMPLICIT_DEF
+    ; VI: [[INSERT1:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV2]](<3 x s16>), 0
+    ; VI: [[UV8:%[0-9]+]]:_(<2 x s16>), [[UV9:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT1]](<4 x s16>)
+    ; VI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV8]](<2 x s16>)
+    ; VI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; VI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C]](s32)
+    ; VI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; VI: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[UV9]](<2 x s16>)
+    ; VI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST3]](s32)
+    ; VI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C]](s32)
+    ; VI: [[INSERT2:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV4]](<3 x s16>), 0
+    ; VI: [[UV10:%[0-9]+]]:_(<2 x s16>), [[UV11:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT2]](<4 x s16>)
+    ; VI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV10]](<2 x s16>)
+    ; VI: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C]](s32)
+    ; VI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[UV11]](<2 x s16>)
+    ; VI: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C]](s32)
+    ; VI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; VI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[BITCAST4]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(s32) = G_AND [[COPY6]], [[C1]]
+    ; VI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[LSHR4]](s32)
+    ; VI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C1]]
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[AND1]], [[C]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[AND]], [[SHL]]
+    ; VI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; VI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[BITCAST5]](s32)
+    ; VI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY8]], [[C1]]
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; VI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[C2]], [[C]](s32)
+    ; VI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[AND2]], [[SHL1]]
+    ; VI: [[BITCAST7:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; VI: [[C3:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[C4:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[C5:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; VI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[C5]](s32)
+    ; VI: [[SHL2:%[0-9]+]]:_(s32) = G_SHL [[C5]], [[C]](s32)
+    ; VI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[COPY9]], [[SHL2]]
+    ; VI: [[BITCAST8:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; VI: [[AND3:%[0-9]+]]:_(<2 x s16>) = G_AND [[BITCAST6]], [[BITCAST8]]
+    ; VI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[AND3]](<2 x s16>)
+    ; VI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; VI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C]](s32)
+    ; VI: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR6]](s32)
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C3]], [[TRUNC6]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s16) = G_SUB [[C3]], [[TRUNC7]]
+    ; VI: [[SHL3:%[0-9]+]]:_(s16) = G_SHL [[TRUNC]], [[SUB]](s16)
+    ; VI: [[SHL4:%[0-9]+]]:_(s16) = G_SHL [[TRUNC1]], [[SUB1]](s16)
+    ; VI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SHL3]](s16)
+    ; VI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SHL4]](s16)
+    ; VI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C]](s32)
+    ; VI: [[OR3:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL5]]
+    ; VI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR3]](s32)
+    ; VI: [[BITCAST11:%[0-9]+]]:_(s32) = G_BITCAST [[AND3]](<2 x s16>)
+    ; VI: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST11]](s32)
+    ; VI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST11]], [[C]](s32)
+    ; VI: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; VI: [[LSHR8:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC3]], [[TRUNC8]](s16)
+    ; VI: [[LSHR9:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC4]], [[TRUNC9]](s16)
+    ; VI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR8]](s16)
+    ; VI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR9]](s16)
+    ; VI: [[SHL6:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C]](s32)
+    ; VI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL6]]
+    ; VI: [[BITCAST12:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; VI: [[OR5:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST10]], [[BITCAST12]]
+    ; VI: [[BITCAST13:%[0-9]+]]:_(s32) = G_BITCAST [[AND3]](<2 x s16>)
+    ; VI: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST13]](s32)
+    ; VI: [[LSHR10:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST13]], [[C]](s32)
+    ; VI: [[TRUNC11:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR10]](s32)
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC10]](s16), [[C4]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC11]](s16), [[C4]]
+    ; VI: [[BITCAST14:%[0-9]+]]:_(s32) = G_BITCAST [[OR5]](<2 x s16>)
+    ; VI: [[TRUNC12:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST14]](s32)
+    ; VI: [[LSHR11:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST14]], [[C]](s32)
+    ; VI: [[TRUNC13:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR11]](s32)
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC3]], [[TRUNC12]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC4]], [[TRUNC13]]
+    ; VI: [[ZEXT4:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; VI: [[ZEXT5:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; VI: [[SHL7:%[0-9]+]]:_(s32) = G_SHL [[ZEXT5]], [[C]](s32)
+    ; VI: [[OR6:%[0-9]+]]:_(s32) = G_OR [[ZEXT4]], [[SHL7]]
+    ; VI: [[BITCAST15:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR6]](s32)
+    ; VI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[C5]](s32)
+    ; VI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[C5]](s32)
+    ; VI: [[SHL8:%[0-9]+]]:_(s32) = G_SHL [[COPY11]], [[C]](s32)
+    ; VI: [[OR7:%[0-9]+]]:_(s32) = G_OR [[COPY10]], [[SHL8]]
+    ; VI: [[BITCAST16:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR7]](s32)
+    ; VI: [[AND4:%[0-9]+]]:_(<2 x s16>) = G_AND [[BITCAST7]], [[BITCAST16]]
+    ; VI: [[BITCAST17:%[0-9]+]]:_(s32) = G_BITCAST [[AND4]](<2 x s16>)
+    ; VI: [[TRUNC14:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST17]](s32)
+    ; VI: [[LSHR12:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST17]], [[C]](s32)
+    ; VI: [[TRUNC15:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR12]](s32)
+    ; VI: [[SUB2:%[0-9]+]]:_(s16) = G_SUB [[C3]], [[TRUNC14]]
+    ; VI: [[SUB3:%[0-9]+]]:_(s16) = G_SUB [[C3]], [[TRUNC15]]
+    ; VI: [[SHL9:%[0-9]+]]:_(s16) = G_SHL [[TRUNC2]], [[SUB2]](s16)
+    ; VI: [[SHL10:%[0-9]+]]:_(s16) = G_SHL [[DEF2]], [[SUB3]](s16)
+    ; VI: [[ZEXT6:%[0-9]+]]:_(s32) = G_ZEXT [[SHL9]](s16)
+    ; VI: [[ZEXT7:%[0-9]+]]:_(s32) = G_ZEXT [[SHL10]](s16)
+    ; VI: [[SHL11:%[0-9]+]]:_(s32) = G_SHL [[ZEXT7]], [[C]](s32)
+    ; VI: [[OR8:%[0-9]+]]:_(s32) = G_OR [[ZEXT6]], [[SHL11]]
+    ; VI: [[BITCAST18:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR8]](s32)
+    ; VI: [[BITCAST19:%[0-9]+]]:_(s32) = G_BITCAST [[AND4]](<2 x s16>)
+    ; VI: [[TRUNC16:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST19]](s32)
+    ; VI: [[LSHR13:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST19]], [[C]](s32)
+    ; VI: [[TRUNC17:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR13]](s32)
+    ; VI: [[LSHR14:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC5]], [[TRUNC16]](s16)
+    ; VI: [[LSHR15:%[0-9]+]]:_(s16) = G_LSHR [[DEF2]], [[TRUNC17]](s16)
+    ; VI: [[ZEXT8:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR14]](s16)
+    ; VI: [[ZEXT9:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR15]](s16)
+    ; VI: [[SHL12:%[0-9]+]]:_(s32) = G_SHL [[ZEXT9]], [[C]](s32)
+    ; VI: [[OR9:%[0-9]+]]:_(s32) = G_OR [[ZEXT8]], [[SHL12]]
+    ; VI: [[BITCAST20:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR9]](s32)
+    ; VI: [[OR10:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST18]], [[BITCAST20]]
+    ; VI: [[BITCAST21:%[0-9]+]]:_(s32) = G_BITCAST [[AND4]](<2 x s16>)
+    ; VI: [[TRUNC18:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST21]](s32)
+    ; VI: [[LSHR16:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST21]], [[C]](s32)
+    ; VI: [[TRUNC19:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR16]](s32)
+    ; VI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC18]](s16), [[C4]]
+    ; VI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC19]](s16), [[C4]]
+    ; VI: [[BITCAST22:%[0-9]+]]:_(s32) = G_BITCAST [[OR10]](<2 x s16>)
+    ; VI: [[TRUNC20:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST22]](s32)
+    ; VI: [[LSHR17:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST22]], [[C]](s32)
+    ; VI: [[TRUNC21:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR17]](s32)
+    ; VI: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC5]], [[TRUNC20]]
+    ; VI: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[DEF2]], [[TRUNC21]]
+    ; VI: [[ZEXT10:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT2]](s16)
+    ; VI: [[ZEXT11:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT3]](s16)
+    ; VI: [[SHL13:%[0-9]+]]:_(s32) = G_SHL [[ZEXT11]], [[C]](s32)
+    ; VI: [[OR11:%[0-9]+]]:_(s32) = G_OR [[ZEXT10]], [[SHL13]]
+    ; VI: [[BITCAST23:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR11]](s32)
+    ; VI: [[CONCAT_VECTORS3:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[BITCAST15]](<2 x s16>), [[BITCAST23]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; VI: [[EXTRACT:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[CONCAT_VECTORS3]](<6 x s16>), 0
+    ; VI: [[EXTRACT1:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[DEF1]](<4 x s16>), 0
+    ; VI: [[CONCAT_VECTORS4:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[EXTRACT]](<3 x s16>), [[EXTRACT1]](<3 x s16>)
+    ; VI: [[UV12:%[0-9]+]]:_(<2 x s16>), [[UV13:%[0-9]+]]:_(<2 x s16>), [[UV14:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS4]](<6 x s16>)
+    ; VI: $vgpr0 = COPY [[UV12]](<2 x s16>)
+    ; VI: $vgpr1 = COPY [[UV13]](<2 x s16>)
+    ; VI: $vgpr2 = COPY [[UV14]](<2 x s16>)
+    ; GFX9-LABEL: name: test_fshr_v3s16_v3s16
+    ; GFX9: [[COPY:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr0
+    ; GFX9: [[COPY1:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr1
+    ; GFX9: [[COPY2:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr2
+    ; GFX9: [[COPY3:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr3
+    ; GFX9: [[COPY4:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr4
+    ; GFX9: [[COPY5:%[0-9]+]]:_(<2 x s16>) = COPY $vgpr5
+    ; GFX9: [[DEF:%[0-9]+]]:_(<2 x s16>) = G_IMPLICIT_DEF
+    ; GFX9: [[CONCAT_VECTORS:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY]](<2 x s16>), [[COPY1]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; GFX9: [[UV:%[0-9]+]]:_(<3 x s16>), [[UV1:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS]](<6 x s16>)
+    ; GFX9: [[CONCAT_VECTORS1:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY2]](<2 x s16>), [[COPY3]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; GFX9: [[UV2:%[0-9]+]]:_(<3 x s16>), [[UV3:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS1]](<6 x s16>)
+    ; GFX9: [[CONCAT_VECTORS2:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[COPY4]](<2 x s16>), [[COPY5]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; GFX9: [[UV4:%[0-9]+]]:_(<3 x s16>), [[UV5:%[0-9]+]]:_(<3 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS2]](<6 x s16>)
+    ; GFX9: [[DEF1:%[0-9]+]]:_(<4 x s16>) = G_IMPLICIT_DEF
+    ; GFX9: [[INSERT:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV]](<3 x s16>), 0
+    ; GFX9: [[UV6:%[0-9]+]]:_(<2 x s16>), [[UV7:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT]](<4 x s16>)
+    ; GFX9: [[BITCAST:%[0-9]+]]:_(s32) = G_BITCAST [[UV6]](<2 x s16>)
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; GFX9: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST]], [[C]](s32)
+    ; GFX9: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[UV7]](<2 x s16>)
+    ; GFX9: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C]](s32)
+    ; GFX9: [[DEF2:%[0-9]+]]:_(s16) = G_IMPLICIT_DEF
+    ; GFX9: [[COPY6:%[0-9]+]]:_(s32) = COPY [[BITCAST]](s32)
+    ; GFX9: [[COPY7:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY6]](s32), [[COPY7]](s32)
+    ; GFX9: [[COPY8:%[0-9]+]]:_(s32) = COPY [[BITCAST1]](s32)
+    ; GFX9: [[DEF3:%[0-9]+]]:_(s32) = G_IMPLICIT_DEF
+    ; GFX9: [[COPY9:%[0-9]+]]:_(s32) = COPY [[DEF3]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC1:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY8]](s32), [[COPY9]](s32)
+    ; GFX9: [[INSERT1:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV2]](<3 x s16>), 0
+    ; GFX9: [[UV8:%[0-9]+]]:_(<2 x s16>), [[UV9:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT1]](<4 x s16>)
+    ; GFX9: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV8]](<2 x s16>)
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; GFX9: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C]](s32)
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; GFX9: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[UV9]](<2 x s16>)
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST3]](s32)
+    ; GFX9: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C]](s32)
+    ; GFX9: [[COPY10:%[0-9]+]]:_(s32) = COPY [[BITCAST2]](s32)
+    ; GFX9: [[COPY11:%[0-9]+]]:_(s32) = COPY [[LSHR2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC2:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY10]](s32), [[COPY11]](s32)
+    ; GFX9: [[COPY12:%[0-9]+]]:_(s32) = COPY [[BITCAST3]](s32)
+    ; GFX9: [[COPY13:%[0-9]+]]:_(s32) = COPY [[DEF3]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC3:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY12]](s32), [[COPY13]](s32)
+    ; GFX9: [[INSERT2:%[0-9]+]]:_(<4 x s16>) = G_INSERT [[DEF1]], [[UV4]](<3 x s16>), 0
+    ; GFX9: [[UV10:%[0-9]+]]:_(<2 x s16>), [[UV11:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[INSERT2]](<4 x s16>)
+    ; GFX9: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV10]](<2 x s16>)
+    ; GFX9: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C]](s32)
+    ; GFX9: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[UV11]](<2 x s16>)
+    ; GFX9: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C]](s32)
+    ; GFX9: [[COPY14:%[0-9]+]]:_(s32) = COPY [[BITCAST4]](s32)
+    ; GFX9: [[COPY15:%[0-9]+]]:_(s32) = COPY [[LSHR4]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC4:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY14]](s32), [[COPY15]](s32)
+    ; GFX9: [[COPY16:%[0-9]+]]:_(s32) = COPY [[BITCAST5]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC5:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY16]](s32), [[DEF3]](s32)
+    ; GFX9: [[COPY17:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY18:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC6:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY17]](s32), [[COPY18]](s32)
+    ; GFX9: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; GFX9: [[COPY19:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC7:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY19]](s32), [[C2]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[BUILD_VECTOR_TRUNC4]], [[BUILD_VECTOR_TRUNC7]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC6]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(<2 x s16>) = G_SHL [[BUILD_VECTOR_TRUNC]], [[SUB]](<2 x s16>)
+    ; GFX9: [[LSHR6:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[BUILD_VECTOR_TRUNC2]], [[AND]](<2 x s16>)
+    ; GFX9: [[OR:%[0-9]+]]:_(<2 x s16>) = G_OR [[SHL]], [[LSHR6]]
+    ; GFX9: [[BITCAST6:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; GFX9: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST6]](s32)
+    ; GFX9: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST6]], [[C]](s32)
+    ; GFX9: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC3]](s16), [[C1]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC4]](s16), [[C1]]
+    ; GFX9: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[OR]](<2 x s16>)
+    ; GFX9: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST7]](s32)
+    ; GFX9: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C]](s32)
+    ; GFX9: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR8]](s32)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC5]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC1]], [[TRUNC6]]
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT1]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC8:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT]](s32), [[ANYEXT1]](s32)
+    ; GFX9: [[COPY20:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY21:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC9:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY20]](s32), [[COPY21]](s32)
+    ; GFX9: [[COPY22:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[COPY23:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC10:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY22]](s32), [[COPY23]](s32)
+    ; GFX9: [[AND1:%[0-9]+]]:_(<2 x s16>) = G_AND [[BUILD_VECTOR_TRUNC5]], [[BUILD_VECTOR_TRUNC10]]
+    ; GFX9: [[SUB1:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC9]], [[AND1]]
+    ; GFX9: [[SHL1:%[0-9]+]]:_(<2 x s16>) = G_SHL [[BUILD_VECTOR_TRUNC1]], [[SUB1]](<2 x s16>)
+    ; GFX9: [[LSHR9:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[BUILD_VECTOR_TRUNC3]], [[AND1]](<2 x s16>)
+    ; GFX9: [[OR1:%[0-9]+]]:_(<2 x s16>) = G_OR [[SHL1]], [[LSHR9]]
+    ; GFX9: [[BITCAST8:%[0-9]+]]:_(s32) = G_BITCAST [[AND1]](<2 x s16>)
+    ; GFX9: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST8]](s32)
+    ; GFX9: [[LSHR10:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST8]], [[C]](s32)
+    ; GFX9: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR10]](s32)
+    ; GFX9: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC7]](s16), [[C1]]
+    ; GFX9: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC8]](s16), [[C1]]
+    ; GFX9: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[OR1]](<2 x s16>)
+    ; GFX9: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; GFX9: [[LSHR11:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C]](s32)
+    ; GFX9: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR11]](s32)
+    ; GFX9: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC2]], [[TRUNC9]]
+    ; GFX9: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[DEF2]], [[TRUNC10]]
+    ; GFX9: [[ANYEXT2:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT2]](s16)
+    ; GFX9: [[ANYEXT3:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT3]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC11:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT2]](s32), [[ANYEXT3]](s32)
+    ; GFX9: [[CONCAT_VECTORS3:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[BUILD_VECTOR_TRUNC8]](<2 x s16>), [[BUILD_VECTOR_TRUNC11]](<2 x s16>), [[DEF]](<2 x s16>)
+    ; GFX9: [[EXTRACT:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[CONCAT_VECTORS3]](<6 x s16>), 0
+    ; GFX9: [[EXTRACT1:%[0-9]+]]:_(<3 x s16>) = G_EXTRACT [[DEF1]](<4 x s16>), 0
+    ; GFX9: [[CONCAT_VECTORS4:%[0-9]+]]:_(<6 x s16>) = G_CONCAT_VECTORS [[EXTRACT]](<3 x s16>), [[EXTRACT1]](<3 x s16>)
+    ; GFX9: [[UV12:%[0-9]+]]:_(<2 x s16>), [[UV13:%[0-9]+]]:_(<2 x s16>), [[UV14:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[CONCAT_VECTORS4]](<6 x s16>)
+    ; GFX9: $vgpr0 = COPY [[UV12]](<2 x s16>)
+    ; GFX9: $vgpr1 = COPY [[UV13]](<2 x s16>)
+    ; GFX9: $vgpr2 = COPY [[UV14]](<2 x s16>)
+    %0:_(<2 x s16>) = COPY $vgpr0
+    %1:_(<2 x s16>) = COPY $vgpr1
+    %2:_(<2 x s16>) = COPY $vgpr2
+    %3:_(<2 x s16>) = COPY $vgpr3
+    %4:_(<2 x s16>) = COPY $vgpr4
+    %5:_(<2 x s16>) = COPY $vgpr5
+    %6:_(<2 x s16>) = G_IMPLICIT_DEF
+    %7:_(<6 x s16>) = G_CONCAT_VECTORS %0(<2 x s16>), %1(<2 x s16>), %6(<2 x s16>)
+    %8:_(<3 x s16>), %9:_(<3 x s16>) = G_UNMERGE_VALUES %7(<6 x s16>)
+    %10:_(<6 x s16>) = G_CONCAT_VECTORS %2(<2 x s16>), %3(<2 x s16>), %6(<2 x s16>)
+    %11:_(<3 x s16>), %12:_(<3 x s16>) = G_UNMERGE_VALUES %10(<6 x s16>)
+    %13:_(<6 x s16>) = G_CONCAT_VECTORS %4(<2 x s16>), %5(<2 x s16>), %6(<2 x s16>)
+    %14:_(<3 x s16>), %15:_(<3 x s16>) = G_UNMERGE_VALUES %13(<6 x s16>)
+    %16:_(<3 x s16>) = G_FSHR %8, %11, %14(<3 x s16>)
+    %17:_(<3 x s16>) = G_IMPLICIT_DEF
+    %18:_(<6 x s16>) = G_CONCAT_VECTORS %16(<3 x s16>), %17(<3 x s16>)
+    %19:_(<2 x s16>), %20:_(<2 x s16>), %21:_(<2 x s16>) = G_UNMERGE_VALUES %18(<6 x s16>)
+    $vgpr0 = COPY %19(<2 x s16>)
+    $vgpr1 = COPY %20(<2 x s16>)
+    $vgpr2 = COPY %21(<2 x s16>)
+...
+
+---
+name: test_fshr_v4s16_v4s16
+body: |
+  bb.0:
+    liveins: $vgpr0_vgpr1, $vgpr2_vgpr3, $vgpr4_vgpr5
+
+    ; SI-LABEL: name: test_fshr_v4s16_v4s16
+    ; SI: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    ; SI: [[COPY1:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    ; SI: [[COPY2:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    ; SI: [[UV:%[0-9]+]]:_(<2 x s16>), [[UV1:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; SI: [[UV2:%[0-9]+]]:_(<2 x s16>), [[UV3:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY1]](<4 x s16>)
+    ; SI: [[UV4:%[0-9]+]]:_(<2 x s16>), [[UV5:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY2]](<4 x s16>)
+    ; SI: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; SI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[C1:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; SI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[C]], [[C1]](s32)
+    ; SI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; SI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; SI: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[UV4]], [[BITCAST]]
+    ; SI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C1]](s32)
+    ; SI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[BITCAST1]](s32)
+    ; SI: [[SUB:%[0-9]+]]:_(s32) = G_SUB [[COPY4]], [[COPY5]]
+    ; SI: [[COPY6:%[0-9]+]]:_(s32) = COPY [[LSHR]](s32)
+    ; SI: [[SUB1:%[0-9]+]]:_(s32) = G_SUB [[C1]], [[COPY6]]
+    ; SI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV]](<2 x s16>)
+    ; SI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C1]](s32)
+    ; SI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 65535
+    ; SI: [[COPY7:%[0-9]+]]:_(s32) = COPY [[SUB]](s32)
+    ; SI: [[AND1:%[0-9]+]]:_(s32) = G_AND [[COPY7]], [[C2]]
+    ; SI: [[COPY8:%[0-9]+]]:_(s32) = COPY [[BITCAST2]](s32)
+    ; SI: [[SHL1:%[0-9]+]]:_(s32) = G_SHL [[COPY8]], [[AND1]](s32)
+    ; SI: [[COPY9:%[0-9]+]]:_(s32) = COPY [[SUB1]](s32)
+    ; SI: [[AND2:%[0-9]+]]:_(s32) = G_AND [[COPY9]], [[C2]]
+    ; SI: [[COPY10:%[0-9]+]]:_(s32) = COPY [[LSHR1]](s32)
+    ; SI: [[SHL2:%[0-9]+]]:_(s32) = G_SHL [[COPY10]], [[AND2]](s32)
+    ; SI: [[COPY11:%[0-9]+]]:_(s32) = COPY [[SHL1]](s32)
+    ; SI: [[AND3:%[0-9]+]]:_(s32) = G_AND [[COPY11]], [[C2]]
+    ; SI: [[COPY12:%[0-9]+]]:_(s32) = COPY [[SHL2]](s32)
+    ; SI: [[AND4:%[0-9]+]]:_(s32) = G_AND [[COPY12]], [[C2]]
+    ; SI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[AND4]], [[C1]](s32)
+    ; SI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[AND3]], [[SHL3]]
+    ; SI: [[BITCAST3:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; SI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV2]](<2 x s16>)
+    ; SI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C1]](s32)
+    ; SI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C1]](s32)
+    ; SI: [[COPY13:%[0-9]+]]:_(s32) = COPY [[BITCAST5]](s32)
+    ; SI: [[AND5:%[0-9]+]]:_(s32) = G_AND [[COPY13]], [[C2]]
+    ; SI: [[COPY14:%[0-9]+]]:_(s32) = COPY [[BITCAST4]](s32)
+    ; SI: [[AND6:%[0-9]+]]:_(s32) = G_AND [[COPY14]], [[C2]]
+    ; SI: [[LSHR4:%[0-9]+]]:_(s32) = G_LSHR [[AND6]], [[AND5]](s32)
+    ; SI: [[COPY15:%[0-9]+]]:_(s32) = COPY [[LSHR3]](s32)
+    ; SI: [[AND7:%[0-9]+]]:_(s32) = G_AND [[COPY15]], [[C2]]
+    ; SI: [[COPY16:%[0-9]+]]:_(s32) = COPY [[LSHR2]](s32)
+    ; SI: [[AND8:%[0-9]+]]:_(s32) = G_AND [[COPY16]], [[C2]]
+    ; SI: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[AND8]], [[AND7]](s32)
+    ; SI: [[COPY17:%[0-9]+]]:_(s32) = COPY [[LSHR4]](s32)
+    ; SI: [[AND9:%[0-9]+]]:_(s32) = G_AND [[COPY17]], [[C2]]
+    ; SI: [[COPY18:%[0-9]+]]:_(s32) = COPY [[LSHR5]](s32)
+    ; SI: [[AND10:%[0-9]+]]:_(s32) = G_AND [[COPY18]], [[C2]]
+    ; SI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[AND10]], [[C1]](s32)
+    ; SI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[AND9]], [[SHL4]]
+    ; SI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; SI: [[OR3:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST3]], [[BITCAST6]]
+    ; SI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; SI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C1]](s32)
+    ; SI: [[COPY19:%[0-9]+]]:_(s32) = COPY [[BITCAST7]](s32)
+    ; SI: [[AND11:%[0-9]+]]:_(s32) = G_AND [[COPY19]], [[C2]]
+    ; SI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 0
+    ; SI: [[COPY20:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND11]](s32), [[COPY20]]
+    ; SI: [[COPY21:%[0-9]+]]:_(s32) = COPY [[LSHR6]](s32)
+    ; SI: [[AND12:%[0-9]+]]:_(s32) = G_AND [[COPY21]], [[C2]]
+    ; SI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND12]](s32), [[C3]]
+    ; SI: [[BITCAST8:%[0-9]+]]:_(s32) = G_BITCAST [[UV2]](<2 x s16>)
+    ; SI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST8]](s32)
+    ; SI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST8]], [[C1]](s32)
+    ; SI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; SI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[OR3]](<2 x s16>)
+    ; SI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; SI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C1]](s32)
+    ; SI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR8]](s32)
+    ; SI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC]], [[TRUNC2]]
+    ; SI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC1]], [[TRUNC3]]
+    ; SI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; SI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; SI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C1]](s32)
+    ; SI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL5]]
+    ; SI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; SI: [[COPY22:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[COPY23:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; SI: [[SHL6:%[0-9]+]]:_(s32) = G_SHL [[COPY23]], [[C1]](s32)
+    ; SI: [[OR5:%[0-9]+]]:_(s32) = G_OR [[COPY22]], [[SHL6]]
+    ; SI: [[BITCAST11:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR5]](s32)
+    ; SI: [[AND13:%[0-9]+]]:_(<2 x s16>) = G_AND [[UV5]], [[BITCAST11]]
+    ; SI: [[BITCAST12:%[0-9]+]]:_(s32) = G_BITCAST [[AND13]](<2 x s16>)
+    ; SI: [[LSHR9:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST12]], [[C1]](s32)
+    ; SI: [[COPY24:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY25:%[0-9]+]]:_(s32) = COPY [[BITCAST12]](s32)
+    ; SI: [[SUB2:%[0-9]+]]:_(s32) = G_SUB [[COPY24]], [[COPY25]]
+    ; SI: [[COPY26:%[0-9]+]]:_(s32) = COPY [[C1]](s32)
+    ; SI: [[COPY27:%[0-9]+]]:_(s32) = COPY [[LSHR9]](s32)
+    ; SI: [[SUB3:%[0-9]+]]:_(s32) = G_SUB [[COPY26]], [[COPY27]]
+    ; SI: [[BITCAST13:%[0-9]+]]:_(s32) = G_BITCAST [[UV1]](<2 x s16>)
+    ; SI: [[LSHR10:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST13]], [[C1]](s32)
+    ; SI: [[COPY28:%[0-9]+]]:_(s32) = COPY [[SUB2]](s32)
+    ; SI: [[AND14:%[0-9]+]]:_(s32) = G_AND [[COPY28]], [[C2]]
+    ; SI: [[COPY29:%[0-9]+]]:_(s32) = COPY [[BITCAST13]](s32)
+    ; SI: [[SHL7:%[0-9]+]]:_(s32) = G_SHL [[COPY29]], [[AND14]](s32)
+    ; SI: [[COPY30:%[0-9]+]]:_(s32) = COPY [[SUB3]](s32)
+    ; SI: [[AND15:%[0-9]+]]:_(s32) = G_AND [[COPY30]], [[C2]]
+    ; SI: [[COPY31:%[0-9]+]]:_(s32) = COPY [[LSHR10]](s32)
+    ; SI: [[SHL8:%[0-9]+]]:_(s32) = G_SHL [[COPY31]], [[AND15]](s32)
+    ; SI: [[COPY32:%[0-9]+]]:_(s32) = COPY [[SHL7]](s32)
+    ; SI: [[AND16:%[0-9]+]]:_(s32) = G_AND [[COPY32]], [[C2]]
+    ; SI: [[COPY33:%[0-9]+]]:_(s32) = COPY [[SHL8]](s32)
+    ; SI: [[AND17:%[0-9]+]]:_(s32) = G_AND [[COPY33]], [[C2]]
+    ; SI: [[SHL9:%[0-9]+]]:_(s32) = G_SHL [[AND17]], [[C1]](s32)
+    ; SI: [[OR6:%[0-9]+]]:_(s32) = G_OR [[AND16]], [[SHL9]]
+    ; SI: [[BITCAST14:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR6]](s32)
+    ; SI: [[BITCAST15:%[0-9]+]]:_(s32) = G_BITCAST [[UV3]](<2 x s16>)
+    ; SI: [[LSHR11:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST15]], [[C1]](s32)
+    ; SI: [[BITCAST16:%[0-9]+]]:_(s32) = G_BITCAST [[AND13]](<2 x s16>)
+    ; SI: [[LSHR12:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST16]], [[C1]](s32)
+    ; SI: [[COPY34:%[0-9]+]]:_(s32) = COPY [[BITCAST16]](s32)
+    ; SI: [[AND18:%[0-9]+]]:_(s32) = G_AND [[COPY34]], [[C2]]
+    ; SI: [[COPY35:%[0-9]+]]:_(s32) = COPY [[BITCAST15]](s32)
+    ; SI: [[AND19:%[0-9]+]]:_(s32) = G_AND [[COPY35]], [[C2]]
+    ; SI: [[LSHR13:%[0-9]+]]:_(s32) = G_LSHR [[AND19]], [[AND18]](s32)
+    ; SI: [[COPY36:%[0-9]+]]:_(s32) = COPY [[LSHR12]](s32)
+    ; SI: [[AND20:%[0-9]+]]:_(s32) = G_AND [[COPY36]], [[C2]]
+    ; SI: [[COPY37:%[0-9]+]]:_(s32) = COPY [[LSHR11]](s32)
+    ; SI: [[AND21:%[0-9]+]]:_(s32) = G_AND [[COPY37]], [[C2]]
+    ; SI: [[LSHR14:%[0-9]+]]:_(s32) = G_LSHR [[AND21]], [[AND20]](s32)
+    ; SI: [[COPY38:%[0-9]+]]:_(s32) = COPY [[LSHR13]](s32)
+    ; SI: [[AND22:%[0-9]+]]:_(s32) = G_AND [[COPY38]], [[C2]]
+    ; SI: [[COPY39:%[0-9]+]]:_(s32) = COPY [[LSHR14]](s32)
+    ; SI: [[AND23:%[0-9]+]]:_(s32) = G_AND [[COPY39]], [[C2]]
+    ; SI: [[SHL10:%[0-9]+]]:_(s32) = G_SHL [[AND23]], [[C1]](s32)
+    ; SI: [[OR7:%[0-9]+]]:_(s32) = G_OR [[AND22]], [[SHL10]]
+    ; SI: [[BITCAST17:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR7]](s32)
+    ; SI: [[OR8:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST14]], [[BITCAST17]]
+    ; SI: [[BITCAST18:%[0-9]+]]:_(s32) = G_BITCAST [[AND13]](<2 x s16>)
+    ; SI: [[LSHR15:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST18]], [[C1]](s32)
+    ; SI: [[COPY40:%[0-9]+]]:_(s32) = COPY [[BITCAST18]](s32)
+    ; SI: [[AND24:%[0-9]+]]:_(s32) = G_AND [[COPY40]], [[C2]]
+    ; SI: [[COPY41:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND24]](s32), [[COPY41]]
+    ; SI: [[COPY42:%[0-9]+]]:_(s32) = COPY [[LSHR15]](s32)
+    ; SI: [[AND25:%[0-9]+]]:_(s32) = G_AND [[COPY42]], [[C2]]
+    ; SI: [[COPY43:%[0-9]+]]:_(s32) = COPY [[C3]](s32)
+    ; SI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[AND25]](s32), [[COPY43]]
+    ; SI: [[BITCAST19:%[0-9]+]]:_(s32) = G_BITCAST [[UV3]](<2 x s16>)
+    ; SI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST19]](s32)
+    ; SI: [[LSHR16:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST19]], [[C1]](s32)
+    ; SI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR16]](s32)
+    ; SI: [[BITCAST20:%[0-9]+]]:_(s32) = G_BITCAST [[OR8]](<2 x s16>)
+    ; SI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST20]](s32)
+    ; SI: [[LSHR17:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST20]], [[C1]](s32)
+    ; SI: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR17]](s32)
+    ; SI: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC4]], [[TRUNC6]]
+    ; SI: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[TRUNC5]], [[TRUNC7]]
+    ; SI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT2]](s16)
+    ; SI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT3]](s16)
+    ; SI: [[SHL11:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C1]](s32)
+    ; SI: [[OR9:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL11]]
+    ; SI: [[BITCAST21:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR9]](s32)
+    ; SI: [[CONCAT_VECTORS:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST10]](<2 x s16>), [[BITCAST21]](<2 x s16>)
+    ; SI: $vgpr0_vgpr1 = COPY [[CONCAT_VECTORS]](<4 x s16>)
+    ; VI-LABEL: name: test_fshr_v4s16_v4s16
+    ; VI: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    ; VI: [[COPY1:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    ; VI: [[COPY2:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    ; VI: [[UV:%[0-9]+]]:_(<2 x s16>), [[UV1:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; VI: [[UV2:%[0-9]+]]:_(<2 x s16>), [[UV3:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY1]](<4 x s16>)
+    ; VI: [[UV4:%[0-9]+]]:_(<2 x s16>), [[UV5:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY2]](<4 x s16>)
+    ; VI: [[C:%[0-9]+]]:_(s16) = G_CONSTANT i16 16
+    ; VI: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; VI: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; VI: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[C3:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; VI: [[SHL:%[0-9]+]]:_(s32) = G_SHL [[C2]], [[C3]](s32)
+    ; VI: [[OR:%[0-9]+]]:_(s32) = G_OR [[COPY3]], [[SHL]]
+    ; VI: [[BITCAST:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR]](s32)
+    ; VI: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[UV4]], [[BITCAST]]
+    ; VI: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; VI: [[LSHR:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C3]](s32)
+    ; VI: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR]](s32)
+    ; VI: [[SUB:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC]]
+    ; VI: [[SUB1:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC1]]
+    ; VI: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[UV]](<2 x s16>)
+    ; VI: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; VI: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C3]](s32)
+    ; VI: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR1]](s32)
+    ; VI: [[SHL1:%[0-9]+]]:_(s16) = G_SHL [[TRUNC2]], [[SUB]](s16)
+    ; VI: [[SHL2:%[0-9]+]]:_(s16) = G_SHL [[TRUNC3]], [[SUB1]](s16)
+    ; VI: [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[SHL1]](s16)
+    ; VI: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[SHL2]](s16)
+    ; VI: [[SHL3:%[0-9]+]]:_(s32) = G_SHL [[ZEXT1]], [[C3]](s32)
+    ; VI: [[OR1:%[0-9]+]]:_(s32) = G_OR [[ZEXT]], [[SHL3]]
+    ; VI: [[BITCAST3:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR1]](s32)
+    ; VI: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV2]](<2 x s16>)
+    ; VI: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST4]](s32)
+    ; VI: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C3]](s32)
+    ; VI: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; VI: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST5]](s32)
+    ; VI: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C3]](s32)
+    ; VI: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; VI: [[LSHR4:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC4]], [[TRUNC6]](s16)
+    ; VI: [[LSHR5:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC5]], [[TRUNC7]](s16)
+    ; VI: [[ZEXT2:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR4]](s16)
+    ; VI: [[ZEXT3:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR5]](s16)
+    ; VI: [[SHL4:%[0-9]+]]:_(s32) = G_SHL [[ZEXT3]], [[C3]](s32)
+    ; VI: [[OR2:%[0-9]+]]:_(s32) = G_OR [[ZEXT2]], [[SHL4]]
+    ; VI: [[BITCAST6:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR2]](s32)
+    ; VI: [[OR3:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST3]], [[BITCAST6]]
+    ; VI: [[BITCAST7:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; VI: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST7]](s32)
+    ; VI: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST7]], [[C3]](s32)
+    ; VI: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR6]](s32)
+    ; VI: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC8]](s16), [[C1]]
+    ; VI: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC9]](s16), [[C1]]
+    ; VI: [[BITCAST8:%[0-9]+]]:_(s32) = G_BITCAST [[UV2]](<2 x s16>)
+    ; VI: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST8]](s32)
+    ; VI: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST8]], [[C3]](s32)
+    ; VI: [[TRUNC11:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; VI: [[BITCAST9:%[0-9]+]]:_(s32) = G_BITCAST [[OR3]](<2 x s16>)
+    ; VI: [[TRUNC12:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST9]](s32)
+    ; VI: [[LSHR8:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST9]], [[C3]](s32)
+    ; VI: [[TRUNC13:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR8]](s32)
+    ; VI: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC10]], [[TRUNC12]]
+    ; VI: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC11]], [[TRUNC13]]
+    ; VI: [[ZEXT4:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT]](s16)
+    ; VI: [[ZEXT5:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT1]](s16)
+    ; VI: [[SHL5:%[0-9]+]]:_(s32) = G_SHL [[ZEXT5]], [[C3]](s32)
+    ; VI: [[OR4:%[0-9]+]]:_(s32) = G_OR [[ZEXT4]], [[SHL5]]
+    ; VI: [[BITCAST10:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR4]](s32)
+    ; VI: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; VI: [[SHL6:%[0-9]+]]:_(s32) = G_SHL [[COPY5]], [[C3]](s32)
+    ; VI: [[OR5:%[0-9]+]]:_(s32) = G_OR [[COPY4]], [[SHL6]]
+    ; VI: [[BITCAST11:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR5]](s32)
+    ; VI: [[AND1:%[0-9]+]]:_(<2 x s16>) = G_AND [[UV5]], [[BITCAST11]]
+    ; VI: [[BITCAST12:%[0-9]+]]:_(s32) = G_BITCAST [[AND1]](<2 x s16>)
+    ; VI: [[TRUNC14:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST12]](s32)
+    ; VI: [[LSHR9:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST12]], [[C3]](s32)
+    ; VI: [[TRUNC15:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR9]](s32)
+    ; VI: [[SUB2:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC14]]
+    ; VI: [[SUB3:%[0-9]+]]:_(s16) = G_SUB [[C]], [[TRUNC15]]
+    ; VI: [[BITCAST13:%[0-9]+]]:_(s32) = G_BITCAST [[UV1]](<2 x s16>)
+    ; VI: [[TRUNC16:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST13]](s32)
+    ; VI: [[LSHR10:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST13]], [[C3]](s32)
+    ; VI: [[TRUNC17:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR10]](s32)
+    ; VI: [[SHL7:%[0-9]+]]:_(s16) = G_SHL [[TRUNC16]], [[SUB2]](s16)
+    ; VI: [[SHL8:%[0-9]+]]:_(s16) = G_SHL [[TRUNC17]], [[SUB3]](s16)
+    ; VI: [[ZEXT6:%[0-9]+]]:_(s32) = G_ZEXT [[SHL7]](s16)
+    ; VI: [[ZEXT7:%[0-9]+]]:_(s32) = G_ZEXT [[SHL8]](s16)
+    ; VI: [[SHL9:%[0-9]+]]:_(s32) = G_SHL [[ZEXT7]], [[C3]](s32)
+    ; VI: [[OR6:%[0-9]+]]:_(s32) = G_OR [[ZEXT6]], [[SHL9]]
+    ; VI: [[BITCAST14:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR6]](s32)
+    ; VI: [[BITCAST15:%[0-9]+]]:_(s32) = G_BITCAST [[UV3]](<2 x s16>)
+    ; VI: [[TRUNC18:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST15]](s32)
+    ; VI: [[LSHR11:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST15]], [[C3]](s32)
+    ; VI: [[TRUNC19:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR11]](s32)
+    ; VI: [[BITCAST16:%[0-9]+]]:_(s32) = G_BITCAST [[AND1]](<2 x s16>)
+    ; VI: [[TRUNC20:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST16]](s32)
+    ; VI: [[LSHR12:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST16]], [[C3]](s32)
+    ; VI: [[TRUNC21:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR12]](s32)
+    ; VI: [[LSHR13:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC18]], [[TRUNC20]](s16)
+    ; VI: [[LSHR14:%[0-9]+]]:_(s16) = G_LSHR [[TRUNC19]], [[TRUNC21]](s16)
+    ; VI: [[ZEXT8:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR13]](s16)
+    ; VI: [[ZEXT9:%[0-9]+]]:_(s32) = G_ZEXT [[LSHR14]](s16)
+    ; VI: [[SHL10:%[0-9]+]]:_(s32) = G_SHL [[ZEXT9]], [[C3]](s32)
+    ; VI: [[OR7:%[0-9]+]]:_(s32) = G_OR [[ZEXT8]], [[SHL10]]
+    ; VI: [[BITCAST17:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR7]](s32)
+    ; VI: [[OR8:%[0-9]+]]:_(<2 x s16>) = G_OR [[BITCAST14]], [[BITCAST17]]
+    ; VI: [[BITCAST18:%[0-9]+]]:_(s32) = G_BITCAST [[AND1]](<2 x s16>)
+    ; VI: [[TRUNC22:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST18]](s32)
+    ; VI: [[LSHR15:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST18]], [[C3]](s32)
+    ; VI: [[TRUNC23:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR15]](s32)
+    ; VI: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC22]](s16), [[C1]]
+    ; VI: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC23]](s16), [[C1]]
+    ; VI: [[BITCAST19:%[0-9]+]]:_(s32) = G_BITCAST [[UV3]](<2 x s16>)
+    ; VI: [[TRUNC24:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST19]](s32)
+    ; VI: [[LSHR16:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST19]], [[C3]](s32)
+    ; VI: [[TRUNC25:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR16]](s32)
+    ; VI: [[BITCAST20:%[0-9]+]]:_(s32) = G_BITCAST [[OR8]](<2 x s16>)
+    ; VI: [[TRUNC26:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST20]](s32)
+    ; VI: [[LSHR17:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST20]], [[C3]](s32)
+    ; VI: [[TRUNC27:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR17]](s32)
+    ; VI: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC24]], [[TRUNC26]]
+    ; VI: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[TRUNC25]], [[TRUNC27]]
+    ; VI: [[ZEXT10:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT2]](s16)
+    ; VI: [[ZEXT11:%[0-9]+]]:_(s32) = G_ZEXT [[SELECT3]](s16)
+    ; VI: [[SHL11:%[0-9]+]]:_(s32) = G_SHL [[ZEXT11]], [[C3]](s32)
+    ; VI: [[OR9:%[0-9]+]]:_(s32) = G_OR [[ZEXT10]], [[SHL11]]
+    ; VI: [[BITCAST21:%[0-9]+]]:_(<2 x s16>) = G_BITCAST [[OR9]](s32)
+    ; VI: [[CONCAT_VECTORS:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BITCAST10]](<2 x s16>), [[BITCAST21]](<2 x s16>)
+    ; VI: $vgpr0_vgpr1 = COPY [[CONCAT_VECTORS]](<4 x s16>)
+    ; GFX9-LABEL: name: test_fshr_v4s16_v4s16
+    ; GFX9: [[COPY:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    ; GFX9: [[COPY1:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    ; GFX9: [[COPY2:%[0-9]+]]:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    ; GFX9: [[UV:%[0-9]+]]:_(<2 x s16>), [[UV1:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY]](<4 x s16>)
+    ; GFX9: [[UV2:%[0-9]+]]:_(<2 x s16>), [[UV3:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY1]](<4 x s16>)
+    ; GFX9: [[UV4:%[0-9]+]]:_(<2 x s16>), [[UV5:%[0-9]+]]:_(<2 x s16>) = G_UNMERGE_VALUES [[COPY2]](<4 x s16>)
+    ; GFX9: [[C:%[0-9]+]]:_(s32) = G_CONSTANT i32 16
+    ; GFX9: [[COPY3:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY4:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY3]](s32), [[COPY4]](s32)
+    ; GFX9: [[C1:%[0-9]+]]:_(s16) = G_CONSTANT i16 0
+    ; GFX9: [[C2:%[0-9]+]]:_(s32) = G_CONSTANT i32 15
+    ; GFX9: [[COPY5:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC1:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY5]](s32), [[C2]](s32)
+    ; GFX9: [[AND:%[0-9]+]]:_(<2 x s16>) = G_AND [[UV4]], [[BUILD_VECTOR_TRUNC1]]
+    ; GFX9: [[SUB:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC]], [[AND]]
+    ; GFX9: [[SHL:%[0-9]+]]:_(<2 x s16>) = G_SHL [[UV]], [[SUB]](<2 x s16>)
+    ; GFX9: [[LSHR:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[UV2]], [[AND]](<2 x s16>)
+    ; GFX9: [[OR:%[0-9]+]]:_(<2 x s16>) = G_OR [[SHL]], [[LSHR]]
+    ; GFX9: [[BITCAST:%[0-9]+]]:_(s32) = G_BITCAST [[AND]](<2 x s16>)
+    ; GFX9: [[TRUNC:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST]](s32)
+    ; GFX9: [[LSHR1:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST]], [[C]](s32)
+    ; GFX9: [[TRUNC1:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR1]](s32)
+    ; GFX9: [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC]](s16), [[C1]]
+    ; GFX9: [[ICMP1:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC1]](s16), [[C1]]
+    ; GFX9: [[BITCAST1:%[0-9]+]]:_(s32) = G_BITCAST [[UV2]](<2 x s16>)
+    ; GFX9: [[TRUNC2:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST1]](s32)
+    ; GFX9: [[LSHR2:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST1]], [[C]](s32)
+    ; GFX9: [[TRUNC3:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR2]](s32)
+    ; GFX9: [[BITCAST2:%[0-9]+]]:_(s32) = G_BITCAST [[OR]](<2 x s16>)
+    ; GFX9: [[TRUNC4:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST2]](s32)
+    ; GFX9: [[LSHR3:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST2]], [[C]](s32)
+    ; GFX9: [[TRUNC5:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR3]](s32)
+    ; GFX9: [[SELECT:%[0-9]+]]:_(s16) = G_SELECT [[ICMP]](s1), [[TRUNC2]], [[TRUNC4]]
+    ; GFX9: [[SELECT1:%[0-9]+]]:_(s16) = G_SELECT [[ICMP1]](s1), [[TRUNC3]], [[TRUNC5]]
+    ; GFX9: [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT]](s16)
+    ; GFX9: [[ANYEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT1]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC2:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT]](s32), [[ANYEXT1]](s32)
+    ; GFX9: [[COPY6:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[COPY7:%[0-9]+]]:_(s32) = COPY [[C]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC3:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY6]](s32), [[COPY7]](s32)
+    ; GFX9: [[COPY8:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[COPY9:%[0-9]+]]:_(s32) = COPY [[C2]](s32)
+    ; GFX9: [[BUILD_VECTOR_TRUNC4:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[COPY8]](s32), [[COPY9]](s32)
+    ; GFX9: [[AND1:%[0-9]+]]:_(<2 x s16>) = G_AND [[UV5]], [[BUILD_VECTOR_TRUNC4]]
+    ; GFX9: [[SUB1:%[0-9]+]]:_(<2 x s16>) = G_SUB [[BUILD_VECTOR_TRUNC3]], [[AND1]]
+    ; GFX9: [[SHL1:%[0-9]+]]:_(<2 x s16>) = G_SHL [[UV1]], [[SUB1]](<2 x s16>)
+    ; GFX9: [[LSHR4:%[0-9]+]]:_(<2 x s16>) = G_LSHR [[UV3]], [[AND1]](<2 x s16>)
+    ; GFX9: [[OR1:%[0-9]+]]:_(<2 x s16>) = G_OR [[SHL1]], [[LSHR4]]
+    ; GFX9: [[BITCAST3:%[0-9]+]]:_(s32) = G_BITCAST [[AND1]](<2 x s16>)
+    ; GFX9: [[TRUNC6:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST3]](s32)
+    ; GFX9: [[LSHR5:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST3]], [[C]](s32)
+    ; GFX9: [[TRUNC7:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR5]](s32)
+    ; GFX9: [[ICMP2:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC6]](s16), [[C1]]
+    ; GFX9: [[ICMP3:%[0-9]+]]:_(s1) = G_ICMP intpred(eq), [[TRUNC7]](s16), [[C1]]
+    ; GFX9: [[BITCAST4:%[0-9]+]]:_(s32) = G_BITCAST [[UV3]](<2 x s16>)
+    ; GFX9: [[TRUNC8:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST4]](s32)
+    ; GFX9: [[LSHR6:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST4]], [[C]](s32)
+    ; GFX9: [[TRUNC9:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR6]](s32)
+    ; GFX9: [[BITCAST5:%[0-9]+]]:_(s32) = G_BITCAST [[OR1]](<2 x s16>)
+    ; GFX9: [[TRUNC10:%[0-9]+]]:_(s16) = G_TRUNC [[BITCAST5]](s32)
+    ; GFX9: [[LSHR7:%[0-9]+]]:_(s32) = G_LSHR [[BITCAST5]], [[C]](s32)
+    ; GFX9: [[TRUNC11:%[0-9]+]]:_(s16) = G_TRUNC [[LSHR7]](s32)
+    ; GFX9: [[SELECT2:%[0-9]+]]:_(s16) = G_SELECT [[ICMP2]](s1), [[TRUNC8]], [[TRUNC10]]
+    ; GFX9: [[SELECT3:%[0-9]+]]:_(s16) = G_SELECT [[ICMP3]](s1), [[TRUNC9]], [[TRUNC11]]
+    ; GFX9: [[ANYEXT2:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT2]](s16)
+    ; GFX9: [[ANYEXT3:%[0-9]+]]:_(s32) = G_ANYEXT [[SELECT3]](s16)
+    ; GFX9: [[BUILD_VECTOR_TRUNC5:%[0-9]+]]:_(<2 x s16>) = G_BUILD_VECTOR_TRUNC [[ANYEXT2]](s32), [[ANYEXT3]](s32)
+    ; GFX9: [[CONCAT_VECTORS:%[0-9]+]]:_(<4 x s16>) = G_CONCAT_VECTORS [[BUILD_VECTOR_TRUNC2]](<2 x s16>), [[BUILD_VECTOR_TRUNC5]](<2 x s16>)
+    ; GFX9: $vgpr0_vgpr1 = COPY [[CONCAT_VECTORS]](<4 x s16>)
+    %0:_(<4 x s16>) = COPY $vgpr0_vgpr1
+    %1:_(<4 x s16>) = COPY $vgpr2_vgpr3
+    %2:_(<4 x s16>) = COPY $vgpr4_vgpr5
+    %3:_(<4 x s16>) = G_FSHR %0, %1, %2
+    $vgpr0_vgpr1 = COPY %3
+...