[DAGCombine] Simplify (truncate (build_pair x, y)) -> (truncate x) or x

[kparzysz]

Summary

In the attached Hexagon test, the vector type <32 x i64> gets scalarized into a number of extracts and build_pairs. It is subsequently not optimized further, and instead of a single HVX instruction, we end up with a lot of scalar code.

Without this patch, before type legalization we have:

Optimized lowered selection DAG: %bb.0 'f0:b0'
SelectionDAG has 15 nodes:
  t0: ch = EntryToken
              t2: v32i32,ch = CopyFromReg t0, Register:v32i32 %0
              t19: v32i32 = splat_vector Constant:i32<1>
            t18: v32i32 = shl t2, t19
          t7: v64i32 = concat_vectors t18, undef:v32i32
        t9: v64i32 = vector_shuffle<0,u,1,u,2,u,3,u,4,u,5,u,6,u,7,u,8,u,9,u,10,u,11,u,12,u,13,u,14,u,15,u,16,u,17,u,18,u,19,u,20,u,21,u,22,u,23,u,24,u,25,u,26,u,27,u,28,u,29,u,30,u,31,u> t7, undef:v64i32
      t10: v32i64 = bitcast t9
    t11: v32i32 = truncate t10
  t13: ch,glue = CopyToReg t0, Register:v32i32 $v0, t11
  t14: ch = HexagonISD::RET_FLAG t13, Register:v32i32 $v0, t13:1

After

Type-legalized selection DAG: %bb.0 'f0:b0'
SelectionDAG has 205 nodes:
  t0: ch = EntryToken
        t2: v32i32,ch = CopyFromReg t0, Register:v32i32 %0
        t19: v32i32 = splat_vector Constant:i32<1>
      t18: v32i32 = shl t2, t19
    t7: v64i32 = concat_vectors t18, undef:v32i32
  t9: v64i32 = vector_shuffle<0,u,1,u,2,u,3,u,4,u,5,u,6,u,7,u,8,u,9,u,10,u,11,u,12,u,13,u,14,u,15,u,16,u,17,u,18,u,19,u,20,u,21,u,22,u,23,u,24,u,25,u,26,u,27,u,28,u,29,u,30,u,31,u> t7, undef:v64i32
          t31: i32 = extract_vector_elt t9, Constant:i32<0>
          t32: i32 = extract_vector_elt t9, Constant:i32<1>
        t157: i64 = build_pair t31, t32
      t227: i32 = truncate t157
          t34: i32 = extract_vector_elt t9, Constant:i32<2>
          t36: i32 = extract_vector_elt t9, Constant:i32<3>
        t158: i64 = build_pair t34, t36
      t229: i32 = truncate t158
          t38: i32 = extract_vector_elt t9, Constant:i32<4>
          t40: i32 = extract_vector_elt t9, Constant:i32<5>
        t159: i64 = build_pair t38, t40
      t231: i32 = truncate t159
          t42: i32 = extract_vector_elt t9, Constant:i32<6>
          t44: i32 = extract_vector_elt t9, Constant:i32<7>
        t160: i64 = build_pair t42, t44
      t233: i32 = truncate t160
[...]

This should be a harmless change, but there is one degradation on x86: llvm/test/CodeGen/X86/test-shrink.ll

The DAG before instruction selection:

SelectionDAG has 14 nodes:
  t0: ch = EntryToken
            t41: i32,ch = load<(load (s16) from %fixed-stack.1, align 4), zext from i16> t0, FrameIndex:i32<-1>, undef:i32
          t40: i32 = and t41, Constant:i32<32768>
        t38: i16 = truncate t40
      t32: i32 = X86ISD::CMP t38, Constant:i16<0>
    t34: ch = X86ISD::BRCOND t0, BasicBlock:ch<no 0x82ec648>, TargetConstant:i8<8>, t32
  t18: ch = br t34, BasicBlock:ch<yes 0x82ec550>

The i32 = load (s16) does not match the pattern that generates AND32rm, and we generate two instructions instead of one.

[craig.topper]

llvm/test/CodeGen/X86/test-shrink.ll

870

The IR here isn't canonical according to InstCombine so it's hard to say if this is a real regression or not.

[pengfei]

llvm/test/CodeGen/X86/test-shrink.ll

870

I see the tests are introduced for shrinking code fold:
https://github.com/llvm/llvm-project/commit/42cd8cd8626a7f5eb14b0b43b866dd90bd33277b
Does the code not work for i16 anymore, or we can use another test case?

[pengfei]

llvm/test/CodeGen/X86/test-shrink.ll

870

It reduces one uop for non-minsize case, but increases 2 bytes for minsize case, which supposes to be degradation? https://godbolt.org/z/4v8z1sh1x

[RKSimon]

Maybe better to initially make this Hexagon only?

[kparzysz]

In D135700#3852225, @RKSimon wrote:

Maybe better to initially make this Hexagon only?

Yes, I have this as a target-specific DAG combine in a local branch, so I'm not depending on this patch.

[RKSimon]

In D135700#3852320, @kparzysz wrote:

In D135700#3852225, @RKSimon wrote:

Maybe better to initially make this Hexagon only?

Yes, I have this as a target-specific DAG combine in a local branch, so I'm not depending on this patch.

What do you want to do next - merge your local branch Hexagon patch into trunk?

[kparzysz]

In D135700#3885112, @RKSimon wrote:

What do you want to do next - merge your local branch Hexagon patch into trunk?

Sorry, missed this comment. The Hexagon-specific code is already merged. As for this, I was planning to fix the x86 codegen to get this case, but I haven't had time to do it yet. I believe this only applies to extending loads, where the consumer accesses the extension bits, and only when the consumer can be modified to access the memory directly. Only existing patterns/selections would need to be updated, because without such explicit change from using register to memory this patch wouldn't have any effect on the code size.

[arsenm]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

13508

Capitalize

13511

Capitalize

13513

Can just unconditionally truncate, it folds out the equality case for you

[arsenm]

LGTM besides the use getNode truncate type check. I'm pretty sure I've tried to do this one before

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

if (N0.getOpcode() == ISD::ZERO_EXTEND ||
     // if the source is larger than the dest, than we just need the truncate.
     if (N0.getOperand(0).getValueType().bitsGT(VT))
       return DAG.getNode(ISD::TRUNCATE, SDLoc(N), VT, N0.getOperand(0));
     // if the source and dest are the same type, we can drop both the extend
     // and the truncate.
     return N0.getOperand(0);
   }
 
+  // fold (truncate (build pair x, y)) -> (truncate x) or x
+  if (N0.getOpcode() == ISD::BUILD_PAIR) {
+    // if we match the low element of the pair, just return it.
+    if (N0.getOperand(0).getValueType() == VT)
+      return N0.getOperand(0);
+    // otherwise, if the low part is still too large, apply the truncate.
+    if (N0.getOperand(0).getValueType().bitsGT(VT))
+      return DAG.getNode(ISD::TRUNCATE, SDLoc(N), VT, N0.getOperand(0));
+  }
+
   // Try to narrow a truncate-of-sext_in_reg to the destination type:
   // trunc (sign_ext_inreg X, iM) to iN --> sign_ext_inreg (trunc X to iN), iM
   if (!LegalTypes && N0.getOpcode() == ISD::SIGN_EXTEND_INREG &&
       N0.hasOneUse()) {
     SDValue X = N0.getOperand(0);
     SDValue ExtVal = N0.getOperand(1);
     EVT ExtVT = cast<VTSDNode>(ExtVal)->getVT();
     if (ExtVT.bitsLT(VT)) {

llvm/test/CodeGen/AMDGPU/llvm.mulo.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck --check-prefix=SI %s
 ; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs < %s | FileCheck --check-prefix=GFX9 %s
 ; RUN: llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs < %s | FileCheck --check-prefix=GFX10 %s
 ; RUN: llc -march=amdgcn -mcpu=gfx1100 -verify-machineinstrs < %s | FileCheck --check-prefix=GFX11 %s
 
 define { i64, i1 } @umulo_i64_v_v(i64 %x, i64 %y) {
 ; SI-LABEL: umulo_i64_v_v:
 ; SI:       ; %bb.0: ; %bb
 ; SI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; SI-NEXT:    v_mul_hi_u32 v4, v1, v2
-; SI-NEXT:    v_mul_lo_u32 v5, v1, v2
-; SI-NEXT:    v_mul_hi_u32 v6, v0, v3
-; SI-NEXT:    v_mul_lo_u32 v7, v0, v3
-; SI-NEXT:    v_mul_hi_u32 v8, v0, v2
-; SI-NEXT:    v_mul_hi_u32 v9, v1, v3
+; SI-NEXT:    v_mul_hi_u32 v4, v1, v3
+; SI-NEXT:    v_mul_hi_u32 v5, v1, v2
+; SI-NEXT:    v_mul_lo_u32 v6, v1, v2
+; SI-NEXT:    v_mul_hi_u32 v7, v0, v3
+; SI-NEXT:    v_mul_lo_u32 v8, v0, v3
+; SI-NEXT:    v_mul_hi_u32 v9, v0, v2
 ; SI-NEXT:    v_mul_lo_u32 v3, v1, v3
 ; SI-NEXT:    v_mul_lo_u32 v0, v0, v2
-; SI-NEXT:    v_add_i32_e32 v1, vcc, v8, v7
-; SI-NEXT:    v_addc_u32_e32 v2, vcc, 0, v6, vcc
-; SI-NEXT:    v_add_i32_e32 v6, vcc, v1, v5
-; SI-NEXT:    v_add_i32_e64 v1, s[4:5], v1, v5
-; SI-NEXT:    v_addc_u32_e32 v2, vcc, v2, v4, vcc
-; SI-NEXT:    v_addc_u32_e32 v4, vcc, 0, v9, vcc
+; SI-NEXT:    v_add_i32_e32 v1, vcc, v9, v8
+; SI-NEXT:    v_addc_u32_e32 v2, vcc, 0, v7, vcc
+; SI-NEXT:    v_add_i32_e32 v7, vcc, v1, v6
+; SI-NEXT:    v_add_i32_e64 v1, s[4:5], v1, v6
+; SI-NEXT:    v_addc_u32_e32 v2, vcc, v2, v5, vcc
+; SI-NEXT:    v_addc_u32_e32 v4, vcc, 0, v4, vcc
 ; SI-NEXT:    v_add_i32_e32 v2, vcc, v2, v3
 ; SI-NEXT:    v_addc_u32_e32 v3, vcc, 0, v4, vcc
 ; SI-NEXT:    v_cmp_ne_u64_e32 vcc, 0, v[2:3]
 ; SI-NEXT:    v_cndmask_b32_e64 v2, 0, 1, vcc
 ; SI-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: umulo_i64_v_v:
 ; GFX9:       ; %bb.0: ; %bb
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_mov_b32_e32 v5, v0
 ; GFX9-NEXT:    v_mov_b32_e32 v4, v1
 ; GFX9-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v5, v2, 0
-; GFX9-NEXT:    v_mad_u64_u32 v[6:7], s[4:5], v5, v3, 0
-; GFX9-NEXT:    v_mad_u64_u32 v[8:9], s[4:5], v4, v2, 0
-; GFX9-NEXT:    v_mov_b32_e32 v10, v1
-; GFX9-NEXT:    v_add_co_u32_e32 v10, vcc, v10, v6
-; GFX9-NEXT:    v_addc_co_u32_e32 v11, vcc, 0, v7, vcc
+; GFX9-NEXT:    v_mad_u64_u32 v[8:9], s[4:5], v5, v3, 0
+; GFX9-NEXT:    v_mad_u64_u32 v[10:11], s[4:5], v4, v2, 0
+; GFX9-NEXT:    v_mov_b32_e32 v12, v1
 ; GFX9-NEXT:    v_mad_u64_u32 v[6:7], s[4:5], v4, v3, 0
-; GFX9-NEXT:    v_add_co_u32_e32 v8, vcc, v10, v8
-; GFX9-NEXT:    v_addc_co_u32_e32 v8, vcc, v11, v9, vcc
+; GFX9-NEXT:    v_add_co_u32_e32 v8, vcc, v12, v8
+; GFX9-NEXT:    v_addc_co_u32_e32 v9, vcc, 0, v9, vcc
+; GFX9-NEXT:    v_add_co_u32_e32 v8, vcc, v8, v10
+; GFX9-NEXT:    v_addc_co_u32_e32 v8, vcc, v9, v11, vcc
 ; GFX9-NEXT:    v_addc_co_u32_e32 v7, vcc, 0, v7, vcc
 ; GFX9-NEXT:    v_mul_lo_u32 v4, v4, v2
 ; GFX9-NEXT:    v_mul_lo_u32 v5, v5, v3
 ; GFX9-NEXT:    v_add_co_u32_e32 v2, vcc, v8, v6
 ; GFX9-NEXT:    v_addc_co_u32_e32 v3, vcc, 0, v7, vcc
 ; GFX9-NEXT:    v_cmp_ne_u64_e32 vcc, 0, v[2:3]
 ; GFX9-NEXT:    v_add3_u32 v1, v1, v5, v4
 ; GFX9-NEXT:    v_cndmask_b32_e64 v2, 0, 1, vcc
bb:
   %umulo = tail call { i64, i1 } @llvm.umul.with.overflow.i64(i64 %x, i64 %y)
   ret { i64, i1 } %umulo
 }
 
 define { i64, i1 } @smulo_i64_v_v(i64 %x, i64 %y) {
 ; SI-LABEL: smulo_i64_v_v:
 ; SI:       ; %bb.0: ; %bb
 ; SI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; SI-NEXT:    v_mul_hi_i32 v7, v1, v3
 ; SI-NEXT:    v_mul_hi_u32 v6, v1, v2
 ; SI-NEXT:    v_mul_lo_u32 v5, v1, v2
-; SI-NEXT:    v_mul_hi_u32 v7, v0, v3
-; SI-NEXT:    v_mul_lo_u32 v8, v0, v3
-; SI-NEXT:    v_mul_hi_u32 v9, v0, v2
-; SI-NEXT:    v_mul_hi_i32 v10, v1, v3
+; SI-NEXT:    v_mul_hi_u32 v8, v0, v3
+; SI-NEXT:    v_mul_lo_u32 v9, v0, v3
+; SI-NEXT:    v_mul_hi_u32 v10, v0, v2
 ; SI-NEXT:    v_mul_lo_u32 v11, v1, v3
 ; SI-NEXT:    v_mul_lo_u32 v4, v0, v2
-; SI-NEXT:    v_add_i32_e32 v8, vcc, v9, v8
-; SI-NEXT:    v_addc_u32_e32 v7, vcc, 0, v7, vcc
-; SI-NEXT:    v_add_i32_e32 v9, vcc, v8, v5
-; SI-NEXT:    v_add_i32_e64 v5, s[4:5], v8, v5
-; SI-NEXT:    v_addc_u32_e32 v8, vcc, v7, v6, vcc
+; SI-NEXT:    v_add_i32_e32 v9, vcc, v10, v9
+; SI-NEXT:    v_addc_u32_e32 v8, vcc, 0, v8, vcc
+; SI-NEXT:    v_add_i32_e32 v10, vcc, v9, v5
+; SI-NEXT:    v_add_i32_e64 v5, s[4:5], v9, v5
+; SI-NEXT:    v_addc_u32_e32 v8, vcc, v8, v6, vcc
 ; SI-NEXT:    v_ashrrev_i32_e32 v6, 31, v5
-; SI-NEXT:    v_addc_u32_e32 v9, vcc, 0, v10, vcc
+; SI-NEXT:    v_addc_u32_e32 v9, vcc, 0, v7, vcc
 ; SI-NEXT:    v_mov_b32_e32 v7, v6
 ; SI-NEXT:    v_add_i32_e32 v8, vcc, v8, v11
 ; SI-NEXT:    v_addc_u32_e32 v9, vcc, 0, v9, vcc
 ; SI-NEXT:    v_sub_i32_e32 v2, vcc, v8, v2
 ; SI-NEXT:    v_subbrev_u32_e32 v10, vcc, 0, v9, vcc
 ; SI-NEXT:    v_cmp_gt_i32_e32 vcc, 0, v1
 ; SI-NEXT:    v_cndmask_b32_e32 v1, v9, v10, vcc
 ; SI-NEXT:    v_cndmask_b32_e32 v2, v8, v2, vcc

 ; SI-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: smulo_i64_v_v:
 ; GFX9:       ; %bb.0: ; %bb
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-NEXT:    v_mov_b32_e32 v5, v0
 ; GFX9-NEXT:    v_mov_b32_e32 v4, v1
 ; GFX9-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v5, v2, 0
-; GFX9-NEXT:    v_mad_u64_u32 v[6:7], s[4:5], v5, v3, 0
-; GFX9-NEXT:    v_mad_u64_u32 v[8:9], s[4:5], v4, v2, 0
-; GFX9-NEXT:    v_mov_b32_e32 v10, v1
-; GFX9-NEXT:    v_add_co_u32_e32 v10, vcc, v10, v6
-; GFX9-NEXT:    v_addc_co_u32_e32 v11, vcc, 0, v7, vcc
+; GFX9-NEXT:    v_mad_u64_u32 v[8:9], s[4:5], v5, v3, 0
+; GFX9-NEXT:    v_mad_u64_u32 v[10:11], s[4:5], v4, v2, 0
+; GFX9-NEXT:    v_mov_b32_e32 v12, v1
 ; GFX9-NEXT:    v_mad_i64_i32 v[6:7], s[4:5], v4, v3, 0
-; GFX9-NEXT:    v_add_co_u32_e32 v8, vcc, v10, v8
-; GFX9-NEXT:    v_addc_co_u32_e32 v8, vcc, v11, v9, vcc
+; GFX9-NEXT:    v_add_co_u32_e32 v8, vcc, v12, v8
+; GFX9-NEXT:    v_addc_co_u32_e32 v9, vcc, 0, v9, vcc
+; GFX9-NEXT:    v_add_co_u32_e32 v8, vcc, v8, v10
+; GFX9-NEXT:    v_addc_co_u32_e32 v8, vcc, v9, v11, vcc
 ; GFX9-NEXT:    v_addc_co_u32_e32 v7, vcc, 0, v7, vcc
 ; GFX9-NEXT:    v_add_co_u32_e32 v6, vcc, v8, v6
 ; GFX9-NEXT:    v_addc_co_u32_e32 v7, vcc, 0, v7, vcc
 ; GFX9-NEXT:    v_sub_co_u32_e32 v8, vcc, v6, v2
 ; GFX9-NEXT:    v_subbrev_co_u32_e32 v9, vcc, 0, v7, vcc
 ; GFX9-NEXT:    v_cmp_gt_i32_e32 vcc, 0, v4
 ; GFX9-NEXT:    v_cndmask_b32_e32 v6, v6, v8, vcc
 ; GFX9-NEXT:    v_cndmask_b32_e32 v7, v7, v9, vcc

llvm/test/CodeGen/AMDGPU/mad_64_32.ll

; GFX11-NEXT:    s_setpc_b64 s[30:31]
   %add = add i64 %mul, %arg2
   ret i64 %add
 }
 
 define i64 @mad_u64_u32_bitops_lhs_mask_small(i64 %arg0, i64 %arg1, i64 %arg2) #0 {
 ; CI-LABEL: mad_u64_u32_bitops_lhs_mask_small:
 ; CI:       ; %bb.0:
 ; CI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; CI-NEXT:    v_and_b32_e32 v3, 1, v1
+; CI-NEXT:    v_and_b32_e32 v1, 1, v1
+; CI-NEXT:    v_mul_lo_u32 v3, v1, v2
 ; CI-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v0, v2, v[4:5]
-; CI-NEXT:    v_mul_lo_u32 v2, v3, v2
-; CI-NEXT:    v_add_i32_e32 v1, vcc, v2, v1
+; CI-NEXT:    v_add_i32_e32 v1, vcc, v3, v1
 ; CI-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; SI-LABEL: mad_u64_u32_bitops_lhs_mask_small:
 ; SI:       ; %bb.0:
 ; SI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; SI-NEXT:    v_and_b32_e32 v1, 1, v1
 ; SI-NEXT:    v_mul_hi_u32 v3, v0, v2
 ; SI-NEXT:    v_mul_lo_u32 v1, v1, v2
; GFX11-NEXT:    s_setpc_b64 s[30:31]
   %add = add i64 %mul, %arg2
   ret i64 %add
 }
 
 define i64 @mad_u64_u32_bitops_rhs_mask_small(i64 %arg0, i64 %arg1, i64 %arg2) #0 {
 ; CI-LABEL: mad_u64_u32_bitops_rhs_mask_small:
 ; CI:       ; %bb.0:
 ; CI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; CI-NEXT:    v_mov_b32_e32 v6, v0
-; CI-NEXT:    v_and_b32_e32 v3, 1, v3
-; CI-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v6, v2, v[4:5]
-; CI-NEXT:    v_mul_lo_u32 v2, v6, v3
-; CI-NEXT:    v_add_i32_e32 v1, vcc, v2, v1
+; CI-NEXT:    v_and_b32_e32 v1, 1, v3
+; CI-NEXT:    v_mul_lo_u32 v3, v0, v1
+; CI-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v0, v2, v[4:5]
+; CI-NEXT:    v_add_i32_e32 v1, vcc, v3, v1
 ; CI-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; SI-LABEL: mad_u64_u32_bitops_rhs_mask_small:
 ; SI:       ; %bb.0:
 ; SI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; SI-NEXT:    v_and_b32_e32 v1, 1, v3
 ; SI-NEXT:    v_mul_hi_u32 v3, v0, v2
 ; SI-NEXT:    v_mul_lo_u32 v1, v0, v1
; GFX11-NEXT:    s_setpc_b64 s[30:31]
   %out = xor i64 %mad, %mul
   ret i64 %out
 }
 
 define i48 @mad_i48_i48(i48 %arg0, i48 %arg1, i48 %arg2) #0 {
 ; CI-LABEL: mad_i48_i48:
 ; CI:       ; %bb.0:
 ; CI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; CI-NEXT:    v_mov_b32_e32 v6, v1
-; CI-NEXT:    v_mov_b32_e32 v7, v0
-; CI-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v7, v2, v[4:5]
-; CI-NEXT:    v_mul_lo_u32 v2, v6, v2
-; CI-NEXT:    v_mul_lo_u32 v3, v7, v3
+; CI-NEXT:    v_mov_b32_e32 v6, v0
+; CI-NEXT:    v_mul_lo_u32 v7, v1, v2
+; CI-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v6, v2, v[4:5]
+; CI-NEXT:    v_mul_lo_u32 v2, v6, v3
+; CI-NEXT:    v_add_i32_e32 v1, vcc, v7, v1
 ; CI-NEXT:    v_add_i32_e32 v1, vcc, v2, v1
-; CI-NEXT:    v_add_i32_e32 v1, vcc, v3, v1
 ; CI-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; SI-LABEL: mad_i48_i48:
 ; SI:       ; %bb.0:
 ; SI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; SI-NEXT:    v_mul_lo_u32 v3, v0, v3
 ; SI-NEXT:    v_mul_hi_u32 v6, v0, v2
+; SI-NEXT:    v_mul_lo_u32 v3, v0, v3
 ; SI-NEXT:    v_mul_lo_u32 v1, v1, v2
 ; SI-NEXT:    v_mul_lo_u32 v0, v0, v2
 ; SI-NEXT:    v_add_i32_e32 v3, vcc, v6, v3
 ; SI-NEXT:    v_add_i32_e32 v1, vcc, v3, v1
 ; SI-NEXT:    v_add_i32_e32 v0, vcc, v0, v4
 ; SI-NEXT:    v_addc_u32_e32 v1, vcc, v1, v5, vcc
 ; SI-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: mad_i48_i48:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT:    v_mov_b32_e32 v6, v1
-; GFX9-NEXT:    v_mov_b32_e32 v7, v0
-; GFX9-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v7, v2, v[4:5]
-; GFX9-NEXT:    v_mul_lo_u32 v3, v7, v3
-; GFX9-NEXT:    v_mul_lo_u32 v2, v6, v2
-; GFX9-NEXT:    v_add3_u32 v1, v2, v1, v3
+; GFX9-NEXT:    v_mul_lo_u32 v3, v0, v3
+; GFX9-NEXT:    v_mul_lo_u32 v6, v1, v2
+; GFX9-NEXT:    v_mad_u64_u32 v[0:1], s[4:5], v0, v2, v[4:5]
+; GFX9-NEXT:    v_add3_u32 v1, v6, v1, v3
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX11-LABEL: mad_i48_i48:
 ; GFX11:       ; %bb.0:
 ; GFX11-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX11-NEXT:    s_waitcnt_vscnt null, 0x0
-; GFX11-NEXT:    v_dual_mov_b32 v6, v1 :: v_dual_mov_b32 v7, v0
-; GFX11-NEXT:    s_delay_alu instid0(VALU_DEP_1) | instskip(SKIP_1) | instid1(VALU_DEP_3)
-; GFX11-NEXT:    v_mad_u64_u32 v[0:1], null, v7, v2, v[4:5]
-; GFX11-NEXT:    v_mul_lo_u32 v3, v7, v3
-; GFX11-NEXT:    v_mul_lo_u32 v2, v6, v2
-; GFX11-NEXT:    s_delay_alu instid0(VALU_DEP_1)
-; GFX11-NEXT:    v_add3_u32 v1, v2, v1, v3
+; GFX11-NEXT:    v_mov_b32_e32 v6, v0
+; GFX11-NEXT:    v_mul_lo_u32 v7, v1, v2
+; GFX11-NEXT:    s_delay_alu instid0(VALU_DEP_2) | instskip(SKIP_1) | instid1(VALU_DEP_1)
+; GFX11-NEXT:    v_mul_lo_u32 v3, v6, v3
+; GFX11-NEXT:    v_mad_u64_u32 v[0:1], null, v6, v2, v[4:5]
+; GFX11-NEXT:    v_add3_u32 v1, v7, v1, v3
 ; GFX11-NEXT:    s_setpc_b64 s[30:31]
   %m = mul i48 %arg0, %arg1
   %a = add i48 %m, %arg2
   ret i48 %a
 }
 
 attributes #0 = { nounwind }
 attributes #1 = { nounwind readnone speculatable }

llvm/test/CodeGen/AMDGPU/select-undef.ll

ret:
   ret void
 }
 
 ; We were expanding undef vectors into zero vectors. Optimizations
 ; would then see we used no elements of the vector, and reform the
 ; undef vector resulting in a combiner loop.
 ; GCN-LABEL: {{^}}inf_loop_undef_vector:
 ; GCN: s_waitcnt
-; GCN-NEXT: v_mad_u64_u32
 ; GCN-NEXT: v_mul_lo_u32
 ; GCN-NEXT: v_mul_lo_u32
+; GCN-NEXT: v_mad_u64_u32
 ; GCN-NEXT: v_add3_u32
 ; GCN-NEXT: global_store_dwordx2
 define void @inf_loop_undef_vector(<6 x float> %arg, float %arg1, i64 %arg2) {
   %i = insertelement <6 x float> %arg, float %arg1, i64 2
   %i3 = bitcast <6 x float> %i to <3 x i64>
   %i4 = extractelement <3 x i64> %i3, i64 0
   %i5 = extractelement <3 x i64> %i3, i64 1
   %i6 = mul i64 %i5, %arg2
   %i7 = add i64 %i6, %i4
   store volatile i64 %i7, i64 addrspace(1)* undef, align 4
   ret void
 }

llvm/test/CodeGen/AMDGPU/shift-i128.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=kaveri -verify-machineinstrs < %s | FileCheck -check-prefix=GCN %s
 
 define i128 @v_shl_i128_vv(i128 %lhs, i128 %rhs) {
 ; GCN-LABEL: v_shl_i128_vv:
 ; GCN:       ; %bb.0:
 ; GCN-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GCN-NEXT:    v_sub_i32_e32 v7, vcc, 64, v4
-; GCN-NEXT:    v_lshl_b64 v[5:6], v[2:3], v4
-; GCN-NEXT:    v_lshr_b64 v[7:8], v[0:1], v7
+; GCN-NEXT:    v_sub_i32_e32 v5, vcc, 64, v4
+; GCN-NEXT:    v_lshr_b64 v[5:6], v[0:1], v5
+; GCN-NEXT:    v_lshl_b64 v[7:8], v[2:3], v4
 ; GCN-NEXT:    v_cmp_eq_u32_e64 s[4:5], 0, v4
-; GCN-NEXT:    v_or_b32_e32 v7, v5, v7
+; GCN-NEXT:    v_or_b32_e32 v7, v7, v5
 ; GCN-NEXT:    v_subrev_i32_e32 v5, vcc, 64, v4
-; GCN-NEXT:    v_or_b32_e32 v8, v6, v8
+; GCN-NEXT:    v_or_b32_e32 v8, v8, v6
 ; GCN-NEXT:    v_lshl_b64 v[5:6], v[0:1], v5
 ; GCN-NEXT:    v_cmp_gt_u32_e32 vcc, 64, v4
 ; GCN-NEXT:    v_cndmask_b32_e32 v5, v5, v7, vcc
 ; GCN-NEXT:    v_lshl_b64 v[0:1], v[0:1], v4
 ; GCN-NEXT:    v_cndmask_b32_e64 v2, v5, v2, s[4:5]
 ; GCN-NEXT:    v_cndmask_b32_e32 v5, v6, v8, vcc
 ; GCN-NEXT:    v_cndmask_b32_e64 v3, v5, v3, s[4:5]
 ; GCN-NEXT:    v_cndmask_b32_e32 v0, 0, v0, vcc
 ; GCN-NEXT:    v_cndmask_b32_e32 v1, 0, v1, vcc
 ; GCN-NEXT:    s_setpc_b64 s[30:31]
   %shl = shl i128 %lhs, %rhs
   ret i128 %shl
 }
 
 define i128 @v_lshr_i128_vv(i128 %lhs, i128 %rhs) {
 ; GCN-LABEL: v_lshr_i128_vv:
 ; GCN:       ; %bb.0:
 ; GCN-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GCN-NEXT:    v_sub_i32_e32 v7, vcc, 64, v4
-; GCN-NEXT:    v_lshr_b64 v[5:6], v[0:1], v4
-; GCN-NEXT:    v_lshl_b64 v[7:8], v[2:3], v7
+; GCN-NEXT:    v_sub_i32_e32 v5, vcc, 64, v4
+; GCN-NEXT:    v_lshl_b64 v[5:6], v[2:3], v5
+; GCN-NEXT:    v_lshr_b64 v[7:8], v[0:1], v4
 ; GCN-NEXT:    v_cmp_eq_u32_e64 s[4:5], 0, v4
-; GCN-NEXT:    v_or_b32_e32 v7, v5, v7
+; GCN-NEXT:    v_or_b32_e32 v7, v7, v5
 ; GCN-NEXT:    v_subrev_i32_e32 v5, vcc, 64, v4
-; GCN-NEXT:    v_or_b32_e32 v8, v6, v8
+; GCN-NEXT:    v_or_b32_e32 v8, v8, v6
 ; GCN-NEXT:    v_lshr_b64 v[5:6], v[2:3], v5
 ; GCN-NEXT:    v_cmp_gt_u32_e32 vcc, 64, v4
 ; GCN-NEXT:    v_cndmask_b32_e32 v5, v5, v7, vcc
 ; GCN-NEXT:    v_lshr_b64 v[2:3], v[2:3], v4
 ; GCN-NEXT:    v_cndmask_b32_e64 v0, v5, v0, s[4:5]
 ; GCN-NEXT:    v_cndmask_b32_e32 v5, v6, v8, vcc
 ; GCN-NEXT:    v_cndmask_b32_e64 v1, v5, v1, s[4:5]
 ; GCN-NEXT:    v_cndmask_b32_e32 v2, 0, v2, vcc
 ; GCN-NEXT:    v_cndmask_b32_e32 v3, 0, v3, vcc
 ; GCN-NEXT:    s_setpc_b64 s[30:31]
 
   %shl = lshr i128 %lhs, %rhs
   ret i128 %shl
 }
 
 define i128 @v_ashr_i128_vv(i128 %lhs, i128 %rhs) {
 ; GCN-LABEL: v_ashr_i128_vv:
 ; GCN:       ; %bb.0:
 ; GCN-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GCN-NEXT:    v_sub_i32_e32 v7, vcc, 64, v4
-; GCN-NEXT:    v_lshr_b64 v[5:6], v[0:1], v4
+; GCN-NEXT:    v_sub_i32_e64 v7, s[4:5], 64, v4
 ; GCN-NEXT:    v_lshl_b64 v[7:8], v[2:3], v7
-; GCN-NEXT:    v_cmp_eq_u32_e64 s[4:5], 0, v4
-; GCN-NEXT:    v_or_b32_e32 v7, v5, v7
-; GCN-NEXT:    v_subrev_i32_e32 v5, vcc, 64, v4
-; GCN-NEXT:    v_or_b32_e32 v8, v6, v8
-; GCN-NEXT:    v_ashr_i64 v[5:6], v[2:3], v5
+; GCN-NEXT:    v_lshr_b64 v[9:10], v[0:1], v4
+; GCN-NEXT:    v_ashrrev_i32_e32 v11, 31, v3
+; GCN-NEXT:    v_or_b32_e32 v8, v10, v8
+; GCN-NEXT:    v_subrev_i32_e64 v10, s[4:5], 64, v4
+; GCN-NEXT:    v_ashr_i64 v[5:6], v[2:3], v4
+; GCN-NEXT:    v_ashr_i64 v[2:3], v[2:3], v10
 ; GCN-NEXT:    v_cmp_gt_u32_e32 vcc, 64, v4
-; GCN-NEXT:    v_cndmask_b32_e32 v5, v5, v7, vcc
-; GCN-NEXT:    v_cndmask_b32_e64 v0, v5, v0, s[4:5]
-; GCN-NEXT:    v_cndmask_b32_e32 v5, v6, v8, vcc
-; GCN-NEXT:    v_cndmask_b32_e64 v1, v5, v1, s[4:5]
-; GCN-NEXT:    v_ashr_i64 v[4:5], v[2:3], v4
-; GCN-NEXT:    v_ashrrev_i32_e32 v3, 31, v3
-; GCN-NEXT:    v_cndmask_b32_e32 v2, v3, v4, vcc
-; GCN-NEXT:    v_cndmask_b32_e32 v3, v3, v5, vcc
+; GCN-NEXT:    v_or_b32_e32 v7, v9, v7
+; GCN-NEXT:    v_cndmask_b32_e32 v2, v2, v7, vcc
+; GCN-NEXT:    v_cmp_eq_u32_e64 s[4:5], 0, v4
+; GCN-NEXT:    v_cndmask_b32_e32 v5, v11, v5, vcc
+; GCN-NEXT:    v_cndmask_b32_e32 v6, v11, v6, vcc
+; GCN-NEXT:    v_cndmask_b32_e64 v0, v2, v0, s[4:5]
+; GCN-NEXT:    v_cndmask_b32_e32 v2, v3, v8, vcc
+; GCN-NEXT:    v_cndmask_b32_e64 v1, v2, v1, s[4:5]
+; GCN-NEXT:    v_mov_b32_e32 v2, v5
+; GCN-NEXT:    v_mov_b32_e32 v3, v6
 ; GCN-NEXT:    s_setpc_b64 s[30:31]
   %shl = ashr i128 %lhs, %rhs
   ret i128 %shl
 }
 
 
 define i128 @v_shl_i128_vk(i128 %lhs) {
 ; GCN-LABEL: v_shl_i128_vk:

llvm/test/CodeGen/AMDGPU/store-weird-sizes.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=hawaii -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=CIVI,HAWAII %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=fiji -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=CIVI,FIJI %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx900 -verify-machineinstrs < %s | FileCheck -enable-var-scope --check-prefix=GFX9 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1010 -verify-machineinstrs < %s | FileCheck -enable-var-scope --check-prefix=GFX10 %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1100 -verify-machineinstrs < %s | FileCheck -enable-var-scope --check-prefix=GFX11 %s
 
 define void @local_store_i56(i56 addrspace(3)* %ptr, i56 %arg) #0 {
 ; CIVI-LABEL: local_store_i56:
 ; CIVI:       ; %bb.0:
 ; CIVI-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; CIVI-NEXT:    s_mov_b32 m0, -1
-; CIVI-NEXT:    ds_write_b16 v0, v2 offset:4
 ; CIVI-NEXT:    ds_write_b32 v0, v1
+; CIVI-NEXT:    ds_write_b16 v0, v2 offset:4
 ; CIVI-NEXT:    v_lshrrev_b32_e32 v1, 16, v2
 ; CIVI-NEXT:    ds_write_b8 v0, v1 offset:6
 ; CIVI-NEXT:    s_waitcnt lgkmcnt(0)
 ; CIVI-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-LABEL: local_store_i56:
 ; GFX9:       ; %bb.0:
 ; GFX9-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT:    ds_write_b32 v0, v1
 ; GFX9-NEXT:    ds_write_b8_d16_hi v0, v2 offset:6
 ; GFX9-NEXT:    ds_write_b16 v0, v2 offset:4
-; GFX9-NEXT:    ds_write_b32 v0, v1
 ; GFX9-NEXT:    s_waitcnt lgkmcnt(0)
 ; GFX9-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX10-LABEL: local_store_i56:
 ; GFX10:       ; %bb.0:
 ; GFX10-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX10-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX10-NEXT:    ds_write_b32 v0, v1
 ; GFX10-NEXT:    ds_write_b8_d16_hi v0, v2 offset:6
 ; GFX10-NEXT:    ds_write_b16 v0, v2 offset:4
-; GFX10-NEXT:    ds_write_b32 v0, v1
 ; GFX10-NEXT:    s_waitcnt lgkmcnt(0)
 ; GFX10-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX11-LABEL: local_store_i56:
 ; GFX11:       ; %bb.0:
 ; GFX11-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX11-NEXT:    s_waitcnt_vscnt null, 0x0
+; GFX11-NEXT:    ds_store_b32 v0, v1
 ; GFX11-NEXT:    ds_store_b8_d16_hi v0, v2 offset:6
 ; GFX11-NEXT:    ds_store_b16 v0, v2 offset:4
-; GFX11-NEXT:    ds_store_b32 v0, v1
 ; GFX11-NEXT:    s_waitcnt lgkmcnt(0)
 ; GFX11-NEXT:    s_setpc_b64 s[30:31]
   store i56 %arg, i56 addrspace(3)* %ptr, align 8
   ret void
 }
 
 define amdgpu_kernel void @local_store_i55(i55 addrspace(3)* %ptr, i55 %arg) #0 {
 ; HAWAII-LABEL: local_store_i55:

llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll

; GFX9-O3-NEXT:    s_setpc_b64 s[30:31]
   call void @llvm.amdgcn.raw.buffer.store.i32(i32 %tmp137, <4 x i32> %tmp14, i32 4, i32 0, i32 0)
   ret void
 }
 
 define amdgpu_gfx i64 @strict_wwm_called_i64(i64 %a) noinline {
 ; GFX9-O0-LABEL: strict_wwm_called_i64:
 ; GFX9-O0:       ; %bb.0:
 ; GFX9-O0-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-O0-NEXT:    v_mov_b32_e32 v2, v0
+; GFX9-O0-NEXT:    v_mov_b32_e32 v2, v1
+; GFX9-O0-NEXT:    v_mov_b32_e32 v3, v0
 ; GFX9-O0-NEXT:    ; implicit-def: $sgpr34
 ; GFX9-O0-NEXT:    ; implicit-def: $sgpr34
-; GFX9-O0-NEXT:    ; kill: def $vgpr1 killed $vgpr1 killed $exec
-; GFX9-O0-NEXT:    ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
-; GFX9-O0-NEXT:    v_mov_b32_e32 v3, v1
-; GFX9-O0-NEXT:    ; implicit-def: $sgpr34_sgpr35
-; GFX9-O0-NEXT:    v_mov_b32_e32 v4, v2
-; GFX9-O0-NEXT:    v_mov_b32_e32 v5, v2
+; GFX9-O0-NEXT:    ; kill: def $vgpr2 killed $vgpr2 killed $exec
 ; GFX9-O0-NEXT:    v_mov_b32_e32 v0, v3
-; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v3
-; GFX9-O0-NEXT:    v_add_co_u32_e64 v4, s[34:35], v4, v5
-; GFX9-O0-NEXT:    v_addc_co_u32_e64 v0, s[34:35], v0, v1, s[34:35]
+; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v2
+; GFX9-O0-NEXT:    ; implicit-def: $sgpr34_sgpr35
+; GFX9-O0-NEXT:    v_mov_b32_e32 v4, v0
+; GFX9-O0-NEXT:    v_mov_b32_e32 v6, v0
+; GFX9-O0-NEXT:    v_mov_b32_e32 v2, v1
+; GFX9-O0-NEXT:    v_mov_b32_e32 v5, v1
+; GFX9-O0-NEXT:    v_add_co_u32_e64 v4, s[34:35], v4, v6
+; GFX9-O0-NEXT:    v_addc_co_u32_e64 v2, s[34:35], v2, v5, s[34:35]
 ; GFX9-O0-NEXT:    ; kill: def $vgpr4 killed $vgpr4 def $vgpr4_vgpr5 killed $exec
-; GFX9-O0-NEXT:    v_mov_b32_e32 v5, v0
+; GFX9-O0-NEXT:    v_mov_b32_e32 v5, v2
 ; GFX9-O0-NEXT:    s_mov_b32 s34, 32
-; GFX9-O0-NEXT:    v_mov_b32_e32 v0, v2
-; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v3
 ; GFX9-O0-NEXT:    v_lshrrev_b64 v[0:1], s34, v[0:1]
 ; GFX9-O0-NEXT:    v_mov_b32_e32 v1, v0
 ; GFX9-O0-NEXT:    v_mov_b32_e32 v0, v4
 ; GFX9-O0-NEXT:    v_mul_lo_u32 v1, v0, v1
-; GFX9-O0-NEXT:    v_mov_b32_e32 v3, v2
 ; GFX9-O0-NEXT:    v_lshrrev_b64 v[6:7], s34, v[4:5]
 ; GFX9-O0-NEXT:    v_mov_b32_e32 v2, v6
 ; GFX9-O0-NEXT:    v_mul_lo_u32 v2, v2, v3
 ; GFX9-O0-NEXT:    v_mad_u64_u32 v[6:7], s[36:37], v0, v3, 0
 ; GFX9-O0-NEXT:    v_mov_b32_e32 v0, v7
 ; GFX9-O0-NEXT:    v_add3_u32 v0, v0, v1, v2
 ; GFX9-O0-NEXT:    ; implicit-def: $sgpr35
 ; GFX9-O0-NEXT:    ; implicit-def: $sgpr36

 ; GFX9-O0-NEXT:    ; kill: def $vgpr1 killed $vgpr1 killed $vgpr1_vgpr2 killed $exec
 ; GFX9-O0-NEXT:    s_setpc_b64 s[30:31]
 ;
 ; GFX9-O3-LABEL: strict_wwm_called_i64:
 ; GFX9-O3:       ; %bb.0:
 ; GFX9-O3-NEXT:    s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
 ; GFX9-O3-NEXT:    v_add_co_u32_e32 v2, vcc, v0, v0
 ; GFX9-O3-NEXT:    v_addc_co_u32_e32 v3, vcc, v1, v1, vcc
-; GFX9-O3-NEXT:    v_mul_lo_u32 v4, v3, v0
-; GFX9-O3-NEXT:    v_mul_lo_u32 v5, v2, v1
+; GFX9-O3-NEXT:    v_mul_lo_u32 v4, v2, v1
+; GFX9-O3-NEXT:    v_mul_lo_u32 v5, v3, v0
 ; GFX9-O3-NEXT:    v_mad_u64_u32 v[0:1], s[34:35], v2, v0, 0
-; GFX9-O3-NEXT:    v_add3_u32 v1, v1, v5, v4
+; GFX9-O3-NEXT:    v_add3_u32 v1, v1, v4, v5
 ; GFX9-O3-NEXT:    v_sub_co_u32_e32 v0, vcc, v0, v2
 ; GFX9-O3-NEXT:    v_subb_co_u32_e32 v1, vcc, v1, v3, vcc
 ; GFX9-O3-NEXT:    s_setpc_b64 s[30:31]
   %add = add i64 %a, %a
   %mul = mul i64 %add, %a
   %sub = sub i64 %mul, %add
   ret i64 %sub
 }

llvm/test/CodeGen/Hexagon/isel-simplify-trunc-buildpair.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -march=hexagon < %s | FileCheck %s
+
+target datalayout = "e-m:e-p:32:32:32-a:0-n16:32-i64:64:64-i32:32:32-i16:16:16-i1:8:8-f32:32:32-f64:64:64-v32:32:32-v64:64:64-v512:512:512-v1024:1024:1024-v2048:2048:2048"
+target triple = "hexagon"
+
+define <32 x i32> @f0(<32 x i32> %a0) #0 {
+; CHECK-LABEL: f0:
+; CHECK:         .cfi_startproc
+; CHECK-NEXT:  // %bb.0: // %b0
+; CHECK-NEXT:    {
+; CHECK-NEXT:     r0 = #1
+; CHECK-NEXT:    }
+; CHECK-NEXT:    {
+; CHECK-NEXT:     v0.w = vasl(v0.w,r0)
+; CHECK-NEXT:    }
+; CHECK-NEXT:    {
+; CHECK-NEXT:     jumpr r31
+; CHECK-NEXT:    }
+b0:
+  %v0 = call <32 x i32> @llvm.fshr.v32i32(<32 x i32> %a0, <32 x i32> poison, <32 x i32> <i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31, i32 31>)
+  %v1 = shufflevector <32 x i32> %v0, <32 x i32> poison, <64 x i32> <i32 0, i32 32, i32 1, i32 33, i32 2, i32 34, i32 3, i32 35, i32 4, i32 36, i32 5, i32 37, i32 6, i32 38, i32 7, i32 39, i32 8, i32 40, i32 9, i32 41, i32 10, i32 42, i32 11, i32 43, i32 12, i32 44, i32 13, i32 45, i32 14, i32 46, i32 15, i32 47, i32 16, i32 48, i32 17, i32 49, i32 18, i32 50, i32 19, i32 51, i32 20, i32 52, i32 21, i32 53, i32 22, i32 54, i32 23, i32 55, i32 24, i32 56, i32 25, i32 57, i32 26, i32 58, i32 27, i32 59, i32 28, i32 60, i32 29, i32 61, i32 30, i32 62, i32 31, i32 63>
+  %v2 = bitcast <64 x i32> %v1 to <32 x i64>
+  %v3 = trunc <32 x i64> %v2 to <32 x i32>
+  ret <32 x i32> %v3
+}
+
+; Function Attrs: nocallback nofree nosync nounwind readnone speculatable willreturn
+declare <32 x i32> @llvm.fshr.v32i32(<32 x i32>, <32 x i32>, <32 x i32>) #1
+
+attributes #0 = { "target-features"="+v62,+hvxv62,+hvx-length128b,-packets" }
+attributes #1 = { nocallback nofree nosync nounwind readnone speculatable willreturn }

llvm/test/CodeGen/X86/64-bit-shift-by-32-minus-y.ll

 ; X32-NOBMI2-LABEL: t5_cse:
 ; X32-NOBMI2:       # %bb.0:
 ; X32-NOBMI2-NEXT:    pushl %ebx
 ; X32-NOBMI2-NEXT:    pushl %edi
 ; X32-NOBMI2-NEXT:    pushl %esi
 ; X32-NOBMI2-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X32-NOBMI2-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X32-NOBMI2-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X32-NOBMI2-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NOBMI2-NEXT:    movl {{[0-9]+}}(%esp), %edi
+; X32-NOBMI2-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NOBMI2-NEXT:    movl %eax, %ebx
 ; X32-NOBMI2-NEXT:    addl $32, %ebx
 ; X32-NOBMI2-NEXT:    adcl $0, %edi
 ; X32-NOBMI2-NEXT:    movl %ebx, (%ecx)
 ; X32-NOBMI2-NEXT:    movl %edi, 4(%ecx)
 ; X32-NOBMI2-NEXT:    movb $32, %cl
 ; X32-NOBMI2-NEXT:    subb %al, %cl
 ; X32-NOBMI2-NEXT:    movl %esi, %eax

 ; X32-BMI2-LABEL: t5_cse:
 ; X32-BMI2:       # %bb.0:
 ; X32-BMI2-NEXT:    pushl %ebx
 ; X32-BMI2-NEXT:    pushl %edi
 ; X32-BMI2-NEXT:    pushl %esi
 ; X32-BMI2-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-BMI2-NEXT:    movl {{[0-9]+}}(%esp), %edx
 ; X32-BMI2-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X32-BMI2-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X32-BMI2-NEXT:    movl {{[0-9]+}}(%esp), %esi
+; X32-BMI2-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X32-BMI2-NEXT:    movl %ebx, %edi
 ; X32-BMI2-NEXT:    addl $32, %edi
 ; X32-BMI2-NEXT:    adcl $0, %esi
 ; X32-BMI2-NEXT:    movl %edi, (%ecx)
 ; X32-BMI2-NEXT:    movl %esi, 4(%ecx)
 ; X32-BMI2-NEXT:    movb $32, %cl
 ; X32-BMI2-NEXT:    subb %bl, %cl
 ; X32-BMI2-NEXT:    shldl %cl, %eax, %edx

llvm/test/CodeGen/X86/combine-bswap.ll

; X64-NEXT:    retq
   %s = shl i64 %z, 48
   %b = call i64 @llvm.bswap.i64(i64 %s)
   ret i64 %b
 }
 
 define i64 @test_bswap64_shift48(i64 %a0) {
 ; X86-LABEL: test_bswap64_shift48:
 ; X86:       # %bb.0:
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    rolw $8, %ax
 ; X86-NEXT:    movzwl %ax, %eax
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: test_bswap64_shift48:
 ; X64:       # %bb.0:
 ; X64-NEXT:    rolw $8, %di

llvm/test/CodeGen/X86/pr49451.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-unknown-unknown | FileCheck %s --check-prefix=X86
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s --check-prefix=X64
 
 @s_0 = external dso_local local_unnamed_addr global i16, align 2
 @s_2 = external dso_local local_unnamed_addr global i16, align 2
 
 define void @func_6(i8 %uc_8, i64 %uli_10) nounwind {
 ; X86-LABEL: func_6:
 ; X86:       # %bb.0: # %entry
 ; X86-NEXT:    pushl %ebx
 ; X86-NEXT:    pushl %esi
-; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl $-1, %ecx
 ; X86-NEXT:    xorl %edx, %edx
 ; X86-NEXT:    xorl %ebx, %ebx
 ; X86-NEXT:    # implicit-def: $si
 ; X86-NEXT:    .p2align 4, 0x90
 ; X86-NEXT:  .LBB0_1: # %for.body612
 ; X86-NEXT:    # =>This Inner Loop Header: Depth=1
 ; X86-NEXT:    testb %dl, %dl

llvm/test/CodeGen/X86/test-shrink.ll

 ; CHECK-WIN32-64-NEXT:  # %bb.1: # %yes
 ; CHECK-WIN32-64-NEXT:    callq bar
 ; CHECK-WIN32-64-NEXT:  .LBB18_2: # %no
 ; CHECK-WIN32-64-NEXT:    addq $40, %rsp
 ; CHECK-WIN32-64-NEXT:    retq
 ;
 ; CHECK-X86-LABEL: and64_trunc_16_sign:
 ; CHECK-X86:       # %bb.0:
-; CHECK-X86-NEXT:    movl $32768, %eax # imm = 0x8000
-; CHECK-X86-NEXT:    andl {{[0-9]+}}(%esp), %eax
+; CHECK-X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-X86-NEXT:    andl $32768, %eax # imm = 0x8000
 ; CHECK-X86-NEXT:    testw %ax, %ax
 ; CHECK-X86-NEXT:    js .LBB18_2
 ; CHECK-X86-NEXT:  # %bb.1: # %yes
 ; CHECK-X86-NEXT:    calll bar@PLT
 ; CHECK-X86-NEXT:  .LBB18_2: # %no
 ; CHECK-X86-NEXT:    retl
   %t = and i64 %x, 32768
   %r = trunc i64 %t to i16

 ; CHECK-WIN32-64-NEXT:  # %bb.1: # %yes
 ; CHECK-WIN32-64-NEXT:    callq bar
 ; CHECK-WIN32-64-NEXT:  .LBB19_2: # %no
 ; CHECK-WIN32-64-NEXT:    addq $40, %rsp
 ; CHECK-WIN32-64-NEXT:    retq
 ;
 ; CHECK-X86-LABEL: and64_trunc_16_sign_minsize:
 ; CHECK-X86:       # %bb.0:
-; CHECK-X86-NEXT:    testw $-32768, {{[0-9]+}}(%esp) # imm = 0x8000
+; CHECK-X86-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
+; CHECK-X86-NEXT:    testw $-32768, %ax # imm = 0x8000
 ; CHECK-X86-NEXT:    js .LBB19_2
 ; CHECK-X86-NEXT:  # %bb.1: # %yes
 ; CHECK-X86-NEXT:    calll bar@PLT
 ; CHECK-X86-NEXT:  .LBB19_2: # %no
 ; CHECK-X86-NEXT:    retl
   %t = and i64 %x, 32768
   %r = trunc i64 %t to i16
   %s = icmp sge i16 %r, 0