[AMDGPU] More codegen patterns for v2i16/v2f16 build_vector

[foad]

Summary

It's simpler to do this at codegen time than to do ad-hoc constant
folding of machine instructions in SIFoldOperands.

[arsenm]

All of this constant folding is really a DAG workaround

llvm/lib/Target/AMDGPU/SIInstructions.td

2051

This needs a move to materialize the constant?

2056

Ditto

[foad]

The only codegen diffs I saw across several thousand graphics shaders were three instances of:

-	s_pack_ll_b32_b16 s3, s2, 0
+	s_and_b32 s3, 0xffff, s2

[foad]

Add a move to materialize the constant.

[rampitec]

Can this appear later in the codegen? It also does not cover global isel, so part in the operand folding probably needs to remain in addition to patterns.

[arsenm]

In D88028#2285683, @rampitec wrote:

Can this appear later in the codegen? It also does not cover global isel, so part in the operand folding probably needs to remain in addition to patterns.

It doesn't cover globalisel because build_vector patterns don't work now (and we use G_BUILD_VECTOR_TRUNC in these cases instead).

This kind of thing should not really be a problem in globalisel.

[foad]

In D88028#2285683, @rampitec wrote:

Can this appear later in the codegen?

I doubt it, and I haven't found any cases where that happens.

It also does not cover global isel, so part in the operand folding probably needs to remain in addition to patterns.

The whole point of instruction selection is to select the best instructions. I really don't think any instruction selector should rely on having this kind of folding run later.

[rampitec]

LGTM. Looks like Matt has no concerns about Global ISel as well.

llvm/lib/Target/AMDGPU/SIFoldOperands.cpp

static bool tryConstantFoldOp(MachineRegisterInfo &MRI,
 
   int Src0Idx = AMDGPU::getNamedOperandIdx(Opc, AMDGPU::OpName::src0);
   MachineOperand *Src0 = getImmOrMaterializedImm(MRI, MI->getOperand(Src0Idx));
   MachineOperand *Src1 = getImmOrMaterializedImm(MRI, MI->getOperand(Src1Idx));
 
   if (!Src0->isImm() && !Src1->isImm())
     return false;
 
-  if (MI->getOpcode() == AMDGPU::V_LSHL_OR_B32 ||
-      MI->getOpcode() == AMDGPU::V_LSHL_ADD_U32 ||
-      MI->getOpcode() == AMDGPU::V_AND_OR_B32) {
-    if (Src0->isImm() && Src0->getImm() == 0) {
-      // v_lshl_or_b32 0, X, Y -> copy Y
-      // v_lshl_or_b32 0, X, K -> v_mov_b32 K
-      // v_lshl_add_b32 0, X, Y -> copy Y
-      // v_lshl_add_b32 0, X, K -> v_mov_b32 K
-      // v_and_or_b32 0, X, Y -> copy Y
-      // v_and_or_b32 0, X, K -> v_mov_b32 K
-      bool UseCopy = TII->getNamedOperand(*MI, AMDGPU::OpName::src2)->isReg();
-      MI->RemoveOperand(Src1Idx);
-      MI->RemoveOperand(Src0Idx);
-
-      MI->setDesc(TII->get(UseCopy ? AMDGPU::COPY : AMDGPU::V_MOV_B32_e32));
-      return true;
-    }
-  }
-
   // and k0, k1 -> v_mov_b32 (k0 & k1)
   // or k0, k1 -> v_mov_b32 (k0 | k1)
   // xor k0, k1 -> v_mov_b32 (k0 ^ k1)
   if (Src0->isImm() && Src1->isImm()) {
     int32_t NewImm;
     if (!evalBinaryInstruction(Opc, NewImm, Src0->getImm(), Src1->getImm()))
       return false;
 

llvm/lib/Target/AMDGPU/SIInstructions.td

 >;
 
 def : GCNPat <
   (v2i16 (build_vector (i16 0), (i16 SReg_32:$src1))),
   (S_LSHL_B32 SReg_32:$src1, (i16 16))
 >;
 
 def : GCNPat <
+  (v2i16 (build_vector (i16 SReg_32:$src1), (i16 0))),
+  (S_AND_B32 (S_MOV_B32 (i32 0xffff)), SReg_32:$src1)
+>;
+
+def : GCNPat <
+  (v2f16 (build_vector (f16 SReg_32:$src1), (f16 FP_ZERO))),
+  (S_AND_B32 (S_MOV_B32 (i32 0xffff)), SReg_32:$src1)
+>;
+
+def : GCNPat <
   (v2i16 (build_vector (i16 SReg_32:$src0), (i16 undef))),
   (COPY_TO_REGCLASS SReg_32:$src0, SReg_32)
 >;
 
 def : GCNPat <
   (v2i16 (build_vector (i16 VGPR_32:$src0), (i16 undef))),
   (COPY_TO_REGCLASS VGPR_32:$src0, VGPR_32)
 >;

llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir

body:             |
 
   bb.3:
     liveins: $vcc
     SI_END_CF %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
     S_ENDPGM 0, implicit $vcc
 
 ...
 ---
-# GCN-LABEL: name: constant_fold_lshl_or_reg0_immreg_reg{{$}}
-# GCN: %2:vgpr_32 = COPY $vgpr0, implicit $exec
-# GCN-NEXT: S_ENDPGM 0, implicit %2
-
-name: constant_fold_lshl_or_reg0_immreg_reg
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-    liveins: $vgpr0
-
-  %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-  %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-  %2:vgpr_32 = V_LSHL_OR_B32 %0,%1, $vgpr0, implicit $exec
-  S_ENDPGM 0, implicit %2
-
-...
-
----
-
-# GCN-LABEL: name: constant_fold_lshl_or_reg0_immreg_imm{{$}}
-# GCN: %2:vgpr_32 = V_MOV_B32_e32 10, implicit $exec
-# GCN-NEXT: S_ENDPGM 0, implicit %2
-
-name: constant_fold_lshl_or_reg0_immreg_imm
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-
-  %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-  %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-  %2:vgpr_32 = V_LSHL_OR_B32 %0, %1, 10, implicit $exec
-  S_ENDPGM 0, implicit %2
-
-...
-
----
-
-# GCN-LABEL: name: constant_fold_lshl_or_reg0_immreg_immreg{{$}}
-# GCN: %3:vgpr_32 = V_MOV_B32_e32 12345, implicit $exec
-# GCN-NEXT: S_ENDPGM 0, implicit %3
-
-name: constant_fold_lshl_or_reg0_immreg_immreg
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-
-  %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-  %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-  %2:vgpr_32 = V_MOV_B32_e32 12345, implicit $exec
-  %3:vgpr_32 = V_LSHL_OR_B32 %0, %1, %2, implicit $exec
-  S_ENDPGM 0, implicit %3
-
-...
-
----
 # GCN-LABEL: name: s_fold_andn2_imm_regimm_32{{$}}
 # GCN: [[VAL:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 1233024, implicit $exec
 # GCN: BUFFER_STORE_DWORD_OFFSET killed [[VAL]],
 name:            s_fold_andn2_imm_regimm_32
 tracksRegLiveness: true
 liveins:
   - { reg: '$sgpr0_sgpr1', virtual-reg: '%0' }
 body:             |
bb.0:
     %26:vgpr_32 = V_LSHLREV_B32_e32 11, %24, implicit $exec
     FLAT_STORE_DWORD %20, %26, 0, 0, 0, 0, implicit $exec, implicit $flat_scr
     %28:vgpr_32 = V_LSHL_B32_e32 %27, %6, implicit $exec
     FLAT_STORE_DWORD %20, %28, 0, 0, 0, 0, implicit $exec, implicit $flat_scr
     S_ENDPGM 0
 
 ...
 
----
-# GCN-LABEL: name: constant_fold_lshl_add_reg0_immreg_reg{{$}}
-# GCN: %2:vgpr_32 = COPY $vgpr0, implicit $exec
-# GCN-NEXT: S_ENDPGM
-
-name:            constant_fold_lshl_add_reg0_immreg_reg
-alignment:       0
-exposesReturnsTwice: false
-legalized:       false
-regBankSelected: false
-selected:        false
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-    liveins: $vgpr0
-
-    %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-    %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-    %2:vgpr_32 = V_LSHL_ADD_U32 %0, %1, $vgpr0, implicit $exec
-    S_ENDPGM 0, implicit %2
-
-...
-
----
-
-# GCN-LABEL: name: constant_fold_lshl_add_reg0_immreg_imm{{$}}
-# GCN: %2:vgpr_32 = V_MOV_B32_e32 10, implicit $exec
-# GCN-NEXT: S_ENDPGM
-
-name:            constant_fold_lshl_add_reg0_immreg_imm
-alignment:       0
-exposesReturnsTwice: false
-legalized:       false
-regBankSelected: false
-selected:        false
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-
-    %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-    %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-    %2:vgpr_32 = V_LSHL_ADD_U32 %0, %1, 10, implicit $exec
-    S_ENDPGM 0, implicit %2
-
-...
-
----
-
-# GCN-LABEL: name: constant_fold_lshl_add_reg0_immreg_immreg{{$}}
-# GCN: %3:vgpr_32 = V_MOV_B32_e32 12345, implicit $exec
-# GCN-NEXT: S_ENDPGM
-
-name:            constant_fold_lshl_add_reg0_immreg_immreg
-alignment:       0
-exposesReturnsTwice: false
-legalized:       false
-regBankSelected: false
-selected:        false
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-
-    %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-    %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-    %2:vgpr_32 = V_MOV_B32_e32 12345, implicit $exec
-    %3:vgpr_32 = V_LSHL_ADD_U32 %0, %1, %2, implicit $exec
-    S_ENDPGM 0, implicit %3
-
-...
-
----
-# GCN-LABEL: name: constant_fold_and_or_reg0_immreg_reg{{$}}
-# GCN: %2:vgpr_32 = COPY $vgpr0, implicit $exec
-# GCN-NEXT: S_ENDPGM
-
-name:            constant_fold_and_or_reg0_immreg_reg
-alignment:       0
-exposesReturnsTwice: false
-legalized:       false
-regBankSelected: false
-selected:        false
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-    liveins: $vgpr0
-
-    %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-    %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-    %2:vgpr_32 = V_AND_OR_B32 %0, %1, $vgpr0, implicit $exec
-    S_ENDPGM 0, implicit %2
-
-...
-
----
-
-# GCN-LABEL: name: constant_fold_and_or_reg0_immreg_imm{{$}}
-# GCN: %2:vgpr_32 = V_MOV_B32_e32 10, implicit $exec
-# GCN-NEXT: S_ENDPGM
-
-name:            constant_fold_and_or_reg0_immreg_imm
-alignment:       0
-exposesReturnsTwice: false
-legalized:       false
-regBankSelected: false
-selected:        false
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-
-    %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-    %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-    %2:vgpr_32 = V_AND_OR_B32 %0, %1, 10, implicit $exec
-    S_ENDPGM 0, implicit %2
-
-...
-
----
-
-# GCN-LABEL: name: constant_fold_and_or_reg0_immreg_immreg{{$}}
-# GCN: %3:vgpr_32 = V_MOV_B32_e32 12345, implicit $exec
-# GCN-NEXT: S_ENDPGM
-
-name:            constant_fold_and_or_reg0_immreg_immreg
-alignment:       0
-exposesReturnsTwice: false
-legalized:       false
-regBankSelected: false
-selected:        false
-tracksRegLiveness: true
-
-body:             |
-  bb.0:
-
-    %0:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
-    %1:vgpr_32 = V_MOV_B32_e32 16, implicit $exec
-    %2:vgpr_32 = V_MOV_B32_e32 12345, implicit $exec
-    %3:vgpr_32 = V_AND_OR_B32 %0, %1, %2, implicit $exec
-    S_ENDPGM 0, implicit %3
-
-...
-
 # This used to incorrectly interpret V_MOV_B32_sdwa as being a move
 # immediate, and interpreting the src0_modifiers field as a
 # materialized immediate.
 
 ---
 # GCN-LABEL: name: no_fold_sdwa_mov_imm
 # GCN: %2:vgpr_32 = V_MOV_B32_sdwa 0, %0, 0, 5, 2, 4, implicit $exec, implicit %0(tied-def 0)
 # GCN-NEXT: [[SHIFT:%[0-9]+]]:vgpr_32 = V_LSHRREV_B32_e64 16, %2, implicit $exec

llvm/test/CodeGen/AMDGPU/fcanonicalize.f16.ll

define <4 x half> @v_test_canonicalize_reg_reg_undef_undef_v4f16(half %val0, half %val1) #1 {
   %vec0 = insertelement <4 x half> undef, half %val0, i32 0
   %vec1 = insertelement <4 x half> %vec0, half %val1, i32 1
   %canonicalized = call <4 x half> @llvm.canonicalize.v4f16(<4 x half> %vec1)
   ret <4 x half> %canonicalized
 }
 
 ; GCN-LABEL: {{^}}v_test_canonicalize_reg_undef_reg_reg_v4f16:
 ; GFX9: s_waitcnt
-; GFX9-NEXT: v_mov_b32_e32 [[MASK:v[0-9]+]], 0xffff
-; GFX9-NEXT: v_and_b32_e32 v1, [[MASK]], v1
+; GFX9-NEXT: v_and_b32_e32 v1, 0xffff, v1
 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v0
 ; GFX9-NEXT: v_lshl_or_b32 v1, v2, 16, v1
-; GFX9-NEXT: v_and_b32_e32 v0, [[MASK]], v0
+; GFX9-NEXT: v_and_b32_e32 v0, 0xffff, v0
 ; GFX9-NEXT: v_pk_max_f16 v1, v1, v1
 ; GFX9-NEXT: s_setpc_b64
 
 ; VI: s_waitcnt
 ; VI-NEXT: v_max_f16_e32 v0, v0, v0
 ; VI-NEXT: v_max_f16_e32 v1, v1, v1
 ; VI-NEXT: v_max_f16_sdwa v2, v2, v2 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
 ; VI-NEXT: v_or_b32_e32 v0, 0x7e000000, v0