[DAGCombiner] Scalarize extend/truncate for splat vector.

[jacquesguan]

Summary

This revision scalarizes extend/truncate for splat vector.

[RKSimon]

llvm/test/CodeGen/RISCV/rvv/vfwmacc-sdnode.ll

28 ↗

(On Diff #419646)

Any luck with these regressions?

llvm/test/CodeGen/RISCV/rvv/vwsub-sdnode.ll

3 ↗

(On Diff #419646)

You need to re-add triple specific checks now that some of these test results diverge

; RUN: llc -mtriple=riscv32 -mattr=+v -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,CHECK32
; RUN: llc -mtriple=riscv64 -mattr=+v -verify-machineinstrs < %s | FileCheck %s --check-prefixes=CHECK,CHECK64

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

443

I don't think UnOp is the right term here. UnaryOperator in IR is the base clang used by fneg. CastOp would probably be better.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

443

Done, thanks.

llvm/test/CodeGen/RISCV/rvv/vfwmacc-sdnode.ll

28 ↗

(On Diff #419646)

Done, thanks.

llvm/test/CodeGen/RISCV/rvv/vwsub-sdnode.ll

3 ↗

(On Diff #419646)

Done, thanks.

[RKSimon]

llvm/test/CodeGen/RISCV/rvv/vwsub-sdnode.ll

548 ↗

(On Diff #421100)

regressions?

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/vwsub-sdnode.ll

548 ↗

(On Diff #421100)

RISC-V is weak on scalar zero extend in the base ISA. We get a zext.w operation with the Zba extension. On the vector side a widening vwsub.wx is probably more expensive than a non-widening vsub.vx

[RKSimon]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

24082

Should we be extracting from a SPLAT_VECTOR node like this or should we be accessing (and maybe truncating?) the N0.getOperand(0) directly?

[jacquesguan]

rebase.

[jacquesguan]

Address comment.

[jacquesguan]

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

24082

Thanks, I changed it to the latter one.

[jacquesguan]

ping

[frasercrmck]

Sorry, I'm a bit rusty. What happens with the equivalent fixed-length vector splats (BUILD_VECTOR)? If they're scalarized can we do the same for SPLAT_VECTOR in the same place? If they're not scalarized, what's the logic for only doing this transform for SPLAT_VECTOR?

[jacquesguan]

rebase main and address comment.

[jacquesguan]

In D122875#3467800, @frasercrmck wrote:

Sorry, I'm a bit rusty. What happens with the equivalent fixed-length vector splats (BUILD_VECTOR)? If they're scalarized can we do the same for SPLAT_VECTOR in the same place? If they're not scalarized, what's the logic for only doing this transform for SPLAT_VECTOR?

I changed some code to support other kinds splat.

[jacquesguan]

ping.

[jacquesguan]

ping.

[jacquesguan]

ping.

[RKSimon]

llvm/test/CodeGen/RISCV/rvv/vnsra-sdnode.ll

33 ↗

(On Diff #447964)

Still a regression - can RISCV64 really not be coaxed into generating something better?

[jacquesguan]

Disable salarize sign_ext and zero_ext for RISCV target.

[jacquesguan]

Fix test.

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/vnsra-sdnode.ll

33 ↗

(On Diff #447964)

In order to generate widening instruction, I disable scalarizing of sign_ext and zero_ext, because the scalar type integer extention is easy to combine with other node, make it won't match the pattern.

[RKSimon]

cheers - its up to the RISCV guys as to whether this is acceptable (and whether it should be pulled out as a parent patch?)

[jacquesguan]

In D122875#3731428, @RKSimon wrote:

cheers - its up to the RISCV guys as to whether this is acceptable (and whether it should be pulled out as a parent patch?)

@craig.topper @frasercrmck What do you about this way, or any advice?

[frasercrmck]

llvm/test/CodeGen/RISCV/rvv/vnsra-sdnode.ll

33 ↗

(On Diff #447964)

Do you think there's any way to fix it?

[jacquesguan]

llvm/test/CodeGen/RISCV/rvv/vnsra-sdnode.ll

33 ↗

(On Diff #447964)

I think if we change the widen pattern to DAG combiner, maybe could generte some, but common DAG folder may fold extend to for others example: ext_inreg firstly. So more condition we need to consider

[jacquesguan]

ping.

[jacquesguan]

ping.

[RKSimon]

@frasercrmck @craig.topper - any more comments?

[RKSimon]

LGTM - cheers

[RKSimon]

@jacquesguan Please can you take a look at https://github.com/llvm/llvm-project/issues/62234 thats related to this patch?

[jacquesguan]

In D122875#4280169, @RKSimon wrote:

@jacquesguan Please can you take a look at https://github.com/llvm/llvm-project/issues/62234 thats related to this patch?

OK.

llvm/include/llvm/CodeGen/TargetLowering.h

public:
   ///   add (add %x, 1), %y
   /// The variant with two add's is IR-canonical.
   /// Some targets may prefer one to the other.
   virtual bool preferIncOfAddToSubOfNot(EVT VT) const {
     // By default, let's assume that everyone prefers the form with two add's.
     return true;
   }
 
+  // Return true if the target wants to transform Op(Splat(X)) -> Splat(Op(X))
+  virtual bool preferScalarizeSplat(unsigned Opc) const { return true; }
+
   /// Return true if the target wants to use the optimization that
   /// turns ext(promotableInst1(...(promotableInstN(load)))) into
   /// promotedInst1(...(promotedInstN(ext(load)))).
   bool enableExtLdPromotion() const { return EnableExtLdPromotion; }
 
   /// Return true if the target can combine store(extractelement VectorTy,
   /// Idx).
   /// \p Cost[out] gives the cost of that transformation when this is true.

llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp

private:
     SDValue visitUMUL_LOHI(SDNode *N);
     SDValue visitMULO(SDNode *N);
     SDValue visitIMINMAX(SDNode *N);
     SDValue visitAND(SDNode *N);
     SDValue visitANDLike(SDValue N0, SDValue N1, SDNode *N);
     SDValue visitOR(SDNode *N);
     SDValue visitORLike(SDValue N0, SDValue N1, SDNode *N);
     SDValue visitXOR(SDNode *N);
+    SDValue SimplifyVCastOp(SDNode *N, const SDLoc &DL);
     SDValue SimplifyVBinOp(SDNode *N, const SDLoc &DL);
     SDValue visitSHL(SDNode *N);
     SDValue visitSRA(SDNode *N);
     SDValue visitSRL(SDNode *N);
     SDValue visitFunnelShift(SDNode *N);
     SDValue visitSHLSAT(SDNode *N);
     SDValue visitRotate(SDNode *N);
     SDValue visitABS(SDNode *N);
SDValue DAGCombiner::foldSextSetcc(SDNode *N) {
   return SDValue();
 }
 
 SDValue DAGCombiner::visitSIGN_EXTEND(SDNode *N) {
   SDValue N0 = N->getOperand(0);
   EVT VT = N->getValueType(0);
   SDLoc DL(N);
 
+  if (VT.isVector())
+    if (SDValue FoldedVOp = SimplifyVCastOp(N, DL))
+      return FoldedVOp;
+
   // sext(undef) = 0 because the top bit will all be the same.
   if (N0.isUndef())
     return DAG.getConstant(0, DL, VT);
 
   if (SDValue Res = tryToFoldExtendOfConstant(N, TLI, DAG, LegalTypes))
     return Res;
 
   // fold (sext (sext x)) -> (sext x)
static SDValue widenCtPop(SDNode *Extend, SelectionDAG &DAG) {
   SDValue NewZext = DAG.getZExtOrTrunc(CtPop.getOperand(0), DL, VT);
   return DAG.getNode(ISD::CTPOP, DL, VT, NewZext);
 }
 
 SDValue DAGCombiner::visitZERO_EXTEND(SDNode *N) {
   SDValue N0 = N->getOperand(0);
   EVT VT = N->getValueType(0);
 
+  if (VT.isVector())
+    if (SDValue FoldedVOp = SimplifyVCastOp(N, SDLoc(N)))
+      return FoldedVOp;
+
   // zext(undef) = 0
   if (N0.isUndef())
     return DAG.getConstant(0, SDLoc(N), VT);
 
   if (SDValue Res = tryToFoldExtendOfConstant(N, TLI, DAG, LegalTypes))
     return Res;
 
   // fold (zext (zext x)) -> (zext x)
SDValue DAGCombiner::visitFP_ROUND(SDNode *N) {
 
   return SDValue();
 }
 
 SDValue DAGCombiner::visitFP_EXTEND(SDNode *N) {
   SDValue N0 = N->getOperand(0);
   EVT VT = N->getValueType(0);
 
+  if (VT.isVector())
+    if (SDValue FoldedVOp = SimplifyVCastOp(N, SDLoc(N)))
+      return FoldedVOp;
+
   // If this is fp_round(fpextend), don't fold it, allow ourselves to be folded.
   if (N->hasOneUse() &&
       N->use_begin()->getOpcode() == ISD::FP_ROUND)
     return SDValue();
 
   // fold (fp_extend c1fp) -> c1fp
   if (DAG.isConstantFPBuildVectorOrConstantFP(N0))
     return DAG.getNode(ISD::FP_EXTEND, SDLoc(N), VT, N0);
if (N0.getOpcode() == ISD::BUILD_VECTOR && N0.getOpcode() == N1.getOpcode() &&
     Ops[Index0] = ScalarBO;
     return DAG.getBuildVector(VT, DL, Ops);
   }
 
   // bo (splat X, Index), (splat Y, Index) --> splat (bo X, Y), Index
   return DAG.getSplat(VT, DL, ScalarBO);
 }
 
+/// Visit a vector cast operation, like FP_EXTEND.
+SDValue DAGCombiner::SimplifyVCastOp(SDNode *N, const SDLoc &DL) {
+  EVT VT = N->getValueType(0);
+  assert(VT.isVector() && "SimplifyVCastOp only works on vectors!");
+  EVT EltVT = VT.getVectorElementType();
+  unsigned Opcode = N->getOpcode();
+
+  SDValue N0 = N->getOperand(0);
+  EVT SrcVT = N0->getValueType(0);
+  EVT SrcEltVT = SrcVT.getVectorElementType();
+  const TargetLowering &TLI = DAG.getTargetLoweringInfo();
+
+  // TODO: promote operation might be also good here?
+  int Index0;
+  SDValue Src0 = DAG.getSplatSourceVector(N0, Index0);
+  if (Src0 &&
+      (N0.getOpcode() == ISD::SPLAT_VECTOR ||
+       TLI.isExtractVecEltCheap(VT, Index0)) &&
+      TLI.isOperationLegalOrCustom(Opcode, EltVT) &&
+      TLI.preferScalarizeSplat(Opcode)) {
+    SDValue IndexC = DAG.getVectorIdxConstant(Index0, DL);
+    SDValue Elt =
+        DAG.getNode(ISD::EXTRACT_VECTOR_ELT, DL, SrcEltVT, Src0, IndexC);
+    SDValue ScalarBO = DAG.getNode(Opcode, DL, EltVT, Elt, N->getFlags());
+    if (VT.isScalableVector())
+      return DAG.getSplatVector(VT, DL, ScalarBO);
+    SmallVector<SDValue, 8> Ops(VT.getVectorNumElements(), ScalarBO);
+    return DAG.getBuildVector(VT, DL, Ops);
+  }
+
+  return SDValue();
+}
+
 /// Visit a binary vector operation, like ADD.
 SDValue DAGCombiner::SimplifyVBinOp(SDNode *N, const SDLoc &DL) {
   EVT VT = N->getValueType(0);
   assert(VT.isVector() && "SimplifyVBinOp only works on vectors!");
 
   SDValue LHS = N->getOperand(0);
   SDValue RHS = N->getOperand(1);
   unsigned Opcode = N->getOpcode();

llvm/lib/Target/RISCV/RISCVISelLowering.h

public:
   bool isOffsetFoldingLegal(const GlobalAddressSDNode *GA) const override;
   bool isFPImmLegal(const APFloat &Imm, EVT VT,
                     bool ForCodeSize) const override;
   bool isExtractSubvectorCheap(EVT ResVT, EVT SrcVT,
                                unsigned Index) const override;
 
   bool isIntDivCheap(EVT VT, AttributeList Attr) const override;
 
+  bool preferScalarizeSplat(unsigned Opc) const override;
+
   bool softPromoteHalfType() const override { return true; }
 
   /// Return the register type for a given MVT, ensuring vectors are treated
   /// as a series of gpr sized integers.
   MVT getRegisterTypeForCallingConv(LLVMContext &Context, CallingConv::ID CC,
                                     EVT VT) const override;
 
   /// Return the number of registers for a given MVT, ensuring vectors are

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

 bool RISCVTargetLowering::isIntDivCheap(EVT VT, AttributeList Attr) const {
   // When aggressively optimizing for code size, we prefer to use a div
   // instruction, as it is usually smaller than the alternative sequence.
   // TODO: Add vector division?
   bool OptSize = Attr.hasFnAttr(Attribute::MinSize);
   return OptSize && !VT.isVector();
 }
 
+bool RISCVTargetLowering::preferScalarizeSplat(unsigned Opc) const {
+  // Scalarize zero_ext and sign_ext might stop match to widening instruction in
+  // some situation.
+  if (Opc == ISD::ZERO_EXTEND || Opc == ISD::SIGN_EXTEND)
+    return false;
+  return true;
+}
+
 #define GET_REGISTER_MATCHER
 #include "RISCVGenAsmMatcher.inc"
 
 Register
 RISCVTargetLowering::getRegisterByName(const char *RegName, LLT VT,
                                        const MachineFunction &MF) const {
   Register Reg = MatchRegisterAltName(RegName);
   if (Reg == RISCV::NoRegister)

llvm/lib/Target/RISCV/RISCVInstrInfoVSDPatterns.td

def : Pat<(fma (wti.Vector (riscv_fpextend_vl_oneuse
                                     (vti.Mask true_mask), (XLenVT srcvalue))),
                    (wti.Vector (riscv_fpextend_vl_oneuse
                                     (vti.Vector vti.RegClass:$rs2),
                                     (vti.Mask true_mask), (XLenVT srcvalue))),
                    (wti.Vector wti.RegClass:$rd)),
               (!cast<Instruction>(instruction_name#"_VV_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.RegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
-    def : Pat<(fma (wti.Vector (riscv_fpextend_vl_oneuse
-                                    (vti.Vector (SplatFPOp vti.ScalarRegClass:$rs1)),
-                                    (vti.Mask true_mask), (XLenVT srcvalue))),
+    def : Pat<(fma (wti.Vector (SplatFPOp
+                                    (fpext_oneuse vti.ScalarRegClass:$rs1))),
                    (wti.Vector (riscv_fpextend_vl_oneuse
                                     (vti.Vector vti.RegClass:$rs2),
                                     (vti.Mask true_mask), (XLenVT srcvalue))),
                    (wti.Vector wti.RegClass:$rd)),
               (!cast<Instruction>(instruction_name#"_V"#vti.ScalarSuffix#"_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.ScalarRegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
   }
 }
 
 multiclass VPatWidenFPNegMulAccSDNode_VV_VF<string instruction_name> {
   foreach vtiToWti = AllWidenableFloatVectors in {
     defvar vti = vtiToWti.Vti;
     defvar wti = vtiToWti.Wti;
     def : Pat<(fma (fneg (wti.Vector (riscv_fpextend_vl_oneuse
                                           (vti.Vector vti.RegClass:$rs1),
                                           (vti.Mask true_mask), (XLenVT srcvalue)))),
                    (riscv_fpextend_vl_oneuse (vti.Vector vti.RegClass:$rs2),
                                              (vti.Mask true_mask), (XLenVT srcvalue)),
                    (fneg wti.RegClass:$rd)),
               (!cast<Instruction>(instruction_name#"_VV_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.RegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
-    def : Pat<(fma (riscv_fpextend_vl_oneuse
-                        (vti.Vector (SplatFPOp vti.ScalarRegClass:$rs1)),
-                        (vti.Mask true_mask), (XLenVT srcvalue)),
+    def : Pat<(fma (SplatFPOp (fpext_oneuse vti.ScalarRegClass:$rs1)),
                    (fneg (wti.Vector (riscv_fpextend_vl_oneuse
                                           (vti.Vector vti.RegClass:$rs2),
                                           (vti.Mask true_mask), (XLenVT srcvalue)))),
                    (fneg wti.RegClass:$rd)),
               (!cast<Instruction>(instruction_name#"_V"#vti.ScalarSuffix#"_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.ScalarRegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
-    def : Pat<(fma (fneg (wti.Vector (riscv_fpextend_vl_oneuse
-                                          (vti.Vector (SplatFPOp vti.ScalarRegClass:$rs1)),
-                                          (vti.Mask true_mask), (XLenVT srcvalue)))),
+    def : Pat<(fma (fneg (wti.Vector (SplatFPOp (fpext_oneuse vti.ScalarRegClass:$rs1)))),
                    (riscv_fpextend_vl_oneuse (vti.Vector vti.RegClass:$rs2),
                                              (vti.Mask true_mask), (XLenVT srcvalue)),
                    (fneg wti.RegClass:$rd)),
               (!cast<Instruction>(instruction_name#"_V"#vti.ScalarSuffix#"_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.ScalarRegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
   }
 }
 
 multiclass VPatWidenFPMulSacSDNode_VV_VF<string instruction_name> {
   foreach vtiToWti = AllWidenableFloatVectors in {
     defvar vti = vtiToWti.Vti;
     defvar wti = vtiToWti.Wti;
     def : Pat<(fma (wti.Vector (riscv_fpextend_vl_oneuse
                                     (vti.Vector vti.RegClass:$rs1),
                                     (vti.Mask true_mask), (XLenVT srcvalue))),
                    (riscv_fpextend_vl_oneuse (vti.Vector vti.RegClass:$rs2),
                                              (vti.Mask true_mask), (XLenVT srcvalue)),
                    (fneg wti.RegClass:$rd)),
               (!cast<Instruction>(instruction_name#"_VV_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.RegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
-    def : Pat<(fma (wti.Vector (riscv_fpextend_vl_oneuse
-                                    (vti.Vector (SplatFPOp vti.ScalarRegClass:$rs1)),
-                                    (vti.Mask true_mask), (XLenVT srcvalue))),
+    def : Pat<(fma (wti.Vector (SplatFPOp (fpext_oneuse vti.ScalarRegClass:$rs1))),
                    (riscv_fpextend_vl_oneuse (vti.Vector vti.RegClass:$rs2),
                                              (vti.Mask true_mask), (XLenVT srcvalue)),
                    (fneg wti.RegClass:$rd)),
               (!cast<Instruction>(instruction_name#"_V"#vti.ScalarSuffix#"_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.ScalarRegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
   }
 }
 
 multiclass VPatWidenFPNegMulSacSDNode_VV_VF<string instruction_name> {
   foreach vtiToWti = AllWidenableFloatVectors in {
     defvar vti = vtiToWti.Vti;
     defvar wti = vtiToWti.Wti;
     def : Pat<(fma (fneg (wti.Vector (riscv_fpextend_vl_oneuse
                                           (vti.Vector vti.RegClass:$rs1),
                                           (vti.Mask true_mask), (XLenVT srcvalue)))),
                    (riscv_fpextend_vl_oneuse (vti.Vector vti.RegClass:$rs2),
                                              (vti.Mask true_mask), (XLenVT srcvalue)),
                    wti.RegClass:$rd),
               (!cast<Instruction>(instruction_name#"_VV_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.RegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
-    def : Pat<(fma (wti.Vector (riscv_fpextend_vl_oneuse
-                                    (vti.Vector (SplatFPOp vti.ScalarRegClass:$rs1)),
-                                    (vti.Mask true_mask), (XLenVT srcvalue))),
+    def : Pat<(fma (wti.Vector (SplatFPOp (fpext_oneuse vti.ScalarRegClass:$rs1))),
                    (fneg (wti.Vector (riscv_fpextend_vl_oneuse
                                           (vti.Vector vti.RegClass:$rs2),
                                           (vti.Mask true_mask), (XLenVT srcvalue)))),
                    wti.RegClass:$rd),
               (!cast<Instruction>(instruction_name#"_V"#vti.ScalarSuffix#"_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.ScalarRegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
-    def : Pat<(fma (fneg (wti.Vector (riscv_fpextend_vl_oneuse
-                                          (vti.Vector (SplatFPOp vti.ScalarRegClass:$rs1)),
-                                          (vti.Mask true_mask), (XLenVT srcvalue)))),
+    def : Pat<(fma (fneg (wti.Vector (SplatFPOp (fpext_oneuse vti.ScalarRegClass:$rs1)))),
                    (riscv_fpextend_vl_oneuse (vti.Vector vti.RegClass:$rs2),
                                              (vti.Mask true_mask), (XLenVT srcvalue)),
                    wti.RegClass:$rd),
               (!cast<Instruction>(instruction_name#"_V"#vti.ScalarSuffix#"_"#vti.LMul.MX)
                  wti.RegClass:$rd, vti.ScalarRegClass:$rs1, vti.RegClass:$rs2,
                  vti.AVL, vti.Log2SEW, TAIL_AGNOSTIC)>;
   }
 }

llvm/test/CodeGen/Hexagon/autohvx/isel-sext-inreg.ll

 ; RUN: llc -march=hexagon < %s | FileCheck %s
 
 ; Check that both functions compile successfully.
 
 
 target datalayout = "e-m:e-p:32:32:32-a:0-n16:32-i64:64:64-i32:32:32-i16:16:16-i1:8:8-f32:32:32-f64:64:64-v32:32:32-v64:64:64-v512:512:512-v1024:1024:1024-v2048:2048:2048"
 target triple = "hexagon"
 
 ; CHECK-LABEL: danny:
-; CHECK: vmem
+; CHECK: memh
 define void @danny(i16* %a0) #0 {
 b0:
   %v1 = load i16, i16* %a0, align 2
   %v2 = insertelement <8 x i16> undef, i16 %v1, i32 6
   %v3 = insertelement <8 x i16> %v2, i16 undef, i32 7
   %v4 = sext <8 x i16> %v3 to <8 x i32>
   %v5 = mul <8 x i32> %v4, <i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410>
   %v6 = add <8 x i32> %v5, <i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768>
   %v7 = add <8 x i32> %v6, zeroinitializer
   %v8 = ashr <8 x i32> %v7, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
   %v9 = add nsw <8 x i32> zeroinitializer, %v8
   %v10 = shl <8 x i32> %v9, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
   %v11 = ashr exact <8 x i32> %v10, <i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16, i32 16>
   %v12 = sub nsw <8 x i32> zeroinitializer, %v11
   %v13 = trunc <8 x i32> %v12 to <8 x i16>
   %v14 = extractelement <8 x i16> %v13, i32 7
   store i16 %v14, i16* %a0, align 2
   ret void
 }
 
 ; CHECK-LABEL: sammy:
-; CHECK: vmem
+; CHECK: memh
 define void @sammy(i16* %a0) #1 {
 b0:
   %v1 = load i16, i16* %a0, align 2
   %v2 = insertelement <16 x i16> undef, i16 %v1, i32 14
   %v3 = insertelement <16 x i16> %v2, i16 undef, i32 15
   %v4 = sext <16 x i16> %v3 to <16 x i32>
   %v5 = mul <16 x i32> %v4, <i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410, i32 -36410>
   %v6 = add <16 x i32> %v5, <i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768, i32 32768>

llvm/test/CodeGen/RISCV/rvv/vfcopysign-sdnode.ll

; CHECK-NEXT:    ret
   %e = fpext <vscale x 1 x half> %vs to <vscale x 1 x float>
   %r = call <vscale x 1 x float> @llvm.copysign.nxv1f32(<vscale x 1 x float> %vm, <vscale x 1 x float> %e)
   ret <vscale x 1 x float> %r
 }
 
 define <vscale x 1 x float> @vfcopysign_exttrunc_vf_nxv1f32_nxv1f16(<vscale x 1 x float> %vm, half %s) {
 ; CHECK-LABEL: vfcopysign_exttrunc_vf_nxv1f32_nxv1f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, mf4, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v10, v9
-; CHECK-NEXT:    vsetvli zero, zero, e32, mf2, ta, ma
-; CHECK-NEXT:    vfsgnj.vv v8, v8, v10
+; CHECK-NEXT:    fcvt.s.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e32, mf2, ta, ma
+; CHECK-NEXT:    vfsgnj.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 1 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 1 x half> %head, <vscale x 1 x half> poison, <vscale x 1 x i32> zeroinitializer
   %esplat = fpext <vscale x 1 x half> %splat to <vscale x 1 x float>
   %r = call <vscale x 1 x float> @llvm.copysign.nxv1f32(<vscale x 1 x float> %vm, <vscale x 1 x float> %esplat)
   ret <vscale x 1 x float> %r
 }
 
; CHECK-NEXT:    ret
   %eneg = fpext <vscale x 1 x half> %n to <vscale x 1 x float>
   %r = call <vscale x 1 x float> @llvm.copysign.nxv1f32(<vscale x 1 x float> %vm, <vscale x 1 x float> %eneg)
   ret <vscale x 1 x float> %r
 }
 
 define <vscale x 1 x float> @vfcopynsign_exttrunc_vf_nxv1f32_nxv1f16(<vscale x 1 x float> %vm, half %s) {
 ; CHECK-LABEL: vfcopynsign_exttrunc_vf_nxv1f32_nxv1f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, mf4, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v10, v9
-; CHECK-NEXT:    vsetvli zero, zero, e32, mf2, ta, ma
-; CHECK-NEXT:    vfsgnjn.vv v8, v8, v10
+; CHECK-NEXT:    fcvt.s.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e32, mf2, ta, ma
+; CHECK-NEXT:    vfsgnjn.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 1 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 1 x half> %head, <vscale x 1 x half> poison, <vscale x 1 x i32> zeroinitializer
   %n = fneg <vscale x 1 x half> %splat
   %eneg = fpext <vscale x 1 x half> %n to <vscale x 1 x float>
   %r = call <vscale x 1 x float> @llvm.copysign.nxv1f32(<vscale x 1 x float> %vm, <vscale x 1 x float> %eneg)
   ret <vscale x 1 x float> %r
 }
; CHECK-NEXT:    ret
   %e = fpext <vscale x 8 x half> %vs to <vscale x 8 x float>
   %r = call <vscale x 8 x float> @llvm.copysign.nxv8f32(<vscale x 8 x float> %vm, <vscale x 8 x float> %e)
   ret <vscale x 8 x float> %r
 }
 
 define <vscale x 8 x float> @vfcopysign_exttrunc_vf_nxv8f32_nxv8f16(<vscale x 8 x float> %vm, half %s) {
 ; CHECK-LABEL: vfcopysign_exttrunc_vf_nxv8f32_nxv8f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, m2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v12, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v16, v12
-; CHECK-NEXT:    vsetvli zero, zero, e32, m4, ta, ma
-; CHECK-NEXT:    vfsgnj.vv v8, v8, v16
+; CHECK-NEXT:    fcvt.s.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e32, m4, ta, ma
+; CHECK-NEXT:    vfsgnj.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 8 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 8 x half> %head, <vscale x 8 x half> poison, <vscale x 8 x i32> zeroinitializer
   %esplat = fpext <vscale x 8 x half> %splat to <vscale x 8 x float>
   %r = call <vscale x 8 x float> @llvm.copysign.nxv8f32(<vscale x 8 x float> %vm, <vscale x 8 x float> %esplat)
   ret <vscale x 8 x float> %r
 }
 
; CHECK-NEXT:    ret
   %eneg = fpext <vscale x 8 x half> %n to <vscale x 8 x float>
   %r = call <vscale x 8 x float> @llvm.copysign.nxv8f32(<vscale x 8 x float> %vm, <vscale x 8 x float> %eneg)
   ret <vscale x 8 x float> %r
 }
 
 define <vscale x 8 x float> @vfcopynsign_exttrunc_vf_nxv8f32_nxv8f16(<vscale x 8 x float> %vm, half %s) {
 ; CHECK-LABEL: vfcopynsign_exttrunc_vf_nxv8f32_nxv8f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, m2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v12, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v16, v12
-; CHECK-NEXT:    vsetvli zero, zero, e32, m4, ta, ma
-; CHECK-NEXT:    vfsgnjn.vv v8, v8, v16
+; CHECK-NEXT:    fcvt.s.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e32, m4, ta, ma
+; CHECK-NEXT:    vfsgnjn.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 8 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 8 x half> %head, <vscale x 8 x half> poison, <vscale x 8 x i32> zeroinitializer
   %n = fneg <vscale x 8 x half> %splat
   %eneg = fpext <vscale x 8 x half> %n to <vscale x 8 x float>
   %r = call <vscale x 8 x float> @llvm.copysign.nxv8f32(<vscale x 8 x float> %vm, <vscale x 8 x float> %eneg)
   ret <vscale x 8 x float> %r
 }
; CHECK-NEXT:    ret
   %e = fpext <vscale x 1 x half> %vs to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %e)
   ret <vscale x 1 x double> %r
 }
 
 define <vscale x 1 x double> @vfcopysign_exttrunc_vf_nxv1f64_nxv1f16(<vscale x 1 x double> %vm, half %s) {
 ; CHECK-LABEL: vfcopysign_exttrunc_vf_nxv1f64_nxv1f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, mf4, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v10, v9
-; CHECK-NEXT:    vsetvli zero, zero, e32, mf2, ta, ma
-; CHECK-NEXT:    vfwcvt.f.f.v v9, v10
-; CHECK-NEXT:    vsetvli zero, zero, e64, m1, ta, ma
-; CHECK-NEXT:    vfsgnj.vv v8, v8, v9
+; CHECK-NEXT:    fcvt.d.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m1, ta, ma
+; CHECK-NEXT:    vfsgnj.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 1 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 1 x half> %head, <vscale x 1 x half> poison, <vscale x 1 x i32> zeroinitializer
   %esplat = fpext <vscale x 1 x half> %splat to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %esplat)
   ret <vscale x 1 x double> %r
 }
 
; CHECK-NEXT:    ret
   %eneg = fpext <vscale x 1 x half> %n to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %eneg)
   ret <vscale x 1 x double> %r
 }
 
 define <vscale x 1 x double> @vfcopynsign_exttrunc_vf_nxv1f64_nxv1f16(<vscale x 1 x double> %vm, half %s) {
 ; CHECK-LABEL: vfcopynsign_exttrunc_vf_nxv1f64_nxv1f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, mf4, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v10, v9
-; CHECK-NEXT:    vsetvli zero, zero, e32, mf2, ta, ma
-; CHECK-NEXT:    vfwcvt.f.f.v v9, v10
-; CHECK-NEXT:    vsetvli zero, zero, e64, m1, ta, ma
-; CHECK-NEXT:    vfsgnjn.vv v8, v8, v9
+; CHECK-NEXT:    fcvt.d.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m1, ta, ma
+; CHECK-NEXT:    vfsgnjn.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 1 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 1 x half> %head, <vscale x 1 x half> poison, <vscale x 1 x i32> zeroinitializer
   %n = fneg <vscale x 1 x half> %splat
   %eneg = fpext <vscale x 1 x half> %n to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %eneg)
   ret <vscale x 1 x double> %r
 }
; CHECK-NEXT:    ret
   %e = fpext <vscale x 1 x float> %vs to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %e)
   ret <vscale x 1 x double> %r
 }
 
 define <vscale x 1 x double> @vfcopysign_exttrunc_vf_nxv1f64_nxv1f32(<vscale x 1 x double> %vm, float %s) {
 ; CHECK-LABEL: vfcopysign_exttrunc_vf_nxv1f64_nxv1f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e32, mf2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v10, v9
-; CHECK-NEXT:    vsetvli zero, zero, e64, m1, ta, ma
-; CHECK-NEXT:    vfsgnj.vv v8, v8, v10
+; CHECK-NEXT:    fcvt.d.s ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m1, ta, ma
+; CHECK-NEXT:    vfsgnj.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 1 x float> poison, float %s, i32 0
   %splat = shufflevector <vscale x 1 x float> %head, <vscale x 1 x float> poison, <vscale x 1 x i32> zeroinitializer
   %esplat = fpext <vscale x 1 x float> %splat to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %esplat)
   ret <vscale x 1 x double> %r
 }
 
; CHECK-NEXT:    ret
   %eneg = fpext <vscale x 1 x float> %n to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %eneg)
   ret <vscale x 1 x double> %r
 }
 
 define <vscale x 1 x double> @vfcopynsign_exttrunc_vf_nxv1f64_nxv1f32(<vscale x 1 x double> %vm, float %s) {
 ; CHECK-LABEL: vfcopynsign_exttrunc_vf_nxv1f64_nxv1f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e32, mf2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v9, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v10, v9
-; CHECK-NEXT:    vsetvli zero, zero, e64, m1, ta, ma
-; CHECK-NEXT:    vfsgnjn.vv v8, v8, v10
+; CHECK-NEXT:    fcvt.d.s ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m1, ta, ma
+; CHECK-NEXT:    vfsgnjn.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 1 x float> poison, float %s, i32 0
   %splat = shufflevector <vscale x 1 x float> %head, <vscale x 1 x float> poison, <vscale x 1 x i32> zeroinitializer
   %n = fneg <vscale x 1 x float> %splat
   %eneg = fpext <vscale x 1 x float> %n to <vscale x 1 x double>
   %r = call <vscale x 1 x double> @llvm.copysign.nxv1f64(<vscale x 1 x double> %vm, <vscale x 1 x double> %eneg)
   ret <vscale x 1 x double> %r
 }
; CHECK-NEXT:    ret
   %e = fpext <vscale x 8 x half> %vs to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %e)
   ret <vscale x 8 x double> %r
 }
 
 define <vscale x 8 x double> @vfcopysign_exttrunc_vf_nxv8f64_nxv8f16(<vscale x 8 x double> %vm, half %s) {
 ; CHECK-LABEL: vfcopysign_exttrunc_vf_nxv8f64_nxv8f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, m2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v16, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v20, v16
-; CHECK-NEXT:    vsetvli zero, zero, e32, m4, ta, ma
-; CHECK-NEXT:    vfwcvt.f.f.v v24, v20
-; CHECK-NEXT:    vsetvli zero, zero, e64, m8, ta, ma
-; CHECK-NEXT:    vfsgnj.vv v8, v8, v24
+; CHECK-NEXT:    fcvt.d.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m8, ta, ma
+; CHECK-NEXT:    vfsgnj.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 8 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 8 x half> %head, <vscale x 8 x half> poison, <vscale x 8 x i32> zeroinitializer
   %esplat = fpext <vscale x 8 x half> %splat to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %esplat)
   ret <vscale x 8 x double> %r
 }
 
; CHECK-NEXT:    ret
   %eneg = fpext <vscale x 8 x half> %n to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %eneg)
   ret <vscale x 8 x double> %r
 }
 
 define <vscale x 8 x double> @vfcopynsign_exttrunc_vf_nxv8f64_nxv8f16(<vscale x 8 x double> %vm, half %s) {
 ; CHECK-LABEL: vfcopynsign_exttrunc_vf_nxv8f64_nxv8f16:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e16, m2, ta, ma
-; CHECK-NEXT:    vfmv.v.f v16, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v20, v16
-; CHECK-NEXT:    vsetvli zero, zero, e32, m4, ta, ma
-; CHECK-NEXT:    vfwcvt.f.f.v v24, v20
-; CHECK-NEXT:    vsetvli zero, zero, e64, m8, ta, ma
-; CHECK-NEXT:    vfsgnjn.vv v8, v8, v24
+; CHECK-NEXT:    fcvt.d.h ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m8, ta, ma
+; CHECK-NEXT:    vfsgnjn.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 8 x half> poison, half %s, i32 0
   %splat = shufflevector <vscale x 8 x half> %head, <vscale x 8 x half> poison, <vscale x 8 x i32> zeroinitializer
   %n = fneg <vscale x 8 x half> %splat
   %eneg = fpext <vscale x 8 x half> %n to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %eneg)
   ret <vscale x 8 x double> %r
 }
; CHECK-NEXT:    ret
   %e = fpext <vscale x 8 x float> %vs to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %e)
   ret <vscale x 8 x double> %r
 }
 
 define <vscale x 8 x double> @vfcopysign_exttrunc_vf_nxv8f64_nxv8f32(<vscale x 8 x double> %vm, float %s) {
 ; CHECK-LABEL: vfcopysign_exttrunc_vf_nxv8f64_nxv8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e32, m4, ta, ma
-; CHECK-NEXT:    vfmv.v.f v16, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v24, v16
-; CHECK-NEXT:    vsetvli zero, zero, e64, m8, ta, ma
-; CHECK-NEXT:    vfsgnj.vv v8, v8, v24
+; CHECK-NEXT:    fcvt.d.s ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m8, ta, ma
+; CHECK-NEXT:    vfsgnj.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 8 x float> poison, float %s, i32 0
   %splat = shufflevector <vscale x 8 x float> %head, <vscale x 8 x float> poison, <vscale x 8 x i32> zeroinitializer
   %esplat = fpext <vscale x 8 x float> %splat to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %esplat)
   ret <vscale x 8 x double> %r
 }
 
; CHECK-NEXT:    ret
   %eneg = fpext <vscale x 8 x float> %n to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %eneg)
   ret <vscale x 8 x double> %r
 }
 
 define <vscale x 8 x double> @vfcopynsign_exttrunc_vf_nxv8f64_nxv8f32(<vscale x 8 x double> %vm, float %s) {
 ; CHECK-LABEL: vfcopynsign_exttrunc_vf_nxv8f64_nxv8f32:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    vsetvli a0, zero, e32, m4, ta, ma
-; CHECK-NEXT:    vfmv.v.f v16, fa0
-; CHECK-NEXT:    vfwcvt.f.f.v v24, v16
-; CHECK-NEXT:    vsetvli zero, zero, e64, m8, ta, ma
-; CHECK-NEXT:    vfsgnjn.vv v8, v8, v24
+; CHECK-NEXT:    fcvt.d.s ft0, fa0
+; CHECK-NEXT:    vsetvli a0, zero, e64, m8, ta, ma
+; CHECK-NEXT:    vfsgnjn.vf v8, v8, ft0
 ; CHECK-NEXT:    ret
   %head = insertelement <vscale x 8 x float> poison, float %s, i32 0
   %splat = shufflevector <vscale x 8 x float> %head, <vscale x 8 x float> poison, <vscale x 8 x i32> zeroinitializer
   %n = fneg <vscale x 8 x float> %splat
   %eneg = fpext <vscale x 8 x float> %n to <vscale x 8 x double>
   %r = call <vscale x 8 x double> @llvm.copysign.nxv8f64(<vscale x 8 x double> %vm, <vscale x 8 x double> %eneg)
   ret <vscale x 8 x double> %r
 }