[RISCV] Support inline asm for vector instructions.

[HsiangKai]

Summary

Types of fractional LMUL and LMUL=1 are all using VR register class. When
using inline asm, it will use the first type in the register class as the
type for the register. It is not necessary the same as the value type. We
need to use INSERT_SUBVECTOR/EXTRACT_SUBVECToR/BITCAST to make it legal
to put the value in the corresponding register class.

[HsiangKai]

Convert V0~V31 to the super register if the value type is belong to M2/M4/M8.

[HsiangKai]

Update test cases.

[craig.topper]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6773

ValueVT.getVectorElementCount().getKnownMinValue() is equivalents to ValueVT.getVectorMinNumElements() I think.

Or would ValueVT.getSizeInBits().getKnownMinValue() replace the whole thing?

[HsiangKai]

Address comments.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6777

In what situations are the element types not equal? Does the PartVT come from the first type in the RC?

6778

Is this guaranteed not to become 0?

6779

Maybe put a /*IsScalable*/ on the true?

llvm/test/CodeGen/RISCV/rvv/inline-asm.ll

3

Nit, but -o - and < %s isn't too common a combination: < %s or -o - %s?

[HsiangKai]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6777

Yes, the PartVT comes from the first type in the RC. Another way to handle it is to search the corresponding type in the RC in GetRegistersForValue(). The problem comes from we reuse VR register class for fractional LMUL types.

6778

Yes, I reordered the supported types in VR. The first type has the element type i8. In addition, I created VM register class for mask register types.

[HsiangKai]

Address @frasercrmck 's comments.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6777

Thanks! (Un)Luckily I've seen the "first type" issue before: I wish there was a better way.

6778

Cool. Then perhaps we can check/assert on this, just so the types aren't accidentally reordered in the future? This seems to me like quite a few implicit dependencies being used and handed to us via this method.

[HsiangKai]

Add an assertion to check the number of element is not zero.

[HsiangKai]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6777

@rogfer01 provided another way to deal with the issue in GetRegistersForValue() in their repository.

He searched the types in the register class for the value type. Do you think it is better to do in this way or provide target hooks to deal with it?

@@ -7906,7 +7906,20 @@ static void GetRegistersForValue(SelectionDAG &DAG, const SDLoc &
   // Get the actual register value type.  This is important, because the user
   // may have asked for (e.g.) the AX register in i32 type.  We need to
   // remember that AX is actually i16 to get the right extension.
-  const MVT RegVT = *TRI.legalclasstypes_begin(*RC);
+  MVT RegVT = *TRI.legalclasstypes_begin(*RC);
+  // Try harder for vectors just in case sizes don't match.
+  // FIXME: This is caused by having nxv1i1 in the same register class as
+  // nxv1i64 in EPI.
+  if (RegVT.isVector() && RefOpInfo.ConstraintVT.isVector() &&
+      RegVT.getSizeInBits() != RefOpInfo.ConstraintVT.getSizeInBits()) {
+    auto E = TRI.legalclasstypes_end(*RC);
+    auto I = std::find_if(TRI.legalclasstypes_begin(*RC), E,
+                          [&RefOpInfo](const MVT::SimpleValueType &VTy) {
+                            return MVT(VTy) == RefOpInfo.ConstraintVT;
+                          });
+    if (I != E)
+      RegVT = *I;
+  }

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6250

I might be missing something: could VM be folded into the loop below?

6777

It's not really specific to vectors, so if we were to go down that route I'd probably expect us to have to offer it up to everything. If I remember correctly, I once had i8,i16,i32,v2i8,v4i8,v2i16 (maybe others) in the same "scalar" register class.

[HsiangKai]

Put VMRegClass before VRRegClass and VRMxRegClass. Handle them in the same loop.

[HsiangKai]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6777

It seems only RISC-V encountered the problem when choosing the type for the register classes. That is why I provided the target hooks to avoid the problems instead of adding checking in GetRegistersForValue().

If you think it is appropriate to add checking here, I think it is a simpler way to solve the problem.

[frasercrmck]

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

6777

I think it's fine as you've got it, thanks.

[HsiangKai]

To select the correct types for values in GetRegistersForValue().

[frasercrmck]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

8087 ↗

(On Diff #329164)

Does this case trigger for any other target?

I'm curious: what made you choose this method in the end?

8088 ↗

(On Diff #329164)

These clang-tidy warnings about const auto * look useful

[HsiangKai]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

8087 ↗

(On Diff #329164)

I prefer to use target hooks to solve the issue.
I think I misunderstood you. I will revert it back.

[HsiangKai]

Revert to use target hooks.

[frasercrmck]

LGTM, though maybe someone else should take a look?

llvm/lib/Target/RISCV/RISCVISelLowering.h

public:
 
   /// Returns true if the target allows unaligned memory accesses of the
   /// specified type.
   bool allowsMisalignedMemoryAccesses(
       EVT VT, unsigned AddrSpace = 0, Align Alignment = Align(1),
       MachineMemOperand::Flags Flags = MachineMemOperand::MONone,
       bool *Fast = nullptr) const override;
 
+  bool splitValueIntoRegisterParts(SelectionDAG &DAG, const SDLoc &DL,
+                                   SDValue Val, SDValue *Parts,
+                                   unsigned NumParts, MVT PartVT,
+                                   Optional<CallingConv::ID> CC) const override;
+
+  SDValue
+  joinRegisterPartsIntoValue(SelectionDAG &DAG, const SDLoc &DL,
+                             const SDValue *Parts, unsigned NumParts,
+                             MVT PartVT, EVT ValueVT,
+                             Optional<CallingConv::ID> CC) const override;
+
   static RISCVVLMUL getLMUL(MVT VT);
   static unsigned getRegClassIDForLMUL(RISCVVLMUL LMul);
   static unsigned getSubregIndexByMVT(MVT VT, unsigned Index);
   static unsigned getRegClassIDForVecVT(MVT VT);
   static std::pair<unsigned, unsigned>
   decomposeSubvectorInsertExtractToSubRegs(MVT VecVT, MVT SubVecVT,
                                            unsigned InsertExtractIdx,
                                            const RISCVRegisterInfo *TRI);

llvm/lib/Target/RISCV/RISCVISelLowering.cpp

 /// constraint it is for this target.
 RISCVTargetLowering::ConstraintType
 RISCVTargetLowering::getConstraintType(StringRef Constraint) const {
   if (Constraint.size() == 1) {
     switch (Constraint[0]) {
     default:
       break;
     case 'f':
+    case 'v':
       return C_RegisterClass;
     case 'I':
     case 'J':
     case 'K':
       return C_Immediate;
     case 'A':
       return C_Memory;
     }
if (Constraint.size() == 1) {
     case 'f':
       if (Subtarget.hasStdExtZfh() && VT == MVT::f16)
         return std::make_pair(0U, &RISCV::FPR16RegClass);
       if (Subtarget.hasStdExtF() && VT == MVT::f32)
         return std::make_pair(0U, &RISCV::FPR32RegClass);
       if (Subtarget.hasStdExtD() && VT == MVT::f64)
         return std::make_pair(0U, &RISCV::FPR64RegClass);
       break;
+    case 'v':
+      for (const auto *RC :
+           {&RISCV::VMRegClass, &RISCV::VRRegClass, &RISCV::VRM2RegClass,
+            &RISCV::VRM4RegClass, &RISCV::VRM8RegClass}) {
+        if (TRI->isTypeLegalForClass(*RC, VT.SimpleTy))
+          return std::make_pair(0U, RC);
+      }
+      break;
     default:
       break;
     }
   }
 
   // Clang will correctly decode the usage of register name aliases into their
   // official names. However, other frontends like `rustc` do not. This allows
   // users of these frontends to use the ABI names for registers in LLVM-style
if (FReg != RISCV::NoRegister) {
         unsigned RegNo = FReg - RISCV::F0_F;
         unsigned DReg = RISCV::F0_D + RegNo;
         return std::make_pair(DReg, &RISCV::FPR64RegClass);
       }
       return std::make_pair(FReg, &RISCV::FPR32RegClass);
     }
   }
 
+  if (Subtarget.hasStdExtV()) {
+    Register VReg = StringSwitch<Register>(Constraint.lower())
+                        .Case("{v0}", RISCV::V0)
+                        .Case("{v1}", RISCV::V1)
+                        .Case("{v2}", RISCV::V2)
+                        .Case("{v3}", RISCV::V3)
+                        .Case("{v4}", RISCV::V4)
+                        .Case("{v5}", RISCV::V5)
+                        .Case("{v6}", RISCV::V6)
+                        .Case("{v7}", RISCV::V7)
+                        .Case("{v8}", RISCV::V8)
+                        .Case("{v9}", RISCV::V9)
+                        .Case("{v10}", RISCV::V10)
+                        .Case("{v11}", RISCV::V11)
+                        .Case("{v12}", RISCV::V12)
+                        .Case("{v13}", RISCV::V13)
+                        .Case("{v14}", RISCV::V14)
+                        .Case("{v15}", RISCV::V15)
+                        .Case("{v16}", RISCV::V16)
+                        .Case("{v17}", RISCV::V17)
+                        .Case("{v18}", RISCV::V18)
+                        .Case("{v19}", RISCV::V19)
+                        .Case("{v20}", RISCV::V20)
+                        .Case("{v21}", RISCV::V21)
+                        .Case("{v22}", RISCV::V22)
+                        .Case("{v23}", RISCV::V23)
+                        .Case("{v24}", RISCV::V24)
+                        .Case("{v25}", RISCV::V25)
+                        .Case("{v26}", RISCV::V26)
+                        .Case("{v27}", RISCV::V27)
+                        .Case("{v28}", RISCV::V28)
+                        .Case("{v29}", RISCV::V29)
+                        .Case("{v30}", RISCV::V30)
+                        .Case("{v31}", RISCV::V31)
+                        .Default(RISCV::NoRegister);
+    if (VReg != RISCV::NoRegister) {
+      if (TRI->isTypeLegalForClass(RISCV::VMRegClass, VT.SimpleTy))
+        return std::make_pair(VReg, &RISCV::VMRegClass);
+      if (TRI->isTypeLegalForClass(RISCV::VRRegClass, VT.SimpleTy))
+        return std::make_pair(VReg, &RISCV::VRRegClass);
+      for (const auto *RC :
+           {&RISCV::VRM2RegClass, &RISCV::VRM4RegClass, &RISCV::VRM8RegClass}) {
+        if (TRI->isTypeLegalForClass(*RC, VT.SimpleTy)) {
+          VReg = TRI->getMatchingSuperReg(VReg, RISCV::sub_vrm1_0, RC);
+          return std::make_pair(VReg, RC);
+        }
+      }
+    }
+  }
+
   return TargetLowering::getRegForInlineAsmConstraint(TRI, Constraint, VT);
 }
 
 unsigned
 RISCVTargetLowering::getInlineAsmMemConstraint(StringRef ConstraintCode) const {
   // Currently only support length 1 constraints.
   if (ConstraintCode.size() == 1) {
     switch (ConstraintCode[0]) {
if (Alignment >= ElemVT.getStoreSize()) {
     if (Fast)
       *Fast = true;
     return true;
   }
 
   return false;
 }
 
+bool RISCVTargetLowering::splitValueIntoRegisterParts(
+    SelectionDAG &DAG, const SDLoc &DL, SDValue Val, SDValue *Parts,
+    unsigned NumParts, MVT PartVT, Optional<CallingConv::ID> CC) const {
+  EVT ValueVT = Val.getValueType();
+  if (ValueVT.isScalableVector() && PartVT.isScalableVector()) {
+    LLVMContext &Context = *DAG.getContext();
+    EVT ValueEltVT = ValueVT.getVectorElementType();
+    EVT PartEltVT = PartVT.getVectorElementType();
+    unsigned ValueVTBitSize = ValueVT.getSizeInBits().getKnownMinSize();
+    unsigned PartVTBitSize = PartVT.getSizeInBits().getKnownMinSize();
+    if (PartVTBitSize % ValueVTBitSize == 0) {
+      // If the element types are different, bitcast to the same element type of
+      // PartVT first.
+      if (ValueEltVT != PartEltVT) {
+        unsigned Count = ValueVTBitSize / PartEltVT.getSizeInBits();
+        assert(Count != 0 && "The number of element should not be zero.");
+        EVT SameEltTypeVT =
+            EVT::getVectorVT(Context, PartEltVT, Count, /*IsScalable=*/true);
+        Val = DAG.getNode(ISD::BITCAST, DL, SameEltTypeVT, Val);
+      }
+      Val = DAG.getNode(ISD::INSERT_SUBVECTOR, DL, PartVT, DAG.getUNDEF(PartVT),
+                        Val, DAG.getConstant(0, DL, Subtarget.getXLenVT()));
+      Parts[0] = Val;
+      return true;
+    }
+  }
+  return false;
+}
+
+SDValue RISCVTargetLowering::joinRegisterPartsIntoValue(
+    SelectionDAG &DAG, const SDLoc &DL, const SDValue *Parts, unsigned NumParts,
+    MVT PartVT, EVT ValueVT, Optional<CallingConv::ID> CC) const {
+  if (ValueVT.isScalableVector() && PartVT.isScalableVector()) {
+    LLVMContext &Context = *DAG.getContext();
+    SDValue Val = Parts[0];
+    EVT ValueEltVT = ValueVT.getVectorElementType();
+    EVT PartEltVT = PartVT.getVectorElementType();
+    unsigned ValueVTBitSize = ValueVT.getSizeInBits().getKnownMinSize();
+    unsigned PartVTBitSize = PartVT.getSizeInBits().getKnownMinSize();
+    if (PartVTBitSize % ValueVTBitSize == 0) {
+      EVT SameEltTypeVT = ValueVT;
+      // If the element types are different, convert it to the same element type
+      // of PartVT.
+      if (ValueEltVT != PartEltVT) {
+        unsigned Count = ValueVTBitSize / PartEltVT.getSizeInBits();
+        assert(Count != 0 && "The number of element should not be zero.");
+        SameEltTypeVT =
+            EVT::getVectorVT(Context, PartEltVT, Count, /*IsScalable=*/true);
+      }
+      Val = DAG.getNode(ISD::EXTRACT_SUBVECTOR, DL, SameEltTypeVT, Val,
+                        DAG.getConstant(0, DL, Subtarget.getXLenVT()));
+      if (ValueEltVT != PartEltVT)
+        Val = DAG.getNode(ISD::BITCAST, DL, ValueVT, Val);
+      return Val;
+    }
+  }
+  return SDValue();
+}
+
 #define GET_REGISTER_MATCHER
 #include "RISCVGenAsmMatcher.inc"
 
 Register
 RISCVTargetLowering::getRegisterByName(const char *RegName, LLT VT,
                                        const MachineFunction &MF) const {
   Register Reg = MatchRegisterAltName(RegName);
   if (Reg == RISCV::NoRegister)

llvm/lib/Target/RISCV/RISCVRegisterInfo.td

class VReg<list<ValueType> regTypes, dag regList, int Vlmul>
   : RegisterClass<"RISCV",
                   regTypes,
                   64, // The maximum supported ELEN is 64.
                   regList> {
   int VLMul = Vlmul;
   int Size = !mul(Vlmul, 64);
 }
 
-def VR : VReg<[vint8mf2_t, vint8mf4_t, vint8mf8_t,
+def VR : VReg<[vint8m1_t, vint16m1_t, vint32m1_t, vint64m1_t,
+               vfloat16m1_t, vfloat32m1_t, vfloat64m1_t,
+               vint8mf2_t, vint8mf4_t, vint8mf8_t,
                vint16mf2_t, vint16mf4_t, vint32mf2_t,
-               vint8m1_t, vint16m1_t, vint32m1_t, vint64m1_t,
-               vfloat16mf4_t, vfloat16mf2_t, vfloat16m1_t,
-               vfloat32mf2_t, vfloat32m1_t, vfloat64m1_t,
+               vfloat16mf4_t, vfloat16mf2_t, vfloat32mf2_t,
                vbool64_t, vbool32_t, vbool16_t, vbool8_t, vbool4_t,
                vbool2_t, vbool1_t],
            (add (sequence "V%u", 25, 31),
                 (sequence "V%u", 8, 24),
                 (sequence "V%u", 0, 7)), 1>;
 
-def VRNoV0 : VReg<[vint8mf2_t, vint8mf4_t, vint8mf8_t,
+def VRNoV0 : VReg<[vint8m1_t, vint16m1_t, vint32m1_t, vint64m1_t,
+                   vfloat16m1_t, vfloat32m1_t, vfloat64m1_t,
+                   vint8mf2_t, vint8mf4_t, vint8mf8_t,
                    vint16mf2_t, vint16mf4_t, vint32mf2_t,
-                   vint8m1_t, vint16m1_t, vint32m1_t, vint64m1_t,
-                   vfloat16mf4_t, vfloat16mf2_t, vfloat16m1_t,
-                   vfloat32mf2_t, vfloat32m1_t, vfloat64m1_t,
+                   vfloat16mf4_t, vfloat16mf2_t, vfloat32mf2_t,
                    vbool64_t, vbool32_t, vbool16_t, vbool8_t, vbool4_t,
                    vbool2_t, vbool1_t],
                (add (sequence "V%u", 25, 31),
                     (sequence "V%u", 8, 24),
                     (sequence "V%u", 1, 7)), 1>;
 
 def VRM2 : VReg<[vint8m2_t, vint16m2_t, vint32m2_t, vint64m2_t,
                  vfloat16m2_t, vfloat32m2_t, vfloat64m2_t],

 
 defvar VMaskVTs = [vbool64_t, vbool32_t, vbool16_t, vbool8_t,
                    vbool4_t, vbool2_t, vbool1_t];
 
 def VMV0 : RegisterClass<"RISCV", VMaskVTs, 64, (add V0)> {
   let Size = 64;
 }
 
+// The register class is added for inline assembly for vector mask types.
+def VM : VReg<[vbool1_t, vbool2_t, vbool4_t, vbool8_t, vbool16_t,
+               vbool32_t, vbool64_t],
+           (add (sequence "V%u", 25, 31),
+                (sequence "V%u", 8, 24),
+                (sequence "V%u", 0, 7)), 1>;
+
 foreach m = LMULList.m in {
   foreach nf = NFList<m>.L in {
     def "VRN" # nf # "M" # m: VReg<[untyped],
                                (add !cast<RegisterTuples>("VN" # nf # "M" # m # "V0"), !cast<RegisterTuples>("VN" # nf # "M" # m # "NoV0")),
                                     !mul(nf, m)>;
     def "VRN" # nf # "M" # m # "NoV0": VReg<[untyped],
                                (add !cast<RegisterTuples>("VN" # nf # "M" # m # "NoV0")),
                                     !mul(nf, m)>;
   }
 }

llvm/test/CodeGen/RISCV/rvv/inline-asm.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=riscv64 -mattr=+experimental-v < %s \
+; RUN:     --verify-machineinstrs | FileCheck %s
+
+define <vscale x 1 x i1> @test_1xi1(<vscale x 1 x i1> %in, <vscale x 1 x i1> %in2) nounwind {
+; CHECK-LABEL: test_1xi1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v0, v8
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 1 x i1> asm "vmand.mm $0, $1, $2", "=v,v,v"(<vscale x 1 x i1> %in, <vscale x 1 x i1> %in2)
+  ret <vscale x 1 x i1> %0
+}
+
+define <vscale x 2 x i1> @test_2xi1(<vscale x 2 x i1> %in, <vscale x 2 x i1> %in2) nounwind {
+; CHECK-LABEL: test_2xi1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v0, v8
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 2 x i1> asm "vmand.mm $0, $1, $2", "=v,v,v"(<vscale x 2 x i1> %in, <vscale x 2 x i1> %in2)
+  ret <vscale x 2 x i1> %0
+}
+
+define <vscale x 4 x i1> @test_4xi1(<vscale x 4 x i1> %in, <vscale x 4 x i1> %in2) nounwind {
+; CHECK-LABEL: test_4xi1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v0, v8
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 4 x i1> asm "vmand.mm $0, $1, $2", "=v,v,v"(<vscale x 4 x i1> %in, <vscale x 4 x i1> %in2)
+  ret <vscale x 4 x i1> %0
+}
+
+define <vscale x 8 x i1> @test_8xi1(<vscale x 8 x i1> %in, <vscale x 8 x i1> %in2) nounwind {
+; CHECK-LABEL: test_8xi1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v0, v8
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 8 x i1> asm "vmand.mm $0, $1, $2", "=v,v,v"(<vscale x 8 x i1> %in, <vscale x 8 x i1> %in2)
+  ret <vscale x 8 x i1> %0
+}
+
+define <vscale x 16 x i1> @test_16xi1(<vscale x 16 x i1> %in, <vscale x 16 x i1> %in2) nounwind {
+; CHECK-LABEL: test_16xi1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v0, v8
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 16 x i1> asm "vmand.mm $0, $1, $2", "=v,v,v"(<vscale x 16 x i1> %in, <vscale x 16 x i1> %in2)
+  ret <vscale x 16 x i1> %0
+}
+
+define <vscale x 32 x i1> @test_32xi1(<vscale x 32 x i1> %in, <vscale x 32 x i1> %in2) nounwind {
+; CHECK-LABEL: test_32xi1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v0, v8
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 32 x i1> asm "vmand.mm $0, $1, $2", "=v,v,v"(<vscale x 32 x i1> %in, <vscale x 32 x i1> %in2)
+  ret <vscale x 32 x i1> %0
+}
+
+define <vscale x 64 x i1> @test_64xi1(<vscale x 64 x i1> %in, <vscale x 64 x i1> %in2) nounwind {
+; CHECK-LABEL: test_64xi1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v0, v8
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 64 x i1> asm "vmand.mm $0, $1, $2", "=v,v,v"(<vscale x 64 x i1> %in, <vscale x 64 x i1> %in2)
+  ret <vscale x 64 x i1> %0
+}
+
+define <vscale x 1 x i64> @test_1xi64(<vscale x 1 x i64> %in, <vscale x 1 x i64> %in2) nounwind {
+; CHECK-LABEL: test_1xi64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 1 x i64> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 1 x i64> %in, <vscale x 1 x i64> %in2)
+  ret <vscale x 1 x i64> %0
+}
+
+define <vscale x 2 x i64> @test_2xi64(<vscale x 2 x i64> %in, <vscale x 2 x i64> %in2) nounwind {
+; CHECK-LABEL: test_2xi64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v10
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 2 x i64> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 2 x i64> %in, <vscale x 2 x i64> %in2)
+  ret <vscale x 2 x i64> %0
+}
+
+define <vscale x 4 x i64> @test_4xi64(<vscale x 4 x i64> %in, <vscale x 4 x i64> %in2) nounwind {
+; CHECK-LABEL: test_4xi64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v12
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 4 x i64> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 4 x i64> %in, <vscale x 4 x i64> %in2)
+  ret <vscale x 4 x i64> %0
+}
+
+define <vscale x 8 x i64> @test_8xi64(<vscale x 8 x i64> %in, <vscale x 8 x i64> %in2) nounwind {
+; CHECK-LABEL: test_8xi64:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 8 x i64> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 8 x i64> %in, <vscale x 8 x i64> %in2)
+  ret <vscale x 8 x i64> %0
+}
+
+define <vscale x 1 x i32> @test_1xi32(<vscale x 1 x i32> %in, <vscale x 1 x i32> %in2) nounwind {
+; CHECK-LABEL: test_1xi32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 1 x i32> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 1 x i32> %in, <vscale x 1 x i32> %in2)
+  ret <vscale x 1 x i32> %0
+}
+
+define <vscale x 2 x i32> @test_2xi32(<vscale x 2 x i32> %in, <vscale x 2 x i32> %in2) nounwind {
+; CHECK-LABEL: test_2xi32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 2 x i32> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 2 x i32> %in, <vscale x 2 x i32> %in2)
+  ret <vscale x 2 x i32> %0
+}
+
+define <vscale x 4 x i32> @test_4xi32(<vscale x 4 x i32> %in, <vscale x 4 x i32> %in2) nounwind {
+; CHECK-LABEL: test_4xi32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v10
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 4 x i32> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 4 x i32> %in, <vscale x 4 x i32> %in2)
+  ret <vscale x 4 x i32> %0
+}
+
+define <vscale x 8 x i32> @test_8xi32(<vscale x 8 x i32> %in, <vscale x 8 x i32> %in2) nounwind {
+; CHECK-LABEL: test_8xi32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v12
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 8 x i32> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 8 x i32> %in, <vscale x 8 x i32> %in2)
+  ret <vscale x 8 x i32> %0
+}
+
+define <vscale x 16 x i32> @test_16xi32(<vscale x 16 x i32> %in, <vscale x 16 x i32> %in2) nounwind {
+; CHECK-LABEL: test_16xi32:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 16 x i32> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 16 x i32> %in, <vscale x 16 x i32> %in2)
+  ret <vscale x 16 x i32> %0
+}
+
+define <vscale x 1 x i16> @test_1xi16(<vscale x 1 x i16> %in, <vscale x 1 x i16> %in2) nounwind {
+; CHECK-LABEL: test_1xi16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 1 x i16> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 1 x i16> %in, <vscale x 1 x i16> %in2)
+  ret <vscale x 1 x i16> %0
+}
+
+define <vscale x 2 x i16> @test_2xi16(<vscale x 2 x i16> %in, <vscale x 2 x i16> %in2) nounwind {
+; CHECK-LABEL: test_2xi16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 2 x i16> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 2 x i16> %in, <vscale x 2 x i16> %in2)
+  ret <vscale x 2 x i16> %0
+}
+
+define <vscale x 4 x i16> @test_4xi16(<vscale x 4 x i16> %in, <vscale x 4 x i16> %in2) nounwind {
+; CHECK-LABEL: test_4xi16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 4 x i16> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 4 x i16> %in, <vscale x 4 x i16> %in2)
+  ret <vscale x 4 x i16> %0
+}
+
+define <vscale x 8 x i16> @test_8xi16(<vscale x 8 x i16> %in, <vscale x 8 x i16> %in2) nounwind {
+; CHECK-LABEL: test_8xi16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v10
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 8 x i16> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 8 x i16> %in, <vscale x 8 x i16> %in2)
+  ret <vscale x 8 x i16> %0
+}
+
+define <vscale x 16 x i16> @test_16xi16(<vscale x 16 x i16> %in, <vscale x 16 x i16> %in2) nounwind {
+; CHECK-LABEL: test_16xi16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v12
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 16 x i16> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 16 x i16> %in, <vscale x 16 x i16> %in2)
+  ret <vscale x 16 x i16> %0
+}
+
+define <vscale x 32 x i16> @test_32xi16(<vscale x 32 x i16> %in, <vscale x 32 x i16> %in2) nounwind {
+; CHECK-LABEL: test_32xi16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 32 x i16> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 32 x i16> %in, <vscale x 32 x i16> %in2)
+  ret <vscale x 32 x i16> %0
+}
+
+define <vscale x 1 x i8> @test_1xi8(<vscale x 1 x i8> %in, <vscale x 1 x i8> %in2) nounwind {
+; CHECK-LABEL: test_1xi8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 1 x i8> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 1 x i8> %in, <vscale x 1 x i8> %in2)
+  ret <vscale x 1 x i8> %0
+}
+
+define <vscale x 2 x i8> @test_2xi8(<vscale x 2 x i8> %in, <vscale x 2 x i8> %in2) nounwind {
+; CHECK-LABEL: test_2xi8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 2 x i8> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 2 x i8> %in, <vscale x 2 x i8> %in2)
+  ret <vscale x 2 x i8> %0
+}
+
+define <vscale x 4 x i8> @test_4xi8(<vscale x 4 x i8> %in, <vscale x 4 x i8> %in2) nounwind {
+; CHECK-LABEL: test_4xi8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 4 x i8> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 4 x i8> %in, <vscale x 4 x i8> %in2)
+  ret <vscale x 4 x i8> %0
+}
+
+define <vscale x 8 x i8> @test_8xi8(<vscale x 8 x i8> %in, <vscale x 8 x i8> %in2) nounwind {
+; CHECK-LABEL: test_8xi8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v9
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 8 x i8> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 8 x i8> %in, <vscale x 8 x i8> %in2)
+  ret <vscale x 8 x i8> %0
+}
+
+define <vscale x 16 x i8> @test_16xi8(<vscale x 16 x i8> %in, <vscale x 16 x i8> %in2) nounwind {
+; CHECK-LABEL: test_16xi8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v10
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 16 x i8> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 16 x i8> %in, <vscale x 16 x i8> %in2)
+  ret <vscale x 16 x i8> %0
+}
+
+define <vscale x 32 x i8> @test_32xi8(<vscale x 32 x i8> %in, <vscale x 32 x i8> %in2) nounwind {
+; CHECK-LABEL: test_32xi8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v12
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 32 x i8> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 32 x i8> %in, <vscale x 32 x i8> %in2)
+  ret <vscale x 32 x i8> %0
+}
+
+define <vscale x 64 x i8> @test_64xi8(<vscale x 64 x i8> %in, <vscale x 64 x i8> %in2) nounwind {
+; CHECK-LABEL: test_64xi8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v8, v8, v16
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 64 x i8> asm "vadd.vv $0, $1, $2", "=v,v,v"(<vscale x 64 x i8> %in, <vscale x 64 x i8> %in2)
+  ret <vscale x 64 x i8> %0
+}
+
+define <vscale x 4 x i8> @test_specify_reg_mf2(<vscale x 4 x i8> %in, <vscale x 4 x i8> %in2) nounwind {
+; CHECK-LABEL: test_specify_reg_mf2:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vmv1r.v v2, v9
+; CHECK-NEXT:    vmv1r.v v1, v8
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v0, v1, v2
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    vmv1r.v v8, v0
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 4 x i8> asm "vadd.vv $0, $1, $2", "={v0},{v1},{v2}"(<vscale x 4 x i8> %in, <vscale x 4 x i8> %in2)
+  ret <vscale x 4 x i8> %0
+}
+
+define <vscale x 8 x i8> @test_specify_reg_m1(<vscale x 8 x i8> %in, <vscale x 8 x i8> %in2) nounwind {
+; CHECK-LABEL: test_specify_reg_m1:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vmv1r.v v2, v9
+; CHECK-NEXT:    vmv1r.v v1, v8
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v0, v1, v2
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    vmv1r.v v8, v0
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 8 x i8> asm "vadd.vv $0, $1, $2", "={v0},{v1},{v2}"(<vscale x 8 x i8> %in, <vscale x 8 x i8> %in2)
+  ret <vscale x 8 x i8> %0
+}
+
+define <vscale x 16 x i8> @test_specify_reg_m2(<vscale x 16 x i8> %in, <vscale x 16 x i8> %in2) nounwind {
+; CHECK-LABEL: test_specify_reg_m2:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vmv2r.v v4, v10
+; CHECK-NEXT:    vmv2r.v v2, v8
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vadd.vv v0, v2, v4
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    vmv2r.v v8, v0
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 16 x i8> asm "vadd.vv $0, $1, $2", "={v0},{v2},{v4}"(<vscale x 16 x i8> %in, <vscale x 16 x i8> %in2)
+  ret <vscale x 16 x i8> %0
+}
+
+define <vscale x 1 x i1> @test_specify_reg_mask(<vscale x 1 x i1> %in, <vscale x 1 x i1> %in2) nounwind {
+; CHECK-LABEL: test_specify_reg_mask:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    vmv1r.v v2, v8
+; CHECK-NEXT:    vmv1r.v v1, v0
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    vmand.mm v0, v1, v2
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    ret
+entry:
+  %0 = tail call <vscale x 1 x i1> asm "vmand.mm $0, $1, $2", "={v0},{v1},{v2}"(<vscale x 1 x i1> %in, <vscale x 1 x i1> %in2)
+  ret <vscale x 1 x i1> %0
+}