[AIX] Add support for non var_arg extended vector ABI calling convention on AIX

[ZarkoCA]

Summary

This patch enables passing non variadic vector type parameters on the caller and callee side and vector return on AIX that are passed in vector registers only.

So far, support is enabled for only the AIX extended Altivec ABI Calling convention.

[ZarkoCA]

Going to work on rebasing this patch.

[sfertile]

llvm/lib/Target/PowerPC/PPCSubtarget.h

329

Can I ask why this got modified?

[ZarkoCA]

llvm/lib/Target/PowerPC/PPCSubtarget.h

329

I was experimenting to see what differences that would make to some test cases I was trying. It shouldn't be in this patch. Sorry for the confusion.

[Xiangling_L]

Hi Zarko, could rebase this patch on the latest master?

[ZarkoCA]

In D86476#2298372, @Xiangling_L wrote:

Hi Zarko, could rebase this patch on the latest master?

Yes, the patch is in the changes planned state. I will rebase when I put it up again.

[sfertile]

We are in a poor position since this depends on D75059 which I never commit-ed since as I realized it was not really NFC because we were wrongly inserting VR_SAVE code for AIX. This had the effect of emitting a fatal error if we used vector arguments, return values and I believe if we selected any vector instructions. I never went back and updated it because there was always something higher priority to work on. I have a non-NFC update I haven't posted which relies on the calling convention lowering to emit an error when passing vector arguments, and I add an error when returning vector arguments. I will post that shortly.

This patch is trying implement 2 related but distinct things

1. Passing vector arguments.
2. Spilling and restoring of the vector.

The test for 1 will have a problem since the dependent patch will change to emitting an error for the return of a vector type. The second thing has no test coverage in this test. I suggest we put this patch on hold, and then proceed with:

1. I post the updated VRSAVE removal patch, which we can review and get commited first.
2. You split out the frame lowering changes from this patch and test them separately. We should be able to land that once D75059 lands.

Then we can either rework the lit test here to not return vector types and try to land this patch first, or enable vector returns (I think it will work as it, but we need lit test coverage).

[ZarkoCA]

This patch now only contains the ISel lowering portion of the previous version. Additionally, lowering on the callee side was implemented since with the work already contained in CC_AIX only the removal of the error in LowerCall_AIX() was required. Return vector types was also enabled. The test case checks the caller, callee and return assembly instructions.

[Xiangling_L]

Hi Zarko, the patch is out of sync with master branch, could you update it?

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

6797–6800

Just want to confirm that this would not lead to a warning when compiled with assertion off?

6820

If I understand correctly, isByVal() section is for aggregate parameter types only? Can we add some comments or even better an assertion here to indicate that?

6920

Are we using v2i64 to support vector long? As ABI says, the vector type with the long keyword are deprecated. Should we emit an error for this type instead?

[ZarkoCA]

Rebased based on new changes in https://reviews.llvm.org/D88676.
Added error for vector long type.

[ZarkoCA]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

6797–6800

I think so, but I wasn't looking to mess with that assert only to avoid it when using vectors.

6820

Yes, that sections only handles arguments that have the ByVal flag.

I think adding comments or an assertion is not related to this patch. We would need a separate review to handle that.

6920

Good catch, thanks.

[ZarkoCA]

Addressed comments and adding parent revisions.

[ZarkoCA]

Rebase to reflect changes in parent revisions.

[DiggerLin]

LGTM , but I think it need other person to approve it before commit

[sfertile]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

6856

I don;'t think we can determine if a v2i64 came from a vector long in the IR since vector long long is a v2i64 as well. Any error handling for the altivec type would have to be done in the front-end I believe.

[ZarkoCA]

Removed vector long error

[ZarkoCA]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

6856

You're right, there was an clang patch that already handled this.

[ZarkoCA]

Removed passing vector parameters to the stack portion of patch, fixed test case and added test cases for errors.

[sfertile]

1 comment, but otherwise LGTM.

[sfertile]

Sorry Zarko, I must have deleted my comment before posting last night I've added it back now.

llvm/test/CodeGen/PowerPC/aix-cc-ext-vec-abi.ll

52 ↗

(On Diff #307720)

I think we should use a variable here to make sure the register used in the toc entry access is the same as used in the vector load:

lwz [[SCRATCH1:[0-9]+]], L..C0(2)
lxvw4x 1, 0, [[SCRATCH1]]

[ZarkoCA]

Updated regex in test cases to check register correct usage.

llvm/lib/Target/PowerPC/PPCCallingConv.td

 
 def CSR_AIX32 : CalleeSavedRegs<(add R13, R14, R15, R16, R17, R18, R19, R20,
                                      R21, R22, R23, R24, R25, R26, R27, R28,
                                      R29, R30, R31, F14, F15, F16, F17, F18,
                                      F19, F20, F21, F22, F23, F24, F25, F26,
                                      F27, F28, F29, F30, F31, CR2, CR3, CR4
                                 )>;
 
+def CSR_AIX32_Altivec: CalleeSavedRegs<(add CSR_AIX32, CSR_Altivec)>;
+
 // Common CalleeSavedRegs for SVR4 and AIX.
 def CSR_PPC64   : CalleeSavedRegs<(add X14, X15, X16, X17, X18, X19, X20,
                                         X21, X22, X23, X24, X25, X26, X27, X28,
                                         X29, X30, X31, F14, F15, F16, F17, F18,
                                         F19, F20, F21, F22, F23, F24, F25, F26,
                                         F27, F28, F29, F30, F31, CR2, CR3, CR4
                                    )>;
 

llvm/lib/Target/PowerPC/PPCFrameLowering.cpp

static const SpillSlot ELFOffsets64[] = {
       CALLEE_SAVED_VRS
   };
 
   static const SpillSlot AIXOffsets32[] = {
       CALLEE_SAVED_FPRS,
       CALLEE_SAVED_GPRS32,
       // Add AIX's extra CSR.
       {PPC::R13, -76},
-      // TODO: Update when we add vector support for AIX.
+      // VRSAVE save area offset.
+      {PPC::VRSAVE, -4},
+      CALLEE_SAVED_VRS
   };
 
   static const SpillSlot AIXOffsets64[] = {
       CALLEE_SAVED_FPRS,
       CALLEE_SAVED_GPRS64,
-      // TODO: Update when we add vector support for AIX.
+      // VRSAVE save area offset.
+      {PPC::VRSAVE, -4},
+      CALLEE_SAVED_VRS
   };
 
   if (Subtarget.is64BitELFABI()) {
     NumEntries = array_lengthof(ELFOffsets64);
     return ELFOffsets64;
   }
 
   if (Subtarget.is32BitELFABI()) {

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

static bool CC_AIX(unsigned ValNo, MVT ValVT, MVT LocVT,
                    CCState &State) {
 
   const PPCSubtarget &Subtarget = static_cast<const PPCSubtarget &>(
       State.getMachineFunction().getSubtarget());
   const bool IsPPC64 = Subtarget.isPPC64();
   const Align PtrAlign = IsPPC64 ? Align(8) : Align(4);
   const MVT RegVT = IsPPC64 ? MVT::i64 : MVT::i32;
 
-  assert((!ValVT.isInteger() ||
-          (ValVT.getSizeInBits() <= RegVT.getSizeInBits())) &&
-         "Integer argument exceeds register size: should have been legalized");
+  if (ValVT.isInteger() && !ValVT.isVector())
+    assert(
+        (ValVT.getSizeInBits() <= RegVT.getSizeInBits()) &&
 
   if (ValVT == MVT::f128)
     report_fatal_error("f128 is unimplemented on AIX.");
 
   if (ArgFlags.isNest())
     report_fatal_error("Nest arguments are unimplemented.");
 
-  if (ValVT.isVector() || LocVT.isVector())
-    report_fatal_error("Vector arguments are unimplemented on AIX.");
-
   static const MCPhysReg GPR_32[] = {// 32-bit registers.
                                      PPC::R3, PPC::R4, PPC::R5, PPC::R6,
                                      PPC::R7, PPC::R8, PPC::R9, PPC::R10};
   static const MCPhysReg GPR_64[] = {// 64-bit registers.
                                      PPC::X3, PPC::X4, PPC::X5, PPC::X6,
                                      PPC::X7, PPC::X8, PPC::X9, PPC::X10};
 
+  static const MCPhysReg VR[] = {// Vector registers.
+                                 PPC::V2,  PPC::V3,  PPC::V4,  PPC::V5,
+                                 PPC::V6,  PPC::V7,  PPC::V8,  PPC::V9,
+                                 PPC::V10, PPC::V11, PPC::V12, PPC::V13};
+
   if (ArgFlags.isByVal()) {
     if (ArgFlags.getNonZeroByValAlign() > PtrAlign)
       report_fatal_error("Pass-by-value arguments with alignment greater than "
                          "register width are not supported.");
 
     const unsigned ByValSize = ArgFlags.getByValSize();
 
     // An empty aggregate parameter takes up no storage and no registers,
     // but needs a MemLoc for a stack slot for the formal arguments side.
if (ArgFlags.isByVal()) {
     }
     return false;
   }
 
   // Arguments always reserve parameter save area.
   switch (ValVT.SimpleTy) {
   default:
     report_fatal_error("Unhandled value type for argument.");
   case MVT::i64:
     // i64 arguments should have been split to i32 for PPC32.
     assert(IsPPC64 && "PPC32 should have split i64 values.");
     LLVM_FALLTHROUGH;
   case MVT::i1:
   case MVT::i32: {
     const unsigned Offset = State.AllocateStack(PtrAlign.value(), PtrAlign);
     // AIX integer arguments are always passed in register width.
     if (ValVT.getSizeInBits() < RegVT.getSizeInBits())
for (unsigned I = 0; I < StoreSize; I += PtrAlign.value()) {
                                              LocInfo)
                  : CCValAssign::getMem(ValNo, ValVT, Offset, LocVT, LocInfo));
         break;
       }
     }
 
     return false;
   }
+  case MVT::v4f32:
+  case MVT::v4i32:
+  case MVT::v8i16:
+  case MVT::v16i8:
+  case MVT::v2f64:
+  case MVT::v2i64:
+  case MVT::v1i128: {
+    if (State.isVarArg())
+      report_fatal_error(
+          "variadic arguments for vector are not yet implemented for AIX");
+
+    const unsigned Offset = State.AllocateStack(16, Align(16));
+    if (unsigned VReg = State.AllocateReg(VR))
+      State.addLoc(CCValAssign::getReg(ValNo, ValVT, VReg, LocVT, LocInfo));
+    else
+      State.addLoc(CCValAssign::getMem(ValNo, ValVT, Offset, LocVT, LocInfo));
+    return false;
+  }
   }
   return true;
 }
 
 static const TargetRegisterClass *getRegClassForSVT(MVT::SimpleValueType SVT,
                                                     bool IsPPC64) {
   assert((IsPPC64 || SVT != MVT::i64) &&
          "i64 should have been split for 32-bit codegen.");
 
   switch (SVT) {
   default:
     report_fatal_error("Unexpected value type for formal argument");
   case MVT::i1:
   case MVT::i32:
   case MVT::i64:
     return IsPPC64 ? &PPC::G8RCRegClass : &PPC::GPRCRegClass;
   case MVT::f32:
     return &PPC::F4RCRegClass;
   case MVT::f64:
     return &PPC::F8RCRegClass;
+  case MVT::v4f32:
+  case MVT::v4i32:
+  case MVT::v8i16:
+  case MVT::v16i8:
+  case MVT::v2f64:
+  case MVT::v2i64:
+  case MVT::v1i128:
+    return &PPC::VRRCRegClass;
   }
 }
 
 static SDValue truncateScalarIntegerArg(ISD::ArgFlagsTy Flags, EVT ValVT,
                                         SelectionDAG &DAG, SDValue ArgValue,
                                         MVT LocVT, const SDLoc &dl) {
   assert(ValVT.isScalarInteger() && LocVT.isScalarInteger());
   assert(ValVT.getSizeInBits() < LocVT.getSizeInBits());

llvm/lib/Target/PowerPC/PPCRegisterInfo.cpp

PPCRegisterInfo::getCalleeSavedRegs(const MachineFunction *MF) const {
   if (TM.isPPC64()) {
     if (Subtarget.hasAltivec())
       return SaveR2 ? CSR_PPC64_R2_Altivec_SaveList
                     : CSR_PPC64_Altivec_SaveList;
     return SaveR2 ? CSR_PPC64_R2_SaveList : CSR_PPC64_SaveList;
   }
   // 32-bit targets.
   if (Subtarget.isAIXABI())
-    return CSR_AIX32_SaveList;
+    return (Subtarget.hasAltivec()) ? CSR_AIX32_Altivec_SaveList
+                                    : CSR_AIX32_SaveList;
   if (Subtarget.hasAltivec())
     return CSR_SVR432_Altivec_SaveList;
   else if (Subtarget.hasSPE())
     return CSR_SVR432_SPE_SaveList;
   return CSR_SVR432_SaveList;
 }
 
 const uint32_t *
 PPCRegisterInfo::getCallPreservedMask(const MachineFunction &MF,
                                       CallingConv::ID CC) const {
   const PPCSubtarget &Subtarget = MF.getSubtarget<PPCSubtarget>();
   if (CC == CallingConv::AnyReg) {
     if (Subtarget.hasVSX())
       return CSR_64_AllRegs_VSX_RegMask;
     if (Subtarget.hasAltivec())
       return CSR_64_AllRegs_Altivec_RegMask;
     return CSR_64_AllRegs_RegMask;
   }
 
   if (Subtarget.isAIXABI()) {
-    assert(!Subtarget.hasAltivec() && "Altivec is not implemented on AIX yet.");
-    return TM.isPPC64() ? CSR_PPC64_RegMask : CSR_AIX32_RegMask;
+    return TM.isPPC64()
+               ? (Subtarget.hasAltivec() ? CSR_64_AllRegs_Altivec_RegMask
+                                         : CSR_PPC64_RegMask)
+               : (Subtarget.hasAltivec() ? CSR_AIX32_Altivec_RegMask
+                                         : CSR_AIX32_RegMask);
   }
 
   if (CC == CallingConv::Cold) {
     return TM.isPPC64() ? (Subtarget.hasAltivec() ? CSR_SVR64_ColdCC_Altivec_RegMask
                                                   : CSR_SVR64_ColdCC_RegMask)
                         : (Subtarget.hasAltivec() ? CSR_SVR32_ColdCC_Altivec_RegMask
                                                   : (Subtarget.hasSPE()
                                                   ? CSR_SVR32_ColdCC_SPE_RegMask

llvm/lib/Target/PowerPC/PPCSubtarget.h

public:
   bool hasHTM() const { return HasHTM; }
   bool hasFloat128() const { return HasFloat128; }
   bool isISA3_0() const { return IsISA3_0; }
   bool isISA3_1() const { return IsISA3_1; }
   bool useLongCalls() const { return UseLongCalls; }
   bool hasFusion() const { return HasFusion; }
   bool hasAddiLoadFusion() const { return HasAddiLoadFusion; }
   bool hasAddisLoadFusion() const { return HasAddisLoadFusion; }
-  bool needsSwapsForVSXMemOps() const {
-    return hasVSX() && isLittleEndian() && !hasP9Vector();
-  }
 
   POPCNTDKind hasPOPCNTD() const { return HasPOPCNTD; }
 
   const Triple &getTargetTriple() const { return TargetTriple; }
 
   bool isTargetELF() const { return TargetTriple.isOSBinFormatELF(); }
   bool isTargetMachO() const { return TargetTriple.isOSBinFormatMachO(); }
   bool isTargetLinux() const { return TargetTriple.isOSLinux(); }
 
   bool isAIXABI() const { return TargetTriple.isOSAIX(); }
   bool isSVR4ABI() const { return !isAIXABI(); }
   bool isELFv2ABI() const;
 
   bool is64BitELFABI() const { return  isSVR4ABI() && isPPC64(); }
   bool is32BitELFABI() const { return  isSVR4ABI() && !isPPC64(); }
   bool isUsingPCRelativeCalls() const;
 
+  bool needsSwapsForVSXMemOps() const {
+    return hasVSX() && (isLittleEndian() || isAIXABI()) && !hasP9Vector();
+  }
   /// Originally, this function return hasISEL(). Now we always enable it,
   /// but may expand the ISEL instruction later.
   bool enableEarlyIfConversion() const override { return true; }
 
   /// Scheduling customization.
   bool enableMachineScheduler() const override;
   /// Pipeliner customization.
   bool enableMachinePipeliner() const override;

llvm/test/CodeGen/PowerPC/aix-cc-vector.ll

+; RUN: llc -verify-machineinstrs -mcpu=pwr7 -mattr=altivec \
+; RUN:  -mtriple powerpc-ibm-aix-xcoff < %s | \
+; RUN: FileCheck --check-prefixes=CHECKASM,ASM32 %s
+
+; RUN: llc -verify-machineinstrs -mcpu=pwr7 -mattr=altivec \
+; RUN:  -mtriple powerpc64-ibm-aix-xcoff < %s | \
+; RUN: FileCheck --check-prefixes=CHECKASM,ASM64 %s
+
+define dso_local <4 x i32> @int(<4 x i32> %int0, <4 x i32> %int1, <4 x i32> %int2, <4 x i32> %int3, <4 x i32> %int4, <4 x i32> %int5, <4 x i32> %int6, <4 x i32> %int7, <4 x i32> %int8, <4 x i32> %int9, <4 x i32> %int10, <4 x i32> %int11, <4 x i32> %int12, <4 x i32> %int13, <4 x i32> %int14) {
+entry:
+  %add = add <4 x i32> %int0, %int1
+  %add1 = add <4 x i32> %add, %int2
+  %add2 = add <4 x i32> %add1, %int3
+  %add3 = add <4 x i32> %add2, %int4
+  %add4 = add <4 x i32> %add3, %int5
+  %add5 = add <4 x i32> %add4, %int6
+  %add6 = add <4 x i32> %add5, %int7
+  %add7 = add <4 x i32> %add6, %int8
+  %add8 = add <4 x i32> %add7, %int9
+  %add9 = add <4 x i32> %add8, %int10
+  %add10 = add <4 x i32> %add9, %int11
+  %add11 = add <4 x i32> %add10, %int12
+  %add12 = add <4 x i32> %add11, %int13
+  %add13 = add <4 x i32> %add12, %int14
+  ret <4 x i32> %add13
+}
+
+; CHECKASM-LABEL:  .int: 
+
+; ASM32-DAG:       addi [[REG1:[0-9]+]], 1, 224
+; ASM32-DAG:       lxvw4x 35, 0, [[REG1]] 
+; ASM32-DAG:       addi [[REG2:[0-9]+]], 1, 240
+; ASM32-DAG:       lxvw4x 36, 0, [[REG2]] 
+; ASM32-DAG:       addi [[REG3:[0-9]+]], 1, 256
+; ASM32-DAG:       lxvw4x 35, 0, [[REG3]] 
+; ASM32-DAG:       vadduwm 2, 2, 3
+; ASM32-DAG:       vadduwm 2, 2, 4
+; ASM32-DAG:       vadduwm 2, 2, 5
+; ASM32-DAG:       vadduwm 2, 2, 6
+; ASM32-DAG:       vadduwm 2, 2, 7
+; ASM32-DAG:       vadduwm 2, 2, 8
+; ASM32-DAG:       vadduwm 2, 2, 9
+; ASM32-DAG:       vadduwm 2, 2, 10
+; ASM32-DAG:       vadduwm 2, 2, 11
+; ASM32-DAG:       vadduwm 2, 2, 12
+; ASM32-DAG:       vadduwm 2, 2, 13
+; ASM32-DAG:       vadduwm 2, 2, 3
+; ASM32-DAG:       vadduwm 2, 2, 4
+; ASM32-DAG:       vadduwm 2, 2, 3
+
+; ASM64-DAG:       addi [[REG1:[0-9]+]], 1, 240
+; ASM64-DAG:       lxvw4x 35, 0, [[REG1]]
+; ASM64-DAG:       addi [[REG2:[0-9]+]], 1, 256
+; ASM64-DAG:       lxvw4x 36, 0, [[REG2]]
+; ASM64-DAG:       addi [[REG3:[0-9]+]], 1, 272
+; ASM64-DAG:       lxvw4x 35, 0, [[REG3]]
+; ASM64-DAG:       vadduwm 2, 2, 3
+; ASM64-DAG:       vadduwm 2, 2, 4
+; ASM64-DAG:       vadduwm 2, 2, 5
+; ASM64-DAG:       vadduwm 2, 2, 6
+; ASM64-DAG:       vadduwm 2, 2, 7
+; ASM64-DAG:       vadduwm 2, 2, 8
+; ASM64-DAG:       vadduwm 2, 2, 9
+; ASM64-DAG:       vadduwm 2, 2, 10
+; ASM64-DAG:       vadduwm 2, 2, 11
+; ASM64-DAG:       vadduwm 2, 2, 12
+; ASM64-DAG:       vadduwm 2, 2, 13
+; ASM64-DAG:       vadduwm 2, 2, 3
+; ASM64-DAG:       vadduwm 2, 2, 4
+; ASM64-DAG:       vadduwm 2, 2, 3
+; ASM64-DAG:       blr
+
+define dso_local <2 x i64> @longint(<2 x i64> %ll0, <2 x i64> %ll1, <2 x i64> %ll2, <2 x i64> %ll3, <2 x i64> %ll4, <2 x i64> %ll5, <2 x i64> %ll6, <2 x i64> %ll7, <2 x i64> %ll8, <2 x i64> %ll9, <2 x i64> %ll10, <2 x i64> %ll11, <2 x i64> %ll12, <2 x i64> %ll13, <2 x i64> %ll14) { entry: %add = add <2 x i64> %ll0, %ll1 %add1 = add <2 x i64> %add, %ll2 %add2 = add <2 x i64> %add1, %ll3
+  %add3 = add <2 x i64> %add2, %ll4
+  %add4 = add <2 x i64> %add3, %ll5
+  %add5 = add <2 x i64> %add4, %ll6
+  %add6 = add <2 x i64> %add5, %ll7
+  %add7 = add <2 x i64> %add6, %ll8
+  %add8 = add <2 x i64> %add7, %ll9
+  %add9 = add <2 x i64> %add8, %ll10
+  %add10 = add <2 x i64> %add9, %ll11
+  %add11 = add <2 x i64> %add10, %ll12
+  %add12 = add <2 x i64> %add11, %ll13
+  %add13 = add <2 x i64> %add12, %ll14
+  ret <2 x i64> %add13
+}
+
+; CHECKASM-LABEL:  .longint:
+
+; ASM32-DAG:       stwu 1, -336(1)
+; ASM32-DAG:       addi 3, 1, 112
+; ASM32-DAG:       addi 4, 1, 96
+; ASM32-DAG:       addi 8, 1, 128
+; ASM32-DAG:       stxvw4x 35, 0, 3
+; ASM32-DAG:       stxvw4x 34, 0, 4
+; ASM32-DAG:       stxvw4x 36, 0, 8
+; ASM32-DAG:       stxvw4x 37, 0, 9
+; ASM32-DAG:       stxvw4x 38, 0, 7
+; ASM32-DAG:       stxvw4x 39, 0, 8
+; ASM32-DAG:       stxvw4x 40, 0, 7
+; ASM32-DAG:       stxvw4x 41, 0, 8
+; ASM32-DAG:       stxvw4x 42, 0, 7
+; ASM32-DAG:       stxvw4x 43, 0, 8
+; ASM32-DAG:       stxvw4x 44, 0, 7
+; ASM32-DAG:       stxvw4x 45, 0, 8
+
+; ASM64-DAG:       stdu 1, -368(1)
+; ASM64-DAG:       xxswapd 1, 34
+; ASM64-DAG:       xxswapd 0, 35
+; ASM64-DAG:       xxswapd 2, 36
+; ASM64-DAG:       xxswapd 0, 37
+; ASM64-DAG:       xxswapd 1, 38
+; ASM64-DAG:       xxswapd 0, 39
+; ASM64-DAG:       xxswapd 1, 40
+; ASM64-DAG:       xxswapd 0, 41
+; ASM64-DAG:       xxswapd 1, 42
+; ASM64-DAG:       xxswapd 2, 43
+; ASM64-DAG:       xxswapd 3, 44
+; ASM64-DAG:       xxswapd 2, 45
+; ASM64-DAG:       xxswapd 34, 0
+; ASM64-DAG:       addi 3, 1, 64
+; ASM64-DAG:       blr
+
+define dso_local <4 x float> @float(<4 x float> %vf0, <4 x float> %vf1, <4 x float> %vf2, <4 x float> %vf3, <4 x float> %vf4, <4 x float> %vf5, <4 x float> %vf6, <4 x float> %vf7, <4 x float> %vf8, <4 x float> %vf9, <4 x float> %vf10, <4 x float> %vf11, <4 x float> %vf12, <4 x float> %vf13, <4 x float> %vf14) {
+entry:
+  %add = fadd <4 x float> %vf0, %vf1
+  %add1 = fadd <4 x float> %add, %vf2
+  %add2 = fadd <4 x float> %add1, %vf3
+  %add3 = fadd <4 x float> %add2, %vf4
+  %add4 = fadd <4 x float> %add3, %vf5
+  %add5 = fadd <4 x float> %add4, %vf6
+  %add6 = fadd <4 x float> %add5, %vf7
+  %add7 = fadd <4 x float> %add6, %vf8
+  %add8 = fadd <4 x float> %add7, %vf9
+  %add9 = fadd <4 x float> %add8, %vf10
+  %add10 = fadd <4 x float> %add9, %vf11
+  %add11 = fadd <4 x float> %add10, %vf12
+  %add12 = fadd <4 x float> %add11, %vf13
+  %add13 = fadd <4 x float> %add12, %vf14
+  ret <4 x float> %add13
+}
+
+; CHECKASM-LABEL:  .float:
+; ASM32-DAG:       addi 3, 1, 224
+; ASM32-DAG:       xvaddsp 0, 34, 35
+; ASM32-DAG:       lxvw4x 1, 0, 3
+; ASM32-DAG:       addi 3, 1, 240
+; ASM32-DAG:       xvaddsp 0, 0, 36
+; ASM32-DAG:       xvaddsp 0, 0, 37
+; ASM32-DAG:       xvaddsp 0, 0, 38
+; ASM32-DAG:       xvaddsp 0, 0, 39
+; ASM32-DAG:       xvaddsp 0, 0, 40
+; ASM32-DAG:       xvaddsp 0, 0, 41
+; ASM32-DAG:       xvaddsp 0, 0, 42
+; ASM32-DAG:       xvaddsp 0, 0, 43
+; ASM32-DAG:       xvaddsp 0, 0, 44
+; ASM32-DAG:       xvaddsp 0, 0, 45
+; ASM32-DAG:       xvaddsp 0, 0, 1
+; ASM32-DAG:       lxvw4x 1, 0, 3
+; ASM32-DAG:       addi 3, 1, 256
+; ASM32-DAG:       xvaddsp 0, 0, 1
+; ASM32-DAG:       lxvw4x 1, 0, 3
+; ASM32-DAG:       xvaddsp 34, 0, 1
+; ASM32-DAG:       blr
+
+; ASM64-DAG:       xvaddsp 0, 34, 35
+; ASM64-DAG:       addi 3, 1, 240
+; ASM64-DAG:       lxvw4x 1, 0, 3
+; ASM64-DAG:       addi 3, 1, 256
+; ASM64-DAG:       xvaddsp 0, 0, 36
+; ASM64-DAG:       xvaddsp 0, 0, 37
+; ASM64-DAG:       xvaddsp 0, 0, 38
+; ASM64-DAG:       xvaddsp 0, 0, 39
+; ASM64-DAG:       xvaddsp 0, 0, 40
+; ASM64-DAG:       xvaddsp 0, 0, 41
+; ASM64-DAG:       xvaddsp 0, 0, 42
+; ASM64-DAG:       xvaddsp 0, 0, 43
+; ASM64-DAG:       xvaddsp 0, 0, 44
+; ASM64-DAG:       xvaddsp 0, 0, 45
+; ASM64-DAG:       xvaddsp 0, 0, 1
+; ASM64-DAG:       lxvw4x 1, 0, 3
+; ASM64-DAG:       addi 3, 1, 272
+; ASM64-DAG:       xvaddsp 0, 0, 1
+; ASM64-DAG:       lxvw4x 1, 0, 3
+; ASM64-DAG:       xvaddsp 34, 0, 1
+; ASM64-DAG:       blr
+
+define dso_local <2 x double> @double(<2 x double> %double0, <2 x double> %double1, <2 x double> %double2, <2 x double> %double3, <2 x double> %double4, <2 x double> %double5, <2 x double> %double6, <2 x double> %double7, <2 x double> %double8, <2 x double> %double9, <2 x double> %double10, <2 x double> %double11, <2 x double> %double12, <2 x double> %double13, <2 x double> %double14) {
+entry:
+  %add = fadd <2 x double> %double0, %double1
+  %add1 = fadd <2 x double> %add, %double2
+  %add2 = fadd <2 x double> %add1, %double3
+  %add3 = fadd <2 x double> %add2, %double4
+  %add4 = fadd <2 x double> %add3, %double5
+  %add5 = fadd <2 x double> %add4, %double6
+  %add6 = fadd <2 x double> %add5, %double7
+  %add7 = fadd <2 x double> %add6, %double8
+  %add8 = fadd <2 x double> %add7, %double9
+  %add9 = fadd <2 x double> %add8, %double10
+  %add10 = fadd <2 x double> %add9, %double11
+  %add11 = fadd <2 x double> %add10, %double12
+  %add12 = fadd <2 x double> %add11, %double13
+  %add13 = fadd <2 x double> %add12, %double14
+  ret <2 x double> %add13
+}
+
+; CHECKASM-LABEL:  .double:
+
+; ASM32-DAG:       xvadddp 0, 34, 35
+; ASM32-DAG:       addi 3, 1, 224
+; ASM32-DAG:       lxvd2x 1, 0, 3
+; ASM32-DAG:       addi 3, 1, 240
+; ASM32-DAG:       lxvd2x 2, 0, 3
+; ASM32-DAG:       addi 3, 1, 256
+; ASM32-DAG:       xxswapd  1, 1
+; ASM32-DAG:       xvadddp 0, 0, 36
+; ASM32-DAG:       xvadddp 0, 0, 37
+; ASM32-DAG:       xvadddp 0, 0, 38
+; ASM32-DAG:       xvadddp 0, 0, 39
+; ASM32-DAG:       xvadddp 0, 0, 40
+; ASM32-DAG:       xvadddp 0, 0, 41
+; ASM32-DAG:       xvadddp 0, 0, 42
+; ASM32-DAG:       xvadddp 0, 0, 43
+; ASM32-DAG:       xvadddp 0, 0, 44
+; ASM32-DAG:       xvadddp 0, 0, 45
+; ASM32-DAG:       xvadddp 0, 0, 1
+; ASM32-DAG:       xxswapd  1, 2
+; ASM32-DAG:       lxvd2x 2, 0, 3
+; ASM32-DAG:       xvadddp 0, 0, 1
+; ASM32-DAG:       xxswapd  1, 2
+; ASM32-DAG:       xvadddp 34, 0, 1
+; ASM32-DAG:       blr
+ 
+; ASM64-DAG:       xvadddp 0, 34, 35
+; ASM64-DAG:       addi 3, 1, 240
+; ASM64-DAG:       lxvd2x 1, 0, 3
+; ASM64-DAG:       addi 3, 1, 256
+; ASM64-DAG:       xvadddp 0, 0, 36
+; ASM64-DAG:       xvadddp 0, 0, 37
+; ASM64-DAG:       xvadddp 0, 0, 38
+; ASM64-DAG:       xvadddp 0, 0, 39
+; ASM64-DAG:       xvadddp 0, 0, 40
+; ASM64-DAG:       xvadddp 0, 0, 41
+; ASM64-DAG:       xvadddp 0, 0, 42
+; ASM64-DAG:       xvadddp 0, 0, 43
+; ASM64-DAG:       xvadddp 0, 0, 44
+; ASM64-DAG:       xvadddp 0, 0, 45
+; ASM64-DAG:       xvadddp 0, 0, 1
+; ASM64-DAG:       lxvd2x 2, 0, 3
+; ASM64-DAG:       addi 3, 1, 272
+; ASM64-DAG:       xvadddp 0, 0, 1
+; ASM64-DAG:       lxvd2x 2, 0, 3
+; ASM64-DAG:       xvadddp 34, 0, 1
+; ASM64-DAG:       blr