[PowerPC][AIX] Implement by-val caller arguments in multiple registers

[cebowleratibm]

Summary

Ths patch builds upon commit rGc21866476e14, which added support for by-value arguments in a single register. With this patch the by-value argument support on AIX is extended to by-value arguments that will fit within available GPRs.

I've moved the 5-8 byte tests, which were aix64 specific into aix-cc-byval.ll because they now work as multi GPR tests in 32-bit. I've added a 64-byte by-value argument as an aix64 only test. When the caller by-value support is complete, the aix-cc-byval-limitation tests will be deleted and the aix64-cc-byval.ll tests should be folded into aix-cc-byval.ll.

Remaining caller by-value argument work:
-by-val args, which require alignment greater than PtrByteSize.
-by-val args, which pass either partially or completely in stack memory.

[sfertile]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

7151

If all we need PeekVA for is its ValNo lets do unsigned ValNo = ArgLocs[I].getValNo()

7166

We do this same operation twice in this function, and the same pattern appears several times elsewhere in the file. I suggest putting it in a well named helper function. Only bother using for the 2 uses in this function for this patch, then we can convert some of the other uses in NFC patches.

7167

If we are setting the no-wrap flags maybe we should set both.

llvm/test/CodeGen/PowerPC/aix-cc-byval.ll

208

I don't see a lot of utility in testing each of 1 through 8 bytes with nearly exactly the same test. The are each so similar that we end up with a long verbose test with not a lot of practical coverage. Instead how about we add

1. one of 5,6,7,8 bytes but with other arguments before/after the by val.
2. A test with a couple byvals mixed with other arguments.
3. One test with a 'large' by val like { [32 x i8] }.

Also use a struct type that isn't just containing an array. Ex if you choose 8 bytes as the size for the test in case 1 use: { i16, i16, i16, i16} as the struct type.

[cebowleratibm]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

7167

Are you referring to setNoSignedWrap?

Digging a bit more, I think we can use SelectionDAG::getObjectPtrOffset

llvm/test/CodeGen/PowerPC/aix-cc-byval.ll

208

1 and 2. I agree.

3. I already have 31, 32 and 64 byte tests.

I'll change the struct layout in some of the tests.

[cebowleratibm]

I still have to update the tests as per Sean's suggestion but I'll post the updated code for now.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

7166

Fair point. I've opted to use a lambda to common this up.

7167

getObjectPtrOffset doesn't setNoSignedWrap but if that's suboptimal I think that's another patch entirely.

[cebowleratibm]

Updated code to address comments.

Test updates still required.

[cebowleratibm]

I split out some of the suggested test improvements to https://reviews.llvm.org/D76875

The patch has been rebased on https://reviews.llvm.org/D76401 ([PowerPC][AIX] ByVal formal argument support: single register.)

Note I've moved the 5-8 byte byval caller tests to aix-cc-byval.ll and left the 5-8 byte byval callee tests in aix64-cc-byval.ll.

I fixed a bug on the byval AllocateStack call in CC_AIX, which needs to reserve ByValSize bytes, rounded up to a multiple of PtrByteSize.

[cebowleratibm]

llvm/test/CodeGen/PowerPC/aix-cc-byval.ll

208

1. is addressed in https://reviews.llvm.org/D76875
2. Was already addressed in https://reviews.llvm.org/D76401 (thanks Sean)
3. This patch has tests for 31 and 32 byte byvals.

[cebowleratibm]

Forgot the 64 byte byval test.

[cebowleratibm]

Clang format.

[sfertile]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

7157

Real minor nit: Pass the Offset in as an argument.

[cebowleratibm]

Tweaked the lambda capture on GetLoad.

I rewrote some of the changes in preparation for the next commit I have planned for caller by-value arguments in stack memory. The changes in this diff should be NFC from the last revision.

[sfertile]

I suggest adding a test along the lines of:

struct F {
  float x, y, z;
};

int callee(struct F);

int caller(void) {
  struct F s = { 0.0f, 0.0f, 0.0f };
  return callee(s);
}

1. To show that homogeneous float aggregates are still passed in gprs.
2. So that not all the tests load a global value for the argument.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

7158

Are 'I != E' and ArgLocs[I].getValNo() == ValNo not both implied by LoadOffset + PtrByteSize <= ByValSize? They are probably better as assertions inside the loop body.

[cebowleratibm]

Added a test for the homogeneous float stuct Sean suggested.

Also made an NFCI change that moved part of a loop condition to an assertion of the loop body.

[ZarkoCA]

llvm/test/CodeGen/PowerPC/aix64-cc-byval.ll

53

I don't see a 64BIT check prefix in this file. I think these should be CHECK instead.

[cebowleratibm]

llvm/test/CodeGen/PowerPC/aix64-cc-byval.ll

53

Good catch. I'll also add the missing assembly expected output.

[cebowleratibm]

Fixed expected output for 64-byte byval arg test.

[sfertile]

LGTM.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

static bool CC_AIX(unsigned ValNo, MVT ValVT, MVT LocVT,
 
   if (ArgFlags.isByVal()) {
     if (ArgFlags.getNonZeroByValAlign() > PtrByteSize)
       report_fatal_error("Pass-by-value arguments with alignment greater than "
                          "register width are not supported.");
 
     const unsigned ByValSize = ArgFlags.getByValSize();
 
-    // An empty aggregate parameter takes up no storage and no registers.
+    // Empty aggregate parameters take no storage and no registers.
     if (ByValSize == 0)
       return false;
 
-    if (ByValSize <= PtrByteSize) {
-      State.AllocateStack(PtrByteSize, PtrByteSize);
-      if (unsigned Reg = State.AllocateReg(IsPPC64 ? GPR_64 : GPR_32)) {
-        State.addLoc(CCValAssign::getReg(ValNo, ValVT, Reg, RegVT, LocInfo));
-        return false;
-      }
-    }
+    State.AllocateStack(ByValSize, PtrByteSize);
 
+    for (unsigned I = 0, E = ByValSize; I < E; I += PtrByteSize) {
+      if (unsigned Reg = State.AllocateReg(IsPPC64 ? GPR_64 : GPR_32))
+        State.addLoc(CCValAssign::getReg(ValNo, ValVT, Reg, RegVT, LocInfo));
+      else
         report_fatal_error(
-        "Pass-by-value arguments are only supported in a single register.");
+            "Pass-by-value arguments are only supported in registers.");
+    }
+    return false;
   }
 
   // Arguments always reserve parameter save area.
   switch (ValVT.SimpleTy) {
   default:
     report_fatal_error("Unhandled value type for argument.");
   case MVT::i64:
     // i64 arguments should have been split to i32 for PPC32.
SDValue PPCTargetLowering::LowerCall_AIX(
 
   // Set up a copy of the stack pointer for loading and storing any
   // arguments that may not fit in the registers available for argument
   // passing.
   const SDValue StackPtr = IsPPC64 ? DAG.getRegister(PPC::X1, MVT::i64)
                                    : DAG.getRegister(PPC::R1, MVT::i32);
 
   for (unsigned I = 0, E = ArgLocs.size(); I != E;) {
-    CCValAssign &VA = ArgLocs[I++];
-
-    SDValue Arg = OutVals[VA.getValNo()];
-    ISD::ArgFlagsTy Flags = Outs[VA.getValNo()].Flags;
-    const MVT LocVT = VA.getLocVT();
-    const MVT ValVT = VA.getValVT();
+    const CCValAssign &PeekVA = ArgLocs[I];
+    unsigned ValNo = PeekVA.getValNo();
+    SDValue Arg = OutVals[ValNo];
+    ISD::ArgFlagsTy Flags = Outs[ValNo].Flags;
 
     if (Flags.isByVal()) {
       const unsigned ByValSize = Flags.getByValSize();
-      assert(
-          VA.isRegLoc() && ByValSize > 0 && ByValSize <= PtrByteSize &&
-          "Pass-by-value arguments are only supported in a single register.");
+      assert(ByValSize > 0 && "Unexpected 0 size pass-by-value argument.");
 
-      // Loads must be a power-of-2 size and cannot be larger than the
-      // ByValSize. For example: a 7 byte by-val arg requires 4, 2 and 1 byte
-      // loads.
-      SDValue RegVal;
-      for (unsigned Bytes = 0; Bytes != ByValSize;) {
-        unsigned N = PowerOf2Floor(ByValSize - Bytes);
+      // Initialize registers, which are fully occupied by the by-val argument.
+      unsigned LoadOffset = 0;
+      for (unsigned E = ((ByValSize / PtrByteSize) * PtrByteSize);
+           LoadOffset != E; LoadOffset += PtrByteSize) {
+        SDValue LoadAddr = Arg;
+        if (LoadOffset != 0) {
+          // Adjust the load offset by the number of bytes read so far.
+          SDNodeFlags AddFlags;
+          AddFlags.setNoUnsignedWrap(true);
+          LoadAddr =
+              DAG.getNode(ISD::ADD, dl, PtrVT, Arg,
+                          DAG.getConstant(LoadOffset, dl, PtrVT), AddFlags);
+        }
+        SDValue Load = DAG.getExtLoad(ISD::EXTLOAD, dl, PtrVT, Chain, LoadAddr,
+                                      MachinePointerInfo(), PtrVT);
+        MemOpChains.push_back(Load.getValue(1));
+        const CCValAssign &ByValVA = ArgLocs[I++];
+        assert(ByValVA.isRegLoc() && ByValVA.getValNo() == PeekVA.getValNo() &&
+               "Unexpected location for pass-by-value argument.");
+        RegsToPass.push_back(std::make_pair(ByValVA.getLocReg(), Load));
+      }
+
+      // Initialize the a final register if there's any residue.
+      // Any residue that occupies the final by-val arg register must be
+      // left-justified on AIX. Loads must be a power-of-2 size and cannot be
+      // larger than the ByValSize. For example: a 7 byte by-val arg requires 4,
+      // 2 and 1 byte loads.
+      SDValue ResidueVal;
+      const unsigned ResidueBytes = ByValSize % PtrByteSize;
+      if (ResidueBytes == 0)
+        continue;
+
+      for (unsigned Bytes = 0; Bytes != ResidueBytes;) {
+        const unsigned N = PowerOf2Floor(ResidueBytes - Bytes);
         const MVT VT =
             N == 1 ? MVT::i8
                    : ((N == 2) ? MVT::i16 : (N == 4 ? MVT::i32 : MVT::i64));
 
         SDValue LoadAddr = Arg;
-        if (Bytes != 0) {
+        if (LoadOffset + Bytes != 0) {
           // Adjust the load offset by the number of bytes read so far.
-          SDNodeFlags Flags;
-          Flags.setNoUnsignedWrap(true);
-          LoadAddr = DAG.getNode(ISD::ADD, dl, LocVT, Arg,
-                                 DAG.getConstant(Bytes, dl, LocVT), Flags);
+          SDNodeFlags AddFlags;
+          AddFlags.setNoUnsignedWrap(true);
+          LoadAddr = DAG.getNode(ISD::ADD, dl, PtrVT, Arg,
+                                 DAG.getConstant(LoadOffset + Bytes, dl, PtrVT),
+                                 AddFlags);
         }
         SDValue Load = DAG.getExtLoad(ISD::ZEXTLOAD, dl, PtrVT, Chain, LoadAddr,
                                       MachinePointerInfo(), VT);
         MemOpChains.push_back(Load.getValue(1));
 
         Bytes += N;
-        assert(LocVT.getSizeInBits() >= (Bytes * 8));
-        if (unsigned NumSHLBits = LocVT.getSizeInBits() - (Bytes * 8)) {
+        assert(PtrVT.getSimpleVT().getSizeInBits() >= (Bytes * 8) &&
+               "Unexpected load emitted during handling of pass-by-value "
+               "argument.");
+        if (unsigned NumSHLBits =
+                PtrVT.getSimpleVT().getSizeInBits() - (Bytes * 8)) {
           // By-val arguments are passed left-justfied in register.
           EVT ShiftAmountTy =
               getShiftAmountTy(Load->getValueType(0), DAG.getDataLayout());
           SDValue SHLAmt = DAG.getConstant(NumSHLBits, dl, ShiftAmountTy);
           SDValue ShiftedLoad =
               DAG.getNode(ISD::SHL, dl, Load.getValueType(), Load, SHLAmt);
-          RegVal = RegVal ? DAG.getNode(ISD::OR, dl, LocVT, RegVal, ShiftedLoad)
+          ResidueVal = ResidueVal ? DAG.getNode(ISD::OR, dl, PtrVT, ResidueVal,
+                                                ShiftedLoad)
                                   : ShiftedLoad;
         } else {
-          assert(!RegVal && Bytes == ByValSize &&
+          assert(!ResidueVal && Bytes == ByValSize &&
                  "Pass-by-value argument handling unexpectedly incomplete.");
-          RegVal = Load;
+          ResidueVal = Load;
         }
       }
 
-      RegsToPass.push_back(std::make_pair(VA.getLocReg(), RegVal));
+      const CCValAssign &ResidueByValVA = ArgLocs[I++];
+      RegsToPass.push_back(
+          std::make_pair(ResidueByValVA.getLocReg(), ResidueVal));
       continue;
     }
 
+    CCValAssign &VA = ArgLocs[I++];
+    const MVT LocVT = VA.getLocVT();
+    const MVT ValVT = VA.getValVT();
+
     switch (VA.getLocInfo()) {
     default:
       report_fatal_error("Unexpected argument extension type.");
     case CCValAssign::Full:
       break;
     case CCValAssign::ZExt:
       Arg = DAG.getNode(ISD::ZERO_EXTEND, dl, VA.getLocVT(), Arg);
       break;

llvm/test/CodeGen/PowerPC/aix-cc-byval-limitation1.ll

 ; RUN: not --crash llc -mtriple powerpc-ibm-aix-xcoff < %s 2>&1 | FileCheck %s
 ; RUN: not --crash llc -mtriple powerpc64-ibm-aix-xcoff < %s 2>&1 | FileCheck %s
 
-%struct.S = type { [9 x i8] }
+%struct.S = type { [65 x i8] }
 
 define void @bar() {
 entry:
   %s1 = alloca %struct.S, align 1
   %agg.tmp = alloca %struct.S, align 1
   call void @foo(%struct.S* byval(%struct.S) align 1 %agg.tmp)
   ret void
 }
 
 declare void @foo(%struct.S* byval(%struct.S) align 1)
 
-; CHECK: LLVM ERROR: Pass-by-value arguments are only supported in a single register.
+; CHECK: LLVM ERROR: Pass-by-value arguments are only supported in registers.

llvm/test/CodeGen/PowerPC/aix-cc-byval-limitation2.ll

 ; RUN: not --crash llc -mtriple powerpc-ibm-aix-xcoff < %s 2>&1 | FileCheck %s
 ; RUN: not --crash llc -mtriple powerpc64-ibm-aix-xcoff < %s 2>&1 | FileCheck %s
 
 %struct.S = type { [1 x i8] }
 
 define void @bar() {
 entry:
   %s1 = alloca %struct.S, align 1
   %agg.tmp = alloca %struct.S, align 1
   call void @foo(i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8, %struct.S* byval(%struct.S) align 1 %agg.tmp)
   ret void
 }
 
 declare void @foo(i32, i32, i32, i32, i32, i32, i32, i32, %struct.S* byval(%struct.S) align 1)
 
-; CHECK: LLVM ERROR: Pass-by-value arguments are only supported in a single register.
+; CHECK: LLVM ERROR: Pass-by-value arguments are only supported in registers.

llvm/test/CodeGen/PowerPC/aix-cc-byval.ll

 
 ; ASM64PWR4:       stdu 1, -112(1)
 ; ASM64PWR4-NEXT:  ld [[REG:[0-9]+]], LC{{[0-9]+}}(2)
 ; ASM64PWR4-NEXT:  lwz 3, 0([[REG]])
 ; ASM64PWR4-NEXT:  sldi 3, 3, 32
 ; ASM64PWR4-NEXT:  bl .test_byval_4Byte
 ; ASM64PWR4-NEXT:  nop
 ; ASM64PWR4-NEXT:  addi 1, 1, 112
+
+%struct.S5 = type { [5 x i8] }
+
+@gS5 = external global %struct.S5, align 1
+
+define void @call_test_byval_5Byte() {
+entry:
+  call void @test_byval_5Byte(%struct.S5* byval(%struct.S5) align 1 @gS5)
+  ret void
+}
+
+declare void @test_byval_5Byte(%struct.S5* byval(%struct.S5) align 1)
+
+; CHECK-LABEL: name: call_test_byval_5Byte{{.*}}
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 32BIT:       ADJCALLSTACKDOWN 56, 0, implicit-def dead $r1, implicit $r1
+; 32BIT-NEXT:  renamable $r[[REGADDR:[0-9]+]] = LWZtoc @gS5, $r2 :: (load 4 from got)
+; 32BIT-DAG:   renamable $r[[REG1:[0-9]+]] = LBZ 4, renamable $r[[REGADDR]] :: (load 1)
+; 32BIT-DAG:   renamable $r3 = LWZ 0, killed renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r4 = RLWINM killed renamable $r[[REG1]], 24, 0, 7
+; 32BIT-NEXT:  BL_NOP <mcsymbol .test_byval_5Byte>, csr_aix32, implicit-def dead $lr, implicit $rm, implicit $r3, implicit $r4, implicit $r2, implicit-def $r1
+; 32BIT-NEXT:   ADJCALLSTACKUP 56, 0, implicit-def dead $r1, implicit $r1
+
+; CHECKASM-LABEL: .call_test_byval_5Byte:
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM32PWR4:       stwu 1, -64(1)
+; ASM32PWR4-NEXT:  lwz [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM32PWR4-DAG:   lbz [[REG1:[0-9]+]], 4([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 3, 0([[REGADDR]])
+; ASM32PWR4-DAG:   slwi 4, [[REG1]], 24
+; ASM32PWR4-NEXT:  bl .test_byval_5Byte
+; ASM32PWR4-NEXT:  nop
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
+; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS5, $x2 :: (load 8 from got)
+; 64BIT-DAG:   renamable $x[[REG1:[0-9]+]] = LWZ8 0, killed renamable $x[[REGADDR]] :: (load 4)
+; 64BIT-DAG:   renamable $x[[REG2:[0-9]+]] = LBZ8 4, renamable $x[[REGADDR]] :: (load 1)
+; 64BIT-DAG:   renamable $x3 = RLWINM8 killed renamable $x[[REG2]], 24, 0, 7
+; 64BIT-DAG:   renamable $x3 = RLDIMI killed renamable $x3, killed renamable $x[[REG1]], 32, 0
+; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_5Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
+; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM64PWR4:       stdu 1, -112(1)
+; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM64PWR4-DAG:   lwz [[REG1:[0-9]+]], 0([[REGADDR]])
+; ASM64PWR4-DAG:   lbz [[REG2:[0-9]+]], 4([[REGADDR]])
+; ASM64PWR4-DAG:   rlwinm 3, [[REG2]], 24, 0, 7
+; ASM64PWR4-DAG:   rldimi 3, [[REG1]], 32, 0
+; ASM64PWR4-NEXT:  bl .test_byval_5Byte
+; ASM64PWR4-NEXT:  nop
+
+%struct.S6 = type { [6 x i8] }
+
+@gS6 = external global %struct.S6, align 1
+
+define void @call_test_byval_6Byte() {
+entry:
+  call void @test_byval_6Byte(%struct.S6* byval(%struct.S6) align 1 @gS6)
+  ret void
+}
+
+declare void @test_byval_6Byte(%struct.S6* byval(%struct.S6) align 1)
+
+; CHECK-LABEL: name: call_test_byval_6Byte{{.*}}
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 32BIT:       ADJCALLSTACKDOWN 56, 0, implicit-def dead $r1, implicit $r1
+; 32BIT-NEXT:  renamable $r[[REGADDR:[0-9]+]] = LWZtoc @gS6, $r2 :: (load 4 from got)
+; 32BIT-DAG:   renamable $r[[REG1:[0-9]+]] = LHZ 4, renamable $r[[REGADDR]] :: (load 2)
+; 32BIT-DAG:   renamable $r3 = LWZ 0, killed renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r4 = RLWINM killed renamable $r[[REG1]], 16, 0, 15
+; 32BIT-NEXT:  BL_NOP <mcsymbol .test_byval_6Byte>, csr_aix32, implicit-def dead $lr, implicit $rm, implicit $r3, implicit $r4, implicit $r2, implicit-def $r1
+; 32BIT-NEXT:  ADJCALLSTACKUP 56, 0, implicit-def dead $r1, implicit $r1
+
+; CHECKASM-LABEL: .call_test_byval_6Byte:
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM32PWR4:       stwu 1, -64(1)
+; ASM32PWR4-NEXT:  lwz [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM32PWR4-DAG:   lhz [[REG1:[0-9]+]], 4([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 3, 0([[REGADDR]])
+; ASM32PWR4-DAG:   slwi 4, [[REG1]], 16
+; ASM32PWR4-NEXT:  bl .test_byval_6Byte
+; ASM32PWR4-NEXT:  nop
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
+; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS6, $x2 :: (load 8 from got)
+; 64BIT-DAG:   renamable $x[[REG1:[0-9]+]] = LWZ8 0, killed renamable $x[[REGADDR]] :: (load 4)
+; 64BIT-DAG:   renamable $x[[REG2:[0-9]+]] = LHZ8 4, renamable $x[[REGADDR]] :: (load 2)
+; 64BIT-DAG:   renamable $x3 = RLWINM8 killed renamable $x[[REG2]], 16, 0, 15
+; 64BIT-DAG:   renamable $x3 = RLDIMI killed renamable $x3, killed renamable $x[[REG1]], 32, 0
+; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_6Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
+; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM64PWR4:       stdu 1, -112(1)
+; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM64PWR4-DAG:   lwz [[REG1:[0-9]+]], 0([[REGADDR]])
+; ASM64PWR4-DAG:   lhz [[REG2:[0-9]+]], 4([[REGADDR]])
+; ASM64PWR4-DAG:   rlwinm 3, [[REG2]], 16, 0, 15
+; ASM64PWR4-DAG:   rldimi 3, [[REG1]], 32, 0
+; ASM64PWR4-NEXT:  bl .test_byval_6Byte
+; ASM64PWR4-NEXT:  nop
+
+%struct.S7 = type { [7 x i8] }
+
+@gS7 = external global %struct.S7, align 1
+
+define void @call_test_byval_7Byte() {
+entry:
+  call void @test_byval_7Byte(%struct.S7* byval(%struct.S7) align 1 @gS7)
+  ret void
+}
+
+declare void @test_byval_7Byte(%struct.S7* byval(%struct.S7) align 1)
+
+; CHECK-LABEL: name: call_test_byval_7Byte{{.*}}
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 32BIT:       ADJCALLSTACKDOWN 56, 0, implicit-def dead $r1, implicit $r1
+; 32BIT-NEXT:  renamable $r[[REGADDR:[0-9]+]] = LWZtoc @gS7, $r2 :: (load 4 from got)
+; 32BIT-DAG:   renamable $r3 = LWZ 0, killed renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r[[REG1:[0-9]+]] = LHZ 4, renamable $r[[REGADDR]] :: (load 2)
+; 32BIT-DAG:   renamable $r[[REG2:[0-9]+]] = LBZ 6, renamable $r[[REGADDR]] :: (load 1)
+; 32BIT-DAG:   renamable $r4 = RLWINM killed renamable $r[[REG2]], 8, 16, 23
+; 32BIT-DAG:   renamable $r4 = RLWIMI killed renamable $r4, killed renamable $r[[REG1]], 16, 0, 15
+; 32BIT-NEXT:  BL_NOP <mcsymbol .test_byval_7Byte>, csr_aix32, implicit-def dead $lr, implicit $rm, implicit $r3, implicit $r4, implicit $r2, implicit-def $r1
+; 32BIT-NEXT:  ADJCALLSTACKUP 56, 0, implicit-def dead $r1, implicit $r1
+
+; CHECKASM-LABEL: .call_test_byval_7Byte:
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM32PWR4:       stwu 1, -64(1)
+; ASM32PWR4-NEXT:  lwz [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM32PWR4-DAG:   lwz 3, 0([[REGADDR]])
+; ASM32PWR4-DAG:   lhz [[REG1:[0-9]+]], 4([[REGADDR]])
+; ASM32PWR4-DAG:   lbz [[REG2:[0-9]+]], 6([[REGADDR]])
+; ASM32PWR4-DAG:   rlwinm 4, [[REG2]], 8, 16, 23
+; ASM32PWR4-DAG:   rlwimi 4, [[REG1]], 16, 0, 15
+; ASM32PWR4-NEXT:  bl .test_byval_7Byte
+; ASM32PWR4-NEXT:  nop
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
+; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS7, $x2 :: (load 8 from got)
+; 64BIT-DAG:   renamable $x[[REG1:[0-9]+]] = LWZ8 0, killed renamable $x[[REGADDR]] :: (load 4)
+; 64BIT-DAG:   renamable $x[[REG2:[0-9]+]] = LHZ8 4, renamable $x[[REGADDR]] :: (load 2)
+; 64BIT-DAG:   renamable $x[[REG3:[0-9]+]] = LBZ8 6, renamable $x[[REGADDR]] :: (load 1)
+; 64BIT-DAG:   renamable $x3 = RLWINM8 killed renamable $x[[REG3]], 8, 16, 23
+; 64BIT-DAG:   renamable $x3 = RLWIMI8 killed renamable $x3, killed renamable $x[[REG2]], 16, 0, 15
+; 64BIT-DAG:   renamable $x3 = RLDIMI killed renamable $x3, killed renamable $x[[REG1]], 32, 0
+; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_7Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
+; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM64PWR4:       stdu 1, -112(1)
+; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM64PWR4-DAG:   lwz [[REG1:[0-9]+]], 0([[REGADDR]])
+; ASM64PWR4-DAG:   lhz [[REG2:[0-9]+]], 4([[REGADDR]])
+; ASM64PWR4-DAG:   lbz [[REG3:[0-9]+]], 6([[REGADDR]])
+; ASM64PWR4-DAG:   rlwinm 3, [[REG3]], 8, 16, 23
+; ASM64PWR4-DAG:   rlwimi 3, [[REG2]], 16, 0, 15
+; ASM64PWR4-DAG:   rldimi 3, [[REG1]], 32, 0
+; ASM64PWR4-NEXT:  bl .test_byval_7Byte
+; ASM64PWR4-NEXT:  nop
+
+%struct.S8 = type { [8 x i8] }
+
+@gS8 = external global %struct.S8, align 1
+
+define void @call_test_byval_8Byte() {
+entry:
+  call void @test_byval_8Byte(%struct.S8* byval(%struct.S8) align 1 @gS8)
+  ret void
+}
+
+declare void @test_byval_8Byte(%struct.S8* byval(%struct.S8) align 1)
+
+; CHECK-LABEL: name: call_test_byval_8Byte{{.*}}
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 32BIT:       ADJCALLSTACKDOWN 56, 0, implicit-def dead $r1, implicit $r1
+; 32BIT-NEXT:  renamable $r[[REGADDR:[0-9]+]] = LWZtoc @gS8, $r2 :: (load 4 from got)
+; 32BIT-DAG:   renamable $r3 = LWZ 0, killed renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r4 = LWZ 4, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-NEXT:  BL_NOP <mcsymbol .test_byval_8Byte>, csr_aix32, implicit-def dead $lr, implicit $rm, implicit $r3, implicit $r4, implicit $r2, implicit-def $r1
+; 32BIT-NEXT:  ADJCALLSTACKUP 56, 0, implicit-def dead $r1, implicit $r1
+
+; CHECKASM-LABEL: .call_test_byval_8Byte:
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM32PWR4:       stwu 1, -64(1)
+; ASM32PWR4-NEXT:  lwz [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM32PWR4-DAG:   lwz 3, 0([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 4, 4([[REGADDR]])
+; ASM32PWR4-NEXT:  bl .test_byval_8Byte
+; ASM32PWR4-NEXT:  nop
+
+; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
+; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS8, $x2 :: (load 8 from got)
+; 64BIT-NEXT:  renamable $x3 = LD 0, killed renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_8Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
+; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
+
+; ASM64PWR4:       stdu 1, -112(1)
+; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM64PWR4-NEXT:  ld 3, 0([[REGADDR]])
+; ASM64PWR4-NEXT:  bl .test_byval_8Byte
+; ASM64PWR4-NEXT:  nop
+
+%struct.S32 = type { [32 x i8] }
+
+@gS32 = external global %struct.S32, align 1
+
+define void @call_test_byval_32Byte() {
+entry:
+  call void @test_byval_32Byte(%struct.S32* byval(%struct.S32) align 1 @gS32)
+  ret void
+}
+
+declare void @test_byval_32Byte(%struct.S32* byval(%struct.S32) align 1)
+
+; CHECK-LABEL: name: call_test_byval_32Byte{{.*}}
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 32BIT:       ADJCALLSTACKDOWN 56, 0, implicit-def dead $r1, implicit $r1
+; 32BIT-NEXT:  renamable $r[[REGADDR:[0-9]+]] = LWZtoc @gS32, $r2 :: (load 4 from got)
+; 32BIT-DAG:   renamable $r3 = LWZ 0, killed renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r4 = LWZ 4, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r5 = LWZ 8, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r6 = LWZ 12, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r7 = LWZ 16, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r8 = LWZ 20, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r9 = LWZ 24, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r10 = LWZ 28, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-NEXT:  BL_NOP <mcsymbol .test_byval_32Byte>, csr_aix32, implicit-def dead $lr, implicit $rm, implicit $r3, implicit $r4, implicit $r5, implicit $r6, implicit $r7, implicit $r8, implicit $r9, implicit $r10, implicit $r2, implicit-def $r1
+; 32BIT-NEXT:  ADJCALLSTACKUP 56, 0, implicit-def dead $r1, implicit $r1
+
+; CHECKASM-LABEL: .call_test_byval_32Byte:
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM32PWR4:       stwu 1, -64(1)
+; ASM32PWR4-NEXT:  lwz [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM32PWR4-DAG:   lwz 3, 0([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 4, 4([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 5, 8([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 6, 12([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 7, 16([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 8, 20([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 9, 24([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 10, 28([[REGADDR]])
+; ASM32PWR4-NEXT:  bl .test_byval_32Byte
+; ASM32PWR4-NEXT:  nop
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
+; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS32, $x2 :: (load 8 from got)
+; 64BIT-DAG:   renamable $x3 = LD 0, killed renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x4 = LD 8, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x5 = LD 16, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x6 = LD 24, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_32Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x4, implicit $x5, implicit $x6, implicit $x2, implicit-def $r1
+; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
+
+; ASM64PWR4:       stdu 1, -112(1)
+; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM64PWR4-DAG:   ld 3, 0([[REGADDR]])
+; ASM64PWR4-DAG:   ld 4, 8([[REGADDR]])
+; ASM64PWR4-DAG:   ld 5, 16([[REGADDR]])
+; ASM64PWR4-DAG:   ld 6, 24([[REGADDR]])
+; ASM64PWR4-NEXT:  bl .test_byval_32Byte
+; ASM64PWR4-NEXT:  nop
+
+%struct.S31 = type { [31 x i8] }
+
+@gS31 = external global %struct.S31, align 1
+
+define void @call_test_byval_31Byte() {
+entry:
+  call void @test_byval_31Byte(%struct.S31* byval(%struct.S31) align 1 @gS31)
+  ret void
+}
+
+declare void @test_byval_31Byte(%struct.S31* byval(%struct.S31) align 1)
+
+; CHECK-LABEL: name: call_test_byval_31Byte{{.*}}
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 32BIT:       ADJCALLSTACKDOWN 56, 0, implicit-def dead $r1, implicit $r1
+; 32BIT-NEXT:  renamable $r[[REGADDR:[0-9]+]] = LWZtoc @gS31, $r2 :: (load 4 from got)
+; 32BIT-DAG:   renamable $r3 = LWZ 0, killed renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r4 = LWZ 4, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r5 = LWZ 8, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r6 = LWZ 12, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r7 = LWZ 16, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r8 = LWZ 20, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r9 = LWZ 24, renamable $r[[REGADDR]] :: (load 4)
+; 32BIT-DAG:   renamable $r[[REG:[0-9]+]] = LHZ 28, renamable $r[[REGADDR]] :: (load 2)
+; 32BIT-DAG:   renamable $r10 = LBZ 30, renamable $r[[REGADDR]] :: (load 1)
+; 32BIT-DAG:   renamable $r10 = RLWINM killed renamable $r10, 8, 16, 23
+; 32BIT-DAG:   renamable $r10 = RLWIMI killed renamable $r10, killed renamable $r[[REG]], 16, 0, 15
+; 32BIT-NEXT:  BL_NOP <mcsymbol .test_byval_31Byte>, csr_aix32, implicit-def dead $lr, implicit $rm, implicit $r3, implicit $r4, implicit $r5, implicit $r6, implicit $r7, implicit $r8, implicit $r9, implicit $r10, implicit $r2, implicit-def $r1
+; 32BIT-NEXT:  ADJCALLSTACKUP 56, 0, implicit-def dead $r1, implicit $r1
+
+; CHECKASM-LABEL: .call_test_byval_31Byte:
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; ASM32PWR4:       stwu 1, -64(1)
+; ASM32PWR4-NEXT:  lwz [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM32PWR4-DAG:   lwz 3, 0([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 4, 4([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 5, 8([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 6, 12([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 7, 16([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 8, 20([[REGADDR]])
+; ASM32PWR4-DAG:   lwz 9, 24([[REGADDR]])
+; ASM32PWR4-DAG:   lbz 10, 30([[REGADDR]])
+; ASM32PWR4-DAG:   lhz [[REG:[0-9]+]], 28([[REGADDR]])
+; ASM32PWR4-DAG:   rlwinm 10, 10, 8, 16, 23
+; ASM32PWR4-DAG:   rlwimi 10, [[REG]], 16, 0, 15
+; ASM32PWR4-NEXT:  bl .test_byval_31Byte
+; ASM32PWR4-NEXT:  nop
+
+; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
+; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
+; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS31, $x2 :: (load 8 from got)
+; 64BIT-DAG:   renamable $x3 = LD 0, killed renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x4 = LD 8, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x5 = LD 16, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x[[REG1:[0-9]+]] = LWZ8 24, renamable $x[[REGADDR]] :: (load 4)
+; 64BIT-DAG:   renamable $x[[REG2:[0-9]+]] = LHZ8 28, renamable $x[[REGADDR]] :: (load 2)
+; 64BIT-DAG:   renamable $x[[REG3:[0-9]+]] = LBZ8 30, renamable $x[[REGADDR]] :: (load 1)
+; 64BIT-DAG:   renamable $x6 = RLWINM8 killed renamable $x[[REG3]], 8, 16, 23
+; 64BIT-DAG:   renamable $x6 = RLWIMI8 killed renamable $x6, killed renamable $x[[REG2]], 16, 0, 15
+; 64BIT-DAG:   renamable $x6 = RLDIMI killed renamable $x6, killed renamable $x[[REG1]], 32, 0
+; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_31Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x4, implicit $x5, implicit $x6, implicit $x2, implicit-def $r1
+; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
+
+; ASM64PWR4:       stdu 1, -112(1)
+; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
+; ASM64PWR4-DAG:   ld 3, 0([[REGADDR]])
+; ASM64PWR4-DAG:   ld 4, 8([[REGADDR]])
+; ASM64PWR4-DAG:   ld 5, 16([[REGADDR]])
+; ASM64PWR4-DAG:   lwz [[REG1:[0-9]+]], 24([[REGADDR]])
+; ASM64PWR4-DAG:   lhz [[REG2:[0-9]+]], 28([[REGADDR]])
+; ASM64PWR4-DAG:   lbz [[REG3:[0-9]+]], 30([[REGADDR]])
+; ASM64PWR4-DAG:   rlwinm 6, [[REG3]], 8, 16, 23
+; ASM64PWR4-DAG:   rlwimi 6, [[REG2]], 16, 0, 15
+; ASM64PWR4-DAG:   rldimi 6, [[REG1]], 32, 0
+; ASM64PWR4-NEXT:  bl .test_byval_31Byte
+; ASM64PWR4-NEXT:  nop

llvm/test/CodeGen/PowerPC/aix64-cc-byval.ll

 ; RUN: llc -mtriple powerpc64-ibm-aix-xcoff -stop-after=machine-cp -verify-machineinstrs < %s | \
 ; RUN: FileCheck --check-prefixes=CHECK,64BIT %s
 
 ; RUN: llc -verify-machineinstrs -mcpu=pwr4 -mattr=-altivec \
 ; RUN:  -mtriple powerpc64-ibm-aix-xcoff < %s | \
 ; RUN: FileCheck --check-prefixes=CHECKASM,ASM64PWR4 %s
 
-%struct.S5 = type { [5 x i8] }
+%struct.S64 = type { [64 x i8] }
 
-@gS5 = external global %struct.S5, align 1
+@gS64 = external global %struct.S64, align 1
 
-define void @call_test_byval_5Byte() {
+define void @call_test_byval_64Byte() {
 entry:
-  call void @test_byval_5Byte(%struct.S5* byval(%struct.S5) align 1 @gS5)
+  call void @test_byval_64Byte(%struct.S64* byval(%struct.S64) align 1 @gS64)
   ret void
 }
 
-declare void @test_byval_5Byte(%struct.S5* byval(%struct.S5) align 1)
+declare void @test_byval_64Byte(%struct.S64* byval(%struct.S64) align 1)
 
-; CHECK-LABEL: name: call_test_byval_5Byte{{.*}}
-
-; CHECKASM-LABEL: .call_test_byval_5Byte:
+; CHECK-LABEL: name: call_test_byval_64Byte{{.*}}
 
 ; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
 ; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
-; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS5, $x2 :: (load 8 from got)
-; 64BIT-DAG:   renamable $x[[REG1:[0-9]+]] = LWZ8 0, killed renamable $x[[REGADDR]] :: (load 4)
-; 64BIT-DAG:   renamable $x[[REG2:[0-9]+]] = LBZ8 4, renamable $x[[REGADDR]] :: (load 1)
-; 64BIT-DAG:   renamable $x3 = RLWINM8 killed renamable $x[[REG2]], 24, 0, 7
-; 64BIT-DAG:   renamable $x3 = RLDIMI killed renamable $x3, killed renamable $x[[REG1]], 32, 0
-; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_5Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
+; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS64, $x2 :: (load 8 from got)
+; 64BIT-DAG:   renamable $x3 = LD 0, killed renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x4 = LD 8, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x5 = LD 16, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x6 = LD 24, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x7 = LD 32, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x8 = LD 40, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x9 = LD 48, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-DAG:   renamable $x10 = LD 56, renamable $x[[REGADDR]] :: (load 8)
+; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_64Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x4, implicit $x5, implicit $x6, implicit $x7, implicit $x8, implicit $x9, implicit $x10, implicit $x2, implicit-def $r1
 ; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
 
-; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
-; ASM64PWR4:       stdu 1, -112(1)
-; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
-; ASM64PWR4-DAG:   lwz [[REG1:[0-9]+]], 0([[REGADDR]])
-; ASM64PWR4-DAG:   lbz [[REG2:[0-9]+]], 4([[REGADDR]])
-; ASM64PWR4-DAG:   rlwinm 3, [[REG2]], 24, 0, 7
-; ASM64PWR4-DAG:   rldimi 3, [[REG1]], 32, 0
-; ASM64PWR4-NEXT:  bl .test_byval_5Byte
-; ASM64PWR4-NEXT:  nop
-
-%struct.S6 = type { [6 x i8] }
-
-@gS6 = external global %struct.S6, align 1
-
-define void @call_test_byval_6Byte() {
-entry:
-  call void @test_byval_6Byte(%struct.S6* byval(%struct.S6) align 1 @gS6)
-  ret void
-}
-
-declare void @test_byval_6Byte(%struct.S6* byval(%struct.S6) align 1)
-
-; CHECK-LABEL: name: call_test_byval_6Byte{{.*}}
-
-; CHECKASM-LABEL: .call_test_byval_6Byte:
-
-; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
-; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
-; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS6, $x2 :: (load 8 from got)
-; 64BIT-DAG:   renamable $x[[REG1:[0-9]+]] = LWZ8 0, killed renamable $x[[REGADDR]] :: (load 4)
-; 64BIT-DAG:   renamable $x[[REG2:[0-9]+]] = LHZ8 4, renamable $x[[REGADDR]] :: (load 2)
-; 64BIT-DAG:   renamable $x3 = RLWINM8 killed renamable $x[[REG2]], 16, 0, 15
-; 64BIT-DAG:   renamable $x3 = RLDIMI killed renamable $x3, killed renamable $x[[REG1]], 32, 0
-; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_6Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
-; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
-
-; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
-; ASM64PWR4:       stdu 1, -112(1)
-; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
-; ASM64PWR4-DAG:   lwz [[REG1:[0-9]+]], 0([[REGADDR]])
-; ASM64PWR4-DAG:   lhz [[REG2:[0-9]+]], 4([[REGADDR]])
-; ASM64PWR4-DAG:   rlwinm 3, [[REG2]], 16, 0, 15
-; ASM64PWR4-DAG:   rldimi 3, [[REG1]], 32, 0
-; ASM64PWR4-NEXT:  bl .test_byval_6Byte
-; ASM64PWR4-NEXT:  nop
-
-%struct.S7 = type { [7 x i8] }
-
-@gS7 = external global %struct.S7, align 1
-
-define void @call_test_byval_7Byte() {
-entry:
-  call void @test_byval_7Byte(%struct.S7* byval(%struct.S7) align 1 @gS7)
-  ret void
-}
-
-declare void @test_byval_7Byte(%struct.S7* byval(%struct.S7) align 1)
-
-; CHECK-LABEL: name: call_test_byval_7Byte{{.*}}
-
-; CHECKASM-LABEL: .call_test_byval_7Byte:
-
-; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
-; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
-; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS7, $x2 :: (load 8 from got)
-; 64BIT-DAG:   renamable $x[[REG1:[0-9]+]] = LWZ8 0, killed renamable $x[[REGADDR]] :: (load 4)
-; 64BIT-DAG:   renamable $x[[REG2:[0-9]+]] = LHZ8 4, renamable $x[[REGADDR]] :: (load 2)
-; 64BIT-DAG:   renamable $x[[REG3:[0-9]+]] = LBZ8 6, renamable $x[[REGADDR]] :: (load 1)
-; 64BIT-DAG:   renamable $x3 = RLWINM8 killed renamable $x[[REG3]], 8, 16, 23
-; 64BIT-DAG:   renamable $x3 = RLWIMI8 killed renamable $x3, killed renamable $x[[REG2]], 16, 0, 15
-; 64BIT-DAG:   renamable $x3 = RLDIMI killed renamable $x3, killed renamable $x[[REG1]], 32, 0
-; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_7Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
-; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
-
-; The DAG block permits some invalid inputs for the benefit of allowing more valid orderings.
-; ASM64PWR4:       stdu 1, -112(1)
-; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
-; ASM64PWR4-DAG:   lwz [[REG1:[0-9]+]], 0([[REGADDR]])
-; ASM64PWR4-DAG:   lhz [[REG2:[0-9]+]], 4([[REGADDR]])
-; ASM64PWR4-DAG:   lbz [[REG3:[0-9]+]], 6([[REGADDR]])
-; ASM64PWR4-DAG:   rlwinm 3, [[REG3]], 8, 16, 23
-; ASM64PWR4-DAG:   rlwimi 3, [[REG2]], 16, 0, 15
-; ASM64PWR4-DAG:   rldimi 3, [[REG1]], 32, 0
-; ASM64PWR4-NEXT:  bl .test_byval_7Byte
-; ASM64PWR4-NEXT:  nop
-
-%struct.S8 = type { [8 x i8] }
-
-@gS8 = external global %struct.S8, align 1
-
-define void @call_test_byval_8Byte() {
-entry:
-  call void @test_byval_8Byte(%struct.S8* byval(%struct.S8) align 1 @gS8)
-  ret void
-}
-
-declare void @test_byval_8Byte(%struct.S8* byval(%struct.S8) align 1)
-
-; CHECK-LABEL: name: call_test_byval_8Byte{{.*}}
-
-; CHECKASM-LABEL: .call_test_byval_8Byte:
-
-; 64BIT:       ADJCALLSTACKDOWN 112, 0, implicit-def dead $r1, implicit $r1
-; 64BIT-NEXT:  renamable $x[[REGADDR:[0-9]+]] = LDtoc @gS8, $x2 :: (load 8 from got)
-; 64BIT-NEXT:  renamable $x3 = LD 0, killed renamable $x[[REGADDR]] :: (load 8)
-; 64BIT-NEXT:  BL8_NOP <mcsymbol .test_byval_8Byte>, csr_aix64, implicit-def dead $lr8, implicit $rm, implicit $x3, implicit $x2, implicit-def $r1
-; 64BIT-NEXT:  ADJCALLSTACKUP 112, 0, implicit-def dead $r1, implicit $r1
+; CHECKASM-LABEL: .call_test_byval_64Byte:
 
 ; ASM64PWR4:       stdu 1, -112(1)
 ; ASM64PWR4-NEXT:  ld [[REGADDR:[0-9]+]], LC{{[0-9]+}}(2)
-; ASM64PWR4-NEXT:  ld 3, 0([[REGADDR]])
-; ASM64PWR4-NEXT:  bl .test_byval_8Byte
+; ASM64PWR4-DAG:   ld 3, 0([[REGADDR]])
+; ASM64PWR4-DAG:   ld 4, 8([[REGADDR]])
+; ASM64PWR4-DAG:   ld 5, 16([[REGADDR]])
+; ASM64PWR4-DAG:   ld 6, 24([[REGADDR]])
+; ASM64PWR4-DAG:   ld 7, 32([[REGADDR]])
+; ASM64PWR4-DAG:   ld 8, 40([[REGADDR]])
+; ASM64PWR4-DAG:   ld 9, 48([[REGADDR]])
+; ASM64PWR4-DAG:   ld 10, 56([[REGADDR]])
+; ASM64PWR4-NEXT:  bl .test_byval_64Byte
 ; ASM64PWR4-NEXT:  nop