[X86] Fix bug in x86_intrcc with arg copy elision

[rnk]

Summary

Use a custom calling convention handler for interrupts instead of fixing
up the locations in LowerMemArgument. This way, the offsets are correct
when constructed and we don't need to account for them in as many
places.

Depends on D56883

Replaces D56275

[rnk]

• rebase

[phil-opp]

I don't know enough about LLVM to review the code, but I just built a custom Rust compiler that uses this patch and can verify that this fixes the passed error code. The passed exception stack frame looks correct too.

Thanks a lot for fixing this @rnk!

[rnk]

@craig.topper, any thoughts? I think we'll want to merge this to 8.0 since Rust wants it.

[craig.topper]

llvm/test/CodeGen/X86/x86-32-intrcc.ll

121 ↗

(On Diff #183175)

Was that stray A at the end of the line supposed to be another sentence? its in the other test file too.

llvm/test/CodeGen/X86/x86-interrupt_cc.ll

32 ↗

(On Diff #183175)

Is there way to prevent the script from regexing over the addresses?

[rnk]

• remove stray character

llvm/test/CodeGen/X86/x86-32-intrcc.ll

121 ↗

(On Diff #183175)

I think it's a stray Vim command for "insert at end of line". :)

llvm/test/CodeGen/X86/x86-interrupt_cc.ll

32 ↗

(On Diff #183175)

No, it always wild-cards stack offsets, but optionally not global / RIP offsets. Do you remember having any particular reason to move to auto-generation in rL324530? I think the test only really needed to look for a few instructions, movk, and a few encodings. It doesn't make sense to me to auto-generate the tests if we card about the encoding, since the wildcarding does nothing to make the test less sensitive to unrelated changes.

[craig.topper]

I think I autogenerated it because I added more run lines and then fixed a bug in the next commit. I'm just in the habit of using the script.

So have we been incorrectly positioning the registers in the AVX512 tests?

[rnk]

In D56944#1374744, @craig.topper wrote:

I think I autogenerated it because I added more run lines and then fixed a bug in the next commit. I'm just in the habit of using the script.

I think auto-generating them is fine, but we should stop checking the instruction encodings since it defeats the wildcarding. If it's valuable to check the encodings, we should have a separate, non-generated test for that.

So have we been incorrectly positioning the registers in the AVX512 tests?

So, after looking at the offsets, it uncovered a bug. I made a mistake to assume that all fixed stack objects come before the return address. I think I can tweak my X86FrameLowering.cpp change to retain the local area offset adjustment for spilled fixed stack objects (ones that would normally be part of the newly allocated frame, not arguments). I need to look into it more though.

[rnk]

• fix spill bug

[phil-opp]

What's the status of this?

[rnk]

In D56944#1408614, @phil-opp wrote:

What's the status of this?

I'm waiting for review from @craig.topper, but I also forgot about it.

[craig.topper]

LGTM

[rnk]

Huh, apparently I spoke too soon, now the tests don't pass. :( Anyway, I'll move it forward.

[rnk]

In D56944#1409973, @rnk wrote:

Huh, apparently I spoke too soon, now the tests don't pass. :( Anyway, I'll move it forward.

I did -= instead of +=, is all.

llvm/trunk/lib/Target/X86/X86CallingConv.cpp

for (auto &It : PendingMembers) {
     State.addLoc(It);
   }
 
   PendingMembers.clear();
 
   return true;
 }
 
+/// X86 interrupt handlers can only take one or two stack arguments, but if
+/// there are two arguments, they are in the opposite order from the standard
+/// convention. Therefore, we have to look at the argument count up front before
+/// allocating stack for each argument.
+static bool CC_X86_Intr(unsigned &ValNo, MVT &ValVT, MVT &LocVT,
+                        CCValAssign::LocInfo &LocInfo,
+                        ISD::ArgFlagsTy &ArgFlags, CCState &State) {
+  const MachineFunction &MF = State.getMachineFunction();
+  size_t ArgCount = State.getMachineFunction().getFunction().arg_size();
+  bool Is64Bit = static_cast<const X86Subtarget &>(MF.getSubtarget()).is64Bit();
+  unsigned SlotSize = Is64Bit ? 8 : 4;
+  unsigned Offset;
+  if (ArgCount == 1 && ValNo == 0) {
+    // If we have one argument, the argument is five stack slots big, at fixed
+    // offset zero.
+    Offset = State.AllocateStack(5 * SlotSize, 4);
+  } else if (ArgCount == 2 && ValNo == 0) {
+    // If we have two arguments, the stack slot is *after* the error code
+    // argument. Pretend it doesn't consume stack space, and account for it when
+    // we assign the second argument.
+    Offset = SlotSize;
+  } else if (ArgCount == 2 && ValNo == 1) {
+    // If this is the second of two arguments, it must be the error code. It
+    // appears first on the stack, and is then followed by the five slot
+    // interrupt struct.
+    Offset = 0;
+    (void)State.AllocateStack(6 * SlotSize, 4);
+  } else {
+    report_fatal_error("unsupported x86 interrupt prototype");
+  }
+
+  // FIXME: This should be accounted for in
+  // X86FrameLowering::getFrameIndexReference, not here.
+  if (Is64Bit && ArgCount == 2)
+    Offset += SlotSize;
+
+  State.addLoc(CCValAssign::getMem(ValNo, ValVT, Offset, LocVT, LocInfo));
+  return true;
+}
+
 // Provides entry points of CC_X86 and RetCC_X86.
 #include "X86GenCallingConv.inc"

llvm/trunk/lib/Target/X86/X86CallingConv.td

def CC_Intel_OCL_BI : CallingConv<[
   // Pass masks in mask registers
   CCIfType<[v16i1, v8i1], CCAssignToReg<[K1]>>,
 
   CCIfSubtarget<"isTargetWin64()", CCDelegateTo<CC_X86_Win64_C>>,
   CCIfSubtarget<"is64Bit()",       CCDelegateTo<CC_X86_64_C>>,
   CCDelegateTo<CC_X86_32_C>
 ]>;
 
-def CC_X86_32_Intr : CallingConv<[
-  CCAssignToStack<4, 4>
-]>;
-
-def CC_X86_64_Intr : CallingConv<[
-  CCAssignToStack<8, 8>
-]>;
-
 //===----------------------------------------------------------------------===//
 // X86 Root Argument Calling Conventions
 //===----------------------------------------------------------------------===//
 
 // This is the root argument convention for the X86-32 backend.
 def CC_X86_32 : CallingConv<[
   // X86_INTR calling convention is valid in MCU target and should override the
   // MCU calling convention. Thus, this should be checked before isTargetMCU().
-  CCIfCC<"CallingConv::X86_INTR", CCDelegateTo<CC_X86_32_Intr>>,
+  CCIfCC<"CallingConv::X86_INTR", CCCustom<"CC_X86_Intr">>,
   CCIfSubtarget<"isTargetMCU()", CCDelegateTo<CC_X86_32_MCU>>,
   CCIfCC<"CallingConv::X86_FastCall", CCDelegateTo<CC_X86_32_FastCall>>,
   CCIfCC<"CallingConv::X86_VectorCall", CCDelegateTo<CC_X86_Win32_VectorCall>>,
   CCIfCC<"CallingConv::X86_ThisCall", CCDelegateTo<CC_X86_32_ThisCall>>,
   CCIfCC<"CallingConv::Fast", CCDelegateTo<CC_X86_32_FastCC>>,
   CCIfCC<"CallingConv::GHC", CCDelegateTo<CC_X86_32_GHC>>,
   CCIfCC<"CallingConv::HiPE", CCDelegateTo<CC_X86_32_HiPE>>,
   CCIfCC<"CallingConv::X86_RegCall", CCDelegateTo<CC_X86_32_RegCall>>,
def CC_X86_64 : CallingConv<[
   CCIfCC<"CallingConv::Win64", CCDelegateTo<CC_X86_Win64_C>>,
   CCIfCC<"CallingConv::X86_64_SysV", CCDelegateTo<CC_X86_64_C>>,
   CCIfCC<"CallingConv::X86_VectorCall", CCDelegateTo<CC_X86_Win64_VectorCall>>,
   CCIfCC<"CallingConv::HHVM", CCDelegateTo<CC_X86_64_HHVM>>,
   CCIfCC<"CallingConv::HHVM_C", CCDelegateTo<CC_X86_64_HHVM_C>>,
   CCIfCC<"CallingConv::X86_RegCall",
     CCIfSubtarget<"isTargetWin64()", CCDelegateTo<CC_X86_Win64_RegCall>>>,
   CCIfCC<"CallingConv::X86_RegCall", CCDelegateTo<CC_X86_SysV64_RegCall>>,
-  CCIfCC<"CallingConv::X86_INTR", CCDelegateTo<CC_X86_64_Intr>>,
+  CCIfCC<"CallingConv::X86_INTR", CCCustom<"CC_X86_Intr">>,
 
   // Mingw64 and native Win64 use Win64 CC
   CCIfSubtarget<"isTargetWin64()", CCDelegateTo<CC_X86_Win64_C>>,
 
   // Otherwise, drop to normal X86-64 CC
   CCDelegateTo<CC_X86_64_C>
 ]>;
 

llvm/trunk/lib/Target/X86/X86FrameLowering.cpp

int X86FrameLowering::getFrameIndexReference(const MachineFunction &MF, int FI,
   int Offset = MFI.getObjectOffset(FI) - getOffsetOfLocalArea();
   const X86MachineFunctionInfo *X86FI = MF.getInfo<X86MachineFunctionInfo>();
   unsigned CSSize = X86FI->getCalleeSavedFrameSize();
   uint64_t StackSize = MFI.getStackSize();
   bool HasFP = hasFP(MF);
   bool IsWin64Prologue = MF.getTarget().getMCAsmInfo()->usesWindowsCFI();
   int64_t FPDelta = 0;
 
+  // In an x86 interrupt, remove the offset we added to account for the return
+  // address from any stack object allocated in the caller's frame. Interrupts
+  // do not have a standard return address. Fixed objects in the current frame,
+  // such as SSE register spills, should not get this treatment.
+  if (MF.getFunction().getCallingConv() == CallingConv::X86_INTR &&
+      Offset >= 0) {
+    Offset += getOffsetOfLocalArea();
+  }
+
   if (IsWin64Prologue) {
     assert(!MFI.hasCalls() || (StackSize % 16) == 8);
 
     // Calculate required stack adjustment.
     uint64_t FrameSize = StackSize - SlotSize;
     // If required, include space for extra hidden slot for stashing base pointer.
     if (X86FI->getRestoreBasePointer())
       FrameSize += SlotSize;

llvm/trunk/lib/Target/X86/X86ISelLowering.cpp

bool ExtendedInMem =
       VA.isExtInLoc() && VA.getValVT().getScalarType() == MVT::i1 &&
       VA.getValVT().getSizeInBits() != VA.getLocVT().getSizeInBits();
 
   if (VA.getLocInfo() == CCValAssign::Indirect || ExtendedInMem)
     ValVT = VA.getLocVT();
   else
     ValVT = VA.getValVT();
 
-  // Calculate SP offset of interrupt parameter, re-arrange the slot normally
-  // taken by a return address.
-  int Offset = 0;
-  if (CallConv == CallingConv::X86_INTR) {
-    // X86 interrupts may take one or two arguments.
-    // On the stack there will be no return address as in regular call.
-    // Offset of last argument need to be set to -4/-8 bytes.
-    // Where offset of the first argument out of two, should be set to 0 bytes.
-    Offset = (Subtarget.is64Bit() ? 8 : 4) * ((i + 1) % Ins.size() - 1);
-    if (Subtarget.is64Bit() && Ins.size() == 2) {
-      // The stack pointer needs to be realigned for 64 bit handlers with error
-      // code, so the argument offset changes by 8 bytes.
-      Offset += 8;
-    }
-  }
-
   // FIXME: For now, all byval parameter objects are marked mutable. This can be
   // changed with more analysis.
   // In case of tail call optimization mark all arguments mutable. Since they
   // could be overwritten by lowering of arguments in case of a tail call.
   if (Flags.isByVal()) {
     unsigned Bytes = Flags.getByValSize();
     if (Bytes == 0) Bytes = 1; // Don't create zero-sized stack objects.
 
     // FIXME: For now, all byval parameter objects are marked as aliasing. This
     // can be improved with deeper analysis.
     int FI = MFI.CreateFixedObject(Bytes, VA.getLocMemOffset(), isImmutable,
                                    /*isAliased=*/true);
-    // Adjust SP offset of interrupt parameter.
-    if (CallConv == CallingConv::X86_INTR) {
-      MFI.setObjectOffset(FI, Offset);
-    }
     return DAG.getFrameIndex(FI, PtrVT);
   }
 
   // This is an argument in memory. We might be able to perform copy elision.
   if (Flags.isCopyElisionCandidate()) {
     EVT ArgVT = Ins[i].ArgVT;
     SDValue PartAddr;
     if (Ins[i].PartOffset == 0) {
X86TargetLowering::LowerMemArgument(SDValue Chain, CallingConv::ID CallConv,
 
   // Set SExt or ZExt flag.
   if (VA.getLocInfo() == CCValAssign::ZExt) {
     MFI.setObjectZExt(FI, true);
   } else if (VA.getLocInfo() == CCValAssign::SExt) {
     MFI.setObjectSExt(FI, true);
   }
 
-  // Adjust SP offset of interrupt parameter.
-  if (CallConv == CallingConv::X86_INTR) {
-    MFI.setObjectOffset(FI, Offset);
-  }
-
   SDValue FIN = DAG.getFrameIndex(FI, PtrVT);
   SDValue Val = DAG.getLoad(
       ValVT, dl, Chain, FIN,
       MachinePointerInfo::getFixedStack(DAG.getMachineFunction(), FI));
   return ExtendedInMem
              ? (VA.getValVT().isVector()
                     ? DAG.getNode(ISD::SCALAR_TO_VECTOR, dl, VA.getValVT(), Val)
                     : DAG.getNode(ISD::TRUNCATE, dl, VA.getValVT(), Val))
SDValue X86TargetLowering::LowerFormalArguments(
   MachineFrameInfo &MFI = MF.getFrameInfo();
   bool Is64Bit = Subtarget.is64Bit();
   bool IsWin64 = Subtarget.isCallingConvWin64(CallConv);
 
   assert(
       !(isVarArg && canGuaranteeTCO(CallConv)) &&
       "Var args not supported with calling conv' regcall, fastcc, ghc or hipe");
 
-  if (CallConv == CallingConv::X86_INTR) {
-    bool isLegal = Ins.size() == 1 ||
-                   (Ins.size() == 2 && ((Is64Bit && Ins[1].VT == MVT::i64) ||
-                                        (!Is64Bit && Ins[1].VT == MVT::i32)));
-    if (!isLegal)
-      report_fatal_error("X86 interrupts may take one or two arguments");
-  }
-
   // Assign locations to all of the incoming arguments.
   SmallVector<CCValAssign, 16> ArgLocs;
   CCState CCInfo(CallConv, isVarArg, MF, ArgLocs, *DAG.getContext());
 
   // Allocate shadow area for Win64.
   if (IsWin64)
     CCInfo.AllocateStack(32, 8);
 

llvm/trunk/test/CodeGen/X86/x86-32-intrcc.ll

 ; RUN: llc -mtriple=i686-unknown-unknown < %s | FileCheck %s
 ; RUN: llc -mtriple=i686-unknown-unknown -O0 < %s | FileCheck %s -check-prefix=CHECK0
 
 %struct.interrupt_frame = type { i32, i32, i32, i32, i32 }
 
[e-mail] = appending global [4 x i8*] [i8* bitcast (void (%struct.interrupt_frame*)* @test_isr_no_ecode to i8*), i8* bitcast (void (%struct.interrupt_frame*, i32)* @test_isr_ecode to i8*), i8* bitcast (void (%struct.interrupt_frame*, i32)* @test_isr_clobbers to i8*), i8* bitcast (void (%struct.interrupt_frame*)* @test_isr_x87 to i8*)], section "llvm.metadata"
+@sink_address = global i32* null
+@sink_i32 = global i32 0
+
 
 ; Spills eax, putting original esp at +4.
 ; No stack adjustment if declared with no error code
 define x86_intrcc void @test_isr_no_ecode(%struct.interrupt_frame* %frame) {
   ; CHECK-LABEL: test_isr_no_ecode:
   ; CHECK: pushl %eax
   ; CHECK: movl 12(%esp), %eax
   ; CHECK: popl %eax
define x86_intrcc void @test_isr_x87(%struct.interrupt_frame* %frame) {
   ; CHECK-NEXT: fstpt f80
   ; CHECK-NEXT: iretl
 entry:
   %ld = load x86_fp80, x86_fp80* @f80, align 4
   %add = fadd x86_fp80 %ld, 0xK3FFF8000000000000000
   store x86_fp80 %add, x86_fp80* @f80, align 4
   ret void
 }
+
+; Use a frame pointer to check the offsets. No return address, arguments start
+; at EBP+4.
+define dso_local x86_intrcc void @test_fp_1(%struct.interrupt_frame* %p) #0 {
+  ; CHECK-LABEL: test_fp_1:
+  ; CHECK: # %bb.0: # %entry
+  ; CHECK-NEXT: pushl %ebp
+  ; CHECK-NEXT: movl %esp, %ebp
+  ; CHECK: cld
+  ; CHECK-DAG: leal 4(%ebp), %[[R1:[^ ]*]]
+  ; CHECK-DAG: leal 20(%ebp), %[[R2:[^ ]*]]
+  ; CHECK: movl %[[R1]], sink_address
+  ; CHECK: movl %[[R2]], sink_address
+  ; CHECK: popl %ebp
+  ; CHECK: iretl
+entry:
+  %arrayidx = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i32 0, i32 0
+  %arrayidx2 = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i32 0, i32 4
+  store volatile i32* %arrayidx, i32** @sink_address
+  store volatile i32* %arrayidx2, i32** @sink_address
+  ret void
+}
+
+; The error code is between EBP and the interrupt_frame.
+define dso_local x86_intrcc void @test_fp_2(%struct.interrupt_frame* %p, i32 %err) #0 {
+  ; CHECK-LABEL: test_fp_2:
+  ; CHECK: # %bb.0: # %entry
+  ; CHECK-NEXT: pushl %ebp
+  ; CHECK-NEXT: movl %esp, %ebp
+  ; CHECK: cld
+  ; CHECK-DAG: movl 4(%ebp), %[[R3:[^ ]*]]
+  ; CHECK-DAG: leal 8(%ebp), %[[R1:[^ ]*]]
+  ; CHECK-DAG: leal 24(%ebp), %[[R2:[^ ]*]]
+  ; CHECK: movl %[[R1]], sink_address
+  ; CHECK: movl %[[R2]], sink_address
+  ; CHECK: movl %[[R3]], sink_i32
+  ; CHECK: popl %ebp
+  ; CHECK: iretl
+entry:
+  %arrayidx = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i32 0, i32 0
+  %arrayidx2 = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i32 0, i32 4
+  store volatile i32* %arrayidx, i32** @sink_address
+  store volatile i32* %arrayidx2, i32** @sink_address
+  store volatile i32 %err, i32* @sink_i32
+  ret void
+}
+
+; Test argument copy elision when copied to a local alloca.
+define x86_intrcc void @test_copy_elide(%struct.interrupt_frame* %frame, i32 %err) #0 {
+  ; CHECK-LABEL: test_copy_elide:
+  ; CHECK: # %bb.0: # %entry
+  ; CHECK-NEXT: pushl %ebp
+  ; CHECK-NEXT: movl %esp, %ebp
+  ; CHECK: cld
+  ; CHECK: leal 4(%ebp), %[[R1:[^ ]*]]
+  ; CHECK: movl %[[R1]], sink_address
+entry:
+  %err.addr = alloca i32, align 4
+  store i32 %err, i32* %err.addr, align 4
+  store volatile i32* %err.addr, i32** @sink_address
+  ret void
+}
+
+attributes #0 = { nounwind "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" }

llvm/trunk/test/CodeGen/X86/x86-64-intrcc.ll

 ; RUN: llc -mtriple=x86_64-unknown-unknown < %s | FileCheck %s
 ; RUN: llc -mtriple=x86_64-unknown-unknown -O0 < %s | FileCheck %s -check-prefix=CHECK0
 
 %struct.interrupt_frame = type { i64, i64, i64, i64, i64 }
 
[e-mail] = appending global [4 x i8*] [i8* bitcast (void (%struct.interrupt_frame*)* @test_isr_no_ecode to i8*), i8* bitcast (void (%struct.interrupt_frame*, i64)* @test_isr_ecode to i8*), i8* bitcast (void (%struct.interrupt_frame*, i64)* @test_isr_clobbers to i8*), i8* bitcast (void (%struct.interrupt_frame*)* @test_isr_x87 to i8*)], section "llvm.metadata"
+@sink_address = global i64* null
+@sink_i32 = global i64 0
 
 ; Spills rax, putting original esp at +8.
 ; No stack adjustment if declared with no error code
 define x86_intrcc void @test_isr_no_ecode(%struct.interrupt_frame* %frame) {
   ; CHECK-LABEL: test_isr_no_ecode:
   ; CHECK: pushq %rax
   ; CHECK: movq 24(%rsp), %rax
   ; CHECK: popq %rax
define x86_intrcc void @test_isr_x87(%struct.interrupt_frame* %frame) {
   ; CHECK-NEXT: fstpt f80
   ; CHECK-NEXT: iretq
 entry:
   %ld = load x86_fp80, x86_fp80* @f80, align 4
   %add = fadd x86_fp80 %ld, 0xK3FFF8000000000000000
   store x86_fp80 %add, x86_fp80* @f80, align 4
   ret void
 }
+
+; Use a frame pointer to check the offsets. No return address, arguments start
+; at RBP+4.
+define dso_local x86_intrcc void @test_fp_1(%struct.interrupt_frame* %p) #0 {
+  ; CHECK-LABEL: test_fp_1:
+  ; CHECK: # %bb.0: # %entry
+  ; CHECK-NEXT: pushq %rbp
+  ; CHECK-NEXT: movq %rsp, %rbp
+  ; CHECK: cld
+  ; CHECK-DAG: leaq 8(%rbp), %[[R1:[^ ]*]]
+  ; CHECK-DAG: leaq 40(%rbp), %[[R2:[^ ]*]]
+  ; CHECK: movq %[[R1]], sink_address
+  ; CHECK: movq %[[R2]], sink_address
+  ; CHECK: popq %rbp
+  ; CHECK: iretq
+entry:
+  %arrayidx = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i64 0, i32 0
+  %arrayidx2 = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i64 0, i32 4
+  store volatile i64* %arrayidx, i64** @sink_address
+  store volatile i64* %arrayidx2, i64** @sink_address
+  ret void
+}
+
+; The error code is between RBP and the interrupt_frame.
+define dso_local x86_intrcc void @test_fp_2(%struct.interrupt_frame* %p, i64 %err) #0 {
+  ; CHECK-LABEL: test_fp_2:
+  ; CHECK: # %bb.0: # %entry
+  ; This RAX push is just to align the stack.
+  ; CHECK-NEXT: pushq %rax
+  ; CHECK-NEXT: pushq %rbp
+  ; CHECK-NEXT: movq %rsp, %rbp
+  ; CHECK: cld
+  ; CHECK-DAG: movq 16(%rbp), %[[R3:[^ ]*]]
+  ; CHECK-DAG: leaq 24(%rbp), %[[R1:[^ ]*]]
+  ; CHECK-DAG: leaq 56(%rbp), %[[R2:[^ ]*]]
+  ; CHECK: movq %[[R1]], sink_address(%rip)
+  ; CHECK: movq %[[R2]], sink_address(%rip)
+  ; CHECK: movq %[[R3]], sink_i32(%rip)
+  ; CHECK: popq %rbp
+  ; Pop off both the error code and the 8 byte alignment adjustment from the
+  ; prologue.
+  ; CHECK: addq $16, %rsp
+  ; CHECK: iretq
+entry:
+  %arrayidx = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i64 0, i32 0
+  %arrayidx2 = getelementptr inbounds %struct.interrupt_frame, %struct.interrupt_frame* %p, i64 0, i32 4
+  store volatile i64* %arrayidx, i64** @sink_address
+  store volatile i64* %arrayidx2, i64** @sink_address
+  store volatile i64 %err, i64* @sink_i32
+  ret void
+}
+
+; Test argument copy elision when copied to a local alloca.
+define x86_intrcc void @test_copy_elide(%struct.interrupt_frame* %frame, i64 %err) #0 {
+  ; CHECK-LABEL: test_copy_elide:
+  ; CHECK: # %bb.0: # %entry
+  ; This RAX push is just to align the stack.
+  ; CHECK-NEXT: pushq %rax
+  ; CHECK-NEXT: pushq %rbp
+  ; CHECK-NEXT: movq %rsp, %rbp
+  ; CHECK: cld
+  ; CHECK: leaq 16(%rbp), %[[R1:[^ ]*]]
+  ; CHECK: movq %[[R1]], sink_address(%rip)
+entry:
+  %err.addr = alloca i64, align 4
+  store i64 %err, i64* %err.addr, align 4
+  store volatile i64* %err.addr, i64** @sink_address
+  ret void
+}
+
+
+attributes #0 = { nounwind "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" }