Reland "[X86][ABI] Don't preserve return regs for preserve_all/preserve_most CCs""

[AntonBikineev]

Summary

The original change mistakenly excluded parameter registers from the
list of callee-saved-registers. This reland fixes it - it only excludes
the return registers for these calling conventions.

Phoebe, could you please take a look? This would be highly appreciated.

Original description:

Currently both calling conventions preserve registers that are used to
store a return value. This causes the returned value to be lost:

define i32 @bar() {
  %1 = call preserve_mostcc i32 @foo()
  ret i32 %1
}

define preserve_mostcc i32 @foo() {
  ret i32 2
  ; preserve_mostcc will restore %rax,
  ; whatever it was before the call.
}

This contradicts the current documentation (preserve_allcc "behaves
identical to the C calling conventions on how arguments and return
values are passed") and also breaks [[clang::preserve_most]].

This change makes CSRs be preserved iff they are not used to store a
return value (e.g. %rax for scalars, {%rax:%rdx} for __int128, %xmm0
for double). For void functions no additional registers are
preserved, i.e. the behaviour is backward compatible with existing
code.

[pengfei]

Add a test case for the previous fail?

[goldstein.w.n]

llvm/lib/Target/X86/X86ISelLowering.cpp

4933

Do you need shouldDisableRetRegFromCSR here? It seems shouldDisableRetRegFromCSR is a superset.

[AntonBikineev]

Thanks for the comments!

In D143425#4108663, @pengfei wrote:

Add a test case for the previous fail?

I've changed the test functions to take parameters, e.g.

define preserve_allcc RETTYPE @preserve_allcc1(i64, i64, double, double)

This covers the broken scenario.

llvm/lib/Target/X86/X86ISelLowering.cpp

4933

Technically it's not needed, but I added it for readability, since we do need to create a separate RegMask for preserve_* calls.

[AntonBikineev]

(slightly edited code in LowerCall for readability)

[pengfei]

This covers the broken scenario.

Can you explain it in more detail? I checked the test locally but found no difference in codegen with or without the new change.

[AntonBikineev]

In D143425#4109955, @pengfei wrote:

This covers the broken scenario.

Can you explain it in more detail? I checked the test locally but found no difference in codegen with or without the new change.

Did you include the original (broken) CL in the baseline? The CL was reverted, so you'd have to apply it to see the codegen diff. But the diff is basically this:

  _preserve_mostcc1:                      ## @preserve_mostcc1
  ## %bb.0:                               ## %entry
        pushq   %r10
        pushq   %r9
        pushq   %r8
+       pushq   %rdi
+       pushq   %rsi
        pushq   %rdx
        pushq   %rcx
        pushq   %rax
        pushq   %rbp
        pushq   %r15
        pushq   %r14
        pushq   %r13
        pushq   %r12
        pushq   %rbx
        ## InlineAsm Start
        ## InlineAsm End
        popq    %rbx
        popq    %r12
        popq    %r13
        popq    %r14
        popq    %r15
        popq    %rbp
        popq    %rax
        popq    %rcx
        popq    %rdx
+       popq    %rsi
+       popq    %rdi
        popq    %r8
        popq    %r9
        popq    %r10
        retq

The original (broken) CL didn't preserve parameter regs (%rdi, %rsi).

[pengfei]

In D143425#4109985, @AntonBikineev wrote:

In D143425#4109955, @pengfei wrote:

This covers the broken scenario.

Can you explain it in more detail? I checked the test locally but found no difference in codegen with or without the new change.

Did you include the original (broken) CL in the baseline? The CL was reverted, so you'd have to apply it to see the codegen diff. But the diff is basically this:

No, I compared between commenting out if (ShouldDisableArgRegs) { and not, and didn't see any difference. Or I misunderstood something?

  _preserve_mostcc1:                      ## @preserve_mostcc1
  ## %bb.0:                               ## %entry
        pushq   %r10
        pushq   %r9
        pushq   %r8
+       pushq   %rdi
+       pushq   %rsi
        pushq   %rdx
        pushq   %rcx
        pushq   %rax
        pushq   %rbp
        pushq   %r15
        pushq   %r14
        pushq   %r13
        pushq   %r12
        pushq   %rbx
        ## InlineAsm Start
        ## InlineAsm End
        popq    %rbx
        popq    %r12
        popq    %r13
        popq    %r14
        popq    %r15
        popq    %rbp
        popq    %rax
        popq    %rcx
        popq    %rdx
+       popq    %rsi
+       popq    %rdi
        popq    %r8
        popq    %r9
        popq    %r10
        retq

The original (broken) CL didn't preserve parameter regs (%rdi, %rsi).

That meets my expectation, I just didn't confirm it.

[AntonBikineev]

In D143425#4110057, @pengfei wrote:

In D143425#4109985, @AntonBikineev wrote:

In D143425#4109955, @pengfei wrote:

This covers the broken scenario.

Can you explain it in more detail? I checked the test locally but found no difference in codegen with or without the new change.

Did you include the original (broken) CL in the baseline? The CL was reverted, so you'd have to apply it to see the codegen diff. But the diff is basically this:

No, I compared between commenting out if (ShouldDisableArgRegs) { and not, and didn't see any difference. Or I misunderstood something?

Indeed, the tests didn't cover this case. Thanks! I added new tests dynamic-regmask-preserve-*.ll to check exactly the preserved RegMask on the caller side. Now the difference between the broken commit is much clearer on the caller side. For example, commenting out if (ShouldDisableArgRegs) { would remove rdi, rsi, rdx, rcx, r8 and their subregisters from the CustomRegMask in the call:

CALL64pcrel32 @callee1, CustomRegMask($bh,$bl,$bp,$bph,$bpl,$bx,$ch,$cl,$cx,$dh,$di,$dih,$dil,$dl,$dx,$ebp,$ebx,$ecx,$edi,$edx,$esi,$hbp,$hbx,$hcx,$hdi,$hdx,$hsi,$rbp,$rbx,$rcx,$rdi,$rdx,$rsi,$si,$sih,$sil,$r8,$r9,$r10,$r12,$r13,$r14,$r15,$r8b,$r9b,$r10b,$r12b,$r13b,$r14b,$r15b,$r8bh,$r9bh,$r10bh,$r12bh,$r13bh,$r14bh,$r15bh,$r8d,$r9d,$r10d,$r12d,$r13d,$r14d,$r15d,$r8w,$r9w,$r10w,$r12w,$r13w,$r14w,$r15w,$r8wh,$r9wh,$r10wh,$r12wh,$r13wh,$r14wh,$r15wh), implicit $rsp, implicit $ssp, implicit $rdi, implicit $rsi, implicit $rdx, implicit $rcx, implicit $r8, implicit-def $rsp, implicit-def $ssp, implicit-def $rax
    ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp

The updated revision also contains a slight functional change: for functions with the sret parameter (function returning a struct by value) I still preserve %rax. The standard SysV CC requires the sret (i.e. %rdi) be copied into %rax, but for the preserve_* CCs this is redundant and is not actually used, since it's always the same as the preserved %rdi.

[AntonBikineev]

(Add tests for checking RegMask and don't preserve %rax for sret functions)

[pengfei]

In D143425#4111499, @AntonBikineev wrote:

In D143425#4110057, @pengfei wrote:

In D143425#4109985, @AntonBikineev wrote:

In D143425#4109955, @pengfei wrote:

This covers the broken scenario.

Can you explain it in more detail? I checked the test locally but found no difference in codegen with or without the new change.

Did you include the original (broken) CL in the baseline? The CL was reverted, so you'd have to apply it to see the codegen diff. But the diff is basically this:

No, I compared between commenting out if (ShouldDisableArgRegs) { and not, and didn't see any difference. Or I misunderstood something?

Indeed, the tests didn't cover this case. Thanks! I added new tests dynamic-regmask-preserve-*.ll to check exactly the preserved RegMask on the caller side. Now the difference between the broken commit is much clearer on the caller side. For example, commenting out if (ShouldDisableArgRegs) { would remove rdi, rsi, rdx, rcx, r8 and their subregisters from the CustomRegMask in the call:

CALL64pcrel32 @callee1, CustomRegMask($bh,$bl,$bp,$bph,$bpl,$bx,$ch,$cl,$cx,$dh,$di,$dih,$dil,$dl,$dx,$ebp,$ebx,$ecx,$edi,$edx,$esi,$hbp,$hbx,$hcx,$hdi,$hdx,$hsi,$rbp,$rbx,$rcx,$rdi,$rdx,$rsi,$si,$sih,$sil,$r8,$r9,$r10,$r12,$r13,$r14,$r15,$r8b,$r9b,$r10b,$r12b,$r13b,$r14b,$r15b,$r8bh,$r9bh,$r10bh,$r12bh,$r13bh,$r14bh,$r15bh,$r8d,$r9d,$r10d,$r12d,$r13d,$r14d,$r15d,$r8w,$r9w,$r10w,$r12w,$r13w,$r14w,$r15w,$r8wh,$r9wh,$r10wh,$r12wh,$r13wh,$r14wh,$r15wh), implicit $rsp, implicit $ssp, implicit $rdi, implicit $rsi, implicit $rdx, implicit $rcx, implicit $r8, implicit-def $rsp, implicit-def $ssp, implicit-def $rax
    ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp

That makes sense to me, thanks for the test!

The updated revision also contains a slight functional change: for functions with the sret parameter (function returning a struct by value) I still preserve %rax. The standard SysV CC requires the sret (i.e. %rdi) be copied into %rax, but for the preserve_* CCs this is redundant and is not actually used, since it's always the same as the preserved %rdi.

I doubt about it, given it is ABI specified. Are there existing callers use RAX to access date rather than rdi?

llvm/test/CodeGen/X86/dynamic-regmask-preserve-most.ll

1 ↗

(On Diff #495675)

Why choose machine-sink as the check point?

3 ↗

(On Diff #495675)

What's this RUN checking for?

[AntonBikineev]

The updated revision also contains a slight functional change: for functions with the sret parameter (function returning a struct by value) I still preserve %rax. The standard SysV CC requires the sret (i.e. %rdi) be copied into %rax, but for the preserve_* CCs this is redundant and is not actually used, since it's always the same as the preserved %rdi.

I doubt about it, given it is ABI specified. Are there existing callers use RAX to access date rather than rdi?

For these CCs there shouldn't be any callers expecting %rax to change since it's anyway been preserved. In that sense the change doesn't break any existing users that may use sret with these CCs.

llvm/test/CodeGen/X86/dynamic-regmask-preserve-most.ll

1 ↗

(On Diff #495675)

Hm, I just copied the code from dynamic-regmask.ll. I guess we can stop after finalize-isel to check CSRs and masks.

3 ↗

(On Diff #495675)

(removed)

[AntonBikineev]

(fix RUN: commands for tests)

[goldstein.w.n]

should there be a test with avx512 for xmm16-31?

llvm/lib/Target/X86/X86ISelLowering.cpp

3389

CallConv == CallingConv::X86_RegCall

[AntonBikineev]

In D143425#4113345, @goldstein.w.n wrote:

should there be a test with avx512 for xmm16-31?

Actually, preserve_all doesn't preserve them, which is apparently missing, since the CCs were introduced short after AVX512 was proposed. It would probably make sense to introduce

def CSR_64_RT_AllRegs_AVX512 : CalleeSavedRegs<(add CSR_64_RT_MostRegs,
                                               (sequence "ZMM%u", 0, 31))>;

but I believe it should go to a separate change.

llvm/lib/Target/X86/X86ISelLowering.cpp

3389

This would also need || MF.getFunction().hasFnAttribute("no_caller_saved_registers"), so I don't think it'd be more readable overall.

[pengfei]

LGTM.

[AntonBikineev]

Thanks a lot for taking a look!

llvm/docs/LangRef.rst

"``preserve_mostcc``" - The `PreserveMost` calling convention
     calling convention on how arguments and return values are passed, but it
     uses a different set of caller/callee-saved registers. This alleviates the
     burden of saving and recovering a large register set before and after the
     call in the caller. If the arguments are passed in callee-saved registers,
     then they will be preserved by the callee across the call. This doesn't
     apply for values returned in callee-saved registers.
 
     - On X86-64 the callee preserves all general purpose registers, except for
-      R11. R11 can be used as a scratch register. Floating-point registers
-      (XMMs/YMMs) are not preserved and need to be saved by the caller.
+      R11 and return registers, if any. R11 can be used as a scratch register.
+      Floating-point registers (XMMs/YMMs) are not preserved and need to be
+      saved by the caller.
 
     The idea behind this convention is to support calls to runtime functions
     that have a hot path and a cold path. The hot path is usually a small piece
     of code that doesn't use many registers. The cold path might need to call out to
     another function and therefore only needs to preserve the caller-saved
     registers, which haven't already been saved by the caller. The
     `PreserveMost` calling convention is very similar to the `cold` calling
     convention in terms of caller/callee-saved registers, but they are used for

llvm/lib/Target/X86/X86CallingConv.td

                                              R8, R9, R10, R11)>;
 
 // CSRs that are handled by prologue, epilogue.
 def CSR_64_CXX_TLS_Darwin_PE : CalleeSavedRegs<(add RBP)>;
 
 // CSRs that are handled explicitly via copies.
 def CSR_64_CXX_TLS_Darwin_ViaCopy : CalleeSavedRegs<(sub CSR_64_TLS_Darwin, RBP)>;
 
-// All GPRs - except r11
+// All GPRs - except r11 and return registers.
 def CSR_64_RT_MostRegs : CalleeSavedRegs<(add CSR_64, RAX, RCX, RDX, RSI, RDI,
                                               R8, R9, R10)>;
 
-// All registers - except r11
+// All registers - except r11 and return registers.
 def CSR_64_RT_AllRegs     : CalleeSavedRegs<(add CSR_64_RT_MostRegs,
                                                  (sequence "XMM%u", 0, 15))>;
 def CSR_64_RT_AllRegs_AVX : CalleeSavedRegs<(add CSR_64_RT_MostRegs,
                                                  (sequence "YMM%u", 0, 15))>;
 
 def CSR_64_MostRegs : CalleeSavedRegs<(add RBX, RCX, RDX, RSI, RDI, R8, R9, R10,
                                            R11, R12, R13, R14, R15, RBP,
                                            (sequence "XMM%u", 0, 15))>;

llvm/lib/Target/X86/X86ISelLowering.cpp

 /// crashing.
 static void errorUnsupported(SelectionDAG &DAG, const SDLoc &dl,
                              const char *Msg) {
   MachineFunction &MF = DAG.getMachineFunction();
   DAG.getContext()->diagnose(
       DiagnosticInfoUnsupported(MF.getFunction(), Msg, dl.getDebugLoc()));
 }
 
+/// Returns true if a CC can dynamically exclude a register from the list of
+/// callee-saved-registers (TargetRegistryInfo::getCalleeSavedRegs()) based on
+/// the return registers.
+static bool shouldDisableRetRegFromCSR(CallingConv::ID CC) {
+  switch (CC) {
+  default:
+    return false;
+  case CallingConv::X86_RegCall:
+  case CallingConv::PreserveMost:
+  case CallingConv::PreserveAll:
+    return true;
+  }
+}
+
+/// Returns true if a CC can dynamically exclude a register from the list of
+/// callee-saved-registers (TargetRegistryInfo::getCalleeSavedRegs()) based on
+/// the parameters.
+static bool shouldDisableArgRegFromCSR(CallingConv::ID CC) {
+  return CC == CallingConv::X86_RegCall;
+}
+
 X86TargetLowering::X86TargetLowering(const X86TargetMachine &TM,
                                      const X86Subtarget &STI)
     : TargetLowering(TM), Subtarget(STI) {
   bool UseX87 = !Subtarget.useSoftFloat() && Subtarget.hasX87();
   MVT PtrVT = MVT::getIntegerVT(TM.getPointerSizeInBits(0));
 
   // Set up the TargetLowering object.
 

                                bool isVarArg,
                                const SmallVectorImpl<ISD::OutputArg> &Outs,
                                const SmallVectorImpl<SDValue> &OutVals,
                                const SDLoc &dl, SelectionDAG &DAG) const {
   MachineFunction &MF = DAG.getMachineFunction();
   X86MachineFunctionInfo *FuncInfo = MF.getInfo<X86MachineFunctionInfo>();
 
   // In some cases we need to disable registers from the default CSR list.
-  // For example, when they are used for argument passing.
+  // For example, when they are used as return registers (preserve_* and X86's
+  // regcall) or for argument passing (X86's regcall).
   bool ShouldDisableCalleeSavedRegister =
-      CallConv == CallingConv::X86_RegCall ||
+      shouldDisableRetRegFromCSR(CallConv) ||
       MF.getFunction().hasFnAttribute("no_caller_saved_registers");
 
   if (CallConv == CallingConv::X86_INTR && !Outs.empty())
     report_fatal_error("X86 interrupts may not return any value");
 
   SmallVector<CCValAssign, 16> RVLocs;
   CCState CCInfo(CallConv, isVarArg, MF, RVLocs, *DAG.getContext());
   CCInfo.AnalyzeReturn(Outs, RetCC_X86);

         DAG.getRegister(RetValReg, getPointerTy(DAG.getDataLayout())));
 
     // Add the returned register to the CalleeSaveDisableRegs list.
     if (ShouldDisableCalleeSavedRegister)
       MF.getRegInfo().disableCalleeSavedRegister(RetValReg);
   }
 
   const X86RegisterInfo *TRI = Subtarget.getRegisterInfo();
   const MCPhysReg *I =
       TRI->getCalleeSavedRegsViaCopy(&DAG.getMachineFunction());
   if (I) {
     for (; *I; ++I) {
       if (X86::GR64RegClass.contains(*I))
         RetOps.push_back(DAG.getRegister(*I, MVT::i64));
       else
         llvm_unreachable("Unexpected register class in CSRsViaCopy!");
     }

       // same, so the size of funclets' (mostly empty) frames is dictated by
       // how far this slot is from the bottom (since they allocate just enough
       // space to accommodate holding this slot at the correct offset).
       int PSPSymFI = MFI.CreateStackObject(8, Align(8), /*isSpillSlot=*/false);
       EHInfo->PSPSymFrameIdx = PSPSymFI;
     }
   }
 
-  if (CallConv == CallingConv::X86_RegCall ||
+  if (shouldDisableArgRegFromCSR(CallConv) ||
       F.hasFnAttribute("no_caller_saved_registers")) {
     MachineRegisterInfo &MRI = MF.getRegInfo();
     for (std::pair<Register, Register> Pair : MRI.liveins())
       MRI.disableCalleeSavedRegister(Pair.first);
   }
 
   return Chain;
 }

       Mask = RegInfo->getNoPreservedMask();
   }
 
   // Define a new register mask from the existing mask.
   uint32_t *RegMask = nullptr;
 
   // In some calling conventions we need to remove the used physical registers
   // from the reg mask.
-  if (CallConv == CallingConv::X86_RegCall || HasNCSR) {
+  if (shouldDisableArgRegFromCSR(CallConv) ||
+      shouldDisableRetRegFromCSR(CallConv) || HasNCSR) {
     const TargetRegisterInfo *TRI = Subtarget.getRegisterInfo();
 
     // Allocate a new Reg Mask and copy Mask.
     RegMask = MF.allocateRegMask();
     unsigned RegMaskSize = MachineOperand::getRegMaskSize(TRI->getNumRegs());
     memcpy(RegMask, Mask, sizeof(RegMask[0]) * RegMaskSize);
 
     // Make sure all sub registers of the argument registers are reset
     // in the RegMask.
-    for (auto const &RegPair : RegsToPass)
-      for (MCSubRegIterator SubRegs(RegPair.first, TRI, /*IncludeSelf=*/true);
-           SubRegs.isValid(); ++SubRegs)
-        RegMask[*SubRegs / 32] &= ~(1u << (*SubRegs % 32));
+    if (shouldDisableArgRegFromCSR(CallConv) || HasNCSR) {
+      for (auto const &RegPair : RegsToPass)
+        for (MCSubRegIterator SubRegs(RegPair.first, TRI, /*IncludeSelf=*/true);
+             SubRegs.isValid(); ++SubRegs)
+          RegMask[*SubRegs / 32] &= ~(1u << (*SubRegs % 32));
+    }
 
     // Create the RegMask Operand according to our updated mask.
     Ops.push_back(DAG.getRegisterMask(RegMask));
   } else {
     // Create the RegMask Operand according to the static mask.
     Ops.push_back(DAG.getRegisterMask(Mask));
   }
 

llvm/test/CodeGen/X86/preserve_allcc64-ret-double.ll

+; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=corei7     | FileCheck --check-prefixes=ALL,SSE %s
+; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=corei7-avx | FileCheck --check-prefixes=ALL,AVX %s
+
+define preserve_allcc double @preserve_allcc1() nounwind {
+entry:
+;ALL-LABEL:   preserve_allcc1
+;SSE:         movaps %xmm1
+;SSE-NOT:     movaps %xmm0
+;AVX:         vmovups %ymm1
+;AVX-NOT:     vmovups %ymm0
+;SSE-NOT:     movaps {{.*}} %xmm0
+;SSE:         movaps {{.*}} %xmm1
+;AVX-NOT:     vmovups {{.*}} %ymm0
+;AVX:         vmovups {{.*}} %ymm1
+  call void asm sideeffect "", "~{rax},~{rbx},~{rcx},~{rdx},~{rsi},~{rdi},~{r8},~{r9},~{r10},~{r11},~{r12},~{r13},~{r14},~{r15},~{rbp},~{xmm0},~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15}"()
+  ret double 0.
+}
+
+; Make sure XMM0 (return register) and R11 are saved before the call
+declare preserve_allcc double @bar_double(i64, i64)
+define void @preserve_allcc2() nounwind {
+entry:
+;SSE-LABEL: preserve_allcc2
+;SSE:       movq %r11, [[REG1:%[a-z0-9]+]]
+;SSE:       movaps %xmm0, [[REG2:[-0-9]*\(%r[sb]p\)]]
+;SSE:       movq [[REG1]], %r11
+;SSE:       movaps [[REG2]], %xmm0
+  %a0 = call i64 asm sideeffect "", "={rax}"() nounwind
+  %a1 = call i64 asm sideeffect "", "={rcx}"() nounwind
+  %a2 = call i64 asm sideeffect "", "={rdx}"() nounwind
+  %a3 = call i64 asm sideeffect "", "={r8}"() nounwind
+  %a4 = call i64 asm sideeffect "", "={r9}"() nounwind
+  %a5 = call i64 asm sideeffect "", "={r10}"() nounwind
+  %a6 = call i64 asm sideeffect "", "={r11}"() nounwind
+  %a10 = call <2 x double> asm sideeffect "", "={xmm0}"() nounwind
+  %a11 = call <2 x double> asm sideeffect "", "={xmm1}"() nounwind
+  %a12 = call <2 x double> asm sideeffect "", "={xmm2}"() nounwind
+  %a13 = call <2 x double> asm sideeffect "", "={xmm3}"() nounwind
+  %a14 = call <2 x double> asm sideeffect "", "={xmm4}"() nounwind
+  %a15 = call <2 x double> asm sideeffect "", "={xmm5}"() nounwind
+  %a16 = call <2 x double> asm sideeffect "", "={xmm6}"() nounwind
+  %a17 = call <2 x double> asm sideeffect "", "={xmm7}"() nounwind
+  %a18 = call <2 x double> asm sideeffect "", "={xmm8}"() nounwind
+  %a19 = call <2 x double> asm sideeffect "", "={xmm9}"() nounwind
+  %a20 = call <2 x double> asm sideeffect "", "={xmm10}"() nounwind
+  %a21 = call <2 x double> asm sideeffect "", "={xmm11}"() nounwind
+  %a22 = call <2 x double> asm sideeffect "", "={xmm12}"() nounwind
+  %a23 = call <2 x double> asm sideeffect "", "={xmm13}"() nounwind
+  %a24 = call <2 x double> asm sideeffect "", "={xmm14}"() nounwind
+  %a25 = call <2 x double> asm sideeffect "", "={xmm15}"() nounwind
+  call preserve_allcc double @bar_double(i64 1, i64 2)
+  call void asm sideeffect "", "{rax},{rcx},{rdx},{r8},{r9},{r10},{r11},{xmm0},{xmm1},{xmm2},{xmm3},{xmm4},{xmm5},{xmm6},{xmm7},{xmm8},{xmm9},{xmm10},{xmm11},{xmm12},{xmm13},{xmm14},{xmm15}"(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, <2 x double> %a10, <2 x double> %a11, <2 x double> %a12, <2 x double> %a13, <2 x double> %a14, <2 x double> %a15, <2 x double> %a16, <2 x double> %a17, <2 x double> %a18, <2 x double> %a19, <2 x double> %a20, <2 x double> %a21, <2 x double> %a22, <2 x double> %a23, <2 x double> %a24, <2 x double> %a25)
+  ret void
+}

llvm/test/CodeGen/X86/preserve_allcc64.ll

-; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=corei7     | FileCheck --check-prefix=SSE %s
-; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=corei7-avx | FileCheck --check-prefix=AVX %s
+; RUN: sed -e "s/RETTYPE/void/;s/RETVAL//" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7 | FileCheck --check-prefixes=ALL,SSE,VOID %s
+; RUN: sed -e "s/RETTYPE/i32/;s/RETVAL/undef/" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7 | FileCheck --check-prefixes=ALL,SSE,INT %s
+; RUN: sed -e "s/RETTYPE/\{i64\,i64\}/;s/RETVAL/undef/" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7 | FileCheck --check-prefixes=ALL,SSE,INT128 %s
+;
+; RUN: sed -e "s/RETTYPE/void/;s/RETVAL//" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7-avx | FileCheck --check-prefixes=ALL,AVX,VOID %s
+; RUN: sed -e "s/RETTYPE/i32/;s/RETVAL/undef/" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7-avx | FileCheck --check-prefixes=ALL,AVX,INT %s
+; RUN: sed -e "s/RETTYPE/\{i64\,i64\}/;s/RETVAL/undef/" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7-avx | FileCheck --check-prefixes=ALL,AVX,INT128 %s
 
-define preserve_allcc void @preserve_allcc1() nounwind {
+define preserve_allcc RETTYPE @preserve_allcc1(i64, i64, double, double) nounwind {
 entry:
-;SSE-LABEL: preserve_allcc1
-;SSE:       pushq %r10
-;SSE-NEXT:  pushq %r9
-;SSE-NEXT:  pushq %r8
-;SSE-NEXT:  pushq %rdi
-;SSE-NEXT:  pushq %rsi
-;SSE-NEXT:  pushq %rdx
-;SSE-NEXT:  pushq %rcx
-;SSE-NEXT:  pushq %rax
-;SSE-NEXT:  pushq %rbp
-;SSE-NEXT:  pushq %r15
-;SSE-NEXT:  pushq %r14
-;SSE-NEXT:  pushq %r13
-;SSE-NEXT:  pushq %r12
-;SSE-NEXT:  pushq %rbx
-;SSE:       movaps %xmm15
-;SSE-NEXT:  movaps %xmm14
-;SSE-NEXT:  movaps %xmm13
-;SSE-NEXT:  movaps %xmm12
-;SSE-NEXT:  movaps %xmm11
-;SSE-NEXT:  movaps %xmm10
-;SSE-NEXT:  movaps %xmm9
-;SSE-NEXT:  movaps %xmm8
-;SSE-NEXT:  movaps %xmm7
-;SSE-NEXT:  movaps %xmm6
-;SSE-NEXT:  movaps %xmm5
-;SSE-NEXT:  movaps %xmm4
-;SSE-NEXT:  movaps %xmm3
-;SSE-NEXT:  movaps %xmm2
-;SSE-NEXT:  movaps %xmm1
-;SSE-NEXT:  movaps %xmm0
-;AVX-LABEL: preserve_allcc1
-;AVX:       pushq %r10
-;AVX-NEXT:  pushq %r9
-;AVX-NEXT:  pushq %r8
-;AVX-NEXT:  pushq %rdi
-;AVX-NEXT:  pushq %rsi
-;AVX-NEXT:  pushq %rdx
-;AVX-NEXT:  pushq %rcx
-;AVX-NEXT:  pushq %rax
-;AVX-NEXT:  pushq %rbp
-;AVX-NEXT:  pushq %r15
-;AVX-NEXT:  pushq %r14
-;AVX-NEXT:  pushq %r13
-;AVX-NEXT:  pushq %r12
-;AVX-NEXT:  pushq %rbx
-;AVX:       vmovups %ymm15
-;AVX-NEXT:  vmovups %ymm14
-;AVX-NEXT:  vmovups %ymm13
-;AVX-NEXT:  vmovups %ymm12
-;AVX-NEXT:  vmovups %ymm11
-;AVX-NEXT:  vmovups %ymm10
-;AVX-NEXT:  vmovups %ymm9
-;AVX-NEXT:  vmovups %ymm8
-;AVX-NEXT:  vmovups %ymm7
-;AVX-NEXT:  vmovups %ymm6
-;AVX-NEXT:  vmovups %ymm5
-;AVX-NEXT:  vmovups %ymm4
-;AVX-NEXT:  vmovups %ymm3
-;AVX-NEXT:  vmovups %ymm2
-;AVX-NEXT:  vmovups %ymm1
-;AVX-NEXT:  vmovups %ymm0
+;ALL-LABEL:   preserve_allcc1
+;ALL:         pushq %r10
+;ALL-NEXT:    pushq %r9
+;ALL-NEXT:    pushq %r8
+;ALL-NEXT:    pushq %rdi
+;ALL-NEXT:    pushq %rsi
+;VOID-NEXT:   pushq %rdx
+;INT-NEXT:    pushq %rdx
+;INT128-NOT:  pushq %rdx
+;ALL-NEXT:    pushq %rcx
+;VOID-NEXT:   pushq %rax
+;INT-NOT:     pushq %rax
+;INT128-NOT:  pushq %rax
+;ALL-NEXT:    pushq %rbp
+;ALL-NEXT:    pushq %r15
+;ALL-NEXT:    pushq %r14
+;ALL-NEXT:    pushq %r13
+;ALL-NEXT:    pushq %r12
+;ALL-NEXT:    pushq %rbx
+;SSE:         movaps %xmm15
+;SSE-NEXT:    movaps %xmm14
+;SSE-NEXT:    movaps %xmm13
+;SSE-NEXT:    movaps %xmm12
+;SSE-NEXT:    movaps %xmm11
+;SSE-NEXT:    movaps %xmm10
+;SSE-NEXT:    movaps %xmm9
+;SSE-NEXT:    movaps %xmm8
+;SSE-NEXT:    movaps %xmm7
+;SSE-NEXT:    movaps %xmm6
+;SSE-NEXT:    movaps %xmm5
+;SSE-NEXT:    movaps %xmm4
+;SSE-NEXT:    movaps %xmm3
+;SSE-NEXT:    movaps %xmm2
+;SSE-NEXT:    movaps %xmm1
+;SSE-NEXT:    movaps %xmm0
+;AVX:         vmovups %ymm15
+;AVX-NEXT:    vmovups %ymm14
+;AVX-NEXT:    vmovups %ymm13
+;AVX-NEXT:    vmovups %ymm12
+;AVX-NEXT:    vmovups %ymm11
+;AVX-NEXT:    vmovups %ymm10
+;AVX-NEXT:    vmovups %ymm9
+;AVX-NEXT:    vmovups %ymm8
+;AVX-NEXT:    vmovups %ymm7
+;AVX-NEXT:    vmovups %ymm6
+;AVX-NEXT:    vmovups %ymm5
+;AVX-NEXT:    vmovups %ymm4
+;AVX-NEXT:    vmovups %ymm3
+;AVX-NEXT:    vmovups %ymm2
+;AVX-NEXT:    vmovups %ymm1
+;AVX-NEXT:    vmovups %ymm0
+;SSE:         movaps {{.*}} %xmm0
+;SSE-NEXT:    movaps {{.*}} %xmm1
+;SSE-NEXT:    movaps {{.*}} %xmm2
+;SSE-NEXT:    movaps {{.*}} %xmm3
+;SSE-NEXT:    movaps {{.*}} %xmm4
+;SSE-NEXT:    movaps {{.*}} %xmm5
+;SSE-NEXT:    movaps {{.*}} %xmm6
+;SSE-NEXT:    movaps {{.*}} %xmm7
+;SSE-NEXT:    movaps {{.*}} %xmm8
+;SSE-NEXT:    movaps {{.*}} %xmm9
+;SSE-NEXT:    movaps {{.*}} %xmm10
+;SSE-NEXT:    movaps {{.*}} %xmm11
+;SSE-NEXT:    movaps {{.*}} %xmm12
+;SSE-NEXT:    movaps {{.*}} %xmm13
+;SSE-NEXT:    movaps {{.*}} %xmm14
+;SSE-NEXT:    movaps {{.*}} %xmm15
+;AVX:         vmovups {{.*}} %ymm0
+;AVX-NEXT:    vmovups {{.*}} %ymm1
+;AVX-NEXT:    vmovups {{.*}} %ymm2
+;AVX-NEXT:    vmovups {{.*}} %ymm3
+;AVX-NEXT:    vmovups {{.*}} %ymm4
+;AVX-NEXT:    vmovups {{.*}} %ymm5
+;AVX-NEXT:    vmovups {{.*}} %ymm6
+;AVX-NEXT:    vmovups {{.*}} %ymm7
+;AVX-NEXT:    vmovups {{.*}} %ymm8
+;AVX-NEXT:    vmovups {{.*}} %ymm9
+;AVX-NEXT:    vmovups {{.*}} %ymm10
+;AVX-NEXT:    vmovups {{.*}} %ymm11
+;AVX-NEXT:    vmovups {{.*}} %ymm12
+;AVX-NEXT:    vmovups {{.*}} %ymm13
+;AVX-NEXT:    vmovups {{.*}} %ymm14
+;AVX-NEXT:    vmovups {{.*}} %ymm15
+;ALL:         popq    %rbx
+;ALL-NEXT:    popq    %r12
+;ALL-NEXT:    popq    %r13
+;ALL-NEXT:    popq    %r14
+;ALL-NEXT:    popq    %r15
+;ALL-NEXT:    popq    %rbp
+;VOID-NEXT:   popq    %rax
+;INT-NOT:     popq    %rax
+;INT128-NOT:  popq    %rax
+;ALL-NEXT:    popq    %rcx
+;VOID-NEXT:   popq    %rdx
+;INT-NEXT:    popq    %rdx
+;INT128-NOT:  popq    %rdx
+;ALL-NEXT:    popq    %rsi
+;ALL-NEXT:    popq    %rdi
+;ALL-NEXT:    popq    %r8
+;ALL-NEXT:    popq    %r9
+;ALL-NEXT:    popq    %r10
   call void asm sideeffect "", "~{rax},~{rbx},~{rcx},~{rdx},~{rsi},~{rdi},~{r8},~{r9},~{r10},~{r11},~{r12},~{r13},~{r14},~{r15},~{rbp},~{xmm0},~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15}"()
-  ret void
+  ret RETTYPE RETVAL
 }
 
-; Make sure only R11 is saved before the call
-declare preserve_allcc void @bar(i64, i64, double, double)
+; Make sure R11 and return registers are saved before the call
+declare preserve_allcc RETTYPE @bar(i64, i64, double, double)
 define void @preserve_allcc2() nounwind {
 entry:
-;SSE-LABEL: preserve_allcc2
-;SSE:       movq %r11, [[REG:%[a-z0-9]+]]
-;SSE-NOT:   movaps %xmm
-;SSE:       movq [[REG]], %r11
+;ALL-LABEL: preserve_allcc2
+;VOID-NOT:  movq %rax, [[REG1:%[a-z0-9]+]]
+;INT:       movq %rax, [[REG1:%[a-z0-9]+]]
+;INT128:    movq %rax, [[REG1:%[a-z0-9]+]]
+;VOID-NOT:  movq %rdx, [[REG2:%[a-z0-9]+]]
+;INT-NOT:   movq %rdx, [[REG2:%[a-z0-9]+]]
+;INT128:    movq %rdx, [[REG2:%[a-z0-9]+]]
+;ALL:       movq %r11, [[REG3:%[a-z0-9]+]]
+;ALL-NOT:   movaps %xmm
+;VOID-NOT:  movq {{.*}}, %rax
+;INT:       movq [[REG1]], %rax
+;INT128:    movq [[REG1]], %rax
+;VOID-NOT:  movq {{.*}}, %rdx
+;INT-NOT:   movq {{.*}}, %rdx
+;INT128:    movq [[REG2]], %rdx
+;ALL:       movq [[REG3]], %r11
   %a0 = call i64 asm sideeffect "", "={rax}"() nounwind
   %a1 = call i64 asm sideeffect "", "={rcx}"() nounwind
   %a2 = call i64 asm sideeffect "", "={rdx}"() nounwind
   %a3 = call i64 asm sideeffect "", "={r8}"() nounwind
   %a4 = call i64 asm sideeffect "", "={r9}"() nounwind
   %a5 = call i64 asm sideeffect "", "={r10}"() nounwind
   %a6 = call i64 asm sideeffect "", "={r11}"() nounwind
   %a10 = call <2 x double> asm sideeffect "", "={xmm2}"() nounwind
   %a11 = call <2 x double> asm sideeffect "", "={xmm3}"() nounwind
   %a12 = call <2 x double> asm sideeffect "", "={xmm4}"() nounwind
   %a13 = call <2 x double> asm sideeffect "", "={xmm5}"() nounwind
   %a14 = call <2 x double> asm sideeffect "", "={xmm6}"() nounwind
   %a15 = call <2 x double> asm sideeffect "", "={xmm7}"() nounwind
   %a16 = call <2 x double> asm sideeffect "", "={xmm8}"() nounwind
   %a17 = call <2 x double> asm sideeffect "", "={xmm9}"() nounwind
   %a18 = call <2 x double> asm sideeffect "", "={xmm10}"() nounwind
   %a19 = call <2 x double> asm sideeffect "", "={xmm11}"() nounwind
   %a20 = call <2 x double> asm sideeffect "", "={xmm12}"() nounwind
   %a21 = call <2 x double> asm sideeffect "", "={xmm13}"() nounwind
   %a22 = call <2 x double> asm sideeffect "", "={xmm14}"() nounwind
   %a23 = call <2 x double> asm sideeffect "", "={xmm15}"() nounwind
-  call preserve_allcc void @bar(i64 1, i64 2, double 3.0, double 4.0)
+  call preserve_allcc RETTYPE @bar(i64 1, i64 2, double 3.0, double 4.0)
   call void asm sideeffect "", "{rax},{rcx},{rdx},{r8},{r9},{r10},{r11},{xmm2},{xmm3},{xmm4},{xmm5},{xmm6},{xmm7},{xmm8},{xmm9},{xmm10},{xmm11},{xmm12},{xmm13},{xmm14},{xmm15}"(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, <2 x double> %a10, <2 x double> %a11, <2 x double> %a12, <2 x double> %a13, <2 x double> %a14, <2 x double> %a15, <2 x double> %a16, <2 x double> %a17, <2 x double> %a18, <2 x double> %a19, <2 x double> %a20, <2 x double> %a21, <2 x double> %a22, <2 x double> %a23)
   ret void
 }

llvm/test/CodeGen/X86/preserve_mostcc64-ret-double.ll

+; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=corei7 | FileCheck %s
+
+; Make sure XMM0 (return register) and R11 are saved before the call
+declare preserve_mostcc double @foo_double(i64, i64)
+define void @preserve_mostcc1() nounwind {
+entry:
+;CHECK-LABEL: preserve_mostcc1
+;CHECK:       movq %r11, [[REG1:%[a-z0-9]+]]
+;CHECK:       movaps %xmm0, [[REG2:[-0-9]*\(%r[sb]p\)]]
+;CHECK:       call
+;CHECK:       movq [[REG1]], %r11
+;CHECK:       movaps [[REG2]], %xmm0
+  %a0 = call i64 asm sideeffect "", "={rax}"() nounwind
+  %a1 = call i64 asm sideeffect "", "={rcx}"() nounwind
+  %a2 = call i64 asm sideeffect "", "={rdx}"() nounwind
+  %a3 = call i64 asm sideeffect "", "={r8}"() nounwind
+  %a4 = call i64 asm sideeffect "", "={r9}"() nounwind
+  %a5 = call i64 asm sideeffect "", "={r10}"() nounwind
+  %a6 = call i64 asm sideeffect "", "={r11}"() nounwind
+  %a10 = call <2 x double> asm sideeffect "", "={xmm0}"() nounwind
+  %a11 = call <2 x double> asm sideeffect "", "={xmm1}"() nounwind
+  %a12 = call <2 x double> asm sideeffect "", "={xmm2}"() nounwind
+  %a13 = call <2 x double> asm sideeffect "", "={xmm3}"() nounwind
+  %a14 = call <2 x double> asm sideeffect "", "={xmm4}"() nounwind
+  %a15 = call <2 x double> asm sideeffect "", "={xmm5}"() nounwind
+  %a16 = call <2 x double> asm sideeffect "", "={xmm6}"() nounwind
+  %a17 = call <2 x double> asm sideeffect "", "={xmm7}"() nounwind
+  %a18 = call <2 x double> asm sideeffect "", "={xmm8}"() nounwind
+  %a19 = call <2 x double> asm sideeffect "", "={xmm9}"() nounwind
+  %a20 = call <2 x double> asm sideeffect "", "={xmm10}"() nounwind
+  %a21 = call <2 x double> asm sideeffect "", "={xmm11}"() nounwind
+  %a22 = call <2 x double> asm sideeffect "", "={xmm12}"() nounwind
+  %a23 = call <2 x double> asm sideeffect "", "={xmm13}"() nounwind
+  %a24 = call <2 x double> asm sideeffect "", "={xmm14}"() nounwind
+  %a25 = call <2 x double> asm sideeffect "", "={xmm15}"() nounwind
+  call preserve_mostcc double @foo_double(i64 1, i64 2)
+  call void asm sideeffect "", "{rax},{rcx},{rdx},{r8},{r9},{r10},{r11},{xmm0},{xmm1},{xmm2},{xmm3},{xmm4},{xmm5},{xmm6},{xmm7},{xmm8},{xmm9},{xmm10},{xmm11},{xmm12},{xmm13},{xmm14},{xmm15}"(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, <2 x double> %a10, <2 x double> %a11, <2 x double> %a12, <2 x double> %a13, <2 x double> %a14, <2 x double> %a15, <2 x double> %a16, <2 x double> %a17, <2 x double> %a18, <2 x double> %a19, <2 x double> %a20, <2 x double> %a21, <2 x double> %a22, <2 x double> %a23, <2 x double> %a24, <2 x double> %a25)
+  ret void
+}

llvm/test/CodeGen/X86/preserve_mostcc64.ll

-; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=corei7     | FileCheck --check-prefix=SSE %s
-; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=corei7-avx | FileCheck --check-prefix=AVX %s
+; RUN: sed -e "s/RETTYPE/void/;s/RETVAL//" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7 | FileCheck --check-prefixes=ALL,VOID %s
+; RUN: sed -e "s/RETTYPE/i32/;s/RETVAL/undef/" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7 | FileCheck --check-prefixes=ALL,INT %s
+; RUN: sed -e "s/RETTYPE/\{i64\,i64\}/;s/RETVAL/undef/" %s | llc -mtriple=x86_64-apple-darwin -mcpu=corei7 | FileCheck --check-prefixes=ALL,INT128 %s
 
-; Every GPR should be saved - except r11
-define preserve_mostcc void @preserve_mostcc1() nounwind {
+; Every GPR should be saved - except r11 and return registers
+define preserve_mostcc RETTYPE @preserve_mostcc1(i64, i64, double, double) nounwind {
 entry:
-;SSE-LABEL: preserve_mostcc1
-;SSE:       pushq %r10
-;SSE-NEXT:  pushq %r9
-;SSE-NEXT:  pushq %r8
-;SSE-NEXT:  pushq %rdi
-;SSE-NEXT:  pushq %rsi
-;SSE-NEXT:  pushq %rdx
-;SSE-NEXT:  pushq %rcx
-;SSE-NEXT:  pushq %rax
-;SSE-NEXT:  pushq %rbp
-;SSE-NEXT:  pushq %r15
-;SSE-NEXT:  pushq %r14
-;SSE-NEXT:  pushq %r13
-;SSE-NEXT:  pushq %r12
-;SSE-NEXT:  pushq %rbx
-;AVX-LABEL: preserve_mostcc1
-;AVX:       pushq %r10
-;AVX-NEXT:  pushq %r9
-;AVX-NEXT:  pushq %r8
-;AVX-NEXT:  pushq %rdi
-;AVX-NEXT:  pushq %rsi
-;AVX-NEXT:  pushq %rdx
-;AVX-NEXT:  pushq %rcx
-;AVX-NEXT:  pushq %rax
-;AVX-NEXT:  pushq %rbp
-;AVX-NEXT:  pushq %r15
-;AVX-NEXT:  pushq %r14
-;AVX-NEXT:  pushq %r13
-;AVX-NEXT:  pushq %r12
-;AVX-NEXT:  pushq %rbx
+;ALL-LABEL:   preserve_mostcc1
+;ALL:         pushq %r10
+;ALL-NEXT:    pushq %r9
+;ALL-NEXT:    pushq %r8
+;ALL-NEXT:    pushq %rdi
+;ALL-NEXT:    pushq %rsi
+;VOID-NEXT:   pushq %rdx
+;INT-NEXT:    pushq %rdx
+;INT128-NOT:  pushq %rdx
+;ALL-NEXT:    pushq %rcx
+;VOID-NEXT:   pushq %rax
+;INT-NOT:     pushq %rax
+;INT128-NOT:  pushq %rax
+;ALL-NEXT:    pushq %rbp
+;ALL-NEXT:    pushq %r15
+;ALL-NEXT:    pushq %r14
+;ALL-NEXT:    pushq %r13
+;ALL-NEXT:    pushq %r12
+;ALL-NEXT:    pushq %rbx
+;ALL:         popq    %rbx
+;ALL-NEXT:    popq    %r12
+;ALL-NEXT:    popq    %r13
+;ALL-NEXT:    popq    %r14
+;ALL-NEXT:    popq    %r15
+;ALL-NEXT:    popq    %rbp
+;VOID-NEXT:   popq    %rax
+;INT-NOT:     popq    %rax
+;INT128-NOT:  popq    %rax
+;ALL-NEXT:    popq    %rcx
+;VOID-NEXT:   popq    %rdx
+;INT-NEXT:    popq    %rdx
+;INT128-NOT:  popq    %rdx
+;ALL-NEXT:    popq    %rsi
+;ALL-NEXT:    popq    %rdi
+;ALL-NEXT:    popq    %r8
+;ALL-NEXT:    popq    %r9
+;ALL-NEXT:    popq    %r10
   call void asm sideeffect "", "~{rax},~{rbx},~{rcx},~{rdx},~{rsi},~{rdi},~{r8},~{r9},~{r10},~{r11},~{r12},~{r13},~{r14},~{r15},~{rbp},~{xmm0},~{xmm1},~{xmm2},~{xmm3},~{xmm4},~{xmm5},~{xmm6},~{xmm7},~{xmm8},~{xmm9},~{xmm10},~{xmm11},~{xmm12},~{xmm13},~{xmm14},~{xmm15}"()
-  ret void
+  ret RETTYPE RETVAL
 }
 
-; Make sure R11 and XMMs are saved before the call
-declare preserve_mostcc void @foo(i64, i64, double, double)
+; Make sure R11, return registers and XMMs are saved before the call
+declare preserve_mostcc RETTYPE @foo(i64, i64, double, double)
 define void @preserve_mostcc2() nounwind {
 entry:
-;SSE-LABEL: preserve_mostcc2
-;SSE:       movq %r11, [[REG:%[a-z0-9]+]]
-;SSE:       movaps %xmm2
-;SSE:       movaps %xmm3
-;SSE:       movaps %xmm4
-;SSE:       movaps %xmm5
-;SSE:       movaps %xmm6
-;SSE:       movaps %xmm7
-;SSE:       movaps %xmm8
-;SSE:       movaps %xmm9
-;SSE:       movaps %xmm10
-;SSE:       movaps %xmm11
-;SSE:       movaps %xmm12
-;SSE:       movaps %xmm13
-;SSE:       movaps %xmm14
-;SSE:       movaps %xmm15
-;SSE:       movq [[REG]], %r11
+;ALL-LABEL: preserve_mostcc2
+;VOID-NOT:  movq %rax, [[REG1:%[a-z0-9]+]]
+;INT:       movq %rax, [[REG1:%[a-z0-9]+]]
+;INT128:    movq %rax, [[REG1:%[a-z0-9]+]]
+;VOID-NOT:  movq %rdx, [[REG2:%[a-z0-9]+]]
+;INT-NOT:   movq %rdx, [[REG2:%[a-z0-9]+]]
+;INT128:    movq %rdx, [[REG2:%[a-z0-9]+]]
+;ALL:       movq %r11, [[REG3:%[a-z0-9]+]]
+;ALL:       movaps %xmm2
+;ALL:       movaps %xmm3
+;ALL:       movaps %xmm4
+;ALL:       movaps %xmm5
+;ALL:       movaps %xmm6
+;ALL:       movaps %xmm7
+;ALL:       movaps %xmm8
+;ALL:       movaps %xmm9
+;ALL:       movaps %xmm10
+;ALL:       movaps %xmm11
+;ALL:       movaps %xmm12
+;ALL:       movaps %xmm13
+;ALL:       movaps %xmm14
+;ALL:       movaps %xmm15
+;ALL:       call
+;VOID-NOT:  movq {{.*}}, %rax
+;INT:       movq [[REG1]], %rax
+;INT128:    movq [[REG1]], %rax
+;VOID-NOT:  movq {{.*}}, %rdx
+;INT-NOT:   movq {{.*}}, %rdx
+;INT128:    movq [[REG2]], %rdx
+;ALL:       movq [[REG3]], %r11
+;ALL:       movaps {{.*}} %xmm2
+;ALL:       movaps {{.*}} %xmm3
+;ALL:       movaps {{.*}} %xmm4
+;ALL:       movaps {{.*}} %xmm5
+;ALL:       movaps {{.*}} %xmm6
+;ALL:       movaps {{.*}} %xmm7
+;ALL:       movaps {{.*}} %xmm8
+;ALL:       movaps {{.*}} %xmm9
+;ALL:       movaps {{.*}} %xmm10
+;ALL:       movaps {{.*}} %xmm11
+;ALL:       movaps {{.*}} %xmm12
+;ALL:       movaps {{.*}} %xmm13
+;ALL:       movaps {{.*}} %xmm14
+;ALL:       movaps {{.*}} %xmm15
   %a0 = call i64 asm sideeffect "", "={rax}"() nounwind
   %a1 = call i64 asm sideeffect "", "={rcx}"() nounwind
   %a2 = call i64 asm sideeffect "", "={rdx}"() nounwind
   %a3 = call i64 asm sideeffect "", "={r8}"() nounwind
   %a4 = call i64 asm sideeffect "", "={r9}"() nounwind
   %a5 = call i64 asm sideeffect "", "={r10}"() nounwind
   %a6 = call i64 asm sideeffect "", "={r11}"() nounwind
   %a10 = call <2 x double> asm sideeffect "", "={xmm2}"() nounwind
   %a11 = call <2 x double> asm sideeffect "", "={xmm3}"() nounwind
   %a12 = call <2 x double> asm sideeffect "", "={xmm4}"() nounwind
   %a13 = call <2 x double> asm sideeffect "", "={xmm5}"() nounwind
   %a14 = call <2 x double> asm sideeffect "", "={xmm6}"() nounwind
   %a15 = call <2 x double> asm sideeffect "", "={xmm7}"() nounwind
   %a16 = call <2 x double> asm sideeffect "", "={xmm8}"() nounwind
   %a17 = call <2 x double> asm sideeffect "", "={xmm9}"() nounwind
   %a18 = call <2 x double> asm sideeffect "", "={xmm10}"() nounwind
   %a19 = call <2 x double> asm sideeffect "", "={xmm11}"() nounwind
   %a20 = call <2 x double> asm sideeffect "", "={xmm12}"() nounwind
   %a21 = call <2 x double> asm sideeffect "", "={xmm13}"() nounwind
   %a22 = call <2 x double> asm sideeffect "", "={xmm14}"() nounwind
   %a23 = call <2 x double> asm sideeffect "", "={xmm15}"() nounwind
-  call preserve_mostcc void @foo(i64 1, i64 2, double 3.0, double 4.0)
+  call preserve_mostcc RETTYPE @foo(i64 1, i64 2, double 3.0, double 4.0)
   call void asm sideeffect "", "{rax},{rcx},{rdx},{r8},{r9},{r10},{r11},{xmm2},{xmm3},{xmm4},{xmm5},{xmm6},{xmm7},{xmm8},{xmm9},{xmm10},{xmm11},{xmm12},{xmm13},{xmm14},{xmm15}"(i64 %a0, i64 %a1, i64 %a2, i64 %a3, i64 %a4, i64 %a5, i64 %a6, <2 x double> %a10, <2 x double> %a11, <2 x double> %a12, <2 x double> %a13, <2 x double> %a14, <2 x double> %a15, <2 x double> %a16, <2 x double> %a17, <2 x double> %a18, <2 x double> %a19, <2 x double> %a20, <2 x double> %a21, <2 x double> %a22, <2 x double> %a23)
   ret void
 }