[X86] Support of no_caller_saved_registers attribute (LLVM part) - restart

[oren_ben_simhon]

Summary

This patch implements the LLVM part for no_caller_saved_registers attribute as appear in interrupt and exception handler proposal.

The review is a update for an old review:
Support of no_caller_saved_registers attribute (LLVM part).

The main difference from the old review is that in this patch we use the dynamic CSR mechanism in order to remove returned /passed arguments from the function regmask/CSR list.

[oren_ben_simhon]

A kind reminder for reviewing this patch :-) Thanks in advance.

[oren_ben_simhon]

Updated the test

[aaboud]

Thanks Oren for resuming the no_caller_saved_registers patch.
Few comments below.

lib/Target/X86/X86FastISel.cpp

3189 ↗

(On Diff #95760)

Why do we need to check the caller function?
Is there a good reason why we would disable fast lowering in such case?

lib/Target/X86/X86ISelLowering.cpp

3810 ↗

(On Diff #95760)

Can you add a comment explaining this line?
For example,
// NoCallerSavedRegisters has same BASE preserved registers as X86_INTR calling convention.

3833 ↗

(On Diff #95760)

No need for this variable, see comment below.
Just fold this check into the "if-condition".

3909 ↗

(On Diff #95760)

No need to pass the flag "RemovePassedReturnedRegs"
If "RemovePassedReturnedRegs=false" then "RegMask=nullptr", so it is enough to check that "RegMask!=nullptr" as we do already.

Please remove "RemovePassedReturnedRegs" from this function.

lib/Target/X86/X86RegisterInfo.cpp

276 ↗

(On Diff #95760)

Add a comment here too.

[oren_ben_simhon]

Implemented comments posted until 04/25 (Thanks Amjad)

[aaboud]

lib/Target/X86/X86FastISel.cpp

3185 ↗

(On Diff #96708)

You are not using this variable anymore, right?

lib/Target/X86/X86ISelLowering.cpp

2260 ↗

(On Diff #96708)

You are using this check 3 times in this function.
It worth moving this to the top of the function, and create a bool variable with well defined name.
E.g.,
bool ShouldDisableCalleeSavedRegister = CallConv == CallingConv::X86_RegCall || MF.getFunction()->hasFnAttribute("no_caller_saved_registers");

lib/Target/X86/X86RegisterInfo.cpp

277 ↗

(On Diff #96708)

[Style]: It is up to you, but I prefer that you align the comment lines to 80 chars, i.e., you can still move words from 2nd line to 1st one.

test/CodeGen/X86/x86-no_caller_saved_registers.ll

1 ↗

(On Diff #96708)

I would add one or two examples regarding disabling preserved registers, similar to the regcall calling convention.
These examples dose not need to have x86_intrcc calling convention, but you should use the no_caller_saved_registers attribute with other calling convention, make sure to have functions with parameters to emphasize the difference.

[oren_ben_simhon]

Implemented comments posted until 04/27 (Thanks Amjad)

[aaboud]

lib/Target/X86/X86ISelLowering.cpp

2242 ↗

(On Diff #96922)

[Style]: Please run clang-format.

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

9 ↗

(On Diff #96922)

Why do we need the "local_unnamed_addr" attribute in this test?

21 ↗

(On Diff #96922)

Please do not use code that you might not control its outcome.
Instead try the inline assembly that allows you to force register usage.

E.g., see other LIT tests you have in the patch:
call void asm sideeffect "", "~{rax},~{rbx},~{rbp},~{r11},~{xmm0}"()

31 ↗

(On Diff #96922)

I would prefer second function that have the "no_caller_saved_registers" but that has no parameters passed in registers, and then you can show that it pushes all used registers.

test/CodeGen/X86/x86-no_caller_saved_registers.ll

1 ↗

(On Diff #96922)

Why did you remove the -O0 command line?
It is needed to check the behavior of FastLowerCall (X86FastISel).

[MatzeB]

Why isn't this "just" a new calling convention at the LLVM level? What are the benefits of having this as an attribute on top of the calling convention?

[aaboud]

In D31876#739596, @MatzeB wrote:

Why isn't this "just" a new calling convention at the LLVM level? What are the benefits of having this as an attribute on top of the calling convention?

Calling convention determines not only what the callee and the caller function saved registers are, but also how we move parameters to the called function.

"no_caller_saved_registers" attribute can be attached to a function with any calling convention, thus it is not a calling convention by itself.
However, this attribute overrides the calling convention callee/caller saved register list.

It is well explained in the Clang part documentation.

[MatzeB]

In D31876#739953, @aaboud wrote:

In D31876#739596, @MatzeB wrote:

Why isn't this "just" a new calling convention at the LLVM level? What are the benefits of having this as an attribute on top of the calling convention?

Calling convention determines not only what the callee and the caller function saved registers are, but also how we move parameters to the called function.

"no_caller_saved_registers" attribute can be attached to a function with any calling convention, thus it is not a calling convention by itself.
However, this attribute overrides the calling convention callee/caller saved register list.

It is well explained in the Clang part documentation.

Ok, may still be worth linking/referencing this in the llvm documentation.

But I am still curious: What motivated the introduction of this attribute, do you have an example where it is used/would be used if it existed?

[aaboud]

Ok, may still be worth linking/referencing this in the llvm documentation.

But I am still curious: What motivated the introduction of this attribute, do you have an example where it is used/would be used if it existed?

interrupt handler calling convention force the interrupt handler function to save all modified registers, as there is no caller that would do that, right?

Now, assume a case where the interrupt handler function "foo()" calls a helper function "bar()", then it needs to save all the registers preserved by callers according to the calling convention of "bar()", even if "bar()" is not going to modify any of them.
However, if we add "no_caller_saved_registers" attribute to "bar()", then we move the responsibility of saving the modified registers to "bar()" which is the function that knows what registers are going to be modified and will save only them.

I hope I explained the example well enough.

[MatzeB]

In D31876#740018, @aaboud wrote:

Ok, may still be worth linking/referencing this in the llvm documentation.

But I am still curious: What motivated the introduction of this attribute, do you have an example where it is used/would be used if it existed?

interrupt handler calling convention force the interrupt handler function to save all modified registers, as there is no caller that would do that, right?

Now, assume a case where the interrupt handler function "foo()" calls a helper function "bar()", then it needs to save all the registers preserved by callers according to the calling convention of "bar()", even if "bar()" is not going to modify any of them.
However, if we add "no_caller_saved_registers" attribute to "bar()", then we move the responsibility of saving the modified registers to "bar()" which is the function that knows what registers are going to be modified and will save only them.

I hope I explained the example well enough.

So this is the old discussion about choosing the right balance between caller and callee saved registers. Sounds to me like whoever uses this should have rather spend the time making sure the compiler can inline his calls and mark all his methods static so the compiler can optimize the calling convention automatically.

Makes me wonder a bit where this should start and end. To play devils advocate: Why stop here and not allow specifying the number or exact list of caller saved registers, whether flags are preserved, what registers to use for arguments, etc. IMO that just increases compiler complexity and hence potential bugs for very little gain.

Anyway seems like others find this useful and are willing to implement and maintain this, so I won't stand in the way.

[oren_ben_simhon]

This comment has been deleted.

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

31 ↗

(On Diff #96922)

We don't need a function without parameters in order to show that the function pushes all used registers.

This is already tested in the function bar1 and bar0 (in the other test file).
In bar0 you can see that it pushes xmm0 because it uses it.
In bar1 you can see that it pushes rdx and xmm1 because it uses it.

So there is no additional value in a function without arguments.

[oren_ben_simhon]

Implemented comments posted until 04/28 (Thanks Amjad and Matthias)

[aaboud]

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

13 ↗

(On Diff #97000)

I am afraid that the order is important here.
Assume that the code would look like this:

pushq %rdx
pushq %rdi
pushq %rsi
movaps %xmm1
...

Then this test will still pass, right?

20 ↗

(On Diff #97000)

I assume that the return value will be passed by %eax, so please check that %rax is not saved either.

31 ↗

(On Diff #96922)

I do not see "bar1" in the other file, and indeed it worth having test that shows that callee do preserve GPR, not only XMM resgister.

What I want to have here is a test that shows we do preserve same registers as in "bar1" above, once they are not used to pass parameters. Otherwise, how can we be sure that the implementation does not always skip these registers you used above?

define x86_64_sysvcc void @bar2() #1 {
; CHECK-LABEL: bar1
; CHECK:     pushq %rdx
; CHECK:     pushq %rax
; CHECK:     pushq  %rdi
; CHECK:     pushq  %rsi
; CHECK:     movaps  %xmm1
; CHECK:     movaps  %xmm0
; CHECK:     movaps {{.*}}, %xmm0
; CHECK:     movaps {{.*}}, %xmm1
; CHECK:     popq %rsi
; CHECK:     popq %rdi
; CHECK:     popq %rax
; CHECK:     popq %rdx
; CHECK:     retq
  call void asm sideeffect "", "~{rdx},~{rax},~{xmm1},~{rdi},~{rsi},~{xmm0}"()
ret void
}

Either way, I do not see the added value of "foo1", why are we testing a function that has no "no_caller_saved_registers" attribute here? it must be checked somewhere else, right?

test/CodeGen/X86/x86-no_caller_saved_registers.ll

24 ↗

(On Diff #97000)

move this to line 27 and add CHECK-LABEL: for @foo0.

[oren_ben_simhon]

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

13 ↗

(On Diff #97000)

That's correct. I will use update_llc_test_checks.py for this function instead.

31 ↗

(On Diff #96922)

I do not see "bar1" in the other file

• bar1 is in this file.
• bar0 is the the other file.

It was probably misunderstood, In all three functions (bar0, bar1 and foo1) we check that parameters that are passed as arguments are not preserved.

For example, in foo1, we see that xmm0 is preserved. Why is it preserved (you ask)?
because it is passed to bar1, which has no_caller_saved_registers, and as such the caller needs to preserve the passed arguments.
This is why foo1 doesn't need to have the attribute no_caller_saved_registers. But still it tests the attribute of the callee.

Either way, I do not see the added value of "foo1", why are we testing a function that has no "no_caller_saved_registers" attribute here? it must be checked somewhere else, right?

Every time you test a calling convention, you need to test what happens in the caller of that CC function (foo0/foo1) and inside the CC function itself (bar1/bar0).

What I want to have here is a test that shows we do preserve same registers as in "bar1" above, once they are not used to pass parameters.

Again, this is already tested, please see my previous comment:

This is already tested in the function bar1 and bar0 (in the other test file).
In bar0 you can see that it pushes xmm0 because it uses it.
In bar1 you can see that it pushes rdx and xmm1 because it uses it.

[oren_ben_simhon]

Implemented comments posted until 04/30 (Thanks Amjad)

[oren_ben_simhon]

Updated the tests.

[aaboud]

Thanks Oren,
Code looks much better now.

llvm/trunk/lib/Target/X86/X86FastISel.cpp

bool X86FastISel::fastLowerCall(CallLoweringInfo &CLI) {
   bool &IsTailCall    = CLI.IsTailCall;
   bool IsVarArg       = CLI.IsVarArg;
   const Value *Callee = CLI.Callee;
   MCSymbol *Symbol = CLI.Symbol;
 
   bool Is64Bit        = Subtarget->is64Bit();
   bool IsWin64        = Subtarget->isCallingConvWin64(CC);
 
+  const CallInst *CI =
+      CLI.CS ? dyn_cast<CallInst>(CLI.CS->getInstruction()) : nullptr;
+  const Function *CalledFn = CI ? CI->getCalledFunction() : nullptr;
+
+  // Functions with no_caller_saved_registers that need special handling.
+  if ((CI && CI->hasFnAttr("no_caller_saved_registers")) ||
+      (CalledFn && CalledFn->hasFnAttribute("no_caller_saved_registers")))
+    return false;
+
   // Handle only C, fastcc, and webkit_js calling conventions for now.
   switch (CC) {
   default: return false;
   case CallingConv::C:
   case CallingConv::Fast:
   case CallingConv::WebKit_JS:
   case CallingConv::Swift:
   case CallingConv::X86_FastCall:

llvm/trunk/lib/Target/X86/X86ISelLowering.cpp

 X86TargetLowering::LowerReturn(SDValue Chain, CallingConv::ID CallConv,
                                bool isVarArg,
                                const SmallVectorImpl<ISD::OutputArg> &Outs,
                                const SmallVectorImpl<SDValue> &OutVals,
                                const SDLoc &dl, SelectionDAG &DAG) const {
   MachineFunction &MF = DAG.getMachineFunction();
   X86MachineFunctionInfo *FuncInfo = MF.getInfo<X86MachineFunctionInfo>();
 
+  // In some cases we need to disable registers from the default CSR list.
+  // For example, when they are used for argument passing.
+  bool ShouldDisableCalleeSavedRegister =
+      CallConv == CallingConv::X86_RegCall ||
+      MF.getFunction()->hasFnAttribute("no_caller_saved_registers");
+
   if (CallConv == CallingConv::X86_INTR && !Outs.empty())
     report_fatal_error("X86 interrupts may not return any value");
 
   SmallVector<CCValAssign, 16> RVLocs;
   CCState CCInfo(CallConv, isVarArg, MF, RVLocs, *DAG.getContext());
   CCInfo.AnalyzeReturn(Outs, RetCC_X86);
 
   SDValue Flag;
   SmallVector<SDValue, 6> RetOps;
   RetOps.push_back(Chain); // Operand #0 = Chain (updated below)
   // Operand #1 = Bytes To Pop
   RetOps.push_back(DAG.getTargetConstant(FuncInfo->getBytesToPopOnReturn(), dl,
                    MVT::i32));
 
   // Copy the result values into the output registers.
   for (unsigned I = 0, OutsIndex = 0, E = RVLocs.size(); I != E;
        ++I, ++OutsIndex) {
     CCValAssign &VA = RVLocs[I];
     assert(VA.isRegLoc() && "Can only return in registers!");
 
     // Add the register to the CalleeSaveDisableRegs list.
-    if (CallConv == CallingConv::X86_RegCall)
+    if (ShouldDisableCalleeSavedRegister)
       MF.getRegInfo().disableCalleeSavedRegister(VA.getLocReg());
 
     SDValue ValToCopy = OutVals[OutsIndex];
     EVT ValVT = ValToCopy.getValueType();
 
     // Promote values to the appropriate types.
     if (VA.getLocInfo() == CCValAssign::SExt)
       ValToCopy = DAG.getNode(ISD::SIGN_EXTEND, dl, VA.getLocVT(), ValToCopy);
if (VA.needsCustom()) {
 
       Passv64i1ArgInRegs(dl, DAG, Chain, ValToCopy, RegsToPass, VA, RVLocs[++I],
                          Subtarget);
 
       assert(2 == RegsToPass.size() &&
              "Expecting two registers after Pass64BitArgInRegs");
 
       // Add the second register to the CalleeSaveDisableRegs list.
-      if (CallConv == CallingConv::X86_RegCall)
+      if (ShouldDisableCalleeSavedRegister)
         MF.getRegInfo().disableCalleeSavedRegister(RVLocs[I].getLocReg());
     } else {
       RegsToPass.push_back(std::make_pair(VA.getLocReg(), ValToCopy));
     }
 
     // Add nodes to the DAG and add the values into the RetOps list
     for (auto &Reg : RegsToPass) {
       Chain = DAG.getCopyToReg(Chain, dl, Reg.first, Reg.second, Flag);
if (unsigned SRetReg = FuncInfo->getSRetReturnReg()) {
     Chain = DAG.getCopyToReg(Chain, dl, RetValReg, Val, Flag);
     Flag = Chain.getValue(1);
 
     // RAX/EAX now acts like a return value.
     RetOps.push_back(
         DAG.getRegister(RetValReg, getPointerTy(DAG.getDataLayout())));
 
     // Add the returned register to the CalleeSaveDisableRegs list.
-    if (CallConv == CallingConv::X86_RegCall)
+    if (ShouldDisableCalleeSavedRegister)
       MF.getRegInfo().disableCalleeSavedRegister(RetValReg);
   }
 
   const X86RegisterInfo *TRI = Subtarget.getRegisterInfo();
   const MCPhysReg *I =
       TRI->getCalleeSavedRegsViaCopy(&DAG.getMachineFunction());
   if (I) {
     for (; *I; ++I) {
SDValue X86TargetLowering::LowerCallResult(
   // Copy all of the result registers out of their specified physreg.
   for (unsigned I = 0, InsIndex = 0, E = RVLocs.size(); I != E;
        ++I, ++InsIndex) {
     CCValAssign &VA = RVLocs[I];
     EVT CopyVT = VA.getLocVT();
 
     // In some calling conventions we need to remove the used registers
     // from the register mask.
-    if (RegMask && CallConv == CallingConv::X86_RegCall) {
+    if (RegMask) {
       for (MCSubRegIterator SubRegs(VA.getLocReg(), TRI, /*IncludeSelf=*/true);
            SubRegs.isValid(); ++SubRegs)
         RegMask[*SubRegs / 32] &= ~(1u << (*SubRegs % 32));
     }
 
     // If this is x86-64, and we disabled SSE, we can't return FP values
     if ((CopyVT == MVT::f32 || CopyVT == MVT::f64 || CopyVT == MVT::f128) &&
         ((Is64Bit || Ins[InsIndex].Flags.isInReg()) && !Subtarget.hasSSE1())) {
if (Personality == EHPersonality::CoreCLR) {
       // same, so the size of funclets' (mostly empty) frames is dictated by
       // how far this slot is from the bottom (since they allocate just enough
       // space to accommodate holding this slot at the correct offset).
       int PSPSymFI = MFI.CreateStackObject(8, 8, /*isSS=*/false);
       EHInfo->PSPSymFrameIdx = PSPSymFI;
     }
   }
 
-  if (CallConv == CallingConv::X86_RegCall) {
+  if (CallConv == CallingConv::X86_RegCall ||
+      Fn->hasFnAttribute("no_caller_saved_registers")) {
     const MachineRegisterInfo &MRI = MF.getRegInfo();
     for (const auto &Pair : make_range(MRI.livein_begin(), MRI.livein_end()))
       MF.getRegInfo().disableCalleeSavedRegister(Pair.first);
   }
 
   return Chain;
 }
 
X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
 
   MachineFunction &MF = DAG.getMachineFunction();
   bool Is64Bit        = Subtarget.is64Bit();
   bool IsWin64        = Subtarget.isCallingConvWin64(CallConv);
   StructReturnType SR = callIsStructReturn(Outs, Subtarget.isTargetMCU());
   bool IsSibcall      = false;
   X86MachineFunctionInfo *X86Info = MF.getInfo<X86MachineFunctionInfo>();
   auto Attr = MF.getFunction()->getFnAttribute("disable-tail-calls");
+  const CallInst *CI =
+      CLI.CS ? dyn_cast<CallInst>(CLI.CS->getInstruction()) : nullptr;
+  const Function *Fn = CI ? CI->getCalledFunction() : nullptr;
+  bool HasNCSR = (CI && CI->hasFnAttr("no_caller_saved_registers")) ||
+                 (Fn && Fn->hasFnAttribute("no_caller_saved_registers"));
 
   if (CallConv == CallingConv::X86_INTR)
     report_fatal_error("X86 interrupts may not be called directly");
 
   if (Attr.getValueAsString() == "true")
     isTailCall = false;
 
   if (Subtarget.isPICStyleGOT() &&
X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
 
   // Add argument registers to the end of the list so that they are known live
   // into the call.
   for (unsigned i = 0, e = RegsToPass.size(); i != e; ++i)
     Ops.push_back(DAG.getRegister(RegsToPass[i].first,
                                   RegsToPass[i].second.getValueType()));
 
   // Add a register mask operand representing the call-preserved registers.
-  const uint32_t *Mask = RegInfo->getCallPreservedMask(MF, CallConv);
+  // If HasNCSR is asserted (attribute NoCallerSavedRegisters exists) then we
+  // set X86_INTR calling convention because it has the same CSR mask
+  // (same preserved registers).
+  const uint32_t *Mask = RegInfo->getCallPreservedMask(
+      MF, HasNCSR ? CallingConv::X86_INTR : CallConv);
   assert(Mask && "Missing call preserved mask for calling convention");
 
   // If this is an invoke in a 32-bit function using a funclet-based
   // personality, assume the function clobbers all registers. If an exception
   // is thrown, the runtime will not restore CSRs.
   // FIXME: Model this more precisely so that we can register allocate across
   // the normal edge and spill and fill across the exceptional edge.
   if (!Is64Bit && CLI.CS && CLI.CS->isInvoke()) {
     const Function *CallerFn = MF.getFunction();
     EHPersonality Pers =
         CallerFn->hasPersonalityFn()
             ? classifyEHPersonality(CallerFn->getPersonalityFn())
             : EHPersonality::Unknown;
     if (isFuncletEHPersonality(Pers))
       Mask = RegInfo->getNoPreservedMask();
   }
 
   // Define a new register mask from the existing mask.
   uint32_t *RegMask = nullptr;
 
   // In some calling conventions we need to remove the used physical registers
   // from the reg mask.
-  if (CallConv == CallingConv::X86_RegCall) {
+  if (CallConv == CallingConv::X86_RegCall || HasNCSR) {
     const TargetRegisterInfo *TRI = Subtarget.getRegisterInfo();
 
     // Allocate a new Reg Mask and copy Mask.
     RegMask = MF.allocateRegisterMask(TRI->getNumRegs());
     unsigned RegMaskSize = (TRI->getNumRegs() + 31) / 32;
     memcpy(RegMask, Mask, sizeof(uint32_t) * RegMaskSize);
 
     // Make sure all sub registers of the argument registers are reset

llvm/trunk/lib/Target/X86/X86RegisterInfo.cpp

X86RegisterInfo::getCalleeSavedRegs(const MachineFunction *MF) const {
   assert(MF && "MachineFunction required");
 
   const X86Subtarget &Subtarget = MF->getSubtarget<X86Subtarget>();
   bool HasSSE = Subtarget.hasSSE1();
   bool HasAVX = Subtarget.hasAVX();
   bool HasAVX512 = Subtarget.hasAVX512();
   bool CallsEHReturn = MF->callsEHReturn();
 
-  switch (MF->getFunction()->getCallingConv()) {
+  CallingConv::ID CC = MF->getFunction()->getCallingConv();
+
+  // If attribute NoCallerSavedRegisters exists then we set X86_INTR calling
+  // convention because it has the CSR list.
+  if (MF->getFunction()->hasFnAttribute("no_caller_saved_registers"))
+    CC = CallingConv::X86_INTR;
+
+  switch (CC) {
   case CallingConv::GHC:
   case CallingConv::HiPE:
     return CSR_NoRegs_SaveList;
   case CallingConv::AnyReg:
     if (HasAVX)
       return CSR_64_AllRegs_AVX_SaveList;
     return CSR_64_AllRegs_SaveList;
   case CallingConv::PreserveMost:

llvm/trunk/test/CodeGen/X86/x86-32-intrcc.ll

define x86_intrcc void @test_isr_ecode(%struct.interrupt_frame* %frame, i32 %ecode) {
   call x86_fastcallcc void asm sideeffect "", "r,r"(i32 %flags, i32 %ecode)
   ret void
 }
 
 ; All clobbered registers must be saved
 define x86_intrcc void @test_isr_clobbers(%struct.interrupt_frame* %frame, i32 %ecode) {
   call void asm sideeffect "", "~{eax},~{ebx},~{ebp}"()
   ; CHECK-LABEL: test_isr_clobbers
-  ; CHECK-SSE-NEXT: pushl %ebp
-  ; CHECK-SSE-NEXT: pushl %ebx
-  ; CHECK-SSE-NEXT; pushl %eax
-  ; CHECK-SSE-NEXT: popl %eax
-  ; CHECK-SSE-NEXT: popl %ebx
-  ; CHECK-SSE-NEXT: popl %ebp
-  ; CHECK-SSE-NEXT: addl $4, %esp
-  ; CHECK-SSE-NEXT: iretl
+  ; CHECK: pushl %ebp
+  ; CHECK: pushl %ebx
+  ; CHECK: pushl %eax
+  ; CHECK: popl %eax
+  ; CHECK: popl %ebx
+  ; CHECK: popl %ebp
+  ; CHECK: addl $4, %esp
+  ; CHECK: iretl
   ; CHECK0-LABEL: test_isr_clobbers
-  ; CHECK0-SSE-NEXT: pushl %ebp
-  ; CHECK0-SSE-NEXT: pushl %ebx
-  ; CHECK0-SSE-NEXT; pushl %eax
-  ; CHECK0-SSE-NEXT: popl %eax
-  ; CHECK0-SSE-NEXT: popl %ebx
-  ; CHECK0-SSE-NEXT: popl %ebp
-  ; CHECK0-SSE-NEXT: addl $4, %esp
-  ; CHECK0-SSE-NEXT: iretl
+  ; CHECK0: pushl %ebp
+  ; CHECK0: pushl %ebx
+  ; CHECK0: pushl %eax
+  ; CHECK0: popl %eax
+  ; CHECK0: popl %ebx
+  ; CHECK0: popl %ebp
+  ; CHECK0: addl $4, %esp
+  ; CHECK0: iretl
   ret void
 }
 
 @f80 = common global x86_fp80 0xK00000000000000000000, align 4
 
 ; Test that the presence of x87 does not crash the FP stackifier
 define x86_intrcc void @test_isr_x87(%struct.interrupt_frame* %frame) {
   ; CHECK-LABEL: test_isr_x87

llvm/trunk/test/CodeGen/X86/x86-64-intrcc.ll

define x86_intrcc void @test_isr_ecode(%struct.interrupt_frame* %frame, i64 %ecode) {
   call void asm sideeffect "", "r,r"(i64 %flags, i64 %ecode)
   ret void
 }
 
 ; All clobbered registers must be saved
 define x86_intrcc void @test_isr_clobbers(%struct.interrupt_frame* %frame, i64 %ecode) {
   call void asm sideeffect "", "~{rax},~{rbx},~{rbp},~{r11},~{xmm0}"()
   ; CHECK-LABEL: test_isr_clobbers
-  ; CHECK-SSE-NEXT: pushq %rax
-  ; CHECK-SSE-NEXT: pushq %rax
-  ; CHECK-SSE-NEXT; pushq %r11
-  ; CHECK-SSE-NEXT: pushq %rbp
-  ; CHECK-SSE-NEXT: pushq %rbx
-  ; CHECK-SSE-NEXT: movaps %xmm0
-  ; CHECK-SSE-NEXT: movaps %xmm0
-  ; CHECK-SSE-NEXT: popq %rbx
-  ; CHECK-SSE-NEXT: popq %rbp
-  ; CHECK-SSE-NEXT: popq %r11
-  ; CHECK-SSE-NEXT: popq %rax
-  ; CHECK-SSE-NEXT: addq $8, %rsp
-  ; CHECK-SSE-NEXT: iretq
+
+  ; CHECK: pushq %rax
+  ; CHECK: pushq %rbp
+  ; CHECK: pushq %r11
+  ; CHECK: pushq %rbx
+  ; CHECK: movaps %xmm0
+  ; CHECK: movaps {{.*}}, %xmm0
+  ; CHECK: popq %rbx
+  ; CHECK: popq %r11
+  ; CHECK: popq %rbp
+  ; CHECK: popq %rax
+  ; CHECK: addq $16, %rsp
+  ; CHECK: iretq
   ; CHECK0-LABEL: test_isr_clobbers
-  ; CHECK0-SSE-NEXT: pushq %rax
-  ; CHECK0-SSE-NEXT; pushq %r11
-  ; CHECK0-SSE-NEXT: pushq %rbp
-  ; CHECK0-SSE-NEXT: pushq %rbx
-  ; CHECK0-SSE-NEXT: movaps %xmm0
-  ; CHECK0-SSE-NEXT: movaps %xmm0
-  ; CHECK0-SSE-NEXT: popq %rbx
-  ; CHECK0-SSE-NEXT: popq %rbp
-  ; CHECK0-SSE-NEXT: popq %r11
-  ; CHECK0-SSE-NEXT: popq %rax
-  ; CHECK0-SSE-NEXT: addq $16, %rsp
-  ; CHECK0-SSE-NEXT: iretq
+
+  ; CHECK0: pushq %rax
+  ; CHECK0: pushq %rbp
+  ; CHECK0: pushq %r11
+  ; CHECK0: pushq %rbx
+  ; CHECK0: movaps %xmm0
+  ; CHECK0: movaps {{.*}}, %xmm0
+  ; CHECK0: popq %rbx
+  ; CHECK0: popq %r11
+  ; CHECK0: popq %rbp
+  ; CHECK0: popq %rax
+  ; CHECK0: addq $16, %rsp
+  ; CHECK0: iretq
   ret void
 }
 
 @f80 = common global x86_fp80 0xK00000000000000000000, align 4
 
 ; Test that the presence of x87 does not crash the FP stackifier
 define x86_intrcc void @test_isr_x87(%struct.interrupt_frame* %frame) {
   ; CHECK-LABEL: test_isr_x87

llvm/trunk/test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py for function "bar"
+; RUN: llc -mtriple=x86_64-unknown-unknown < %s | FileCheck %s
+
+;; In functions with 'no_caller_saved_registers' attribute, all registers should
+;; be preserved except for registers used for passing/returning arguments.
+;; In the following function registers %RDI, %RSI and %XMM0 are used to store
+;; arguments %a0, %a1 and %b0 accordingally. The value is returned in %RAX.
+;; The above registers should not be preserved, however other registers
+;; (that are modified by the function) should be preserved (%RDX and %XMM1).
+define x86_64_sysvcc i32 @bar(i32 %a0, i32 %a1, float %b0) #0 {
+; CHECK-LABEL: bar:
+; CHECK:       # BB#0:
+; CHECK-NEXT:    pushq %rdx
+; CHECK-NEXT:  .Lcfi0:
+; CHECK-NEXT:    .cfi_def_cfa_offset 16
+; CHECK-NEXT:    movaps %xmm1, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; CHECK-NEXT:  .Lcfi1:
+; CHECK-NEXT:    .cfi_offset %rdx, -16
+; CHECK-NEXT:  .Lcfi2:
+; CHECK-NEXT:    .cfi_offset %xmm1, -32
+; CHECK-NEXT:    #APP
+; CHECK-NEXT:    #NO_APP
+; CHECK-NEXT:    movl $4, %eax
+; CHECK-NEXT:    movaps -{{[0-9]+}}(%rsp), %xmm1 # 16-byte Reload
+; CHECK-NEXT:    popq %rdx
+; CHECK-NEXT:    retq
+  call void asm sideeffect "", "~{rax},~{rdx},~{xmm1},~{rdi},~{rsi},~{xmm0}"()
+  ret i32 4
+}
+
+;; Because "bar" has 'no_caller_saved_registers' attribute, function "foo"
+;; doesn't need to preserve registers except for the arguments passed 
+;; to "bar" (%ESI, %EDI and %XMM0).
+define x86_64_sysvcc float @foo(i32 %a0, i32 %a1, float %b0) {
+; CHECK-LABEL: foo
+; CHECK:       movaps  %xmm0, %xmm1
+; CHECK-NEXT:  movl  %esi, %ecx
+; CHECK-NEXT:  movl  %edi, %edx
+; CHECK-NEXT:  callq bar
+; CHECK-NEXT:  addl  %edx, %eax
+; CHECK-NEXT:  addl  %ecx, %eax
+; CHECK-NEXT:  xorps %xmm0, %xmm0
+; CHECK-NEXT:  cvtsi2ssl %eax, %xmm0
+; CHECK-NEXT:  addss %xmm0, %xmm1
+; CHECK:       retq
+  %call = call i32 @bar(i32 %a0, i32 %a1, float %b0) #0
+  %c0   = add i32 %a0, %call
+  %c1   = add i32 %c0, %a1
+  %c2 = sitofp i32 %c1 to float
+  %c3 = fadd float %c2, %b0
+  ret float %c3
+}
+
+attributes #0 = { "no_caller_saved_registers" }

llvm/trunk/test/CodeGen/X86/x86-no_caller_saved_registers.ll

+; RUN: llc -mtriple=x86_64-unknown-unknown < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-unknown-unknown -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=i686-unknown-unknown -mattr=+sse2 < %s | FileCheck %s
+; RUN: llc -mtriple=i686-unknown-unknown -mattr=+sse2 -O0 < %s | FileCheck %s
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; In functions with 'no_caller_saved_registers' attribute, all registers should
+;; be preserved except for registers used for passing/returning arguments.
+;; The test checks that function "bar" preserves xmm0 register.
+;; It also checks that caller function "foo" does not store registers for callee 
+;; "bar". For example, there is no store/load/access to xmm registers.
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+define i32 @bar(i32 %a0, i32 %a1, i32 %a2, i32 %a3, i32 %a4, i32 %a5, i32 %a6, i32 %a7, i32 %a8) #0 {
+; CHECK-LABEL: bar
+; CHECK:       mov{{.*}}  %xmm0
+; CHECK:       mov{{.*}} {{.*}}, %xmm0
+; CHECK:       ret
+  call void asm sideeffect "", "~{xmm0}"()
+  ret i32 1
+}
+
+define x86_intrcc void @foo(i8* nocapture readnone %c) {
+; CHECK-LABEL: foo
+; CHECK-NOT: xmm
+entry:
+  tail call i32 @bar(i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8) #0
+  ret void
+}
+
+attributes #0 = { "no_caller_saved_registers" }