[X86] Support of no_caller_saved_registers attribute (LLVM part) - restart

[oren_ben_simhon]

Summary

This patch implements the LLVM part for no_caller_saved_registers attribute as appear in interrupt and exception handler proposal.

The review is a update for an old review:
Support of no_caller_saved_registers attribute (LLVM part).

The main difference from the old review is that in this patch we use the dynamic CSR mechanism in order to remove returned /passed arguments from the function regmask/CSR list.

[oren_ben_simhon]

A kind reminder for reviewing this patch :-) Thanks in advance.

[oren_ben_simhon]

Updated the test

[aaboud]

Thanks Oren for resuming the no_caller_saved_registers patch.
Few comments below.

lib/Target/X86/X86FastISel.cpp

3189

Why do we need to check the caller function?
Is there a good reason why we would disable fast lowering in such case?

lib/Target/X86/X86ISelLowering.cpp

3810

Can you add a comment explaining this line?
For example,
// NoCallerSavedRegisters has same BASE preserved registers as X86_INTR calling convention.

3833–3835

No need for this variable, see comment below.
Just fold this check into the "if-condition".

3909

No need to pass the flag "RemovePassedReturnedRegs"
If "RemovePassedReturnedRegs=false" then "RegMask=nullptr", so it is enough to check that "RegMask!=nullptr" as we do already.

Please remove "RemovePassedReturnedRegs" from this function.

lib/Target/X86/X86RegisterInfo.cpp

276

Add a comment here too.

[oren_ben_simhon]

Implemented comments posted until 04/25 (Thanks Amjad)

[aaboud]

lib/Target/X86/X86FastISel.cpp

3185

You are not using this variable anymore, right?

lib/Target/X86/X86ISelLowering.cpp

2260–2261

You are using this check 3 times in this function.
It worth moving this to the top of the function, and create a bool variable with well defined name.
E.g.,
bool ShouldDisableCalleeSavedRegister = CallConv == CallingConv::X86_RegCall || MF.getFunction()->hasFnAttribute("no_caller_saved_registers");

lib/Target/X86/X86RegisterInfo.cpp

277

[Style]: It is up to you, but I prefer that you align the comment lines to 80 chars, i.e., you can still move words from 2nd line to 1st one.

test/CodeGen/X86/x86-no_caller_saved_registers.ll

2

I would add one or two examples regarding disabling preserved registers, similar to the regcall calling convention.
These examples dose not need to have x86_intrcc calling convention, but you should use the no_caller_saved_registers attribute with other calling convention, make sure to have functions with parameters to emphasize the difference.

[oren_ben_simhon]

Implemented comments posted until 04/27 (Thanks Amjad)

[aaboud]

lib/Target/X86/X86ISelLowering.cpp

2242

[Style]: Please run clang-format.

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

9 ↗

(On Diff #96922)

Why do we need the "local_unnamed_addr" attribute in this test?

21 ↗

(On Diff #96922)

Please do not use code that you might not control its outcome.
Instead try the inline assembly that allows you to force register usage.

E.g., see other LIT tests you have in the patch:
call void asm sideeffect "", "~{rax},~{rbx},~{rbp},~{r11},~{xmm0}"()

31 ↗

(On Diff #96922)

I would prefer second function that have the "no_caller_saved_registers" but that has no parameters passed in registers, and then you can show that it pushes all used registers.

test/CodeGen/X86/x86-no_caller_saved_registers.ll

2

Why did you remove the -O0 command line?
It is needed to check the behavior of FastLowerCall (X86FastISel).

[MatzeB]

Why isn't this "just" a new calling convention at the LLVM level? What are the benefits of having this as an attribute on top of the calling convention?

[aaboud]

In D31876#739596, @MatzeB wrote:

Why isn't this "just" a new calling convention at the LLVM level? What are the benefits of having this as an attribute on top of the calling convention?

Calling convention determines not only what the callee and the caller function saved registers are, but also how we move parameters to the called function.

"no_caller_saved_registers" attribute can be attached to a function with any calling convention, thus it is not a calling convention by itself.
However, this attribute overrides the calling convention callee/caller saved register list.

It is well explained in the Clang part documentation.

[MatzeB]

In D31876#739953, @aaboud wrote:

In D31876#739596, @MatzeB wrote:

Why isn't this "just" a new calling convention at the LLVM level? What are the benefits of having this as an attribute on top of the calling convention?

Calling convention determines not only what the callee and the caller function saved registers are, but also how we move parameters to the called function.

"no_caller_saved_registers" attribute can be attached to a function with any calling convention, thus it is not a calling convention by itself.
However, this attribute overrides the calling convention callee/caller saved register list.

It is well explained in the Clang part documentation.

Ok, may still be worth linking/referencing this in the llvm documentation.

But I am still curious: What motivated the introduction of this attribute, do you have an example where it is used/would be used if it existed?

[aaboud]

Ok, may still be worth linking/referencing this in the llvm documentation.

But I am still curious: What motivated the introduction of this attribute, do you have an example where it is used/would be used if it existed?

interrupt handler calling convention force the interrupt handler function to save all modified registers, as there is no caller that would do that, right?

Now, assume a case where the interrupt handler function "foo()" calls a helper function "bar()", then it needs to save all the registers preserved by callers according to the calling convention of "bar()", even if "bar()" is not going to modify any of them.
However, if we add "no_caller_saved_registers" attribute to "bar()", then we move the responsibility of saving the modified registers to "bar()" which is the function that knows what registers are going to be modified and will save only them.

I hope I explained the example well enough.

[MatzeB]

In D31876#740018, @aaboud wrote:

Ok, may still be worth linking/referencing this in the llvm documentation.

But I am still curious: What motivated the introduction of this attribute, do you have an example where it is used/would be used if it existed?

interrupt handler calling convention force the interrupt handler function to save all modified registers, as there is no caller that would do that, right?

Now, assume a case where the interrupt handler function "foo()" calls a helper function "bar()", then it needs to save all the registers preserved by callers according to the calling convention of "bar()", even if "bar()" is not going to modify any of them.
However, if we add "no_caller_saved_registers" attribute to "bar()", then we move the responsibility of saving the modified registers to "bar()" which is the function that knows what registers are going to be modified and will save only them.

I hope I explained the example well enough.

So this is the old discussion about choosing the right balance between caller and callee saved registers. Sounds to me like whoever uses this should have rather spend the time making sure the compiler can inline his calls and mark all his methods static so the compiler can optimize the calling convention automatically.

Makes me wonder a bit where this should start and end. To play devils advocate: Why stop here and not allow specifying the number or exact list of caller saved registers, whether flags are preserved, what registers to use for arguments, etc. IMO that just increases compiler complexity and hence potential bugs for very little gain.

Anyway seems like others find this useful and are willing to implement and maintain this, so I won't stand in the way.

[oren_ben_simhon]

This comment has been deleted.

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

31 ↗

(On Diff #96922)

We don't need a function without parameters in order to show that the function pushes all used registers.

This is already tested in the function bar1 and bar0 (in the other test file).
In bar0 you can see that it pushes xmm0 because it uses it.
In bar1 you can see that it pushes rdx and xmm1 because it uses it.

So there is no additional value in a function without arguments.

[oren_ben_simhon]

Implemented comments posted until 04/28 (Thanks Amjad and Matthias)

[aaboud]

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

31 ↗

(On Diff #96922)

I do not see "bar1" in the other file, and indeed it worth having test that shows that callee do preserve GPR, not only XMM resgister.

What I want to have here is a test that shows we do preserve same registers as in "bar1" above, once they are not used to pass parameters. Otherwise, how can we be sure that the implementation does not always skip these registers you used above?

define x86_64_sysvcc void @bar2() #1 {
; CHECK-LABEL: bar1
; CHECK:     pushq %rdx
; CHECK:     pushq %rax
; CHECK:     pushq  %rdi
; CHECK:     pushq  %rsi
; CHECK:     movaps  %xmm1
; CHECK:     movaps  %xmm0
; CHECK:     movaps {{.*}}, %xmm0
; CHECK:     movaps {{.*}}, %xmm1
; CHECK:     popq %rsi
; CHECK:     popq %rdi
; CHECK:     popq %rax
; CHECK:     popq %rdx
; CHECK:     retq
  call void asm sideeffect "", "~{rdx},~{rax},~{xmm1},~{rdi},~{rsi},~{xmm0}"()
ret void
}

Either way, I do not see the added value of "foo1", why are we testing a function that has no "no_caller_saved_registers" attribute here? it must be checked somewhere else, right?

13 ↗

(On Diff #97000)

I am afraid that the order is important here.
Assume that the code would look like this:

pushq %rdx
pushq %rdi
pushq %rsi
movaps %xmm1
...

Then this test will still pass, right?

20 ↗

(On Diff #97000)

I assume that the return value will be passed by %eax, so please check that %rax is not saved either.

test/CodeGen/X86/x86-no_caller_saved_registers.ll

25

move this to line 27 and add CHECK-LABEL: for @foo0.

[oren_ben_simhon]

test/CodeGen/X86/x86-no_caller_saved_registers-preserve.ll

31 ↗

(On Diff #96922)

I do not see "bar1" in the other file

• bar1 is in this file.
• bar0 is the the other file.

It was probably misunderstood, In all three functions (bar0, bar1 and foo1) we check that parameters that are passed as arguments are not preserved.

For example, in foo1, we see that xmm0 is preserved. Why is it preserved (you ask)?
because it is passed to bar1, which has no_caller_saved_registers, and as such the caller needs to preserve the passed arguments.
This is why foo1 doesn't need to have the attribute no_caller_saved_registers. But still it tests the attribute of the callee.

Either way, I do not see the added value of "foo1", why are we testing a function that has no "no_caller_saved_registers" attribute here? it must be checked somewhere else, right?

Every time you test a calling convention, you need to test what happens in the caller of that CC function (foo0/foo1) and inside the CC function itself (bar1/bar0).

What I want to have here is a test that shows we do preserve same registers as in "bar1" above, once they are not used to pass parameters.

Again, this is already tested, please see my previous comment:

This is already tested in the function bar1 and bar0 (in the other test file).
In bar0 you can see that it pushes xmm0 because it uses it.
In bar1 you can see that it pushes rdx and xmm1 because it uses it.

13 ↗

(On Diff #97000)

That's correct. I will use update_llc_test_checks.py for this function instead.

[oren_ben_simhon]

Implemented comments posted until 04/30 (Thanks Amjad)

[oren_ben_simhon]

Updated the tests.

[aaboud]

Thanks Oren,
Code looks much better now.

lib/Target/X86/X86FastISel.cpp

bool X86FastISel::fastLowerCall(CallLoweringInfo &CLI) {
   bool &IsTailCall    = CLI.IsTailCall;
   bool IsVarArg       = CLI.IsVarArg;
   const Value *Callee = CLI.Callee;
   MCSymbol *Symbol = CLI.Symbol;
 
   bool Is64Bit        = Subtarget->is64Bit();
   bool IsWin64        = Subtarget->isCallingConvWin64(CC);
 
+  const CallInst *CI =
+      CLI.CS ? dyn_cast<CallInst>(CLI.CS->getInstruction()) : nullptr;
+  const Function *CalledFn = CI ? CI->getCalledFunction() : nullptr;
+  const Function *CallerFn = CI ? CI->getParent()->getParent() : nullptr;
+  bool HasNCSR =
+      (CI && CI->hasFnAttr("no_caller_saved_registers")) ||
+      (CalledFn && CalledFn->hasFnAttribute("no_caller_saved_registers")) ||
+      (CallerFn && CallerFn->hasFnAttribute("no_caller_saved_registers"));
+
+  if (HasNCSR)
+    return false;
+
   // Handle only C, fastcc, and webkit_js calling conventions for now.
   switch (CC) {
   default: return false;
   case CallingConv::C:
   case CallingConv::Fast:
   case CallingConv::WebKit_JS:
   case CallingConv::Swift:
   case CallingConv::X86_FastCall:

lib/Target/X86/X86ISelLowering.h

private:
     void addLegalFPImmediate(const APFloat& Imm) {
       LegalFPImmediates.push_back(Imm);
     }
 
     SDValue LowerCallResult(SDValue Chain, SDValue InFlag,
                             CallingConv::ID CallConv, bool isVarArg,
                             const SmallVectorImpl<ISD::InputArg> &Ins,
                             const SDLoc &dl, SelectionDAG &DAG,
-                            SmallVectorImpl<SDValue> &InVals,
-                            uint32_t *RegMask) const;
+                            SmallVectorImpl<SDValue> &InVals, uint32_t *RegMask,
+                            bool RemoveReturnedRegs) const;
     SDValue LowerMemArgument(SDValue Chain, CallingConv::ID CallConv,
                              const SmallVectorImpl<ISD::InputArg> &ArgInfo,
                              const SDLoc &dl, SelectionDAG &DAG,
                              const CCValAssign &VA, MachineFrameInfo &MFI,
                              unsigned i) const;
     SDValue LowerMemOpCallTo(SDValue Chain, SDValue StackPtr, SDValue Arg,
                              const SDLoc &dl, SelectionDAG &DAG,
                              const CCValAssign &VA,

lib/Target/X86/X86ISelLowering.cpp

X86TargetLowering::LowerReturn(SDValue Chain, CallingConv::ID CallConv,
                                const SmallVectorImpl<SDValue> &OutVals,
                                const SDLoc &dl, SelectionDAG &DAG) const {
   MachineFunction &MF = DAG.getMachineFunction();
   X86MachineFunctionInfo *FuncInfo = MF.getInfo<X86MachineFunctionInfo>();
 
   if (CallConv == CallingConv::X86_INTR && !Outs.empty())
     report_fatal_error("X86 interrupts may not return any value");
 
   SmallVector<CCValAssign, 16> RVLocs;
   CCState CCInfo(CallConv, isVarArg, MF, RVLocs, *DAG.getContext());
   CCInfo.AnalyzeReturn(Outs, RetCC_X86);
 
   SDValue Flag;
   SmallVector<SDValue, 6> RetOps;
   RetOps.push_back(Chain); // Operand #0 = Chain (updated below)
   // Operand #1 = Bytes To Pop
   RetOps.push_back(DAG.getTargetConstant(FuncInfo->getBytesToPopOnReturn(), dl,
                    MVT::i32));
 
   // Copy the result values into the output registers.
   for (unsigned I = 0, OutsIndex = 0, E = RVLocs.size(); I != E;
        ++I, ++OutsIndex) {
     CCValAssign &VA = RVLocs[I];
     assert(VA.isRegLoc() && "Can only return in registers!");
 
     // Add the register to the CalleeSaveDisableRegs list.
-    if (CallConv == CallingConv::X86_RegCall)
+    if (CallConv == CallingConv::X86_RegCall ||
+        MF.getFunction()->hasFnAttribute("no_caller_saved_registers"))
       MF.getRegInfo().disableCalleeSavedRegister(VA.getLocReg());
 
     SDValue ValToCopy = OutVals[OutsIndex];
     EVT ValVT = ValToCopy.getValueType();
 
     // Promote values to the appropriate types.
     if (VA.getLocInfo() == CCValAssign::SExt)
       ValToCopy = DAG.getNode(ISD::SIGN_EXTEND, dl, VA.getLocVT(), ValToCopy);
if (VA.needsCustom()) {
 
       Passv64i1ArgInRegs(dl, DAG, Chain, ValToCopy, RegsToPass, VA, RVLocs[++I],
                          Subtarget);
 
       assert(2 == RegsToPass.size() &&
              "Expecting two registers after Pass64BitArgInRegs");
 
       // Add the second register to the CalleeSaveDisableRegs list.
-      if (CallConv == CallingConv::X86_RegCall)
+      if (CallConv == CallingConv::X86_RegCall ||
+          MF.getFunction()->hasFnAttribute("no_caller_saved_registers"))
         MF.getRegInfo().disableCalleeSavedRegister(RVLocs[I].getLocReg());
     } else {
       RegsToPass.push_back(std::make_pair(VA.getLocReg(), ValToCopy));
     }
 
     // Add nodes to the DAG and add the values into the RetOps list
     for (auto &Reg : RegsToPass) {
       Chain = DAG.getCopyToReg(Chain, dl, Reg.first, Reg.second, Flag);
if (unsigned SRetReg = FuncInfo->getSRetReturnReg()) {
     Chain = DAG.getCopyToReg(Chain, dl, RetValReg, Val, Flag);
     Flag = Chain.getValue(1);
 
     // RAX/EAX now acts like a return value.
     RetOps.push_back(
         DAG.getRegister(RetValReg, getPointerTy(DAG.getDataLayout())));
 
     // Add the returned register to the CalleeSaveDisableRegs list.
-    if (CallConv == CallingConv::X86_RegCall)
+    if (CallConv == CallingConv::X86_RegCall ||
+        MF.getFunction()->hasFnAttribute("no_caller_saved_registers"))
       MF.getRegInfo().disableCalleeSavedRegister(RetValReg);
   }
 
   const X86RegisterInfo *TRI = Subtarget.getRegisterInfo();
   const MCPhysReg *I =
       TRI->getCalleeSavedRegsViaCopy(&DAG.getMachineFunction());
   if (I) {
     for (; *I; ++I) {

 }
 
 /// Lower the result values of a call into the
 /// appropriate copies out of appropriate physical registers.
 ///
 SDValue X86TargetLowering::LowerCallResult(
     SDValue Chain, SDValue InFlag, CallingConv::ID CallConv, bool isVarArg,
     const SmallVectorImpl<ISD::InputArg> &Ins, const SDLoc &dl,
-    SelectionDAG &DAG, SmallVectorImpl<SDValue> &InVals,
-    uint32_t *RegMask) const {
+    SelectionDAG &DAG, SmallVectorImpl<SDValue> &InVals, uint32_t *RegMask,
+    bool RemoveReturnedRegs) const {
 
   const TargetRegisterInfo *TRI = Subtarget.getRegisterInfo();
   // Assign locations to each value returned by this call.
   SmallVector<CCValAssign, 16> RVLocs;
   bool Is64Bit = Subtarget.is64Bit();
   CCState CCInfo(CallConv, isVarArg, DAG.getMachineFunction(), RVLocs,
                  *DAG.getContext());
   CCInfo.AnalyzeCallResult(Ins, RetCC_X86);
 
   // Copy all of the result registers out of their specified physreg.
   for (unsigned I = 0, InsIndex = 0, E = RVLocs.size(); I != E;
        ++I, ++InsIndex) {
     CCValAssign &VA = RVLocs[I];
     EVT CopyVT = VA.getLocVT();
 
     // In some calling conventions we need to remove the used registers
     // from the register mask.
-    if (RegMask && CallConv == CallingConv::X86_RegCall) {
+    if (RegMask && RemoveReturnedRegs) {
       for (MCSubRegIterator SubRegs(VA.getLocReg(), TRI, /*IncludeSelf=*/true);
            SubRegs.isValid(); ++SubRegs)
         RegMask[*SubRegs / 32] &= ~(1u << (*SubRegs % 32));
     }
 
     // If this is x86-64, and we disabled SSE, we can't return FP values
     if ((CopyVT == MVT::f32 || CopyVT == MVT::f64 || CopyVT == MVT::f128) &&
         ((Is64Bit || Ins[InsIndex].Flags.isInReg()) && !Subtarget.hasSSE1())) {
if (Personality == EHPersonality::CoreCLR) {
       // same, so the size of funclets' (mostly empty) frames is dictated by
       // how far this slot is from the bottom (since they allocate just enough
       // space to accommodate holding this slot at the correct offset).
       int PSPSymFI = MFI.CreateStackObject(8, 8, /*isSS=*/false);
       EHInfo->PSPSymFrameIdx = PSPSymFI;
     }
   }
 
-  if (CallConv == CallingConv::X86_RegCall) {
+  if (CallConv == CallingConv::X86_RegCall ||
+      Fn->hasFnAttribute("no_caller_saved_registers")) {
     const MachineRegisterInfo &MRI = MF.getRegInfo();
     for (const auto &Pair : make_range(MRI.livein_begin(), MRI.livein_end()))
       MF.getRegInfo().disableCalleeSavedRegister(Pair.first);
   }
 
   return Chain;
 }
 
X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
 
   MachineFunction &MF = DAG.getMachineFunction();
   bool Is64Bit        = Subtarget.is64Bit();
   bool IsWin64        = Subtarget.isCallingConvWin64(CallConv);
   StructReturnType SR = callIsStructReturn(Outs, Subtarget.isTargetMCU());
   bool IsSibcall      = false;
   X86MachineFunctionInfo *X86Info = MF.getInfo<X86MachineFunctionInfo>();
   auto Attr = MF.getFunction()->getFnAttribute("disable-tail-calls");
+  const CallInst *CI =
+      CLI.CS ? dyn_cast<CallInst>(CLI.CS->getInstruction()) : nullptr;
+  const Function *Fn = CI ? CI->getCalledFunction() : nullptr;
+  bool HasNCSR = (CI && CI->hasFnAttr("no_caller_saved_registers")) ||
+                 (Fn && Fn->hasFnAttribute("no_caller_saved_registers"));
 
   if (CallConv == CallingConv::X86_INTR)
     report_fatal_error("X86 interrupts may not be called directly");
 
   if (Attr.getValueAsString() == "true")
     isTailCall = false;
 
   if (Subtarget.isPICStyleGOT() &&
X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
 
   // Add argument registers to the end of the list so that they are known live
   // into the call.
   for (unsigned i = 0, e = RegsToPass.size(); i != e; ++i)
     Ops.push_back(DAG.getRegister(RegsToPass[i].first,
                                   RegsToPass[i].second.getValueType()));
 
   // Add a register mask operand representing the call-preserved registers.
-  const uint32_t *Mask = RegInfo->getCallPreservedMask(MF, CallConv);
+  const uint32_t *Mask = RegInfo->getCallPreservedMask(
+      MF, HasNCSR ? CallingConv::X86_INTR : CallConv);
   assert(Mask && "Missing call preserved mask for calling convention");
 
   // If this is an invoke in a 32-bit function using a funclet-based
   // personality, assume the function clobbers all registers. If an exception
   // is thrown, the runtime will not restore CSRs.
   // FIXME: Model this more precisely so that we can register allocate across
   // the normal edge and spill and fill across the exceptional edge.
   if (!Is64Bit && CLI.CS && CLI.CS->isInvoke()) {
     const Function *CallerFn = MF.getFunction();
     EHPersonality Pers =
         CallerFn->hasPersonalityFn()
             ? classifyEHPersonality(CallerFn->getPersonalityFn())
             : EHPersonality::Unknown;
     if (isFuncletEHPersonality(Pers))
       Mask = RegInfo->getNoPreservedMask();
   }
 
   // Define a new register mask from the existing mask.
   uint32_t *RegMask = nullptr;
 
   // In some calling conventions we need to remove the used physical registers
   // from the reg mask.
-  if (CallConv == CallingConv::X86_RegCall) {
+  bool RemovePassedReturnedRegs =
+      (CallConv == CallingConv::X86_RegCall || HasNCSR);
+  if (RemovePassedReturnedRegs) {
     const TargetRegisterInfo *TRI = Subtarget.getRegisterInfo();
 
     // Allocate a new Reg Mask and copy Mask.
     RegMask = MF.allocateRegisterMask(TRI->getNumRegs());
     unsigned RegMaskSize = (TRI->getNumRegs() + 31) / 32;
     memcpy(RegMask, Mask, sizeof(uint32_t) * RegMaskSize);
 
     // Make sure all sub registers of the argument registers are reset
Chain = DAG.getCALLSEQ_END(Chain,
                                                      true),
                                InFlag, dl);
     InFlag = Chain.getValue(1);
   }
 
   // Handle result values, copying them out of physregs into vregs that we
   // return.
   return LowerCallResult(Chain, InFlag, CallConv, isVarArg, Ins, dl, DAG,
-                         InVals, RegMask);
+                         InVals, RegMask, RemovePassedReturnedRegs);
 }
 
 //===----------------------------------------------------------------------===//
 //                Fast Calling Convention (tail call) implementation
 //===----------------------------------------------------------------------===//
 
 //  Like std call, callee cleans arguments, convention except that ECX is
 //  reserved for storing the tail called function address. Only 2 registers are

lib/Target/X86/X86RegisterInfo.cpp

X86RegisterInfo::getCalleeSavedRegs(const MachineFunction *MF) const {
   assert(MF && "MachineFunction required");
 
   const X86Subtarget &Subtarget = MF->getSubtarget<X86Subtarget>();
   bool HasSSE = Subtarget.hasSSE1();
   bool HasAVX = Subtarget.hasAVX();
   bool HasAVX512 = Subtarget.hasAVX512();
   bool CallsEHReturn = MF->callsEHReturn();
 
-  switch (MF->getFunction()->getCallingConv()) {
+  CallingConv::ID CC = MF->getFunction()->getCallingConv();
+  if (MF->getFunction()->hasFnAttribute("no_caller_saved_registers"))
+    CC = CallingConv::X86_INTR;
+  switch (CC) {
   case CallingConv::GHC:
   case CallingConv::HiPE:
     return CSR_NoRegs_SaveList;
   case CallingConv::AnyReg:
     if (HasAVX)
       return CSR_64_AllRegs_AVX_SaveList;
     return CSR_64_AllRegs_SaveList;
   case CallingConv::PreserveMost:

test/CodeGen/X86/x86-32-intrcc.ll

define x86_intrcc void @test_isr_ecode(%struct.interrupt_frame* %frame, i32 %ecode) {
   call x86_fastcallcc void asm sideeffect "", "r,r"(i32 %flags, i32 %ecode)
   ret void
 }
 
 ; All clobbered registers must be saved
 define x86_intrcc void @test_isr_clobbers(%struct.interrupt_frame* %frame, i32 %ecode) {
   call void asm sideeffect "", "~{eax},~{ebx},~{ebp}"()
   ; CHECK-LABEL: test_isr_clobbers
-  ; CHECK-SSE-NEXT: pushl %ebp
-  ; CHECK-SSE-NEXT: pushl %ebx
-  ; CHECK-SSE-NEXT; pushl %eax
-  ; CHECK-SSE-NEXT: popl %eax
-  ; CHECK-SSE-NEXT: popl %ebx
-  ; CHECK-SSE-NEXT: popl %ebp
-  ; CHECK-SSE-NEXT: addl $4, %esp
-  ; CHECK-SSE-NEXT: iretl
+  ; CHECK: pushl %ebp
+  ; CHECK: pushl %ebx
+  ; CHECK: pushl %eax
+  ; CHECK: popl %eax
+  ; CHECK: popl %ebx
+  ; CHECK: popl %ebp
+  ; CHECK: addl $4, %esp
+  ; CHECK: iretl
   ; CHECK0-LABEL: test_isr_clobbers
-  ; CHECK0-SSE-NEXT: pushl %ebp
-  ; CHECK0-SSE-NEXT: pushl %ebx
-  ; CHECK0-SSE-NEXT; pushl %eax
-  ; CHECK0-SSE-NEXT: popl %eax
-  ; CHECK0-SSE-NEXT: popl %ebx
-  ; CHECK0-SSE-NEXT: popl %ebp
-  ; CHECK0-SSE-NEXT: addl $4, %esp
-  ; CHECK0-SSE-NEXT: iretl
+  ; CHECK0: pushl %ebp
+  ; CHECK0: pushl %ebx
+  ; CHECK0: pushl %eax
+  ; CHECK0: popl %eax
+  ; CHECK0: popl %ebx
+  ; CHECK0: popl %ebp
+  ; CHECK0: addl $4, %esp
+  ; CHECK0: iretl
   ret void
 }
 
 @f80 = common global x86_fp80 0xK00000000000000000000, align 4
 
 ; Test that the presence of x87 does not crash the FP stackifier
 define x86_intrcc void @test_isr_x87(%struct.interrupt_frame* %frame) {
   ; CHECK-LABEL: test_isr_x87

test/CodeGen/X86/x86-64-intrcc.ll

define x86_intrcc void @test_isr_ecode(%struct.interrupt_frame* %frame, i64 %ecode) {
   call void asm sideeffect "", "r,r"(i64 %flags, i64 %ecode)
   ret void
 }
 
 ; All clobbered registers must be saved
 define x86_intrcc void @test_isr_clobbers(%struct.interrupt_frame* %frame, i64 %ecode) {
   call void asm sideeffect "", "~{rax},~{rbx},~{rbp},~{r11},~{xmm0}"()
   ; CHECK-LABEL: test_isr_clobbers
-  ; CHECK-SSE-NEXT: pushq %rax
-  ; CHECK-SSE-NEXT: pushq %rax
-  ; CHECK-SSE-NEXT; pushq %r11
-  ; CHECK-SSE-NEXT: pushq %rbp
-  ; CHECK-SSE-NEXT: pushq %rbx
-  ; CHECK-SSE-NEXT: movaps %xmm0
-  ; CHECK-SSE-NEXT: movaps %xmm0
-  ; CHECK-SSE-NEXT: popq %rbx
-  ; CHECK-SSE-NEXT: popq %rbp
-  ; CHECK-SSE-NEXT: popq %r11
-  ; CHECK-SSE-NEXT: popq %rax
-  ; CHECK-SSE-NEXT: addq $8, %rsp
-  ; CHECK-SSE-NEXT: iretq
+
+  ; CHECK: pushq %rax
+  ; CHECK: pushq %rbp
+  ; CHECK: pushq %r11
+  ; CHECK: pushq %rbx
+  ; CHECK: movaps %xmm0
+  ; CHECK: movaps {{.*}}, %xmm0
+  ; CHECK: popq %rbx
+  ; CHECK: popq %r11
+  ; CHECK: popq %rbp
+  ; CHECK: popq %rax
+  ; CHECK: addq $16, %rsp
+  ; CHECK: iretq
   ; CHECK0-LABEL: test_isr_clobbers
-  ; CHECK0-SSE-NEXT: pushq %rax
-  ; CHECK0-SSE-NEXT; pushq %r11
-  ; CHECK0-SSE-NEXT: pushq %rbp
-  ; CHECK0-SSE-NEXT: pushq %rbx
-  ; CHECK0-SSE-NEXT: movaps %xmm0
-  ; CHECK0-SSE-NEXT: movaps %xmm0
-  ; CHECK0-SSE-NEXT: popq %rbx
-  ; CHECK0-SSE-NEXT: popq %rbp
-  ; CHECK0-SSE-NEXT: popq %r11
-  ; CHECK0-SSE-NEXT: popq %rax
-  ; CHECK0-SSE-NEXT: addq $16, %rsp
-  ; CHECK0-SSE-NEXT: iretq
+
+  ; CHECK0: pushq %rax
+  ; CHECK0: pushq %rbp
+  ; CHECK0: pushq %r11
+  ; CHECK0: pushq %rbx
+  ; CHECK0: movaps %xmm0
+  ; CHECK0: movaps {{.*}}, %xmm0
+  ; CHECK0: popq %rbx
+  ; CHECK0: popq %r11
+  ; CHECK0: popq %rbp
+  ; CHECK0: popq %rax
+  ; CHECK0: addq $16, %rsp
+  ; CHECK0: iretq
   ret void
 }
 
 @f80 = common global x86_fp80 0xK00000000000000000000, align 4
 
 ; Test that the presence of x87 does not crash the FP stackifier
 define x86_intrcc void @test_isr_x87(%struct.interrupt_frame* %frame) {
   ; CHECK-LABEL: test_isr_x87

test/CodeGen/X86/x86-no_caller_saved_registers.ll

+; RUN: llc -mtriple=x86_64-unknown-unknown < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-unknown-unknown -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=i686-unknown-unknown -mattr=+sse2 < %s | FileCheck %s
+; RUN: llc -mtriple=i686-unknown-unknown -O0 -mattr=+sse2 < %s | FileCheck %s
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; Checks that caller function "foo" does not store registers for callee "bar".
+;; Where:
+;;   bar0 is marked as "no_caller_saved_registers" on call-site only.
+;;   bar1 is marked as "no_caller_saved_registers" on function declaration only.
+;; For example, there is no store/load/access to xmm registers.
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; Command line: -S -o - -emit-llvm -O2
+;; Source:
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;;extern void bar (int, int, int, int, int, int, int, int, int) 
+;;   __attribute__ ((no_caller_saved_registers)); 
+;;
+;;void 
+;;foo (void* c) __attribute__((interrupt))
+;;{ 
+;;  bar (0, 1, 2, 3, 4, 5, 6, 7, 8); 
+;;} 
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
[e-mail] = appending global [2 x i8*] [i8* bitcast (void (i8*)* @foo1 to i8*), i8* bitcast (void (i8*)* @foo1 to i8*)], section "llvm.metadata"
+
+; CHECK-NOT: xmm
+
+define x86_intrcc void @foo0(i8* nocapture readnone %c) #0 {
+entry:
+  tail call void @bar0(i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8) #1
+  ret void
+}
+
+define x86_intrcc void @foo1(i8* nocapture readnone %c) #0 {
+entry:
+  tail call i32 @bar1(i32 0, i32 1, i32 2, i32 3, i32 4, i32 5, i32 6, i32 7, i32 8)
+  ret void
+}
+
+declare void @bar0(i32, i32, i32, i32, i32, i32, i32, i32, i32)
+define i32 @bar1(i32 %a0, i32 %a1, i32 %a2, i32 %a3, i32 %a4, i32 %a5, i32 %a6, i32 %a7, i32 %a8) local_unnamed_addr #1 {
+; CHECK-LABEL: bar1
+; CHECK:       mov{{.*}}  %xmm0
+; CHECK:       mov{{.*}} {{.*}}, %xmm0
+; CHECK:       ret
+entry:
+  %div = sdiv i32 %a0, %a1
+  %conv = sitofp i32 %div to float
+  %conv1 = fptosi float %conv to i32
+  ret i32 %conv1
+}
+
+attributes #0 = { "disable-tail-calls"="true" }
+attributes #1 = { "no_caller_saved_registers" }