[X86] Disabling PLT in Regcall CC Functions

[oren_ben_simhon]

Summary

According to x86-64 psABI, xmm0-xmm7 can be used to pass function parameters.
However regcall calling convention uses also xmm8-xmm15 to pass function
parameters which violates x86-64 psABI.
Detail info can be found at bugzilla.

We encounter the violation symptom when using PIC with lazy binding optimization.
In that case the PLT mechanism as described in x86_64 psABI will
not preserve xmm8-xmm15 registers and will lead to miscompilation.

The agreed solution is to disable PLT for Regcall calling convention in linux 64 bit.

[aaboud]

lib/Target/X86/X86Subtarget.cpp

146

[Style]: Fix format (clang-format).

[oren_ben_simhon]

Implemented comments posted until 04/30 (Thanks Amjad)

[oren_ben_simhon]

Added the fix to all SystemV ELF based operating systems (Thanks Rafael and David)

lib/Target/X86/X86Subtarget.cpp

 
 unsigned char
 X86Subtarget::classifyGlobalFunctionReference(const GlobalValue *GV,
                                               const Module &M) const {
   if (TM.shouldAssumeDSOLocal(M, GV))
     return X86II::MO_NO_FLAG;
 
   assert(!isTargetCOFF());
+  const Function *F = dyn_cast_or_null<Function>(GV);
 
-  if (isTargetELF())
+  if (isTargetELF()) {
+    if (is64Bit() && F && (CallingConv::X86_RegCall == F->getCallingConv()))
+      // According to psABI, PLT stub clobbers XMM8-XMM15.
+      // In Regcall calling convention those registers are used for passing
+      // parameters. Thus we need to prevent lazy binding in Regcall.
+      return X86II::MO_GOTPCREL;
     return X86II::MO_PLT;
+  }
 
   if (is64Bit()) {
-    auto *F = dyn_cast_or_null<Function>(GV);
     if (F && F->hasFnAttribute(Attribute::NonLazyBind))
       // If the function is marked as non-lazy, generate an indirect call
       // which loads from the GOT directly. This avoids runtime overhead
       // at the cost of eager binding (and one extra byte of encoding).
       return X86II::MO_GOTPCREL;
     return X86II::MO_NO_FLAG;
   }
 

test/CodeGen/X86/regcall-no-plt.ll

+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -relocation-model=pic < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-freebsd -relocation-model=pic < %s | FileCheck %s
+
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+;; According to x86-64 psABI, xmm0-xmm7 can be used to pass function parameters.  
+;; However regcall calling convention uses also xmm8-xmm15 to pass function  
+;; parameters which violates x86-64 psABI. 
+;; Detail info about it can be found at:
+;; https://sourceware.org/bugzilla/show_bug.cgi?id=21265
+;;
+;; We encounter the violation symptom when using PIC with lazy binding 
+;; optimization.
+;; In that case the PLT mechanism as described in x86_64 psABI will
+;; not preserve xmm8-xmm15 registers and will lead to miscompilation.
+;;
+;; The agreed solution is to disable PLT for regcall calling convention for 
+;; SystemV using ELF format.
+;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
+
+declare void @lazy()
+declare x86_regcallcc void @regcall_not_lazy()
+
+; CHECK-LABEL: foo:
+; CHECK:  callq lazy@PLT
+; CHECK:  callq *regcall_not_lazy@GOTPCREL(%rip)
+define void @foo() nounwind {
+  call void @lazy()
+  call void @regcall_not_lazy()
+  ret void
+}
+
+; CHECK-LABEL: tail_call_regcall:
+; CHECK:   jmpq *regcall_not_lazy@GOTPCREL(%rip)
+define void @tail_call_regcall() nounwind {
+  tail call void @regcall_not_lazy()
+  ret void
+}
+
+; CHECK-LABEL: tail_call_regular:
+; CHECK:   jmp lazy
+define void @tail_call_regular() nounwind {
+  tail call void @lazy()
+  ret void
+}