Calling conventions for HHVM

[maksfb]

Summary

HHVM calling convention, hhvmcc, is used by HHVM JIT for calling
functions in translated cache. We currently support LLVM back end to
generate code for X86-64 and may support other architectures in the
future.

In HHVM calling convention any GP register could be used to pass and
return values, with the exception of R12 which is reserved for
thread-local area and is callee-saved. Other than R12, we always
pass RBX and RBP as args, which are our virtual machine's stack pointer
and frame pointer respectively.

When we enter translation cache via hhvmcc function, we expect
the stack to be aligned at 16 bytes, i.e. skewed by 8 bytes as opposed
to standard ABI alignment. This affects stack object alignment and stack
adjustments for calls.

The second calling convention, hhvm_ccc, is used to call C++ helpers from
HHVM's translation cache. It is almost identical to standard C calling
convention with an exception that we pass our virtual machine's frame pointer
in RBP as the first argument to a function, and then use standard
registers/stack (RDI, RSI, etc.) for the rest of the arguments.

[sanjoy]

By no means a "proper" review, but two things I noticed on a quick scan:

include/llvm/Support/MathExtras.h

619

Should this have a test in Support/MathExtrasTest.cpp?

lib/CodeGen/PrologEpilogInserter.cpp

570

I'd skip the braces for the single line body, like elsewhere in LLVM.

[maksfb]

include/llvm/Support/MathExtras.h

619

Thanks! Didn't know about it.

lib/CodeGen/PrologEpilogInserter.cpp

570

Sure.

[maksfb]

This comment was removed by maksfb.

[maksfb]

Restore the diff + suggestions by Sanjoy Das.

[mcrosier]

Looks to be in pretty good shape, but I'm not confident enough in my review to give the final LGTM. One minor nit.

lib/Target/X86/X86ISelLowering.cpp

3874

While this looks correct, I'm not sure I agree with the refactoring.

[maksfb]

Cool. Thanks.

lib/Target/X86/X86ISelLowering.cpp

3874

Which part of it? The logic here is to check for a tail call calling convention, and I thought it'd better to maintain a single list of those.

[mcrosier]

lib/Target/X86/X86ISelLowering.cpp

3874

Nevermind. You make a good point. :)

[reames]

Most of this looks pretty straight forward. The only part which worries me is the Skew implementation and how much that has to touch. Could you give a bit of background on why you need this skew, and why you chose to implement it this way?

Fair warning, I'm not going to be able to sign off on the skew parts. Everything else is in areas I'm familiar with, but that's really not.

include/llvm/IR/CallingConv.h

154

This sentence isn't clear to me. Is this analogous to anyreg where the register allocator can put arguments in any register and the runtime has to parse the stack map to figure out where they are? Or are there defined argument and return registers? If the later, I'd just emit this sentence.

lib/CodeGen/AsmPrinter/AsmPrinterDwarf.cpp

50

Is this a stray change? Or an unrelated bug fix? If the former, please remove. If the later, please submit separately.

lib/CodeGen/PrologEpilogInserter.cpp

515

It would be cleaner to separate a refactoring change which used the helper function, then extended the argument list later. I'd prefer you did this.

566

I don't have a concrete objection here, but this section feels suspicious. Why do you need to skew each and every element within the frame layout rather than inserting a dummy slot to ensure the skew is respected? Also, do you really want individual arguments passed at Align + Skew? Having an vector argument being passed at align (32) + 8 just seems weird to me.

[maksfb]

Thanks for the review. The stack skew is due to the deviation from x64 ABI and is related to the way we enter our translation cache which happens from a piece of assembly code. Typically on a function entry %rsp mod 16 == 8, for us it's %rsp mod 16 == 0. From LLVM point of view all the objects on the stack with alignment over 8 bytes have to be placed differently relative to %rsp. Also, if there happens to be a call to a regular function, we don't have to adjust %rsp when the stack frame is empty (on x64 you have to issue a dummy push or sub 8, %rsp). There's more info in inline comments on why we have to change the alignment rules for implementation.

include/llvm/IR/CallingConv.h

154

Unlike anyreg where LLVM chooses the registers, we'd like to give a similar level of control to the runtime (i.e. HHVM). If you think this sentence is confusing, I will remove it.

lib/CodeGen/AsmPrinter/AsmPrinterDwarf.cpp

50

Documentation bugfix.

lib/CodeGen/PrologEpilogInserter.cpp

515

Sounds good. I will submit refactoring change separately.

566

That's exactly what we need to do. Inserting dummy object wouldn't give us the same result. On x64 you need to push 8 bytes on a stack before making a call. Our stack have already been adjusted on the entry to HHVM translation cache function, so we can simply do the call without doing the stack adjustment. This changes alignment rules for all the objects on the stack though.

[maksfb]

Updated the patch based on Philip's feedback.

[qcolombet]

Hi Maksim,

LGTM with a couple of nitpicks.

Cheers,
-Quentin

include/llvm/Support/MathExtras.h

621

In the comment you said that Skew must be less than Align.
So, this line of code seems useless.
Either keep that line and change the comment to actually reflect what is happening or change that into an assert Skew < Align.

lib/CodeGen/PrologEpilogInserter.cpp

570

I would rather have a target hook for that: getSkew, so that we do not expose (more) target code in the generic part.

[maksfb]

Thanks for the review, Quentin!

include/llvm/Support/MathExtras.h

621

Good point. I'll change the comment.

lib/CodeGen/PrologEpilogInserter.cpp

570

Makes sense. Will move the logic to TargetFrameLowering.

[maksfb]

Updated the patch based on Quentin's feedback, and rebased.

[qcolombet]

Small nitpicks and questions.

include/llvm/Target/TargetFrameLowering.h

99 ↗

(On Diff #35933)

Do not repeat the method name. This was the old style comment.

102 ↗

(On Diff #35933)

Should be virtual.

lib/CodeGen/TargetFrameLoweringImpl.cpp

91 ↗

(On Diff #35933)

Is HHVM a generic calling convention or is it x86_64 specific.

If the latter, move this code in a x86 frame lowering override.

[maksfb]

Thanks!

include/llvm/Target/TargetFrameLowering.h

99 ↗

(On Diff #35933)

Ok.

102 ↗

(On Diff #35933)

Ok.

lib/CodeGen/TargetFrameLoweringImpl.cpp

91 ↗

(On Diff #35933)

It is a generic calling convention and for other architectures we will also have a return address removed from the stack before we enter the function.

[maksfb]

More feedback from Quentin.

[qcolombet]

Thanks for your patience.

Looks good, please go ahead!

Q.

include/llvm/IR/CallingConv.h

enum {
     /// \brief The C convention as implemented on Windows/x86-64. This
     /// convention differs from the more common \c X86_64_SysV convention
     /// in a number of ways, most notably in that XMM registers used to pass
     /// arguments are shadowed by GPRs, and vice versa.
     X86_64_Win64 = 79,
 
     /// \brief MSVC calling convention that passes vectors and vector aggregates
     /// in SSE registers.
-    X86_VectorCall = 80
+    X86_VectorCall = 80,
+
+    /// \brief Calling convention used by HipHop Virtual Machine (HHVM) to
+    /// perform calls to and from translation cache, and for calling PHP
+    /// functions. It can use most of GP registers for arguments and
+    /// return values.
+    /// HHVM calling convention supports tail/sibling call elimination.
+    HHVM = 81,
+
+    /// \brief HHVM calling convention for invoking C/C++ helpers.
+    HHVM_C = 82
   };
 } // End CallingConv namespace
 
 } // End llvm namespace
 
 #endif

include/llvm/Support/MathExtras.h

 inline uint64_t PowerOf2Floor(uint64_t A) {
   if (!A) return 0;
   return 1ull << (63 - countLeadingZeros(A, ZB_Undefined));
 }
 
 /// Returns the next integer (mod 2**64) that is greater than or equal to
 /// \p Value and is a multiple of \p Align. \p Align must be non-zero.
 ///
+/// If non-zero \p Skew is specified, the return value will be a minimal
+/// integer that is greater than or equal to \p Value and equal to
+/// \p Align * N + \p Skew for some integer N. The \p Skew must be less
+/// than \p Align.
+///
 /// Examples:
 /// \code
 ///   RoundUpToAlignment(5, 8) = 8
 ///   RoundUpToAlignment(17, 8) = 24
 ///   RoundUpToAlignment(~0LL, 8) = 0
 ///   RoundUpToAlignment(321, 255) = 510
+///
+///   RoundUpToAlignment(5, 8, 7) = 7
+///   RoundUpToAlignment(17, 8, 1) = 17
+///   RoundUpToAlignment(~0LL, 8, 3) = 3
+///   RoundUpToAlignment(321, 255, 42) = 552
 /// \endcode
-inline uint64_t RoundUpToAlignment(uint64_t Value, uint64_t Align) {
-  return (Value + Align - 1) / Align * Align;
+inline uint64_t RoundUpToAlignment(uint64_t Value, uint64_t Align,
+                                   uint64_t Skew = 0) {
+  Skew %= Align;
+  return (Value + Align - 1 - Skew) / Align * Align + Skew;
 }
 
 /// Returns the offset to the next integer (mod 2**64) that is greater than
 /// or equal to \p Value and is a multiple of \p Align. \p Align must be
 /// non-zero.
 inline uint64_t OffsetToAlignment(uint64_t Value, uint64_t Align) {
   return RoundUpToAlignment(Value, Align) - Value;
 }

lib/AsmParser/LLLexer.cpp

#define KEYWORD(STR)                                                           \
   KEYWORD(intel_ocl_bicc);
   KEYWORD(x86_64_sysvcc);
   KEYWORD(x86_64_win64cc);
   KEYWORD(webkit_jscc);
   KEYWORD(anyregcc);
   KEYWORD(preserve_mostcc);
   KEYWORD(preserve_allcc);
   KEYWORD(ghccc);
+  KEYWORD(hhvmcc);
+  KEYWORD(hhvm_ccc);
 
   KEYWORD(cc);
   KEYWORD(c);
 
   KEYWORD(attributes);
 
   KEYWORD(alwaysinline);
   KEYWORD(argmemonly);

lib/AsmParser/LLParser.cpp

 ///   ::= 'spir_kernel'
 ///   ::= 'x86_64_sysvcc'
 ///   ::= 'x86_64_win64cc'
 ///   ::= 'webkit_jscc'
 ///   ::= 'anyregcc'
 ///   ::= 'preserve_mostcc'
 ///   ::= 'preserve_allcc'
 ///   ::= 'ghccc'
+///   ::= 'hhvmcc'
+///   ::= 'hhvm_ccc'
 ///   ::= 'cc' UINT
 ///
 bool LLParser::ParseOptionalCallingConv(unsigned &CC) {
   switch (Lex.getKind()) {
   default:                       CC = CallingConv::C; return false;
   case lltok::kw_ccc:            CC = CallingConv::C; break;
   case lltok::kw_fastcc:         CC = CallingConv::Fast; break;
   case lltok::kw_coldcc:         CC = CallingConv::Cold; break;
bool LLParser::ParseOptionalCallingConv(unsigned &CC) {
   case lltok::kw_intel_ocl_bicc: CC = CallingConv::Intel_OCL_BI; break;
   case lltok::kw_x86_64_sysvcc:  CC = CallingConv::X86_64_SysV; break;
   case lltok::kw_x86_64_win64cc: CC = CallingConv::X86_64_Win64; break;
   case lltok::kw_webkit_jscc:    CC = CallingConv::WebKit_JS; break;
   case lltok::kw_anyregcc:       CC = CallingConv::AnyReg; break;
   case lltok::kw_preserve_mostcc:CC = CallingConv::PreserveMost; break;
   case lltok::kw_preserve_allcc: CC = CallingConv::PreserveAll; break;
   case lltok::kw_ghccc:          CC = CallingConv::GHC; break;
+  case lltok::kw_hhvmcc:         CC = CallingConv::HHVM; break;
+  case lltok::kw_hhvm_ccc:       CC = CallingConv::HHVM_C; break;
   case lltok::kw_cc: {
       Lex.Lex();
       return ParseUInt32(CC);
     }
   }
 
   Lex.Lex();
   return false;

lib/AsmParser/LLToken.h

enum Kind {
     kw_arm_apcscc, kw_arm_aapcscc, kw_arm_aapcs_vfpcc,
     kw_msp430_intrcc,
     kw_ptx_kernel, kw_ptx_device,
     kw_spir_kernel, kw_spir_func,
     kw_x86_64_sysvcc, kw_x86_64_win64cc,
     kw_webkit_jscc, kw_anyregcc,
     kw_preserve_mostcc, kw_preserve_allcc,
     kw_ghccc,
+    kw_hhvmcc, kw_hhvm_ccc,
 
     // Attributes:
     kw_attributes,
     kw_alwaysinline,
     kw_argmemonly,
     kw_sanitize_address,
     kw_builtin,
     kw_byval,

lib/CodeGen/AsmPrinter/AsmPrinterDwarf.cpp

 /// EmitSLEB128 - emit the specified signed leb128 value.
 void AsmPrinter::EmitSLEB128(int64_t Value, const char *Desc) const {
   if (isVerbose() && Desc)
     OutStreamer->AddComment(Desc);
 
   OutStreamer->EmitSLEB128IntValue(Value);
 }
 
-/// EmitULEB128 - emit the specified signed leb128 value.
+/// EmitULEB128 - emit the specified unsigned leb128 value.
 void AsmPrinter::EmitULEB128(uint64_t Value, const char *Desc,
                              unsigned PadTo) const {
   if (isVerbose() && Desc)
     OutStreamer->AddComment(Desc);
 
   OutStreamer->EmitULEB128IntValue(Value, PadTo);
 }
 

lib/CodeGen/PrologEpilogInserter.cpp

for (MachineBasicBlock *MBB : RestoreBlocks) {
     }
   }
 }
 
 /// AdjustStackOffset - Helper function used to adjust the stack frame offset.
 static inline void
 AdjustStackOffset(MachineFrameInfo *MFI, int FrameIdx,
                   bool StackGrowsDown, int64_t &Offset,
-                  unsigned &MaxAlign) {
+                  unsigned &MaxAlign, unsigned Skew) {
   // If the stack grows down, add the object size to find the lowest address.
   if (StackGrowsDown)
     Offset += MFI->getObjectSize(FrameIdx);
 
   unsigned Align = MFI->getObjectAlignment(FrameIdx);
 
   // If the alignment of this object is greater than that of the stack, then
   // increase the stack alignment to match.
   MaxAlign = std::max(MaxAlign, Align);
 
   // Adjust to alignment boundary.
-  Offset = (Offset + Align - 1) / Align * Align;
+  Offset = RoundUpToAlignment(Offset, Align, Skew);
 
   if (StackGrowsDown) {
     DEBUG(dbgs() << "alloc FI(" << FrameIdx << ") at SP[" << -Offset << "]\n");
     MFI->setObjectOffset(FrameIdx, -Offset); // Set the computed offset
   } else {
     DEBUG(dbgs() << "alloc FI(" << FrameIdx << ") at SP[" << Offset << "]\n");
     MFI->setObjectOffset(FrameIdx, Offset);
     Offset += MFI->getObjectSize(FrameIdx);
   }
 }
 
 /// AssignProtectedObjSet - Helper function to assign large stack objects (i.e.,
 /// those required to be close to the Stack Protector) to stack offsets.
 static void
 AssignProtectedObjSet(const StackObjSet &UnassignedObjs,
                       SmallSet<int, 16> &ProtectedObjs,
                       MachineFrameInfo *MFI, bool StackGrowsDown,
-                      int64_t &Offset, unsigned &MaxAlign) {
+                      int64_t &Offset, unsigned &MaxAlign, unsigned Skew) {
 
   for (StackObjSet::const_iterator I = UnassignedObjs.begin(),
         E = UnassignedObjs.end(); I != E; ++I) {
     int i = *I;
-    AdjustStackOffset(MFI, i, StackGrowsDown, Offset, MaxAlign);
+    AdjustStackOffset(MFI, i, StackGrowsDown, Offset, MaxAlign, Skew);
     ProtectedObjs.insert(i);
   }
 }
 
 /// calculateFrameObjectOffsets - Calculate actual frame offsets for all of the
 /// abstract stack objects.
 ///
 void PEI::calculateFrameObjectOffsets(MachineFunction &Fn) {
void PEI::calculateFrameObjectOffsets(MachineFunction &Fn) {
   // of stack growth -- so it's always nonnegative.
   int LocalAreaOffset = TFI.getOffsetOfLocalArea();
   if (StackGrowsDown)
     LocalAreaOffset = -LocalAreaOffset;
   assert(LocalAreaOffset >= 0
          && "Local area offset should be in direction of stack growth");
   int64_t Offset = LocalAreaOffset;
 
+  // For HHVM calling convention we need to apply skew to alignment of all
+  // objects on the stack because enter the translation cache at a different
+  // alignment than most functions.
+  unsigned Skew = 0;
+  if (LLVM_UNLIKELY(Fn.getFunction()->getCallingConv() == CallingConv::HHVM)) {
+    Skew = 8;
+  }
+
   // If there are fixed sized objects that are preallocated in the local area,
   // non-fixed objects can't be allocated right at the start of local area.
   // We currently don't support filling in holes in between fixed sized
   // objects, so we adjust 'Offset' to point to the end of last fixed sized
   // preallocated object.
   for (int i = MFI->getObjectIndexBegin(); i != 0; ++i) {
     int64_t FixedOff;
     if (StackGrowsDown) {
void PEI::calculateFrameObjectOffsets(MachineFunction &Fn) {
   if (StackGrowsDown) {
     for (unsigned i = MinCSFrameIndex; i <= MaxCSFrameIndex; ++i) {
       // If the stack grows down, we need to add the size to find the lowest
       // address of the object.
       Offset += MFI->getObjectSize(i);
 
       unsigned Align = MFI->getObjectAlignment(i);
       // Adjust to alignment boundary
-      Offset = RoundUpToAlignment(Offset, Align);
+      Offset = RoundUpToAlignment(Offset, Align, Skew);
 
       MFI->setObjectOffset(i, -Offset);        // Set the computed offset
     }
   } else {
     int MaxCSFI = MaxCSFrameIndex, MinCSFI = MinCSFrameIndex;
     for (int i = MaxCSFI; i >= MinCSFI ; --i) {
       unsigned Align = MFI->getObjectAlignment(i);
       // Adjust to alignment boundary
-      Offset = RoundUpToAlignment(Offset, Align);
+      Offset = RoundUpToAlignment(Offset, Align, Skew);
 
       MFI->setObjectOffset(i, Offset);
       Offset += MFI->getObjectSize(i);
     }
   }
 
   unsigned MaxAlign = MFI->getMaxAlignment();
 
   // Make sure the special register scavenging spill slot is closest to the
   // incoming stack pointer if a frame pointer is required and is closer
   // to the incoming rather than the final stack pointer.
   const TargetRegisterInfo *RegInfo = Fn.getSubtarget().getRegisterInfo();
   bool EarlyScavengingSlots = (TFI.hasFP(Fn) &&
                                TFI.isFPCloseToIncomingSP() &&
                                RegInfo->useFPForScavengingIndex(Fn) &&
                                !RegInfo->needsStackRealignment(Fn));
   if (RS && EarlyScavengingSlots) {
     SmallVector<int, 2> SFIs;
     RS->getScavengingFrameIndices(SFIs);
     for (SmallVectorImpl<int>::iterator I = SFIs.begin(),
            IE = SFIs.end(); I != IE; ++I)
-      AdjustStackOffset(MFI, *I, StackGrowsDown, Offset, MaxAlign);
+      AdjustStackOffset(MFI, *I, StackGrowsDown, Offset, MaxAlign, Skew);
   }
 
   // FIXME: Once this is working, then enable flag will change to a target
   // check for whether the frame is large enough to want to use virtual
   // frame index registers. Functions which don't want/need this optimization
   // will continue to use the existing code path.
   if (MFI->getUseLocalStackAllocationBlock()) {
     unsigned Align = MFI->getLocalFrameMaxAlign();
 
     // Adjust to alignment boundary.
-    Offset = RoundUpToAlignment(Offset, Align);
+    Offset = RoundUpToAlignment(Offset, Align, Skew);
 
     DEBUG(dbgs() << "Local frame base offset: " << Offset << "\n");
 
     // Resolve offsets for objects in the local block.
     for (unsigned i = 0, e = MFI->getLocalFrameObjectCount(); i != e; ++i) {
       std::pair<int, int64_t> Entry = MFI->getLocalFrameObjectMap(i);
       int64_t FIOffset = (StackGrowsDown ? -Offset : Offset) + Entry.second;
       DEBUG(dbgs() << "alloc FI(" << Entry.first << ") at SP[" <<
void PEI::calculateFrameObjectOffsets(MachineFunction &Fn) {
   // stack.
   SmallSet<int, 16> ProtectedObjs;
   if (MFI->getStackProtectorIndex() >= 0) {
     StackObjSet LargeArrayObjs;
     StackObjSet SmallArrayObjs;
     StackObjSet AddrOfObjs;
 
     AdjustStackOffset(MFI, MFI->getStackProtectorIndex(), StackGrowsDown,
-                      Offset, MaxAlign);
+                      Offset, MaxAlign, Skew);
 
     // Assign large stack objects first.
     for (unsigned i = 0, e = MFI->getObjectIndexEnd(); i != e; ++i) {
       if (MFI->isObjectPreAllocated(i) &&
           MFI->getUseLocalStackAllocationBlock())
         continue;
       if (i >= MinCSFrameIndex && i <= MaxCSFrameIndex)
         continue;
for (unsigned i = 0, e = MFI->getObjectIndexEnd(); i != e; ++i) {
       case StackProtector::SSPLK_LargeArray:
         LargeArrayObjs.insert(i);
         continue;
       }
       llvm_unreachable("Unexpected SSPLayoutKind.");
     }
 
     AssignProtectedObjSet(LargeArrayObjs, ProtectedObjs, MFI, StackGrowsDown,
-                          Offset, MaxAlign);
+                          Offset, MaxAlign, Skew);
     AssignProtectedObjSet(SmallArrayObjs, ProtectedObjs, MFI, StackGrowsDown,
-                          Offset, MaxAlign);
+                          Offset, MaxAlign, Skew);
     AssignProtectedObjSet(AddrOfObjs, ProtectedObjs, MFI, StackGrowsDown,
-                          Offset, MaxAlign);
+                          Offset, MaxAlign, Skew);
   }
 
   // Then assign frame offsets to stack objects that are not used to spill
   // callee saved registers.
   for (unsigned i = 0, e = MFI->getObjectIndexEnd(); i != e; ++i) {
     if (MFI->isObjectPreAllocated(i) &&
         MFI->getUseLocalStackAllocationBlock())
       continue;
     if (i >= MinCSFrameIndex && i <= MaxCSFrameIndex)
       continue;
     if (RS && RS->isScavengingFrameIndex((int)i))
       continue;
     if (MFI->isDeadObjectIndex(i))
       continue;
     if (MFI->getStackProtectorIndex() == (int)i)
       continue;
     if (ProtectedObjs.count(i))
       continue;
 
-    AdjustStackOffset(MFI, i, StackGrowsDown, Offset, MaxAlign);
+    AdjustStackOffset(MFI, i, StackGrowsDown, Offset, MaxAlign, Skew);
   }
 
   // Make sure the special register scavenging spill slot is closest to the
   // stack pointer.
   if (RS && !EarlyScavengingSlots) {
     SmallVector<int, 2> SFIs;
     RS->getScavengingFrameIndices(SFIs);
     for (SmallVectorImpl<int>::iterator I = SFIs.begin(),
            IE = SFIs.end(); I != IE; ++I)
-      AdjustStackOffset(MFI, *I, StackGrowsDown, Offset, MaxAlign);
+      AdjustStackOffset(MFI, *I, StackGrowsDown, Offset, MaxAlign, Skew);
   }
 
   if (!TFI.targetHandlesStackFrameRounding()) {
     // If we have reserved argument space for call sites in the function
     // immediately on entry to the current function, count it as part of the
     // overall stack size.
     if (MFI->adjustsStack() && TFI.hasReservedCallFrame(Fn))
       Offset += MFI->getMaxCallFrameSize();
 
     // Round up the size to a multiple of the alignment.  If the function has
     // any calls or alloca's, align to the target's StackAlignment value to
     // ensure that the callee's frame or the alloca data is suitably aligned;
     // otherwise, for leaf functions, align to the TransientStackAlignment
     // value.
     unsigned StackAlign;
     if (MFI->adjustsStack() || MFI->hasVarSizedObjects() ||
         (RegInfo->needsStackRealignment(Fn) && MFI->getObjectIndexEnd() != 0))
       StackAlign = TFI.getStackAlignment();
     else
       StackAlign = TFI.getTransientStackAlignment();
 
     // If the frame pointer is eliminated, all frame offsets will be relative to
     // SP not FP. Align to MaxAlign so this works.
     StackAlign = std::max(StackAlign, MaxAlign);
-    Offset = RoundUpToAlignment(Offset, StackAlign);
+    Offset = RoundUpToAlignment(Offset, StackAlign, Skew);
   }
 
   // Update frame info to pretend that this is part of the stack...
   int64_t StackSize = Offset - LocalAreaOffset;
   MFI->setStackSize(StackSize);
   NumBytesStackSpace += StackSize;
 }
 

lib/IR/AsmWriter.cpp

static void PrintCallingConv(unsigned cc, raw_ostream &Out) {
   case CallingConv::ARM_AAPCS_VFP: Out << "arm_aapcs_vfpcc"; break;
   case CallingConv::MSP430_INTR:   Out << "msp430_intrcc"; break;
   case CallingConv::PTX_Kernel:    Out << "ptx_kernel"; break;
   case CallingConv::PTX_Device:    Out << "ptx_device"; break;
   case CallingConv::X86_64_SysV:   Out << "x86_64_sysvcc"; break;
   case CallingConv::X86_64_Win64:  Out << "x86_64_win64cc"; break;
   case CallingConv::SPIR_FUNC:     Out << "spir_func"; break;
   case CallingConv::SPIR_KERNEL:   Out << "spir_kernel"; break;
+  case CallingConv::HHVM:          Out << "hhvmcc"; break;
+  case CallingConv::HHVM_C:        Out << "hhvm_ccc"; break;
   }
 }
 
 // PrintEscapedString - Print each character of the specified string, escaping
 // it if it is not printable or if it is an escape char.
 static void PrintEscapedString(StringRef Name, raw_ostream &Out) {
   for (unsigned i = 0, e = Name.size(); i != e; ++i) {
     unsigned char C = Name[i];

lib/Target/X86/X86CallingConv.td

 //
 // This calling convention is currently only supported by the stackmap and
 // patchpoint intrinsics. All other uses will result in an assert on Debug
 // builds. On Release builds we fallback to the X86 C calling convention.
 def RetCC_X86_64_AnyReg : CallingConv<[
   CCCustom<"CC_X86_AnyReg_Error">
 ]>;
 
+// X86-64 HHVM return-value convention.
+def RetCC_X86_64_HHVM: CallingConv<[
+  // Promote all types to i64
+  CCIfType<[i8, i16, i32], CCPromoteToType<i64>>,
+
+  // Return: could return in any GP register save RSP and R12.
+  CCIfType<[i64], CCAssignToReg<[RBX, RBP, RDI, RSI, RDX, RCX, R8, R9,
+                                 RAX, R10, R11, R13, R14, R15]>>
+]>;
+
 // This is the root return-value convention for the X86-32 backend.
 def RetCC_X86_32 : CallingConv<[
   // If FastCC, use RetCC_X86_32_Fast.
   CCIfCC<"CallingConv::Fast", CCDelegateTo<RetCC_X86_32_Fast>>,
   // If HiPE, use RetCC_X86_32_HiPE.
   CCIfCC<"CallingConv::HiPE", CCDelegateTo<RetCC_X86_32_HiPE>>,
   CCIfCC<"CallingConv::X86_VectorCall", CCDelegateTo<RetCC_X86_32_VectorCall>>,
 
def RetCC_X86_64 : CallingConv<[
   // Handle JavaScript calls.
   CCIfCC<"CallingConv::WebKit_JS", CCDelegateTo<RetCC_X86_64_WebKit_JS>>,
   CCIfCC<"CallingConv::AnyReg", CCDelegateTo<RetCC_X86_64_AnyReg>>,
 
   // Handle explicit CC selection
   CCIfCC<"CallingConv::X86_64_Win64", CCDelegateTo<RetCC_X86_Win64_C>>,
   CCIfCC<"CallingConv::X86_64_SysV", CCDelegateTo<RetCC_X86_64_C>>,
 
+  // Handle HHVM calls.
+  CCIfCC<"CallingConv::HHVM", CCDelegateTo<RetCC_X86_64_HHVM>>,
+
   // Mingw64 and native Win64 use Win64 CC
   CCIfSubtarget<"isTargetWin64()", CCDelegateTo<RetCC_X86_Win64_C>>,
 
   // Otherwise, drop to normal X86-64 CC
   CCDelegateTo<RetCC_X86_64_C>
 ]>;
 
 // This is the return-value convention used for the entire X86 backend.
def CC_X86_64_C : CallingConv<[
   CCIfType<[v32i8, v16i16, v8i32, v4i64, v8f32, v4f64],
            CCAssignToStack<32, 32>>,
 
   // 512-bit vectors get 64-byte stack slots that are 64-byte aligned.
   CCIfType<[v16i32, v8i64, v16f32, v8f64],
            CCAssignToStack<64, 64>>
 ]>;
 
+// Calling convention for X86-64 HHVM.
+def CC_X86_64_HHVM : CallingConv<[
+  // Use all/any GP registers for args, except RSP.
+  CCIfType<[i64], CCAssignToReg<[RBX, R12, RBP, R15,
+                                 RDI, RSI, RDX, RCX, R8, R9,
+                                 RAX, R10, R11, R13, R14]>>
+]>;
+
+// Calling convention for helper functions in HHVM.
+def CC_X86_64_HHVM_C : CallingConv<[
+  // Pass the first argument in RBP.
+  CCIfType<[i64], CCAssignToReg<[RBP]>>,
+
+  // Otherwise it's the same as the regular C calling convention.
+  CCDelegateTo<CC_X86_64_C>
+]>;
+
 // Calling convention used on Win64
 def CC_X86_Win64_C : CallingConv<[
   // FIXME: Handle byval stuff.
   // FIXME: Handle varargs.
 
   // Promote i1/i8/i16 arguments to i32.
   CCIfType<[i1, i8, i16], CCPromoteToType<i32>>,
 

 def CC_X86_64 : CallingConv<[
   CCIfCC<"CallingConv::GHC", CCDelegateTo<CC_X86_64_GHC>>,
   CCIfCC<"CallingConv::HiPE", CCDelegateTo<CC_X86_64_HiPE>>,
   CCIfCC<"CallingConv::WebKit_JS", CCDelegateTo<CC_X86_64_WebKit_JS>>,
   CCIfCC<"CallingConv::AnyReg", CCDelegateTo<CC_X86_64_AnyReg>>,
   CCIfCC<"CallingConv::X86_64_Win64", CCDelegateTo<CC_X86_Win64_C>>,
   CCIfCC<"CallingConv::X86_64_SysV", CCDelegateTo<CC_X86_64_C>>,
   CCIfCC<"CallingConv::X86_VectorCall", CCDelegateTo<CC_X86_Win64_VectorCall>>,
+  CCIfCC<"CallingConv::HHVM", CCDelegateTo<CC_X86_64_HHVM>>,
+  CCIfCC<"CallingConv::HHVM_C", CCDelegateTo<CC_X86_64_HHVM_C>>,
 
   // Mingw64 and native Win64 use Win64 CC
   CCIfSubtarget<"isTargetWin64()", CCDelegateTo<CC_X86_Win64_C>>,
 
   // Otherwise, drop to normal X86-64 CC
   CCDelegateTo<CC_X86_64_C>
 ]>;
 

 
 //Standard C + YMM 8-15
 def CSR_64_Intel_OCL_BI_AVX    : CalleeSavedRegs<(add CSR_64,
                                                   (sequence "YMM%u", 8, 15))>;
 
 def CSR_64_Intel_OCL_BI_AVX512 : CalleeSavedRegs<(add RBX, RDI, RSI, R14, R15,
                                                   (sequence "ZMM%u", 16, 31),
                                                   K4, K5, K6, K7)>;
+
+// Only R12 is preserved for PHP calls in HHVM.
+def CSR_64_HHVM : CalleeSavedRegs<(add R12)>;

lib/Target/X86/X86ISelLowering.cpp

return DAG.getMemcpy(Chain, dl, Dst, Src, SizeNode, Flags.getByValAlign(),
                        /*isTailCall*/false,
                        MachinePointerInfo(), MachinePointerInfo());
 }
 
 /// Return true if the calling convention is one that
 /// supports tail call optimization.
 static bool IsTailCallConvention(CallingConv::ID CC) {
   return (CC == CallingConv::Fast || CC == CallingConv::GHC ||
-          CC == CallingConv::HiPE);
+          CC == CallingConv::HiPE || CC == CallingConv::HHVM);
 }
 
 /// \brief Return true if the calling convention is a C calling convention.
 static bool IsCCallConvention(CallingConv::ID CC) {
   return (CC == CallingConv::C || CC == CallingConv::X86_64_Win64 ||
           CC == CallingConv::X86_64_SysV);
 }
 
bool X86::isOffsetSuitableForCodeModel(int64_t Offset, CodeModel::Model M,
 
   return false;
 }
 
 /// Determines whether the callee is required to pop its own arguments.
 /// Callee pop is necessary to support tail calls.
 bool X86::isCalleePop(CallingConv::ID CallingConv,
                       bool is64Bit, bool IsVarArg, bool TailCallOpt) {
+
+  if (IsTailCallConvention(CallingConv))
+    return IsVarArg ? false : TailCallOpt;
+
   switch (CallingConv) {
   default:
     return false;
   case CallingConv::X86_StdCall:
   case CallingConv::X86_FastCall:
   case CallingConv::X86_ThisCall:
     return !is64Bit;
-  case CallingConv::Fast:
-  case CallingConv::GHC:
-  case CallingConv::HiPE:
-    if (IsVarArg)
-      return false;
-    return TailCallOpt;
   }
 }
 
 /// \brief Return true if the condition is an unsigned comparison operation.
 static bool isX86CCUnsigned(unsigned X86CC) {
   switch (X86CC) {
   default: llvm_unreachable("Invalid integer condition!");
   case X86::COND_E:     return true;

lib/Target/X86/X86RegisterInfo.cpp

case CallingConv::Intel_OCL_BI: {
     if (HasAVX && IsWin64)
       return CSR_Win64_Intel_OCL_BI_AVX_SaveList;
     if (HasAVX && Is64Bit)
       return CSR_64_Intel_OCL_BI_AVX_SaveList;
     if (!HasAVX && !IsWin64 && Is64Bit)
       return CSR_64_Intel_OCL_BI_SaveList;
     break;
   }
+  case CallingConv::HHVM:
+    return CSR_64_HHVM_SaveList;
   case CallingConv::Cold:
     if (Is64Bit)
       return CSR_64_MostRegs_SaveList;
     break;
   case CallingConv::X86_64_Win64:
     return CSR_Win64_SaveList;
   case CallingConv::X86_64_SysV:
     if (CallsEHReturn)
case CallingConv::Intel_OCL_BI: {
     if (HasAVX && IsWin64)
       return CSR_Win64_Intel_OCL_BI_AVX_RegMask;
     if (HasAVX && Is64Bit)
       return CSR_64_Intel_OCL_BI_AVX_RegMask;
     if (!HasAVX && !IsWin64 && Is64Bit)
       return CSR_64_Intel_OCL_BI_RegMask;
     break;
   }
+  case CallingConv::HHVM:
+    return CSR_64_HHVM_RegMask;
   case CallingConv::Cold:
     if (Is64Bit)
       return CSR_64_MostRegs_RegMask;
     break;
   default:
     break;
   case CallingConv::X86_64_Win64:
     return CSR_Win64_RegMask;

test/CodeGen/X86/hhvm-cc.ll

+; RUN: llc < %s | FileCheck %s
+
+target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare hhvmcc i64 @bar(i64, i64, i64) nounwind
+
+; Simply check we can modify %rbx and %rbp before returning via call to bar.
+define hhvmcc i64 @foo(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  foo:
+; CHECK-DAG:    movl $1, %ebx
+; CHECK-DAG:    movl $3, %ebp
+; CHECK:        jmp bar
+  %ret = musttail call hhvmcc i64 @bar(i64 1, i64 %b, i64 3)
+  ret i64 %ret
+}
+
+; Check that we can read and modify %rbx returned from PHP function.
+define hhvmcc i64 @mod_return(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  mod_return:
+; CHECK-NEXT:   {{^#.*}}
+; CHECK-NEXT:   callq bar
+; CHECK-NEXT:   incq %rbx
+  %tmp = call hhvmcc i64 @bar(i64 %a, i64 %b, i64 %c)
+  %retval = add i64 %tmp, 1
+  ret i64 %retval
+}
+
+%rettype = type { i64, i64, i64, i64, i64, i64, i64,
+                  i64, i64, i64, i64, i64, i64, i64
+}
+
+; Check that we can return up to 14 64-bit args in registers.
+define hhvmcc %rettype @return_all(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  return_all:
+; CHECK-DAG:    movl $1, %ebx
+; CHECK-DAG:    movl $2, %ebp
+; CHECK-DAG:    movl $3, %edi
+; CHECK-DAG:    movl $4, %esi
+; CHECK-DAG:    movl $5, %edx
+; CHECK-DAG:    movl $6, %ecx
+; CHECK-DAG:    movl $7, %r8
+; CHECK-DAG:    movl $8, %r9
+; CHECK-DAG:    movl $9, %eax
+; CHECK-DAG:    movl $10, %r10
+; CHECK-DAG:    movl $11, %r11
+; CHECK-DAG:    movl $12, %r13
+; CHECK-DAG:    movl $13, %r14
+; CHECK-DAG:    movl $14, %r15
+; CHECK:        retq
+  %r1 = insertvalue %rettype zeroinitializer, i64 1, 0
+  %r2 = insertvalue %rettype %r1, i64 2, 1
+  %r3 = insertvalue %rettype %r2, i64 3, 2
+  %r4 = insertvalue %rettype %r3, i64 4, 3
+  %r5 = insertvalue %rettype %r4, i64 5, 4
+  %r6 = insertvalue %rettype %r5, i64 6, 5
+  %r7 = insertvalue %rettype %r6, i64 7, 6
+  %r8 = insertvalue %rettype %r7, i64 8, 7
+  %r9 = insertvalue %rettype %r8, i64 9, 8
+  %r10 = insertvalue %rettype %r9, i64 10, 9
+  %r11 = insertvalue %rettype %r10, i64 11, 10
+  %r12 = insertvalue %rettype %r11, i64 12, 11
+  %r13 = insertvalue %rettype %r12, i64 13, 12
+  %r14 = insertvalue %rettype %r13, i64 14, 13
+  ret %rettype %r14
+}
+
+declare hhvmcc void @return_all_tc(i64, i64, i64, i64, i64, i64, i64, i64,
+                                 i64, i64, i64, i64, i64, i64, i64)
+
+; Check that we can return up to 14 64-bit args in registers via tail call.
+define hhvmcc void @test_return_all_tc(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_return_all_tc:
+; CHECK-NEXT:   {{^#.*}}
+; CHECK-DAG:    movl $1, %ebx
+; CHECK-DAG:    movl $3, %ebp
+; CHECK-DAG:    movl $4, %r15
+; CHECK-DAG:    movl $5, %edi
+; CHECK-DAG:    movl $6, %esi
+; CHECK-DAG:    movl $7, %edx
+; CHECK-DAG:    movl $8, %ecx
+; CHECK-DAG:    movl $9, %r8
+; CHECK-DAG:    movl $10, %r9
+; CHECK-DAG:    movl $11, %eax
+; CHECK-DAG:    movl $12, %r10
+; CHECK-DAG:    movl $13, %r11
+; CHECK-DAG:    movl $14, %r13
+; CHECK-DAG:    movl $15, %r14
+; CHECK:        jmp  return_all_tc
+  tail call hhvmcc void @return_all_tc(
+    i64 1, i64 %b, i64 3, i64 4, i64 5, i64 6, i64 7,
+    i64 8, i64 9, i64 10, i64 11, i64 12, i64 13, i64 14, i64 15)
+  ret void
+}
+
+declare hhvmcc {i64, i64} @php_short(i64, i64, i64, i64)
+
+define hhvmcc i64 @test_php_short(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_php_short:
+; CHECK-NEXT:   {{^#.*}}
+; CHECK-NEXT:   movl $42, %r15
+; CHECK-NEXT:   callq php_short
+; CHECK-NEXT:   leaq (%rbp,%r12), %rbx
+; CHECK-NEXT:   retq
+  %pair = call hhvmcc {i64, i64} @php_short(i64 %a, i64 %b, i64 %c, i64 42)
+  %fp = extractvalue {i64, i64} %pair, 1
+  %rv = add i64 %fp, %b
+  ret i64 %rv
+}
+
+declare hhvmcc %rettype @php_all(i64, i64, i64, i64, i64, i64, i64,
+                                 i64, i64, i64, i64, i64, i64, i64, i64)
+
+; Check that we can pass 15 arguments in registers.
+; Also check that %r12 (2nd arg) is not spilled.
+define hhvmcc i64 @test_php_all(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_php_all:
+; CHECK-NEXT:   {{^#.*}}
+; CHECK-NOT:    sub
+; CHECK-NOT:    sub
+; CHECK-DAG:    movl $1, %ebx
+; CHECK-DAG:    movl $3, %ebp
+; CHECK-DAG:    movl $4, %r15
+; CHECK-DAG:    movl $5, %edi
+; CHECK-DAG:    movl $6, %esi
+; CHECK-DAG:    movl $7, %edx
+; CHECK-DAG:    movl $8, %ecx
+; CHECK-DAG:    movl $9, %r8
+; CHECK-DAG:    movl $10, %r9
+; CHECK-DAG:    movl $11, %eax
+; CHECK-DAG:    movl $12, %r10
+; CHECK-DAG:    movl $13, %r11
+; CHECK-DAG:    movl $14, %r13
+; CHECK-DAG:    movl $15, %r14
+; CHECK:        callq php_all
+  %pair = call hhvmcc %rettype @php_all(
+    i64 1, i64 %b, i64 3, i64 4, i64 5, i64 6, i64 7,
+    i64 8, i64 9, i64 10, i64 11, i64 12, i64 13, i64 14, i64 15)
+  %fp = extractvalue %rettype %pair, 1
+  %rv = add i64 %fp, %b
+  ret i64 %rv
+}
+
+declare hhvmcc void @svcreq(i64, i64, i64, i64, i64, i64, i64, i64, i64, i64,
+                             i64, i64)
+
+define hhvmcc void @test_svcreq(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_svcreq:
+; CHECK-DAG:    movl $42, %r10
+; CHECK-DAG:    movl $1, %edi
+; CHECK-DAG:    movl $2, %esi
+; CHECK-DAG:    movl $3, %edx
+; CHECK-DAG:    movl $4, %ecx
+; CHECK-DAG:    movl $5, %r8
+; CHECK-DAG:    movl $6, %r9
+; CHECK:        jmp svcreq
+  tail call hhvmcc void @svcreq(i64 %a, i64 %b, i64 %c, i64 undef, i64 1,
+                                i64 2, i64 3, i64 4, i64 5, i64 6, i64 undef,
+                                i64 42)
+  ret void
+}
+
+declare hhvm_ccc void @helper_short(i64, i64, i64, i64, i64, i64, i64)
+
+; Pass all arguments in registers and check that we don't adjust stack
+; for the call.
+define hhvmcc void @test_helper_short(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_helper_short:
+; CHECK-NOT:    push
+; CHECK-NOT:    sub
+; CHECK-DAG:    movl $1, %edi
+; CHECK-DAG:    movl $2, %esi
+; CHECK-DAG:    movl $3, %edx
+; CHECK-DAG:    movl $4, %ecx
+; CHECK-DAG:    movl $5, %r8
+; CHECK-DAG:    movl $6, %r9
+; CHECK:        callq helper_short
+  call hhvm_ccc void @helper_short(i64 %c, i64 1, i64 2, i64 3, i64 4,
+                                   i64 5, i64 6)
+  ret void
+}
+
+declare hhvm_ccc void @helper(i64, i64, i64, i64, i64, i64, i64, i64, i64, i64)
+
+define hhvmcc void @test_helper(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_helper:
+; CHECK-DAG:    movl $1, %edi
+; CHECK-DAG:    movl $2, %esi
+; CHECK-DAG:    movl $3, %edx
+; CHECK-DAG:    movl $4, %ecx
+; CHECK-DAG:    movl $5, %r8
+; CHECK-DAG:    movl $6, %r9
+; CHECK:        callq helper
+  call hhvm_ccc void @helper(i64 %c, i64 1, i64 2, i64 3, i64 4, i64 5, i64 6,
+                             i64 7, i64 8, i64 9)
+  ret void
+}
+
+; When we enter function with HHVM calling convention, the stack is aligned
+; at 16 bytes. This means we align objects on the stack differently and
+; adjust the stack differently for calls.
+declare hhvm_ccc void @stack_helper(i64, i64, i64)
+declare hhvm_ccc void @stack_helper2(<2 x double>, i64)
+
+define hhvmcc void @test_stack_helper(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_stack_helper:
+; CHECK-NOT:    push
+; CHECK:        subq $32, %rsp
+; CHECK:        movaps  16(%rsp), %xmm0
+; CHECK:        callq stack_helper2
+  %t1 = alloca <2 x double>, align 16
+  %t2 = alloca i64, align 8
+  %t3 = alloca i64, align 8
+  %load3 = load i64, i64 *%t3
+  call hhvm_ccc void @stack_helper(i64 %c, i64 %load3, i64 42)
+  %load = load <2 x double>, <2 x double> *%t1
+  %load2 = load i64, i64 *%t2
+  call hhvm_ccc void @stack_helper2(<2 x double> %load, i64 %load2)
+  ret void
+}
+
+; Check that we are not adjusting the stack before calling the helper.
+define hhvmcc void @test_stack_helper2(i64 %a, i64 %b, i64 %c) nounwind {
+entry:
+; CHECK-LABEL:  test_stack_helper2:
+; CHECK-NOT:    push
+; CHECK-NOT:    subq
+  call hhvm_ccc void @stack_helper(i64 %c, i64 7, i64 42)
+  ret void
+}
+

test/Feature/callingconventions.ll

 
 declare ghccc void @ghc_callee()
 
 define void @ghc_caller() {
   call ghccc void @ghc_callee()
   ret void
 }
 
+declare hhvm_ccc void @hhvm_c_callee()
+
+define hhvmcc void @hhvm_caller() {
+  call hhvm_ccc void @hhvm_c_callee()
+  ret void
+}
+
 declare i32 @__gxx_personality_v0(...)

utils/vim/syntax/llvm.vim

 
 " Keywords.
 syn keyword llvmKeyword acq_rel acquire sanitize_address addrspace alias align
 syn keyword llvmKeyword alignstack alwaysinline appending arm_aapcs_vfpcc
 syn keyword llvmKeyword arm_aapcscc arm_apcscc asm atomic available_externally
 syn keyword llvmKeyword blockaddress byval c catch cc ccc cleanup coldcc common
 syn keyword llvmKeyword constant datalayout declare default define deplibs
 syn keyword llvmKeyword distinct dllexport dllimport except extern_weak external
-syn keyword llvmKeyword externally_initialized fastcc filter gc global hidden
-syn keyword llvmKeyword initialexec inlinehint inreg intel_ocl_bicc inteldialect
-syn keyword llvmKeyword internal linkonce linkonce_odr localdynamic localexec
-syn keyword llvmKeyword minsize module monotonic msp430_intrcc naked nest
+syn keyword llvmKeyword externally_initialized fastcc filter gc global hhvmcc
+syn keyword llvmKeyword hhvm_ccc hidden initialexec inlinehint inreg
+syn keyword llvmKeyword intel_ocl_bicc inteldialect internal linkonce
+syn keyword llvmKeyword linkonce_odr localdynamic localexec minsize module
+syn keyword llvmKeyword monotonic msp430_intrcc musttail naked nest
 syn keyword llvmKeyword noalias nocapture noimplicitfloat noinline nonlazybind
 syn keyword llvmKeyword noredzone noreturn nounwind optnone optsize personality
 syn keyword llvmKeyword private protected ptx_device ptx_kernel readnone
 syn keyword llvmKeyword readonly release returns_twice sanitize_thread
 syn keyword llvmKeyword sanitize_memory section seq_cst sideeffect signext
 syn keyword llvmKeyword singlethread spir_func spir_kernel sret ssp sspreq
 syn keyword llvmKeyword sspstrong tail target thread_local to triple
 syn keyword llvmKeyword unnamed_addr unordered uwtable volatile weak weak_odr