[COFF, ARM64] Implement Intrinsic.sponentry for AArch64

[yinma]

Summary

This patch adds Intrinsic.sponentry. This intrinsic is required to correctly support setjmp for AArch64 Windows platform.

[mstorsjo]

How is the corresponding case handled for x86_64, where you afaik also need the original stack pointer?

[mgrang]

lib/Target/AArch64/AArch64FastISel.cpp

3466

Period at end of comments.

[yinma]

I debugged MS ARM64 setjmp/longjmp functions and ARM64 EH runtime. If you looks like assembly output from CL for ARM64, it is using SP instead of FP.

[mstorsjo]

In D53673#1275103, @yinma wrote:

I debugged MS ARM64 setjmp/longjmp functions and ARM64 EH runtime. If you looks like assembly output from CL for ARM64, it is using SP instead of FP.

Hmm, in my test it seems like it doesn't even use the SP as it was on on entry, it uses SP on entry + 16:

$ cat frame.c 
#include <setjmp.h>
void other(char* buf);
jmp_buf jb;
void func(int a) {
    char buf[100];
    setjmp(jb);
    other(buf);
}
$ cl -c frame.c -O2
Microsoft (R) C/C++ Optimizing Compiler Version 19.15.26726 for ARM64
Copyright (C) Microsoft Corporation.  All rights reserved.

frame.c
$ dumpbin -nologo -disasm frame.obj

Dump of file frame.obj

File Type: COFF OBJECT

func:
  0000000000000000: F81F0FFE  str         lr,[sp,#-0x10]!
  0000000000000004: 94000000  bl          __security_push_cookie
  0000000000000008: D10183FF  sub         sp,sp,#0x60
  000000000000000C: 90000008  adrp        x8,jb
  0000000000000010: 91000100  add         x0,x8,jb
  0000000000000014: 910203E1  add         x1,sp,#0x80
  0000000000000018: 94000000  bl          _setjmpex
  000000000000001C: 910003E0  mov         x0,sp
  0000000000000020: 94000000  bl          other
  0000000000000024: 910183FF  add         sp,sp,#0x60
  0000000000000028: 94000000  bl          __security_pop_cookie
  000000000000002C: F84107FE  ldr         lr,[sp],#0x10
  0000000000000030: D65F03C0  ret

In this case, we first subtract sp by 0x10 and later by 0x60, but in the parameter to _setjmpex we pass sp + 0x80.

test/CodeGen/AArch64/sponentry.ll

2

This test should test both with fastisel and dagisel, and should test cases both with and without fixed objects.

[yinma]

Because __security_push_cookie has extra 0x10 subtraction on SP, if you removes buf[100], you will see no security_push_cookie is used. it will be sp on entry.

__security_push_cookie:

000000014000D680: D10043FF  sub         sp,sp,#0x10
000000014000D684: F0000471  adrp        xip1,__scrt_native_dllmain_reason
000000014000D688: 91002231  add         xip1,xip1,#8
000000014000D68C: F9400231  ldr         xip1,[xip1]
000000014000D690: CB3163F1  sub         xip1,sp,xip1
000000014000D694: F90007F1  str         xip1,[sp,#8]
000000014000D698: D65F03C0  ret

__security_pop_cookie:

000000014000D69C: F0000471  adrp        xip1,__scrt_native_dllmain_reason
000000014000D6A0: 91002231  add         xip1,xip1,#8
000000014000D6A4: F94007F0  ldr         xip0,[sp,#8]
000000014000D6A8: F9400231  ldr         xip1,[xip1]
000000014000D6AC: CB3063F0  sub         xip0,sp,xip0
000000014000D6B0: EB11021F  cmp         xip0,xip1
000000014000D6B4: 54000061  bne         000000014000D6C0
000000014000D6B8: 910043FF  add         sp,sp,#0x10
000000014000D6BC: D65F03C0  ret
000000014000D6C0: AA1003E0  mov         x0,xip0

[mstorsjo]

In D53673#1276123, @yinma wrote:

Because __security_push_cookie has extra 0x10 subtraction on SP, if you removes buf[100], you will see no security_push_cookie is used. it will be sp on entry.

Ah, I see. Ok, then I guess this aspect makes sense.

[yinma]

Please review again

[mstorsjo]

Except for the concern about the test, this looks sensible to me. As this adds a new intrinsic in the public IR, I'd prefer if someone else with a bit more experience with these things to approve it as well though - @efriedma?

test/CodeGen/AArch64/sponentry.ll

23

Are you sure this actually triggers the fixed object codepath? (This looks a lot like the test example I posted earlier, and that wasn't at least designed to trigger fixed objects.) I think at least a function with varargs has fixed objects.

[mstorsjo]

test/CodeGen/AArch64/sponentry.ll

23

Indeed, I tested this, and both functions show numFixed = 0. So is the other codepath actually tested with some other code, or still untested?

[efriedma]

All new target-independent intrinsics need to be documented in LangRef.

Thinking about it a bit, this seems like a reasonable intrinsic to support across all (or at least most) targets; the meaning is well-defined, and it can't be computed from any of the other frame-related intrinsics, so I think it makes sense to support cross-platform.

That said, maybe instead of adding a new intrinsic, we should rename llvm.addressofreturnaddress? It's the same thing on x86, and addressofreturnaddress doesn't really make sense on other targets anyway.

[rnk]

lib/Target/AArch64/AArch64ISelLowering.cpp

5186–5190

What invariant are you relying on for fixed objects here? This seems like it could be fragile. Stack protectors and exception handling schemes often introduce new fixed objects. You might want to scan for the first or last parameter or CSR or whatever it is that you're really trying to be relative to.

[yinma]

add var_arg test case

[efriedma]

That said, maybe instead of adding a new intrinsic, we should rename llvm.addressofreturnaddress? It's the same thing on x86, and addressofreturnaddress doesn't really make sense on other targets anyway.

Nevermind, ignore this; apparently we actually need a separate llvm.addressofreturnaddress() to return the address of the return address on non-x86 targets, to implement the MSVC _AddressOfReturnAddress().

[efriedma]

lib/Target/AArch64/AArch64ISelLowering.cpp

5186–5190

You can also get fixed objects by just having a function with too many arguments to fit in registers. (Please add a testcase.)

There isn't really any existing fixed object with the right offset, in general. But I think you can unconditionally compute the correct offset with something like DAG.getFrameIndex(MFI.CreateFixedObject(1, 0, true), VT). Or I guess you could use a pseudo-instruction, and expand it after frame lowering, but that doesn't seem obviously better.

[efriedma]

lib/Target/AArch64/AArch64ISelLowering.cpp

5184

This might become unnecessary. But if it doesn't, please use setFrameAddressIsTaken to force a frame pointer to be emitted, instead of checking whether frame pointer elimination is enabled.

5189

Do you need the "if" statement here? CreateFixedObject should work whether or not there are other fixed objects, I think.

[efriedma]

LGTM

docs/LangRef.rst

**Integer constants**
     :ref:`integer <t_integer>` type. Negative numbers may be used with
     integer types.
 **Floating-point constants**
     Floating-point constants use standard decimal notation (e.g.
     123.421), exponential notation (e.g. 1.23421e+2), or a more precise
     hexadecimal notation (see below). The assembler requires the exact
     decimal value of a floating-point constant. For example, the
     assembler accepts 1.25 but rejects 1.3 because 1.3 is a repeating
-    decimal in binary. Floating-point constants must have a 
+    decimal in binary. Floating-point constants must have a
     :ref:`floating-point <t_floating>` type.
 **Null pointer constants**
     The identifier '``null``' is recognized as a null pointer constant
     and must be of :ref:`pointer type <t_pointer>`.
 **Token constants**
     The identifier '``none``' is recognized as an empty token constant
     and must be of :ref:`token type <t_token>`.
 

 ``fptosi (CST to TYPE)``
     Convert a floating-point constant to the corresponding signed
     integer constant. TYPE must be a scalar or vector integer type. CST
     must be of scalar or vector floating-point type. Both CST and TYPE
     must be scalars, or vectors of the same number of elements. If the
     value won't fit in the integer type, the result is a
     :ref:`poison value <poisonvalues>`.
 ``uitofp (CST to TYPE)``
-    Convert an unsigned integer constant to the corresponding 
+    Convert an unsigned integer constant to the corresponding
     floating-point constant. TYPE must be a scalar or vector floating-point
     type.  CST must be of scalar or vector integer type. Both CST and TYPE must
     be scalars, or vectors of the same number of elements.
 ``sitofp (CST to TYPE)``
     Convert a signed integer constant to the corresponding floating-point
     constant. TYPE must be a scalar or vector floating-point type.
     CST must be of scalar or vector integer type. Both CST and TYPE must
     be scalars, or vectors of the same number of elements.
.. code-block:: llvm
     ...
     !0 = !{"loop_header_weight", i64 100}
 
 Irreducible loop header weights are typically based on profile data.
 
 '``invariant.group``' Metadata
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-The experimental ``invariant.group`` metadata may be attached to 
+The experimental ``invariant.group`` metadata may be attached to
 ``load``/``store`` instructions referencing a single metadata with no entries.
 The existence of the ``invariant.group`` metadata on the instruction tells
 the optimizer that every ``load`` and ``store`` to the same pointer operand
 can be assumed to load or store the same
 value (but see the ``llvm.launder.invariant.group`` intrinsic which affects
 when two pointers are considered the same). Pointers returned by bitcast or
 getelementptr with only zero indices are considered the same.
 

 """""""""
 
 The '``fadd``' instruction returns the sum of its two operands.
 
 Arguments:
 """"""""""
 
 The two arguments to the '``fadd``' instruction must be
-:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of 
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
 floating-point values. Both arguments must have identical types.
 
 Semantics:
 """"""""""
 
 The value produced is the floating-point sum of the two operands.
 This instruction is assumed to execute in the default :ref:`floating-point
-environment <floatenv>`. 
+environment <floatenv>`.
 This instruction can also take any number of :ref:`fast-math
 flags <fastmath>`, which are optimization hints to enable otherwise
 unsafe floating-point optimizations:
 
 Example:
 """"""""
 
 .. code-block:: text

 
 Note that the '``fsub``' instruction is used to represent the '``fneg``'
 instruction present in most other intermediate representations.
 
 Arguments:
 """"""""""
 
 The two arguments to the '``fsub``' instruction must be
-:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of 
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
 floating-point values. Both arguments must have identical types.
 
 Semantics:
 """"""""""
 
 The value produced is the floating-point difference of the two operands.
 This instruction is assumed to execute in the default :ref:`floating-point
-environment <floatenv>`. 
+environment <floatenv>`.
 This instruction can also take any number of :ref:`fast-math
 flags <fastmath>`, which are optimization hints to enable otherwise
 unsafe floating-point optimizations:
 
 Example:
 """"""""
 
 .. code-block:: text

 """""""""
 
 The '``fmul``' instruction returns the product of its two operands.
 
 Arguments:
 """"""""""
 
 The two arguments to the '``fmul``' instruction must be
-:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of 
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
 floating-point values. Both arguments must have identical types.
 
 Semantics:
 """"""""""
 
 The value produced is the floating-point product of the two operands.
 This instruction is assumed to execute in the default :ref:`floating-point
-environment <floatenv>`. 
+environment <floatenv>`.
 This instruction can also take any number of :ref:`fast-math
 flags <fastmath>`, which are optimization hints to enable otherwise
 unsafe floating-point optimizations:
 
 Example:
 """"""""
 
 .. code-block:: text

 """""""""
 
 The '``fdiv``' instruction returns the quotient of its two operands.
 
 Arguments:
 """"""""""
 
 The two arguments to the '``fdiv``' instruction must be
-:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of 
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
 floating-point values. Both arguments must have identical types.
 
 Semantics:
 """"""""""
 
 The value produced is the floating-point quotient of the two operands.
 This instruction is assumed to execute in the default :ref:`floating-point
-environment <floatenv>`. 
+environment <floatenv>`.
 This instruction can also take any number of :ref:`fast-math
 flags <fastmath>`, which are optimization hints to enable otherwise
 unsafe floating-point optimizations:
 
 Example:
 """"""""
 
 .. code-block:: text

 
 The '``frem``' instruction returns the remainder from the division of
 its two operands.
 
 Arguments:
 """"""""""
 
 The two arguments to the '``frem``' instruction must be
-:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of 
+:ref:`floating-point <t_floating>` or :ref:`vector <t_vector>` of
 floating-point values. Both arguments must have identical types.
 
 Semantics:
 """"""""""
 
 The value produced is the floating-point remainder of the two operands.
 This is the same output as a libm '``fmod``' function, but without any
-possibility of setting ``errno``. The remainder has the same sign as the 
+possibility of setting ``errno``. The remainder has the same sign as the
 dividend.
 This instruction is assumed to execute in the default :ref:`floating-point
-environment <floatenv>`. 
+environment <floatenv>`.
 This instruction can also take any number of :ref:`fast-math
 flags <fastmath>`, which are optimization hints to enable otherwise
 unsafe floating-point optimizations:
 
 Example:
 """"""""
 
 .. code-block:: text

 The size of ``value`` must be larger than the size of ``ty2``. This
 implies that ``fptrunc`` cannot be used to make a *no-op cast*.
 
 Semantics:
 """"""""""
 
 The '``fptrunc``' instruction casts a ``value`` from a larger
 :ref:`floating-point <t_floating>` type to a smaller :ref:`floating-point
-<t_floating>` type.  
+<t_floating>` type.
 This instruction is assumed to execute in the default :ref:`floating-point
 environment <floatenv>`.
 
 Example:
 """"""""
 
 .. code-block:: llvm
 

 """"""""""
 
 Note that calling this intrinsic does not prevent function inlining or
 other aggressive transformations, so the value returned may not be that
 of the obvious source-language caller.
 
 This intrinsic is only implemented for x86.
 
+'``llvm.sponentry``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare i8* @llvm.sponentry()
+
+Overview:
+"""""""""
+
+The '``llvm.sponentry``' intrinsic returns the stack pointer value at
+the entry of the current function calling this intrinsic.
+
+Semantics:
+""""""""""
+
+Note in order to compute the correct stack pointer value, frame pointer
+must not be elminated.
+
+This intrinsic is only implemented for AArch64.
+
 '``llvm.frameaddress``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 
 ::
 
::
       declare i67 @llvm.fshl.i67(i67 %a, i67 %b, i67 %c)
       declare <2 x i32> @llvm.fshl.v2i32(<2 x i32> %a, <2 x i32> %b, <2 x i32> %c)
 
 Overview:
 """""""""
 
 The '``llvm.fshl``' family of intrinsic functions performs a funnel shift left:
 the first two values are concatenated as { %a : %b } (%a is the most significant
-bits of the wide value), the combined value is shifted left, and the most 
-significant bits are extracted to produce a result that is the same size as the 
-original arguments. If the first 2 arguments are identical, this is equivalent 
-to a rotate left operation. For vector types, the operation occurs for each 
-element of the vector. The shift argument is treated as an unsigned amount 
+bits of the wide value), the combined value is shifted left, and the most
+significant bits are extracted to produce a result that is the same size as the
+original arguments. If the first 2 arguments are identical, this is equivalent
+to a rotate left operation. For vector types, the operation occurs for each
+element of the vector. The shift argument is treated as an unsigned amount
 modulo the element size of the arguments.
 
 Arguments:
 """"""""""
 
 The first two arguments are the values to be concatenated. The third
 argument is the shift amount. The arguments may be any integer type or a
 vector with integer element type. All arguments and the return value must
::
       declare i67 @llvm.fshr.i67(i67 %a, i67 %b, i67 %c)
       declare <2 x i32> @llvm.fshr.v2i32(<2 x i32> %a, <2 x i32> %b, <2 x i32> %c)
 
 Overview:
 """""""""
 
 The '``llvm.fshr``' family of intrinsic functions performs a funnel shift right:
 the first two values are concatenated as { %a : %b } (%a is the most significant
-bits of the wide value), the combined value is shifted right, and the least 
-significant bits are extracted to produce a result that is the same size as the 
-original arguments. If the first 2 arguments are identical, this is equivalent 
-to a rotate right operation. For vector types, the operation occurs for each 
-element of the vector. The shift argument is treated as an unsigned amount 
+bits of the wide value), the combined value is shifted right, and the least
+significant bits are extracted to produce a result that is the same size as the
+original arguments. If the first 2 arguments are identical, this is equivalent
+to a rotate right operation. For vector types, the operation occurs for each
+element of the vector. The shift argument is treated as an unsigned amount
 modulo the element size of the arguments.
 
 Arguments:
 """"""""""
 
 The first two arguments are the values to be concatenated. The third
 argument is the shift amount. The arguments may be any integer type or a
 vector with integer element type. All arguments and the return value must

 
 .. code-block:: llvm
 
     ; Load several elements from array B and expand them in a vector.
     ; The number of loaded elements is equal to the number of '1' elements in the Mask.
     %Tmp = call <8 x double> @llvm.masked.expandload.v8f64(double* %Bptr, <8 x i1> %Mask, <8 x double> undef)
     ; Store the result in A
     call void @llvm.masked.store.v8f64.p0v8f64(<8 x double> %Tmp, <8 x double>* %Aptr, i32 8, <8 x i1> %Mask)
-    
     ; %Bptr should be increased on each iteration according to the number of '1' elements in the Mask.
     %MaskI = bitcast <8 x i1> %Mask to i8
     %MaskIPopcnt = call i8 @llvm.ctpop.i8(i8 %MaskI)
     %MaskI64 = zext i8 %MaskIPopcnt to i64
     %BNextInd = add i64 %BInd, %MaskI64
 
 
 Other targets may support this intrinsic differently, for example, by lowering it into a sequence of conditional scalar load operations and shuffles.

 
 
 .. code-block:: llvm
 
     ; Load elements from A.
     %Tmp = call <8 x double> @llvm.masked.load.v8f64.p0v8f64(<8 x double>* %Aptr, i32 8, <8 x i1> %Mask, <8 x double> undef)
     ; Store all selected elements consecutively in array B
     call <void> @llvm.masked.compressstore.v8f64(<8 x double> %Tmp, double* %Bptr, <8 x i1> %Mask)
-    
     ; %Bptr should be increased on each iteration according to the number of '1' elements in the Mask.
     %MaskI = bitcast <8 x i1> %Mask to i8
     %MaskIPopcnt = call i8 @llvm.ctpop.i8(i8 %MaskI)
     %MaskI64 = zext i8 %MaskIPopcnt to i64
     %BNextInd = add i64 %BInd, %MaskI64
 
 
 Other targets may support this intrinsic differently, for example, by lowering it into a sequence of branches that guard scalar store operations.
::
                                           metadata <rounding mode>,
                                           metadata <exception behavior>)
 
 Overview:
 """""""""
 
 The '``llvm.experimental.constrained.powi``' intrinsic returns the first operand
 raised to the (positive or negative) power specified by the second operand. The
-order of evaluation of multiplications is not defined. When a vector of 
+order of evaluation of multiplications is not defined. When a vector of
 floating-point type is used, the second argument remains a scalar integer value.
 
 
 Arguments:
 """"""""""
 
 The first argument and the return value are floating-point numbers of the same
 type.  The second argument is a 32-bit signed integer specifying the power to
::
       @llvm.experimental.constrained.nearbyint(<type> <op1>,
                                                metadata <rounding mode>,
                                                metadata <exception behavior>)
 
 Overview:
 """""""""
 
 The '``llvm.experimental.constrained.nearbyint``' intrinsic returns the first
-operand rounded to the nearest integer. It will not raise an inexact 
+operand rounded to the nearest integer. It will not raise an inexact
 floating-point exception if the operand is not an integer.
 
 
 Arguments:
 """"""""""
 
 The first argument and the return value are floating-point numbers of the same
 type.

include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
     /// The address of the GOT
     GLOBAL_OFFSET_TABLE,
 
     /// FRAMEADDR, RETURNADDR - These nodes represent llvm.frameaddress and
     /// llvm.returnaddress on the DAG.  These nodes take one operand, the index
     /// of the frame or return address to return.  An index of zero corresponds
     /// to the current function's frame or return address, an index of one to
     /// the parent's frame or return address, and so on.
-    FRAMEADDR, RETURNADDR, ADDROFRETURNADDR,
+    FRAMEADDR, RETURNADDR, ADDROFRETURNADDR, SPONENTRY,
 
     /// LOCAL_RECOVER - Represents the llvm.localrecover intrinsic.
     /// Materializes the offset from the local object pointer of another
     /// function to a particular local object passed to llvm.localescape. The
     /// operand is the MCSymbol label used to represent this offset, since
     /// typically the offset is not known until after code generation of the
     /// parent.
     LOCAL_RECOVER,

include/llvm/IR/Intrinsics.td

def int_gcwrite : Intrinsic<[],
                             [llvm_ptr_ty, llvm_ptr_ty, llvm_ptrptr_ty],
                             [IntrArgMemOnly, NoCapture<1>, NoCapture<2>]>;
 
 //===--------------------- Code Generator Intrinsics ----------------------===//
 //
 def int_returnaddress : Intrinsic<[llvm_ptr_ty], [llvm_i32_ty], [IntrNoMem]>;
 def int_addressofreturnaddress : Intrinsic<[llvm_ptr_ty], [], [IntrNoMem]>;
 def int_frameaddress  : Intrinsic<[llvm_ptr_ty], [llvm_i32_ty], [IntrNoMem]>;
+def int_sponentry  : Intrinsic<[llvm_ptr_ty], [], [IntrNoMem]>;
 def int_read_register  : Intrinsic<[llvm_anyint_ty], [llvm_metadata_ty],
                                    [IntrReadMem], "llvm.read_register">;
 def int_write_register : Intrinsic<[], [llvm_metadata_ty, llvm_anyint_ty],
                                    [], "llvm.write_register">;
 
 // Gets the address of the local variable area. This is typically a copy of the
 // stack, frame, or base pointer depending on the type of prologue.
 def int_localaddress : Intrinsic<[llvm_ptr_ty], [], [IntrNoMem]>;

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

case ISD::EH_SJLJ_SETUP_DISPATCH:
     if (Action == TargetLowering::Legal)
       Action = TargetLowering::Expand;
     break;
   case ISD::INIT_TRAMPOLINE:
   case ISD::ADJUST_TRAMPOLINE:
   case ISD::FRAMEADDR:
   case ISD::RETURNADDR:
   case ISD::ADDROFRETURNADDR:
+  case ISD::SPONENTRY:
     // These operations lie about being legal: when they claim to be legal,
     // they should actually be custom-lowered.
     Action = TLI.getOperationAction(Node->getOpcode(), Node->getValueType(0));
     if (Action == TargetLowering::Legal)
       Action = TargetLowering::Custom;
     break;
   case ISD::READCYCLECOUNTER:
     // READCYCLECOUNTER returns an i64, even if type legalization might have

lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::returnaddress:
     setValue(&I, DAG.getNode(ISD::RETURNADDR, sdl,
                              TLI.getPointerTy(DAG.getDataLayout()),
                              getValue(I.getArgOperand(0))));
     return nullptr;
   case Intrinsic::addressofreturnaddress:
     setValue(&I, DAG.getNode(ISD::ADDROFRETURNADDR, sdl,
                              TLI.getPointerTy(DAG.getDataLayout())));
     return nullptr;
+  case Intrinsic::sponentry:
+    setValue(&I, DAG.getNode(ISD::SPONENTRY, sdl,
+                             TLI.getPointerTy(DAG.getDataLayout())));
+    return nullptr;
   case Intrinsic::frameaddress:
     setValue(&I, DAG.getNode(ISD::FRAMEADDR, sdl,
                              TLI.getPointerTy(DAG.getDataLayout()),
                              getValue(I.getArgOperand(0))));
     return nullptr;
   case Intrinsic::read_register: {
     Value *Reg = I.getArgOperand(0);
     SDValue Chain = getRoot();

lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp

#endif
   case ISD::GlobalAddress:              return "GlobalAddress";
   case ISD::GlobalTLSAddress:           return "GlobalTLSAddress";
   case ISD::FrameIndex:                 return "FrameIndex";
   case ISD::JumpTable:                  return "JumpTable";
   case ISD::GLOBAL_OFFSET_TABLE:        return "GLOBAL_OFFSET_TABLE";
   case ISD::RETURNADDR:                 return "RETURNADDR";
   case ISD::ADDROFRETURNADDR:           return "ADDROFRETURNADDR";
   case ISD::FRAMEADDR:                  return "FRAMEADDR";
+  case ISD::SPONENTRY:                  return "SPONENTRY";
   case ISD::LOCAL_RECOVER:              return "LOCAL_RECOVER";
   case ISD::READ_REGISTER:              return "READ_REGISTER";
   case ISD::WRITE_REGISTER:             return "WRITE_REGISTER";
   case ISD::FRAME_TO_ARGS_OFFSET:       return "FRAME_TO_ARGS_OFFSET";
   case ISD::EH_DWARF_CFA:               return "EH_DWARF_CFA";
   case ISD::EH_RETURN:                  return "EH_RETURN";
   case ISD::EH_SJLJ_SETJMP:             return "EH_SJLJ_SETJMP";
   case ISD::EH_SJLJ_LONGJMP:            return "EH_SJLJ_LONGJMP";

lib/Target/AArch64/AArch64FastISel.cpp

while (Depth--) {
                                 SrcReg, /*IsKill=*/true, 0);
       assert(DestReg && "Unexpected LDR instruction emission failure.");
       SrcReg = DestReg;
     }
 
     updateValueMap(II, SrcReg);
     return true;
   }
+  case Intrinsic::sponentry: {
+    MachineFrameInfo &MFI = FuncInfo.MF->getFrameInfo();
+    MFI.setFrameAddressIsTaken(true);
+
+    // Make sure the computation is correct.
+    if (!Subtarget->getFrameLowering()->noFramePointerElim(*FuncInfo.MF))
+      report_fatal_error("Intrinsic::sponentry requires frame pointer.");
+
+    // SP = FP + Fixed Object + 16
+    unsigned numFixed = MFI.getNumFixedObjects();
+    MVT VT = TLI.getPointerTy(DL);
+    if (numFixed) {
+      // First fixed objected.
+      int FI = MFI.CreateFixedObject(4, 0, false);
+      unsigned ResultReg = createResultReg(TLI.getRegClassFor(VT));
+      BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
+              TII.get(AArch64::ADDXri), ResultReg)
+              .addFrameIndex(FI)
+              .addImm(0)
+              .addImm(0);
+
+      updateValueMap(II, ResultReg);
+    } else {
+      // SP = FP + 16
+      const AArch64RegisterInfo *RegInfo = Subtarget->getRegisterInfo();
+      unsigned FramePtr = RegInfo->getFrameRegister(*(FuncInfo.MF));
+      unsigned SrcReg = MRI.createVirtualRegister(&AArch64::GPR64RegClass);
+      BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
+              TII.get(TargetOpcode::COPY), SrcReg).addReg(FramePtr);
+
+      unsigned AddReg = emitAdd_ri_(VT, SrcReg, /*IsKill=*/false, 16);
+      SrcReg = AddReg;
+
+      updateValueMap(II, SrcReg);
+    }
+    return true;
+  }
   case Intrinsic::memcpy:
   case Intrinsic::memmove: {
     const auto *MTI = cast<MemTransferInst>(II);
     // Don't handle volatile.
     if (MTI->isVolatile())
       return false;
 
     // Disable inlining for memmove before calls to ComputeAddress.  Otherwise,

lib/Target/AArch64/AArch64ISelLowering.h

private:
   SDValue LowerBlockAddress(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerAAPCS_VASTART(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerDarwin_VASTART(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerWin64_VASTART(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerVASTART(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerVACOPY(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerVAARG(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
+  SDValue LowerSPONENTRY(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerEXTRACT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerSCALAR_TO_VECTOR(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerBUILD_VECTOR(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerVECTOR_SHUFFLE(SDValue Op, SelectionDAG &DAG) const;
   SDValue LowerEXTRACT_SUBVECTOR(SDValue Op, SelectionDAG &DAG) const;

lib/Target/AArch64/AArch64ISelLowering.cpp

SDValue AArch64TargetLowering::LowerOperation(SDValue Op,
   case ISD::FDIV:
     return LowerF128Call(Op, DAG, RTLIB::DIV_F128);
   case ISD::FP_ROUND:
     return LowerFP_ROUND(Op, DAG);
   case ISD::FP_EXTEND:
     return LowerFP_EXTEND(Op, DAG);
   case ISD::FRAMEADDR:
     return LowerFRAMEADDR(Op, DAG);
+  case ISD::SPONENTRY:
+    return LowerSPONENTRY(Op, DAG);
   case ISD::RETURNADDR:
     return LowerRETURNADDR(Op, DAG);
   case ISD::INSERT_VECTOR_ELT:
     return LowerINSERT_VECTOR_ELT(Op, DAG);
   case ISD::EXTRACT_VECTOR_ELT:
     return LowerEXTRACT_VECTOR_ELT(Op, DAG);
   case ISD::BUILD_VECTOR:
     return LowerBUILD_VECTOR(Op, DAG);
SDValue AArch64TargetLowering::LowerFRAMEADDR(SDValue Op,
   SDValue FrameAddr =
       DAG.getCopyFromReg(DAG.getEntryNode(), DL, AArch64::FP, VT);
   while (Depth--)
     FrameAddr = DAG.getLoad(VT, DL, DAG.getEntryNode(), FrameAddr,
                             MachinePointerInfo());
   return FrameAddr;
 }
 
+SDValue AArch64TargetLowering::LowerSPONENTRY(SDValue Op,
+                                              SelectionDAG &DAG) const {
+  MachineFrameInfo &MFI = DAG.getMachineFunction().getFrameInfo();
+  MFI.setFrameAddressIsTaken(true);
+
+  // Make sure the computation is correct.
+  if (!Subtarget->getFrameLowering()->noFramePointerElim(
+                                                      DAG.getMachineFunction()))
+      report_fatal_error("Intrinsic::sponentry requires frame pointer.");
+
+  unsigned numFixed = MFI.getNumFixedObjects();
+  EVT VT = getPointerTy(DAG.getDataLayout());
+  SDLoc DL(Op);
+  if (numFixed) {
+    int FI = MFI.CreateFixedObject(4, 0, false);
+    return DAG.getFrameIndex(FI, VT);
+  } else {
+    SDValue Offset = DAG.getConstant(16, DL, MVT::i64);
+    SDValue FrameAddr =
+        DAG.getCopyFromReg(DAG.getEntryNode(), DL, AArch64::FP, VT);
+    return DAG.getNode(ISD::ADD, DL, VT, FrameAddr, Offset);
+  }
+}
+
 // FIXME? Maybe this could be a TableGen attribute on some registers and
 // this table could be generated automatically from RegInfo.
 unsigned AArch64TargetLowering::getRegisterByName(const char* RegName, EVT VT,
                                                   SelectionDAG &DAG) const {
   unsigned Reg = StringSwitch<unsigned>(RegName)
                        .Case("sp", AArch64::SP)
                        .Case("x1", AArch64::X1)
                        .Case("w1", AArch64::W1)

test/CodeGen/AArch64/sponentry.ll

+; RUN: llc -mtriple=aarch64-windows-msvc -disable-fp-elim %s -o - | FileCheck %s
+; RUN: not llc -mtriple=aarch64-windows-msvc %s -o - 2>&1 | FileCheck %s --check-prefix=NOTWORK
+; RUN: llc -mtriple=aarch64-windows-msvc -fast-isel -disable-fp-elim %s -o - | FileCheck %s
+; RUN: not llc -mtriple=aarch64-windows-msvc -fast-isel %s -o - 2>&1 | FileCheck %s --check-prefix=NOTWORK
+
+@env2 = common dso_local global [24 x i64]* null, align 8
+
+define dso_local void @bar() {
+  %1 = call i8* @llvm.sponentry()
+  %2 = load [24 x i64]*, [24 x i64]** @env2, align 8
+  %3 = getelementptr inbounds [24 x i64], [24 x i64]* %2, i32 0, i32 0
+  %4 = bitcast i64* %3 to i8*
+  %5 = call i32 @_setjmpex(i8* %4, i8* %1) #2
+  ret void
+}
+
+; CHECK: bar:
+; CHECK: mov     x29, sp
+; CHECK: add     x1, x29, #16
+; CEHCK: bl      _setjmpex
+
+define dso_local void @foo([24 x i64]*) {
+  %2 = alloca [24 x i64]*, align 8
+  %3 = alloca i32, align 4
+  %4 = alloca [100 x i32], align 4
+  store [24 x i64]* %0, [24 x i64]** %2, align 8
+  %5 = call i8* @llvm.sponentry()
+  %6 = load [24 x i64]*, [24 x i64]** %2, align 8
+  %7 = getelementptr inbounds [24 x i64], [24 x i64]* %6, i32 0, i32 0
+  %8 = bitcast i64* %7 to i8*
+  %9 = call i32 @_setjmpex(i8* %8, i8* %5)
+  store i32 %9, i32* %3, align 4
+  ret void
+}
+
+; CHECK: foo:
+; CHECK: sub     sp, sp, #448
+; CHECK: add     x29, sp, #432
+; CHECK: add     x1, x29, #16
+; CEHCK: bl      _setjmpex
+
+define dso_local void @var_args(i8*, ...) {
+  %2 = alloca i8*, align 8
+  %3 = alloca i8*, align 8
+  store i8* %0, i8** %2, align 8
+  %4 = bitcast i8** %3 to i8*
+  call void @llvm.va_start(i8* %4)
+  %5 = load i8*, i8** %3, align 8
+  %6 = getelementptr inbounds i8, i8* %5, i64 8
+  store i8* %6, i8** %3, align 8
+  %7 = bitcast i8* %5 to i32*
+  %8 = load i32, i32* %7, align 8
+  %9 = bitcast i8** %3 to i8*
+  call void @llvm.va_end(i8* %9)
+  %10 = call i8* @llvm.sponentry()
+  %11 = load [24 x i64]*, [24 x i64]** @env2, align 8
+  %12 = getelementptr inbounds [24 x i64], [24 x i64]* %11, i32 0, i32 0
+  %13 = bitcast i64* %12 to i8*
+  %14 = call i32 @_setjmpex(i8* %13, i8* %10) #3
+  ret void
+}
+
+; CHECK: var_args:
+; CHECK: sub     sp, sp, #96
+; CHECK: add     x29, sp, #16
+; CHECK: add     x1, x29, #80
+; CEHCK: bl      _setjmpex
+
+define dso_local void @manyargs(i64 %x1, i64 %x2, i64 %x3, i64 %x4, i64 %x5, i64 %x6, i64 %x7, i64 %x8, i64 %x9, i64 %x10) {
+  %1 = call i8* @llvm.sponentry()
+  %2 = load [24 x i64]*, [24 x i64]** @env2, align 8
+  %3 = getelementptr inbounds [24 x i64], [24 x i64]* %2, i32 0, i32 0
+  %4 = bitcast i64* %3 to i8*
+  %5 = call i32 @_setjmpex(i8* %4, i8* %1) #2
+  ret void
+}
+
+; CHECK: manyargs:
+; CHECK: stp     x29, x30, [sp, #-16]!
+; CHECK: add     x1, x29, #16
+
+; Function Attrs: nounwind readnone
+declare i8* @llvm.sponentry()
+
+; Function Attrs: returns_twice
+declare dso_local i32 @_setjmpex(i8*, i8*)
+
+; Function Attrs: nounwind
+declare void @llvm.va_start(i8*) #1
+
+; Function Attrs: nounwind
+declare void @llvm.va_end(i8*) #1
+
+; NOTWORK: Intrinsic::sponentry requires frame pointer